<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">Hi Bruno,</div><br class=""><div><blockquote type="cite" class=""><div class="">On Dec 2, 2015, at 7:29 PM, Bruno Cardoso Lopes <<a href="mailto:bruno.cardoso@gmail.com" class="">bruno.cardoso@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">bruno created this revision.<br class="">bruno added reviewers: doug.gregor, akyrtzi.<br class="">bruno added subscribers: cfe-commits, dexonsmith.<br class=""><br class="">Consider the following ObjC++ snippet:<br class=""><br class=""> @protocol PA;<br class=""> @protocol PB;<br class=""><br class=""> @class NSArray<ObjectType>;<br class=""> typedef int some_t;<br class=""><br class=""> id<PA> FA(NSArray<id<PB>> *h, some_t group);<br class=""><br class="">This would hit an assertion in the parser after generating an annotation token while<br class="">trying to update the token cache:<br class=""><br class="">Assertion failed: (CachedTokens[CachedLexPos-1].getLastLoc() == Tok.getAnnotationEndLoc() && "The annotation should be until the most recent cached token")<br class="">...<br class="">7 clang::Preprocessor::AnnotatePreviousCachedTokens(clang::Token const&) + 494<br class="">8 clang::Parser::TryAnnotateTypeOrScopeTokenAfterScopeSpec(bool, bool, clang::CXXScopeSpec&, bool) + 1163<br class="">9 clang::Parser::TryAnnotateTypeOrScopeToken(bool, bool) + 361<br class="">10 clang::Parser::isCXXDeclarationSpecifier(clang::Parser::TPResult, bool*) + 598<br class="">...<br class=""><br class="">The cached preprocessor token in this case is:<br class=""><br class="">greatergreater '>>' Loc=<testcase.mm:7:24> <br class=""><br class="">while the annotation ("NSArray<id<PB>>") ends at "testcase.mm:7:25", hence the assertion.<br class="">The mismatch only happens because of the cached token length and the assertion should account for that.<br class=""><br class=""><a href="http://reviews.llvm.org/D15173" class="">http://reviews.llvm.org/D15173</a><br class=""><br class="">Files:<br class=""> lib/Lex/PPCaching.cpp<br class=""> test/Parser/objcxx11-protocol-in-template.mm<br class=""><br class="">Index: test/Parser/objcxx11-protocol-in-template.mm<br class="">===================================================================<br class="">--- test/Parser/objcxx11-protocol-in-template.mm<br class="">+++ test/Parser/objcxx11-protocol-in-template.mm<br class="">@@ -8,3 +8,11 @@<br class=""><br class=""> vector<id<P>> v;<br class=""> vector<vector<id<P>>> v2;<br class="">+<br class="">+@protocol PA;<br class="">+@protocol PB;<br class="">+<br class="">+@class NSArray<ObjectType>;<br class="">+typedef int some_t;<br class="">+<br class="">+id<PA> FA(NSArray<id<PB>> *h, some_t group);<br class="">Index: lib/Lex/PPCaching.cpp<br class="">===================================================================<br class="">--- lib/Lex/PPCaching.cpp<br class="">+++ lib/Lex/PPCaching.cpp<br class="">@@ -97,8 +97,19 @@<br class=""> void Preprocessor::AnnotatePreviousCachedTokens(const Token &Tok) {<br class=""> assert(Tok.isAnnotation() && "Expected annotation token");<br class=""> assert(CachedLexPos != 0 && "Expected to have some cached tokens");<br class="">- assert(CachedTokens[CachedLexPos-1].getLastLoc() == Tok.getAnnotationEndLoc()<br class="">- && "The annotation should be until the most recent cached token");<br class="">+<br class="">+ // The annotation should be until the most recent cached token. Since<br class="">+ // `Tok` length could be bigger than one (e.g. greatergreater '>>'), account<br class="">+ // for that cases before checking the assertion.<br class="">+ Token CachedLastTok = CachedTokens[CachedLexPos - 1];<br class="">+ unsigned CachedLastTokLoc = CachedLastTok.getLastLoc().getRawEncoding();<br class="">+ unsigned TokAnnEndLoc = Tok.getAnnotationEndLoc().getRawEncoding();<br class="">+ if (CachedLastTokLoc != TokAnnEndLoc && !CachedLastTok.isAnnotation())<br class="">+ CachedLastTokLoc += CachedLastTok.getLength() - 1;<br class="">+ (void)CachedLastTokLoc;<br class="">+ (void)TokAnnEndLoc;<br class="">+ assert(CachedLastTokLoc == TokAnnEndLoc &&<br class="">+ "The annotation should be until the most recent cached token");<br class=""><br class=""> // Start from the end of the cached tokens list and look for the token<br class=""> // that is the beginning of the annotation token.<br class=""></div></div></blockquote><div><br class=""></div><div>This would be better to be wrapped inside a "#ifndef NDEBUG” section.</div><div>Also the above is a bit ad-hoc, not quite general.</div><div><br class=""></div><div>How about doing something like:</div><div><br class=""></div></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div><div>Token CachedLastTok = CachedTokens[CachedLexPos - 1];</div></div><div><div>unsigned CachedLastTokLoc = CachedLastTok.getLastLoc().getLocWithOffset(CachedLastTok.getLength())</div></div><div><div><br class=""></div></div><div><div><div>unsigned TokAnnEndLoc = Tok.getAnnotationEndLoc();</div></div></div><div><div><div><div>unsigned TokAnnEndLocSize = Lexer:::MeasureTokenLength(SourceMgr.getSpellingLoc(Tok.getAnnotationEndLoc()))</div></div></div></div><div><div><div><div class="">TokAnnEndLocSize = TokAnnEndLoc.getLocWithOffset(TokAnnEndLocSize)</div></div></div></div></blockquote><div><div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">and compare the locations ? The above tries to account for possible different token sizes but the end locations should be the same.</div></div><div><br class=""></div><div><br class=""></div><blockquote type="cite" class=""><div class=""><div class=""><br class=""><br class=""><span id="cid:4ED373B5-7C77-42CE-8CA0-B52509556A6A@apple.com"><D15173.41709.patch></span></div></div></blockquote></div><br class=""></body></html>