<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Jan 10, 2014 at 2:35 PM, Alp Toker <span dir="ltr"><<a href="mailto:alp@nuanti.com" target="_blank">alp@nuanti.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
On 10/01/2014 20:50, Sean Silva wrote:<div><div class="h5"><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
<br>
On Fri, Jan 10, 2014 at 7:37 AM, Alp Toker <<a href="mailto:alp@nuanti.com" target="_blank">alp@nuanti.com</a> <mailto:<a href="mailto:alp@nuanti.com" target="_blank">alp@nuanti.com</a>>> wrote:<br>
<br>
Author: alp<br>
Date: Fri Jan 10 08:37:02 2014<br>
New Revision: 198942<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=198942&view=rev" target="_blank">http://llvm.org/viewvc/llvm-<u></u>project?rev=198942&view=rev</a><br>
Log:<br>
Remove unexpected code completion handling from ConsumeToken()<br>
<br>
With this change tok::code_completion is finally handled<br>
exclusively as a<br>
special token kind like other tokens that need special treatment.<br>
<br>
All callers have been updated to use the specific token<br>
consumption methods and<br>
the parser has a clear idea the current token isn't special by the<br>
time<br>
ConsumeToken() gets called, so this has been unreachable for some<br>
time.<br>
<br>
ConsumeAnyToken() behaviour is unchanged and will continue to support<br>
unexpected code completion as part of the special token path.<br>
<br>
This survived an amount of fuzzing and validation, but please ping<br>
the list if<br>
you hit a code path that previously relied on the old unexpected<br>
handler and<br>
now asserts.<br>
<br>
Modified:<br>
cfe/trunk/include/clang/Parse/<u></u>Parser.h<br>
<br>
Modified: cfe/trunk/include/clang/Parse/<u></u>Parser.h<br>
URL:<br>
<a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Parse/Parser.h?rev=198942&r1=198941&r2=198942&view=diff" target="_blank">http://llvm.org/viewvc/llvm-<u></u>project/cfe/trunk/include/<u></u>clang/Parse/Parser.h?rev=<u></u>198942&r1=198941&r2=198942&<u></u>view=diff</a><br>
==============================<u></u>==============================<u></u>==================<br>
--- cfe/trunk/include/clang/Parse/<u></u>Parser.h (original)<br>
+++ cfe/trunk/include/clang/Parse/<u></u>Parser.h Fri Jan 10 08:37:02 2014<br>
@@ -273,16 +273,12 @@ public:<br>
bool ParseTopLevelDecl(<u></u>DeclGroupPtrTy &Result);<br>
<br>
/// ConsumeToken - Consume the current 'peek token' and lex the<br>
next one.<br>
- /// This does not work with all kinds of tokens: strings and<br>
specific other<br>
- /// tokens must be consumed with custom methods below. This<br>
returns the<br>
- /// location of the consumed token.<br>
+ /// This does not work with special tokens: string literals,<br>
code completion<br>
+ /// and balanced tokens must be handled using the specific<br>
consume methods.<br>
+ /// Returns the location of the consumed token.<br>
SourceLocation ConsumeToken() {<br>
assert(!isTokenSpecial() &&<br>
"Should consume special tokens with Consume*Token");<br>
-<br>
- if (LLVM_UNLIKELY(Tok.is(tok::<u></u>code_completion)))<br>
- return handleUnexpectedCodeCompletion<u></u>Token();<br>
<br>
<br>
Is there some central place where we can mark tok::code_completion as unlikely?<br>
</blockquote>
<br></div></div>
The unexpected code completion tokens that used to end up here are checked for precisely now, so the parser will efficiently reach code completion in every case without needing to get as far as ConsumeToken().<br>
<br>
The checks themselves are of the form Tok.is(tok::code_completion) or otherwise exist in switch statements. Perhaps try an LLVM_UNLIKELY() in Tok.is() / Tok.isNot()? Not sure if it'd add value -- there are plenty of other equally unlikely tokens like tok::eof after all.<br>
<br>
To venture a guess, the next lowest hanging fruit might rather be ConsumeAnyToken() which has to test and branch for various kinds of special token kinds and gets called in tight loops. I don't have numbers and it's never showed up on profiles so I'd be still be cautious to spend too much time micro-optimizing there.<br>
</blockquote><div><br></div><div>Ok. Sounds like these code sequences will be a good fit for the upcoming PGO work.</div><div><br></div><div>-- Sean Silva</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Alp.<br>
<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
<br>
-- Sean Silva<br>
<br>
-<br>
PrevTokLocation = Tok.getLocation();<br>
PP.Lex(Tok);<br>
return PrevTokLocation;<br>
@@ -329,7 +325,7 @@ private:<br>
/// isTokenSpecial - True if this token requires special<br>
consumption methods.<br>
bool isTokenSpecial() const {<br>
return isTokenStringLiteral() || isTokenParen() ||<br>
isTokenBracket() ||<br>
- isTokenBrace();<br>
+ isTokenBrace() || Tok.is(tok::code_completion);<br>
}<br>
<br>
/// \brief Returns true if the current token is '=' or is a<br>
type of '='.<br>
<br>
<br>
______________________________<u></u>_________________<br>
cfe-commits mailing list<br></div>
<a href="mailto:cfe-commits@cs.uiuc.edu" target="_blank">cfe-commits@cs.uiuc.edu</a> <mailto:<a href="mailto:cfe-commits@cs.uiuc.edu" target="_blank">cfe-commits@cs.uiuc.<u></u>edu</a>><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits" target="_blank">http://lists.cs.uiuc.edu/<u></u>mailman/listinfo/cfe-commits</a><br>
<br>
<br><span class="HOEnZb"><font color="#888888">
</font></span></blockquote><span class="HOEnZb"><font color="#888888">
<br>
-- <br>
<a href="http://www.nuanti.com" target="_blank">http://www.nuanti.com</a><br>
the browser experts<br>
<br>
</font></span></blockquote></div><br></div></div>