akyrtzi at gmail.com
Wed Jul 9 16:50:16 PDT 2008
Chris Lattner wrote:
>>> +++ lib/Lex/PPLexerChange.cpp (working copy)
>>> @@ -85,6 +85,10 @@
>>> // size we new to a multiple of 16 tokens. If the previous buffer
>>> has space
>>> // left, we can just grow it. This means we only have to do the
>>> new 1/16th as
>>> // often.
>>> + // Optimized LookAhead(0) case.
>>> + if (N == 0)
>>> + return LookNext();
>>> Token *LookaheadTokens = new Token[N+1];
>>> Is this correct for the LookAhead(1) case?
>> I don't quite understand what you mean, LookAhead(1) isn't affected
>> by this change, can you elaborate ?
> I haven't looked at the code, I just want to make sure that
> LookAhead(1) or LookAhead(2) doesn't get off-by-one due to the peeked
LookAhead isn't called recursively, it collects all tokens through
Lex(), into the LookaheadTokens array.
If a peeked token exists, like when calling "LookAhead(0);
LookAhead(1);", it goes like this:
There's a peeked token now.
-LookAhead consumes through Lex() the peeked token
-Peeked token is invalidated, Lex() will not use it again.
-LookAhead consumes one more token by 'normal' Lex().
-Previously peeked token plus one more enter the token stream.
-Previously peeked token will be the next available token.
So, everything works as expected :-)
More information about the cfe-dev