[cfe-dev] Preprocessor::LookNext

[Argiris Kirtzidis]

Chris Lattner wrote:
> Is there a specific problem with calling LookAhead(0)?  To me, this is 
> a performance optimization tweak... I'd rather do this sort of thing 
> when more of the C++ front-end is in place.  This would give us better 
> grounds for profiling and determine whether it is actually a win in 
> the end.

Yes, it is only a performance optimization tweak; I wanted to find out 
if it is acceptable to optimize just LookAhead(0) (which, by the way, is 
currently the only use for LookAhead).

>
> Adding a "LookNext()" method to the preprocessor as a helper for 
> LookAhead(0) would be ok if you think it would be significantly useful 
> though.  Also, LookAhead could obviously be improved :).
>
> [....]    While always having peektok available is nice, it adds 
> significant cost to the hotest part of the C preprocessor.  I would 
> expect a significant slowdown from the patch on large .i files with 
> -Eonly for example (which spends almost all time in this code).

With a slight modification, the PeekedToken check can be avoided for the 
common case:

>   void Lex(Token &Result) {
>     if (CurLexer)
>       CurLexer->Lex(Result);
>     else
>       CurTokenLexer->Lex(Result);
>     else {
>       // We have a peeked token that hasn't been consumed yet.
>       Result = PeekedToken;
>       ConsumedPeekedToken();
>       return;
>     }
>   }

LookNext would cache CurLexer,CurTokenLexer before setting them to null, 
and ConsumedPeekedToken would restore them.

> are you sure it would be completely unreasonable to refactor the C++ 
> grammar to avoid the look-ahead in the common case?

I'll go ahead with GetLookAhead; after I post a patch and the code is 
there for reviewing, it will be a lot easier to discuss about this.


-Argiris