[cfe-dev] Preprocessor::LookNext

[Chris Lattner]

On Jul 8, 2008, at 7:28 AM, Argiris Kirtzidis wrote:

> Chris Lattner wrote:
>> Is there a specific problem with calling LookAhead(0)?  To me, this  
>> is a performance optimization tweak... I'd rather do this sort of  
>> thing when more of the C++ front-end is in place.  This would give  
>> us better grounds for profiling and determine whether it is  
>> actually a win in the end.
>
> Yes, it is only a performance optimization tweak; I wanted to find  
> out if it is acceptable to optimize just LookAhead(0) (which, by the  
> way, is currently the only use for LookAhead).

Ok.

>> [....]    While always having peektok available is nice, it adds  
>> significant cost to the hotest part of the C preprocessor.  I would  
>> expect a significant slowdown from the patch on large .i files with  
>> -Eonly for example (which spends almost all time in this code).
>
> With a slight modification, the PeekedToken check can be avoided for  
> the common case:
>
>>  void Lex(Token &Result) {
>>    if (CurLexer)
>>      CurLexer->Lex(Result);
>>    else if (CurTokenLexer)
>>      CurTokenLexer->Lex(Result);
>>    else {
>>      // We have a peeked token that hasn't been consumed yet.
>>      Result = PeekedToken;
>>      ConsumedPeekedToken();
>>      return;
>>    }
>>  }
>
> LookNext would cache CurLexer,CurTokenLexer before setting them to  
> null, and ConsumedPeekedToken would restore them.

I like this approach about 10 times better than the other one :) it is  
very clever and simple!  The good thing about this is that it is an  
easy thing to measure.  Can you whip up a patch that implements this  
approach?  I don't think it will have a measurable performance impact  
on the preprocessor (unlike the other approach).  If so, I would be  
very happy to commit it and simplify the (existing and future) parser  
to use it.

-Chris