[cfe-commits] r108175 - in /cfe/trunk: include/clang/Lex/Preprocessor.h lib/Lex/PPCaching.cpp

[Argyrios Kyrtzidis]

On Jul 12, 2010, at 10:19 PM, Chris Lattner wrote:

> 
> On Jul 12, 2010, at 11:49 AM, Argyrios Kyrtzidis wrote:
> 
>> Author: akirtzidis
>> Date: Mon Jul 12 13:49:30 2010
>> New Revision: 108175
>> 
>> URL: http://llvm.org/viewvc/llvm-project?rev=108175&view=rev
>> Log:
>> If we are past tok::eof and in caching lex mode, avoid caching repeated tok::eofs.
> 
> I'm confused about this: why is someone calling Lex after tok::eof got returned?

Why not ? :-)
You are right, the check is totally unnecessary, I removed it at r108199.

-Argiris

> 
> -Chris
> 
> 
>> 
>> Modified:
>>   cfe/trunk/include/clang/Lex/Preprocessor.h
>>   cfe/trunk/lib/Lex/PPCaching.cpp
>> 
>> Modified: cfe/trunk/include/clang/Lex/Preprocessor.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/Preprocessor.h?rev=108175&r1=108174&r2=108175&view=diff
>> ==============================================================================
>> --- cfe/trunk/include/clang/Lex/Preprocessor.h (original)
>> +++ cfe/trunk/include/clang/Lex/Preprocessor.h Mon Jul 12 13:49:30 2010
>> @@ -871,7 +871,11 @@
>>  //===--------------------------------------------------------------------===//
>>  // Caching stuff.
>>  void CachingLex(Token &Result);
>> -  bool InCachingLexMode() const { return CurPPLexer == 0 && CurTokenLexer == 0;}
>> +  bool InCachingLexMode() const {
>> +    // If the Lexer pointers are 0 and IncludeMacroStack is empty, it means
>> +    // that we are past EOF, not that we are in CachingLex mode.
>> +    return CurPPLexer == 0 && CurTokenLexer == 0 && !IncludeMacroStack.empty();
>> +  }
>>  void EnterCachingLexMode();
>>  void ExitCachingLexMode() {
>>    if (InCachingLexMode())
>> 
>> Modified: cfe/trunk/lib/Lex/PPCaching.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PPCaching.cpp?rev=108175&r1=108174&r2=108175&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/Lex/PPCaching.cpp (original)
>> +++ cfe/trunk/lib/Lex/PPCaching.cpp Mon Jul 12 13:49:30 2010
>> @@ -45,13 +45,19 @@
>> }
>> 
>> void Preprocessor::CachingLex(Token &Result) {
>> +  if (!InCachingLexMode())
>> +    return;
>> +
>>  if (CachedLexPos < CachedTokens.size()) {
>>    Result = CachedTokens[CachedLexPos++];
>>    return;
>>  }
>> 
>>  ExitCachingLexMode();
>> -  Lex(Result);
>> +  // True if we consumed everything already.
>> +  bool PastEOF =  CurPPLexer == 0 && CurTokenLexer == 0;
>> +  if (!PastEOF)
>> +    Lex(Result);
>> 
>>  if (!isBacktrackEnabled()) {
>>    // All cached tokens were consumed.
>> @@ -60,10 +66,12 @@
>>    return;
>>  }
>> 
>> -  // Cache the lexed token.
>> +  // Cache the lexed token if it's not a repeated tok::eof.
>>  EnterCachingLexMode();
>> -  CachedTokens.push_back(Result);
>> -  ++CachedLexPos;
>> +  if (!PastEOF) {
>> +    CachedTokens.push_back(Result);
>> +    ++CachedLexPos;
>> +  }
>> }
>> 
>> void Preprocessor::EnterCachingLexMode() {
>> 
>> 
>> _______________________________________________
>> cfe-commits mailing list
>> cfe-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>