[cfe-commits] r108175 - in /cfe/trunk: include/clang/Lex/Preprocessor.h lib/Lex/PPCaching.cpp

[Chris Lattner]

On Jul 12, 2010, at 11:49 AM, Argyrios Kyrtzidis wrote:

> Author: akirtzidis
> Date: Mon Jul 12 13:49:30 2010
> New Revision: 108175
> 
> URL: http://llvm.org/viewvc/llvm-project?rev=108175&view=rev
> Log:
> If we are past tok::eof and in caching lex mode, avoid caching repeated tok::eofs.

I'm confused about this: why is someone calling Lex after tok::eof got returned?

-Chris


> 
> Modified:
>    cfe/trunk/include/clang/Lex/Preprocessor.h
>    cfe/trunk/lib/Lex/PPCaching.cpp
> 
> Modified: cfe/trunk/include/clang/Lex/Preprocessor.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/Preprocessor.h?rev=108175&r1=108174&r2=108175&view=diff
> ==============================================================================
> --- cfe/trunk/include/clang/Lex/Preprocessor.h (original)
> +++ cfe/trunk/include/clang/Lex/Preprocessor.h Mon Jul 12 13:49:30 2010
> @@ -871,7 +871,11 @@
>   //===--------------------------------------------------------------------===//
>   // Caching stuff.
>   void CachingLex(Token &Result);
> -  bool InCachingLexMode() const { return CurPPLexer == 0 && CurTokenLexer == 0;}
> +  bool InCachingLexMode() const {
> +    // If the Lexer pointers are 0 and IncludeMacroStack is empty, it means
> +    // that we are past EOF, not that we are in CachingLex mode.
> +    return CurPPLexer == 0 && CurTokenLexer == 0 && !IncludeMacroStack.empty();
> +  }
>   void EnterCachingLexMode();
>   void ExitCachingLexMode() {
>     if (InCachingLexMode())
> 
> Modified: cfe/trunk/lib/Lex/PPCaching.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PPCaching.cpp?rev=108175&r1=108174&r2=108175&view=diff
> ==============================================================================
> --- cfe/trunk/lib/Lex/PPCaching.cpp (original)
> +++ cfe/trunk/lib/Lex/PPCaching.cpp Mon Jul 12 13:49:30 2010
> @@ -45,13 +45,19 @@
> }
> 
> void Preprocessor::CachingLex(Token &Result) {
> +  if (!InCachingLexMode())
> +    return;
> +
>   if (CachedLexPos < CachedTokens.size()) {
>     Result = CachedTokens[CachedLexPos++];
>     return;
>   }
> 
>   ExitCachingLexMode();
> -  Lex(Result);
> +  // True if we consumed everything already.
> +  bool PastEOF =  CurPPLexer == 0 && CurTokenLexer == 0;
> +  if (!PastEOF)
> +    Lex(Result);
> 
>   if (!isBacktrackEnabled()) {
>     // All cached tokens were consumed.
> @@ -60,10 +66,12 @@
>     return;
>   }
> 
> -  // Cache the lexed token.
> +  // Cache the lexed token if it's not a repeated tok::eof.
>   EnterCachingLexMode();
> -  CachedTokens.push_back(Result);
> -  ++CachedLexPos;
> +  if (!PastEOF) {
> +    CachedTokens.push_back(Result);
> +    ++CachedLexPos;
> +  }
> }
> 
> void Preprocessor::EnterCachingLexMode() {
> 
> 
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits