[cfe-dev] Lexing empty macros

[Chris Lattner]

On Apr 25, 2008, at 11:35 AM, Emerson Murphy-Hill wrote:

> I'm having a bit of difficulty with the lexer.  It's been working  
> dandy for me for the most part, but has some problems when lexing  
> over macro expansions that are empty.

Please paste the code you're using to lex over this.  The "clang -dump- 
tokens" option presumably works on this, right?

> is no problem.  But defining the macro thusly:
>
> #ifndef MyMacro
> #define MyMacro
> #endif
>
> or:
>
> #ifndef MyMacro
> #define MyMacro /*some comment*/
> #endif
>
> causes a EXC_BAD_ACCESS here:
>
> #0	clang::TokenLexer::isAtEnd at TokenLexer.h:122
> #1	clang::TokenLexer::Lex at TokenLexer.cpp:279
> #2	clang::Preprocessor::Lex at Preprocessor.h:262
> #3	clang::Preprocessor::HandleMacroExpandedIdentifier at  
> PPMacroExpansion.cpp:205
> #4	clang::Preprocessor::HandleIdentifier at Preprocessor.cpp:532
> #5	clang::Lexer::LexIdentifier at Lexer.cpp:534
> #6	clang::Lexer::LexTokenInternal at Lexer.cpp:1314
> #7	clang::Lexer::Lex at Lexer.h:141
>
> (Actually, in the comment case, I'm not entirely sure if the comment  
> gets expanded or simply removed by the preprocessor before expansion.)

Are you sure you're not lexing past EOF?

> This problem seems odd because the source file is getting parsed by  
> clang correctly the first time around (for instance, I can run the  
> clang driver and emit HTML, no problem).  But when I make a new  
> lexer (re-using the preprocessor from the original parsing), I get  
> the above errors.  Is there some lexer or preprocessor setting that  
> I'm missing?
>
when reusing a preprocessor like this, be aware that you're starting  
out with the full t-u full of macro definitions already installed.   
This means that the macro is already defined, so the #ifndef block  
isn't entered.

-Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20080425/c4ff0b4f/attachment.html>