[cfe-dev] Lexing empty macros

[Emerson Murphy-Hill]

On Apr 25, 2008, at 2:45 PM, Chris Lattner wrote:
>
> On Apr 25, 2008, at 11:35 AM, Emerson Murphy-Hill wrote:
>> I'm having a bit of difficulty with the lexer.  It's been working  
>> dandy for me for the most part, but has some problems when lexing  
>> over macro expansions that are empty.
>
> Please paste the code you're using to lex over this.  The "clang - 
> dump-tokens" option presumably works on this, right?

dump-tokens works.  Here's some crashing code:

    Lexer* lexer = new Lexer(LocStart,*(provider- 
 >preprocessor()),startBuf,endBuf);

    Token current;
    do {
       lexer->Lex(current);
     }while(!current.is(tok::eof) && !current.is(tok::r_brace));

Works fine when the macro has something in it, crashes when it's empty.

>>
>> (Actually, in the comment case, I'm not entirely sure if the  
>> comment gets expanded or simply removed by the preprocessor before  
>> expansion.)
>
> Are you sure you're not lexing past EOF?

Pretty sure.

>
>> This problem seems odd because the source file is getting parsed by  
>> clang correctly the first time around (for instance, I can run the  
>> clang driver and emit HTML, no problem).  But when I make a new  
>> lexer (re-using the preprocessor from the original parsing), I get  
>> the above errors.  Is there some lexer or preprocessor setting that  
>> I'm missing?
>>
> when reusing a preprocessor like this, be aware that you're starting  
> out with the full t-u full of macro definitions already installed.   
> This means that the macro is already defined, so the #ifndef block  
> isn't entered.

That makes sense.

Thanks,

e

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20080425/5b505453/attachment.html>