[cfe-dev] Using the Lexer...

[Chris Lattner]

On Feb 11, 2009, at 6:31 PM, Alexei Svitkine wrote:

> Well obviously I wasn't using it right - I'm just saying it would be
> helpful (for people just starting with the API) if this was detected
> rather than waiting to crash.
>
> Basically what was happening was I was creating the Lexer on the stack
> and passing in the preprocessor as a parameter, rather than using the
> Preprocessor to lex like the driver does.
>
> Since EnterMainSourceFile() was never called, CurLexer was never set
> in PPLexerChange.cpp, and when Preprocessor::HandleEndOfFile() was
> called it assumed since CurLexer was NULL, that there was a
> CurPTHLexer and tried to use that, resulting in a crash. This would
> only happen when we got to the EOF since before that everything was
> running fine.
>
> I'm just suggesting that since such a use is clearly wrong, this
> should be detected early on. Perhaps in the constructor of Lexer that
> takes the Preprocessor parameter. Or when trying to get a token
> without first calling EnterMainSourceFile(). Or some other even better
> place.

I'd be happy to add an assert.  Can you please suggest one that would  
have caught this?

-Chris