[cfe-commits] r61172 - /cfe/trunk/lib/Lex/PTHLexer.cpp
clattner at apple.com
Wed Dec 17 16:56:47 PST 2008
On Dec 17, 2008, at 4:09 PM, Daniel Dunbar wrote:
> I'm not sure this is a good idea. At the least, I think it should be
> behind a routine like PeekNextTokenKind(). The file format should be
> kept separate from the internal implementation, and this is spreading
> the parsing of the file format to new places.
I agree that factoring this better would be nice.
> In addition, what does this save? In the common case won't we end up
> discarding 0 tokens? If the PTHLexer always had the next token
> available, it seems to me that this means that in the common case we
> would never be doing any unnecessary work. But I don't really grok
> this code, so...
It saves identifier lookup, and reduce dependencies on GetToken. Ted
and I discussed that GetToken and the local one-token cache should go
away for perf reasons. The PTH generator should compute and insert an
EOM token explicitly in the stream where needed.
More information about the cfe-commits