[cfe-dev] Annotation tokens

[Chris Lattner]

On Aug 26, 2008, at 10:31 AM, Argiris Kirtzidis wrote:

> Doug Gregor wrote:
>> Performance-wise, tentative parsing can be improved by allowing it to
>> annotate the token stream with new "already parsed as a..." tokens as
>> it goes through. Good candidates for this kind of annotation are
>> qualified names (which Chris mentioned) and template-ids (GCC does  
>> the
>> latter).
>>
>
> I think this is a good idea that can be incorporated in Clang. This  
> won't only be useful for improving performance of tentative parsing  
> but also as a way to cleanly parse qualified names.

To be honest, I'm not sure exactly how this will shape up, and I have  
a hard time understanding how it will fit into the rest of the  
parser.  I'm slightly concerned that this is an abuse of  the Token  
abstraction (which represents things coming out of the lexer), and we  
don't want to make token any larger.  Perhaps a new magic token Kind,  
where the identifier pointer was a pointer to a "DeclTy" or something  
like that would be ok.

If you can make it clean, go for it.  :)

-Chris