[cfe-dev] annotation token documentation

[Chris Lattner]

On Jan 5, 2009, at 10:35 PM, Douglas Gregor wrote:
>> Argiris, I've come around to being a true believer in the approach, I
>> think it is very nifty.  I have a couple of concerns about the  
>> current
>> implementation though: first it looks like the template-id handling
>> code is not really complete.
>
> The completely-broken template-id handling is my fault. There's a  
> whole bunch of code that needs to get written, but it's relatively  
> low priority (for now).

Ah ok.  I'm completely fine with it being broken for now, can you  
please review the comments I added to the internals doc and update if  
they are wrong?

>> Second, I think it would be interesting to consider handling the
>> "negative" case (when an identifier is not a type) by turning the
>> identifier into an annotation token as well, representing the  
>> variable
>> it resolves to (which could be null if it refers to nothing).  Right
>> now, if we see an identifier in the token stream, we don't know if
>> that means it is unresolved or whether it means that it is a variable
>> and backtracking already analyzed but did not change it.
>
>
> This would be great. Right now, we have isTypeName,  
> isCurrentClassName, and isTemplateName, all of which call LookupDecl  
> and then return a single yes or no answer. Some code paths call  
> these routines multiple times. We'd be better off with a "what does  
> this identifier?" action that tells us what we're looking at... and  
> then we can store that information in an annotation token.

Cool, I completely agree.

> One thing we have to be careful of is that a given identifier can be  
> looked up in different ways in different contexts, so we have to  
> know when to throw away the results of a previous lookup. For  
> example, and identifier 'x' prior to a '::' is looked up differently  
> that 'x' if it is followed by something else.

Is this a purely lexical thing?  Can backtracking change the decision  
about what form a token is?  The current code does a pretty good job  
(afaik) handling a different than a::b using lookahead for example, is  
this sufficient or are there other crazy cases beyond what we're  
already handling?

-Chris