[cfe-dev] annotation token documentation

[Chris Lattner]

After my little fun hacking on annotation tokens, I decided to write  
up some documentation so that I'd remember how they work the next time  
around:
http://clang.llvm.org/docs/InternalsManual.html#AnnotationToken

Argiris, I've come around to being a true believer in the approach, I  
think it is very nifty.  I have a couple of concerns about the current  
implementation though: first it looks like the template-id handling  
code is not really complete.  I added some FIXME's to the  
documentation above, I assume that the implementation is just in  
progress.  I'd appreciate it if someone clueful could review the doc.

Second, I think it would be interesting to consider handling the  
"negative" case (when an identifier is not a type) by turning the  
identifier into an annotation token as well, representing the variable  
it resolves to (which could be null if it refers to nothing).  Right  
now, if we see an identifier in the token stream, we don't know if  
that means it is unresolved or whether it means that it is a variable  
and backtracking already analyzed but did not change it.

Argiris, do you think it would make sense to make a new annot_variable  
that contains a (potentially null) pointer to a variable decl?  Even  
without backtracking, this would reduce name lookups in C: right now  
we try to decide if something is a type (which requires a lookup) then  
decide its not, so we handle it as a variable (requiring another  
lookup).  Specifically, things like:

int x = sizeof(x);

first call Sema::isTypeName then call Sema::LookupDecl.  If the parser  
just called a "lookup the decl for this identifier" method, it could  
annotate it once and avoid the second lookup.

-Chris