[cfe-dev] Identifier text in raw lexing

[Chris Lattner]

On Dec 4, 2010, at 2:36 AM, Abramo Bagnara wrote:
> Currently, during lexing in LexingRawMode only the literal text is
> collected (setLiteralData/getLiteralData), while the identifier text is
> lost.

Hi Abramo,

I'm not sure what you're trying to accomplish here: it sounds like you want to get the "identifier text" from the token itself without getting the spelling.  Is this correct? If so, are you looking for a performance win, a functionality improvement, or something else?  What problem are you trying to solve?

-Chris

> 
> We'd like to save the start of identifier text in raw lexing mode using
> the same shared pointer to permit to raw lexing client to avoid to do a
> lot of extra work to get the identifier text.
> 
> To implement that:
> 
> - we'd insert a raw_identifer token kind
> - we'd add Token::isIdentifier method
> - we'd add Token::get/setRawIdentiferData methods
> - we'd change Token::getIdentifierInfo to return 0 on raw_identifier
> - we'd replace occurrences of check for tok::identifier with
> Token::isIdentifer()
> 
> The performance impact should be definitely zero, but we are willing to
> do the benchmark you ask to proof that.
> 
> A patch following the above guidelines have some chances to be accepted
> upstream?
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev