[cfe-dev] Identifier text in raw lexing

Abramo Bagnara abramo.bagnara at gmail.com
Sat Dec 4 02:36:07 PST 2010


Currently, during lexing in LexingRawMode only the literal text is
collected (setLiteralData/getLiteralData), while the identifier text is
lost.

We'd like to save the start of identifier text in raw lexing mode using
the same shared pointer to permit to raw lexing client to avoid to do a
lot of extra work to get the identifier text.

To implement that:

- we'd insert a raw_identifer token kind
- we'd add Token::isIdentifier method
- we'd add Token::get/setRawIdentiferData methods
- we'd change Token::getIdentifierInfo to return 0 on raw_identifier
- we'd replace occurrences of check for tok::identifier with
Token::isIdentifer()

The performance impact should be definitely zero, but we are willing to
do the benchmark you ask to proof that.

A patch following the above guidelines have some chances to be accepted
upstream?



More information about the cfe-dev mailing list