[cfe-commits] [Request for approval] Identifier text in raw lexing

[Abramo Bagnara]

Il 05/12/2010 08:42, Abramo Bagnara ha scritto:
> Il 05/12/2010 00:26, Chris Lattner ha scritto:
>> On Dec 4, 2010, at 2:36 AM, Abramo Bagnara wrote:
>>> Currently, during lexing in LexingRawMode only the literal text is
>>> collected (setLiteralData/getLiteralData), while the identifier text is
>>> lost.
>>
>> Hi Abramo,
>>
>> I'm not sure what you're trying to accomplish here: it sounds like you want to get the "identifier text" from the token itself without getting the spelling.  Is this correct? If so, are you looking for a performance win, a functionality improvement, or something else?  What problem are you trying to solve?
> 
> Yes, the idea is to have access to identifier text just like we have now
> access to LiteralData.
> 
> We are looking for a performance win (and secondarily for a simmetric
> interface wrt literals) for our raw lexer clients (currently they need
> to use something like the above to get the text for each identifier):
> 
>   token_begin = sm.getSpellingLoc(token_begin);
> 
>   const char* current_char = sm.getCharacterData(token_begin);
>   const char* end = current_char + length;
> 
>   while (current_char < end) {
>     char c = clang::Lexer::getCharAndSizeNoWarn(current_char, length, lo);
>     output += c;
>     current_char += length;
>   }

I've attached the working patch for approval.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: raw-identifier.patch
Type: text/x-patch
Size: 52686 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20101207/9f32dc51/attachment.bin>