[cfe-dev] Identifier text in raw lexing

[Abramo Bagnara]

Il 05/12/2010 00:26, Chris Lattner ha scritto:
> On Dec 4, 2010, at 2:36 AM, Abramo Bagnara wrote:
>> Currently, during lexing in LexingRawMode only the literal text is
>> collected (setLiteralData/getLiteralData), while the identifier text is
>> lost.
> 
> Hi Abramo,
> 
> I'm not sure what you're trying to accomplish here: it sounds like you want to get the "identifier text" from the token itself without getting the spelling.  Is this correct? If so, are you looking for a performance win, a functionality improvement, or something else?  What problem are you trying to solve?

Yes, the idea is to have access to identifier text just like we have now
access to LiteralData.

We are looking for a performance win (and secondarily for a simmetric
interface wrt literals) for our raw lexer clients (currently they need
to use something like the above to get the text for each identifier):

  token_begin = sm.getSpellingLoc(token_begin);

  const char* current_char = sm.getCharacterData(token_begin);
  const char* end = current_char + length;

  while (current_char < end) {
    char c = clang::Lexer::getCharAndSizeNoWarn(current_char, length, lo);
    output += c;
    current_char += length;
  }