[cfe-dev] Relexing more than one tokens

[Abramo Bagnara]

Chris Lattner ha scritto:
> 
>>>>> However, not all hope is lost.  It is very reasonable for an
>>>>> ASTConsumer
>>>>> to construct ASTs for a translation unit *AND* then preprocess the
>>>>> whole
>>>>> file again to get the tokens in a big vector.  Given that, you
>>>>> could map
>>>>> from the AST node to an index in the vector, then scan around in the
>>>>> vector of tokens looking for what you want.
>>>>
>>>> How you'd map the AST node to the relexed tokens vector?
>>>
>>> Just compare the source locations.
>>
>> Are you meaning that preprocessing the whole translation unit again I'd
>> get the same SourceLocation opaque ID?
> 
> Hopefully.  If not, you can do a "deep comparison" along the lines of
> SourceManager::isBeforeInTranslationUnit.

Instead of preprocessing the whole translation unit, wouldn't be more
effective to transform Preprocessor::Lex() in a virtual method and
permit to applications to use a derived class of Preprocessor that have
a Lex method that after calling Preprocessor::Lex() saves a copy of each
tokens in the big vector you said above?