[cfe-dev] Patch to allow comment translators implementation

[Abramo Bagnara]

Il 29/12/2009 23:51, Abramo Bagnara ha scritto:
> Il 29/12/2009 21:08, Chris Lattner ha scritto:
>>
>> On Dec 26, 2009, at 7:58 AM, Abramo Bagnara wrote:
>>
>>>
>>> This small patch change comments handler in a simple way to permit to
>>> implement quite easily comment translators.
>>>
>>> Once applied this patch, a CommentHandler is allowed to build a first
>>> token to be returned to Lexer and to push a TokenStream for the others,
>>> then allowing generic comment -> tokens transformer.
>>>
>>> This can be useful to transform comment shaped program annotation that
>>> should be translated to source code and also other interesting applications.
>>
>> This is an interesting approach.  The only major concern I have is
>> that this only allows you to translate comments into exactly one
>> token.  In the case of openmp pragmas (for example) this doesn't seem

Do I've been sufficient clear explaining that the comments may be
translated to an arbitrary number of tokens calling EnterTokenStream
inside the CommentHandler?

>> rich enough.  A different approach would be to allow the handler to
>> push an arbitrary number of tokens into the parser's lookahead
>> buffer.  Would this work for what you're trying to do?
> 
> Yes, but perhaps this is not needed: as I wrote the CommentHandler could
> return a first token *and* produce the other tokens to be read and push
> them to lexer stack using EnterTokenStream.
> 
> I've already tried this with success in a sample implementation that
> simply lex the comment content without modify it:

Still I've not got any feedback: do you think that the patch in original
mail will be applied as is? Should I improve it in some way?

-- 
Abramo Bagnara

Opera Unica                          Phone: +39.0546.656023
Via Borghesi, 16
48014 Castel Bolognese (RA) - Italy