[cfe-dev] Patch to allow comment translators implementation

[Chris Lattner]

On Jan 12, 2010, at 4:47 AM, Abramo Bagnara wrote:
>> Yes, but I find the protocol for introducing tokens via a comment
>> handler to be very confusing. Could we instead eliminate the Token
>> &token argument, and just make the protocol: to "parse" the  
>> contents of
>> the comment, use EnterTokenStream and then return true?
>
> The only proposal I see that would not change a lot of things is to
> remove Token argument from CommentHandler and to specify that they
> always shall use EnterTokenStream also when they need to insert only  
> one
> token.
> Then at end of Preprocessor::HandleComment if any of the comment
> handlers has informed the caller to have inserted some tokens the  
> first
> one available is fetched and returned by Preprocessor::HandleComment.

Yes, I think this is a good plan.

>> I think it's okay if the HandleComment protocol can be simplified a  
>> bit
>> and if it is documented, although I'd like to hear from Chris. I'd  
>> feel
>> much better if we actually had some kind of use of this code path  
>> within
>> Clang itself. For example, would it be possible for the keep-comments
>> mode to be implemented outside of the lexer using your changes to
>> HandleComment? That might actually simplify the lexer while making it
>> more general.
>
> ... hmmm, probably it's feasible, but I'm not so sure it's a good idea
> because this means we should always have a CommentHandler loaded and  
> we
> have the problem of CommentHandler execution order, as only the first
> CommentHandler that want to return a first Token is allowed to do that
> (unless we implement the variant above).

I don't think it's necessary to reimplement -C.  This functionality  
would be useful to implement the OpenMP pragma language, for example.   
What do you plan to use this for?  Would any of it be coming back to  
mainline at some point?

-Chris