[cfe-dev] parsing code snippets

[Douglas Gregor]

On May 6, 2011, at 1:56 AM, Olaf Krzikalla wrote:

> Hi,
> 
> Am 05.05.2011 16:01, schrieb Douglas Gregor:
>> 
>> On May 5, 2011, at 12:43 AM, Olaf Krzikalla wrote:
>> Why not turn the "#pragma scout vectorize" token sequence into a special token (tok::pragma_scout_vectorize), then have the parser recognize that token (in whatever contexts is makes sense, e.g., a statement context for this case) and handle the actual #pragma parsing?
> 
> Hmm, that still means heavy changes in the parser itself, doesn't it? At 
> least I haven't found any callback or customization machinery there.
> Another thing I have to consider is that I want to switch to C0xx 
> attributes as fast as possible. That is, in the future the code shall 
> look like this:
> 
>    [[scout::vectorize, scout::aligned(a)]]
>    for (int i = 0; i < 100; ++i)
>      a[i] = 0.0;
> 
> My current approach is a dirty hack, which however worked out so far:
> I published Parser::ParseExpression and Parser::ConsumeAnyToken (the 
> latter is necessary in order to move Parser::Tok away from the "pragma")
> and store a global pointer to the parser in clang::ParseAST, which I 
> later retrieve from my pragma handler. Of course this is 
> proof-of-concept only. However, judging from N2418, sec.10:
> 
> attribute-argument: assignment-expression | type-id
> 
> clang will need such a feature in the future anyway in order to support 
> custom attributes (I hope that such a feature is planned).
> My questions are:
> 1. Do you have any headaches if the Parser have two public functions 
> ParseExpression and ParseTypeName (which internally might do some 
> preparation work before calling their private counterparts)?

To put this into the open-source tree, I'd want some evidence that this is the right way to support the kind of extensibility you're looking for. I think there's probably a better way.

> 2. How can we expose the current parser obejct to custom handlers 
> (attribute or pragma)?


I don't have a good answer. For attributes, the idea was that we would have a way to describe the grammar of the attribute (with a *very* restrictive syntax), and have tblgen generate the attribute parsers from that. But, as far as I know, nobody is working on this at the moment.

	- Doug