[PATCH] D86169: Initial support for letting plugins perform custom parsing of attribute arguments.

[Aaron Ballman via Phabricator via cfe-commits]

aaron.ballman added a comment.

In D86169#2242673 <https://reviews.llvm.org/D86169#2242673>, @jonathan.protzenko wrote:

> In D86169#2241830 <https://reviews.llvm.org/D86169#2241830>, @aaron.ballman wrote:
>
>>> - Upon encountering `[[ attr(X) ]]` where `attr` is a plugin-handled attribute: clang parses `X` as token soup, only caring about finding a matching parenthesis
>>
>> Just to be clear, what you mean is to lex all of the tokens comprising `X` and store them off for later?
>
> Correct.
>
>>> Does this stand any chance of working?
>>
>> This is along the same lines of what I was thinking, but there is one potential snag. Clang layers our library components and Sema should never refer back to the Parser (it doesn't link the library in). Because you have an odd requirement where you need to do further parsing during the semantics stage, you may run into awkwardness from this. Also, you may have to worry about needing this functionality recursively. e.g., `int x [[your_attr(int y [[your_attr(int z;)]];)]];` which could be... interesting.
>
> I guess I'd have to i) call the action to create, say, a Decl, then later ii) call the plugin and extend the Decl with fresh attributes, assuming the attribute vector on the Decl can be grown *after* the Decl is created -- hopefully this can be driven from the parser, without a back-edge in the dependency graph? Or maybe I'm missing something?

I can't think of another case where we've needed to parse from within Sema, so I'm not certain what issues you will or won't run into. The concern I have is that the parser calls into sema for semantic checking, so this causes a dependency cycle that the components may not handle well (or they may handle it just fine). e.g., `int x [[whatever(int y;)]];` the parser will parse the `int x [[...]]` bit, then call into Sema to check the attribute, which would then spawn a parser to parse the `int y;` bit, which calls into Sema again for semantic handling. It's that last step that may be an issue (for instance, it may introduce a new AST node for `int y;` along with the semantic checking of the declaration).

> Do you have any example somewhere of how to store tokens and re-fill the lex buffer with them later on? This is much more involved than my current patch so may take me a while to figure out.

We don't really have examples of it quite in this way, but we do have a `TokenLexer` object that you would likely need to use. I think you'd squirrel away the tokens when parsing the attribute argument clause, and then make a new `TokenLexer` that is used by the parser's `Preprocessor` member for parsing the tokens. This gets used when expanding tokens from `_Pragma`, for instance.

>>> Bonus question: seeing `diagAppertainsToDecl`, the name of this function led me to believe that for now, plugins could only handle attributes attached to declarations. Is this the case?
>>
>> Correct, the plugin system does not yet handle type or statement attributes (it was focusing on declaration attributes first, which are the lion's share of attributes in Clang).
>
> Ok, I was asking because if there's no plans to handle type or statement attributes via a plugin, then we're somewhat over-engineering this.

There are plans to support those, but I'm not certain if anyone is actively working on adding that support. I want to avoid adding anything that would make supporting statement and type attributes harder, and I'd be worried that passing in the `Declarator *` would make that project harder to get started on.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D86169/new/

https://reviews.llvm.org/D86169