[cfe-dev] Keeping invalid AST nodes

[Douglas Gregor]

On Nov 5, 2012, at 10:02 AM, Eli Friedman <eli.friedman at gmail.com> wrote:

> On Mon, Nov 5, 2012 at 4:18 AM, Erik Verbruggen <erik.verbruggen at me.com> wrote:
>> When using libclang for syntax highlighting, I noticed that no AST nodes get built for e.g. invalid expressions. For example, in the following C++ code:
>> 
>> int func(int i) {
>>    int j = undefinedFunction(i) + i;
>>    return j;
>> }
>> 
>> the whole initialisation for int j gets dropped, including the (somewhat) valid references to i. I understand that this is done to keep the AST valid after parsing, but an IDE or refactoring tools cannot help out with fixing the problem in that snippet. My question: is there some way to keep the invalid AST nodes, or is somebody already working on this?
> 
> No, there's no way to keep them, and AFAIK nobody is working on this.

I agree with this…

> I'm also not sure it's worthwhile: it's a ton of work for a use case
> which doesn't seem very important, given that the vast majority of
> code people care about is valid.


… but not necessarily this. You won't get any sympathy from me if you want to, say, refactor invalid code, since you could never validate the results. But syntax highlighting and related code queries do matter, and it makes sense to try to keep invalid AST nodes around.

There are several caveats. Obviously, we don't want to slow the compiler down. Moreover, we need to be careful not to keep invalid ASTs around for template instantiations, where we encounter invalid ASTs all the time but don't want to hold on to them.

	- Doug