[cfe-dev] Token lookahead without the preprocessor

[Jordan Rose]

Hi, all. I've been trying to come up with a useful recovery for this case (<rdar://problem/11602405> for Apple folks):

void foo();
{
	// note the spurious semicolon above
}

The trouble is, having a semicolon there is a perfectly good way to end a declaration. It's clear that if there's a brace on the next line, it was actually supposed to be a definition (because C/C++ don't have top-level braces). But we get in trouble in this case (from test/CodeGen/pragma-weak.c):

void __both2(void);
void both2(void) __attribute((alias("__both2"))); // first, wins
#pragma weak both2 = __both2
void __both2(void) {}
// CHECK: @both2 = alias void ()* @__both2
// CHECK: define void @__both2()

The lookahead after the semicolon has to go all the way to the next 'void' to get another token, and meanwhile the Lexer and Preprocessor have seen and recorded the #pragma weak.

There are similar problems in test/SemaCXX/warn-thread-safety-analysis.cpp, though I haven't specifically tracked them down.

Any ideas on what's the right thing to do here? I'd be fine with "there's a preprocessing directive in the way; don't bother" or "the next token is 'void' but you're gonna have to re-Lex from where you are" but I don't think we have a good way to do either one. (Raw mode /almost/ works except I'm not sure of the right way to go into raw mode from Parser.)

Thanks for any advice,
Jordan