[llvm] [MC] Skip AsmToken::Comment at statement start in AsmParser (PR #218456)

[Matt Turner via llvm-commits]

mattst88 wrote:

I gave that a try. It does fix the bug (all three new tests pass), but it regresses four in-tree tests and adds an assertion failure. Here's exactly what I tested, on top of commit 35efd5d809c9 with this PR's `parseStatement()` hunk dropped:

```diff
--- a/llvm/lib/MC/MCParser/AsmParser.cpp
+++ b/llvm/lib/MC/MCParser/AsmParser.cpp
@@ -1088,11 +1088,11 @@
 /// Throw away the rest of the line for testing purposes.
 void AsmParser::eatToEndOfStatement() {
   while (Lexer.isNot(AsmToken::EndOfStatement) && Lexer.isNot(AsmToken::Eof))
-    Lexer.Lex();
+    Lex();

   // Eat EOL.
   if (Lexer.is(AsmToken::EndOfStatement))
-    Lexer.Lex();
+    Lex();
 }

 StringRef AsmParser::parseStringToEndOfStatement() {
```

`Lex()` reports `AsmToken::Error` via `Error()`, which pushes onto `PendingErrors`. `eatToEndOfStatement()` is raw for two reasons that depend on that not happening.

1. Lexing errors in deferred bodies must be ignored. `parseDirectiveMacro` contains this comment that says as much:

```cpp
  // Consuming deferred text, so use Lexer.Lex to ignore Lexing Errors
```
A macro/.rept/.irpc body is only valid after substitution, so scanning it must not diagnose. Four tests regress on exactly this:

- MC/AsmParser/macro_parsing.s: `int $0x\num`  now produces "invalid hexadecimal number"
- MC/AsmParser/macro-irpc.s: `.long 0x\foo` now produces "invalid hexadecimal number"
- MC/ELF/compress-debug-sections-zlib.s: `.irpc j, 0123456789` now produces "invalid octal number"
- MC/ELF/compress-debug-sections-zstd.s: same


2. `Run()` prints pending errors before the recovery call (AsmParser.cpp:1006-1011), so an error raised during the eat survives into the next iteration and trips `parseStatement`'s entry assert:

```
$ cat t.s
      .byte 1 2 ` 3
      nop
$ llvm-mc -triple=x86_64 t.s -o /dev/null
t.s:1:10: error: unexpected token
llvm-mc: AsmParser.cpp:1737: ...parseStatement(...):
  Assertion `!hasPendingError() && "parseStatement started with pending error"' failed.
Aborted (core dumped)
```

Without assertions that's a spurious second diagnostic, the same kind of problem this PR fixes.

On consolidation: the function doesn't actually get simpler (both loops stay, only the callee changes), and the comment handling wouldn't converge either. Routing deferred text through `Lex()` makes `-preserve-comments` start emitting comments from macro bodies and skipped `.if` branches at scan time, which is a separate behavior change.

For reference, with the PR as-is: MC suite 1446 passed / 6 failed, and all six are MC/COFF/cv-* failing because `llvm-pdbutil` isn't in my X86-only build.

https://github.com/llvm/llvm-project/pull/218456