[clang] [Clang][Parse] Fix assertion when annotating a failed decltype-specifier (PR #211221)

[via cfe-commits]

llvmorg-github-actions[bot] wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-clang

Author: Akash Agrawal (akashagrwl)

<details>
<summary>Changes</summary>

When `ParseOptionalCXXScopeSpecifier` sees a leading `decltype`, it parses the decltype-specifier and then calls `AnnotateExistingDecltypeSpecifier` to turn the consumed tokens into a single `annot_decltype` token. If the decltype-specifier fails to parse (e.g. `decltype` is not followed by `(`), error recovery may skip tokens, leaving the `EndLoc` returned by `ParseDecltypeSpecifier` stale. Annotating with that stale end location no longer matches the most recent cached token and trips the invariant asserted in `Preprocessor::AnnotatePreviousCachedTokens`:

```cpp
int decltype;      // crashed
int *decltype = 0; // crashed
```

Fix it the same way the sibling pack-indexing path (`AnnotateExistingIndexedTypeNamePack`) already does: on error, re-derive `EndLoc` from the last cached token. Unlike the pack-indexing branch, this path can be entered with a single `decltype` token, so the backtracking cache may be empty after `RevertCachedTokens`; guard the query with a new `Preprocessor::hasCachedTokenLocation()` helper so we never call `getLastCachedTokenLocation()` (which asserts `CachedLexPos != 0`) on an empty cache.

Fixes #<!-- -->211207

---
Full diff: https://github.com/llvm/llvm-project/pull/211221.diff


3 Files Affected:

- (modified) clang/include/clang/Lex/Preprocessor.h (+4) 
- (modified) clang/lib/Parse/ParseDeclCXX.cpp (+14) 
- (added) clang/test/Parser/decltype-crash.cpp (+14) 


``````````diff
diff --git a/clang/include/clang/Lex/Preprocessor.h b/clang/include/clang/Lex/Preprocessor.h
index 1c917dcfe7b7e..95a8074215014 100644
--- a/clang/include/clang/Lex/Preprocessor.h
+++ b/clang/include/clang/Lex/Preprocessor.h
@@ -1991,6 +1991,10 @@ class Preprocessor {
     return CachedTokens[CachedLexPos-1].getLastLoc();
   }
 
+  /// Whether there is a cached token whose location
+  /// getLastCachedTokenLocation() can return.
+  bool hasCachedTokenLocation() const { return CachedLexPos != 0; }
+
   /// Whether \p Tok is the most recent token (`CachedLexPos - 1`) in
   /// CachedTokens.
   bool IsPreviousCachedToken(const Token &Tok) const;
diff --git a/clang/lib/Parse/ParseDeclCXX.cpp b/clang/lib/Parse/ParseDeclCXX.cpp
index 150987dae332d..dd57495634218 100644
--- a/clang/lib/Parse/ParseDeclCXX.cpp
+++ b/clang/lib/Parse/ParseDeclCXX.cpp
@@ -1136,6 +1136,20 @@ void Parser::AnnotateExistingDecltypeSpecifier(const DeclSpec &DS,
   // make sure we have a token we can turn into an annotation token
   if (PP.isBacktrackEnabled()) {
     PP.RevertCachedTokens(1);
+    if (DS.getTypeSpecType() == TST_error && PP.hasCachedTokenLocation()) {
+      // The decltype-specifier failed to parse (e.g. 'decltype' not followed
+      // by '('), so error recovery may have skipped tokens to resynchronize.
+      // The EndLoc computed before that skip is now stale and no longer
+      // matches the most recent cached token, which would trip the invariant
+      // in Preprocessor::AnnotatePreviousCachedTokens. Annotate up to the last
+      // cached token instead, as AnnotateExistingIndexedTypeNamePack does.
+      // Unlike that pack-indexing path (which needs several lookahead tokens
+      // to be reached), this one can fire with just the 'decltype' token
+      // cached, so the cache may be empty after RevertCachedTokens; guard
+      // against that, since getLastCachedTokenLocation() requires a cached
+      // token.
+      EndLoc = PP.getLastCachedTokenLocation();
+    }
   } else
     PP.EnterToken(Tok, /*IsReinject*/ true);
 
diff --git a/clang/test/Parser/decltype-crash.cpp b/clang/test/Parser/decltype-crash.cpp
new file mode 100644
index 0000000000000..c892c18069d87
--- /dev/null
+++ b/clang/test/Parser/decltype-crash.cpp
@@ -0,0 +1,14 @@
+// RUN: %clang_cc1 -fsyntax-only -verify -std=c++11 %s
+// RUN: %clang_cc1 -fsyntax-only -verify -std=c++20 %s
+
+// These used to assert in Preprocessor::AnnotatePreviousCachedTokens instead
+// of just diagnosing the error: ParseOptionalCXXScopeSpecifier annotated a
+// decltype-specifier that failed to parse (no '(' after 'decltype'), using a
+// stale end-location left over from error recovery.
+int decltype = 0;
+// expected-error at -1 {{expected '(' after 'decltype'}}
+// expected-error at -2 {{expected unqualified-id}}
+
+int *decltype = 0;
+// expected-error at -1 {{expected '(' after 'decltype'}}
+// expected-error at -2 {{expected unqualified-id}}

``````````

</details>


https://github.com/llvm/llvm-project/pull/211221