[clang] [clang][ExtractAPI] Fix assertion on invalid UTF-8 in doc comments (PR #212394)

[Patryk Stefanski via cfe-commits]

https://github.com/patrykstefanski updated https://github.com/llvm/llvm-project/pull/212394

>From e7cf2a605acf31d95b031e2267dea3f92b955741 Mon Sep 17 00:00:00 2001
From: Patryk Stefanski <patryk.stefanski at protonmail.com>
Date: Mon, 27 Jul 2026 15:50:45 -0700
Subject: [PATCH 1/2] [clang][ExtractAPI] Fix assertion on invalid UTF-8 in doc
 comments

A doc comment can contain invalid UTF-8. The raw bytes reach
serializeDocComment, which assigned them to a llvm::json::Value,
tripping its valid-UTF-8 assertion. This is hit by legacy headers that
are not UTF-8 encoded, which compile fine but crash -extract-api.

Sanitize with json::isUTF8/fixUTF8 before serializing.

Fixes #212393
---
 clang/docs/ReleaseNotes.md                       |  2 ++
 .../Serialization/SymbolGraphSerializer.cpp      |  6 +++++-
 clang/test/ExtractAPI/invalid_utf8_doc_comment.c | 16 ++++++++++++++++
 3 files changed, 23 insertions(+), 1 deletion(-)
 create mode 100644 clang/test/ExtractAPI/invalid_utf8_doc_comment.c

diff --git a/clang/docs/ReleaseNotes.md b/clang/docs/ReleaseNotes.md
index ac2688886c1ee..7dfd186abcdc5 100644
--- a/clang/docs/ReleaseNotes.md
+++ b/clang/docs/ReleaseNotes.md
@@ -397,6 +397,8 @@ features cannot lower the translation-unit ABI level;
 
 #### Miscellaneous Clang Crashes Fixed
 - Fixed a crash when instantiating an invalid dependent friend destructor declaration in a class template. (#GH210234)
+- Fixed an assertion failure in `-extract-api` when a documentation comment
+  contains invalid UTF-8. (#GH212393)
 
 ### OpenACC Specific Changes
 
diff --git a/clang/lib/ExtractAPI/Serialization/SymbolGraphSerializer.cpp b/clang/lib/ExtractAPI/Serialization/SymbolGraphSerializer.cpp
index d3df9eb604f27..8edaaec0d1c7b 100644
--- a/clang/lib/ExtractAPI/Serialization/SymbolGraphSerializer.cpp
+++ b/clang/lib/ExtractAPI/Serialization/SymbolGraphSerializer.cpp
@@ -262,7 +262,11 @@ std::optional<Object> serializeDocComment(const DocComment &Comment) {
   Array LinesArray;
   for (const auto &CommentLine : Comment) {
     Object Line;
-    Line["text"] = CommentLine.Text;
+    // Source files are not required to be valid UTF-8. JSON values must be
+    // valid UTF-8, so replace any invalid sequences before serializing.
+    Line["text"] = json::isUTF8(CommentLine.Text)
+                       ? CommentLine.Text
+                       : json::fixUTF8(CommentLine.Text);
     serializeObject(Line, "range",
                     serializeSourceRange(CommentLine.Begin, CommentLine.End));
     LinesArray.emplace_back(std::move(Line));
diff --git a/clang/test/ExtractAPI/invalid_utf8_doc_comment.c b/clang/test/ExtractAPI/invalid_utf8_doc_comment.c
new file mode 100644
index 0000000000000..8532aea704d02
--- /dev/null
+++ b/clang/test/ExtractAPI/invalid_utf8_doc_comment.c
@@ -0,0 +1,16 @@
+// RUN: rm -rf %t
+// RUN: %clang_cc1 -extract-api --pretty-sgf --emit-sgf-symbol-labels-for-testing \
+// RUN:   -triple arm64-apple-macosx -x c-header %s -o %t/output.symbols.json
+// RUN: FileCheck %s --input-file %t/output.symbols.json
+
+// This file is purposefully NOT valid UTF-8. The doc comment below contains a
+// raw 0xD5 byte. Be careful when modifying.
+
+/*! @brief The sender�s storage. */
+int foo(void);
+
+// CHECK:      "docComment": {
+// CHECK-NEXT:   "lines": [
+// CHECK:          "text": "@brief The sender�s storage. "
+// CHECK:        ]
+// CHECK-NEXT: },

>From 0777051f4ef06499227eb347ba52105e45cace30 Mon Sep 17 00:00:00 2001
From: Patryk Stefanski <patryk.stefanski at protonmail.com>
Date: Tue, 28 Jul 2026 08:52:07 -0700
Subject: [PATCH 2/2] Apply snprajwal's suggestions

---
 clang/lib/ExtractAPI/Serialization/SymbolGraphSerializer.cpp | 2 +-
 clang/test/ExtractAPI/invalid_utf8_doc_comment.c             | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/clang/lib/ExtractAPI/Serialization/SymbolGraphSerializer.cpp b/clang/lib/ExtractAPI/Serialization/SymbolGraphSerializer.cpp
index 8edaaec0d1c7b..6228f2e687be2 100644
--- a/clang/lib/ExtractAPI/Serialization/SymbolGraphSerializer.cpp
+++ b/clang/lib/ExtractAPI/Serialization/SymbolGraphSerializer.cpp
@@ -262,7 +262,7 @@ std::optional<Object> serializeDocComment(const DocComment &Comment) {
   Array LinesArray;
   for (const auto &CommentLine : Comment) {
     Object Line;
-    // Source files are not required to be valid UTF-8. JSON values must be
+    // Comments in source files may contain invalid UTF-8. JSON values must be
     // valid UTF-8, so replace any invalid sequences before serializing.
     Line["text"] = json::isUTF8(CommentLine.Text)
                        ? CommentLine.Text
diff --git a/clang/test/ExtractAPI/invalid_utf8_doc_comment.c b/clang/test/ExtractAPI/invalid_utf8_doc_comment.c
index 8532aea704d02..f3f09ea9cbb56 100644
--- a/clang/test/ExtractAPI/invalid_utf8_doc_comment.c
+++ b/clang/test/ExtractAPI/invalid_utf8_doc_comment.c
@@ -1,11 +1,13 @@
 // RUN: rm -rf %t
-// RUN: %clang_cc1 -extract-api --pretty-sgf --emit-sgf-symbol-labels-for-testing \
+// RUN: %clang_cc1 -extract-api --pretty-sgf -verify \
 // RUN:   -triple arm64-apple-macosx -x c-header %s -o %t/output.symbols.json
 // RUN: FileCheck %s --input-file %t/output.symbols.json
 
 // This file is purposefully NOT valid UTF-8. The doc comment below contains a
 // raw 0xD5 byte. Be careful when modifying.
 
+// expected-no-diagnostics
+
 /*! @brief The sender�s storage. */
 int foo(void);