[clang] [llvm] [Distributed ThinLTO] Support Unicode characters (PR #193931)

[Ben Dunbobbin via llvm-commits]

https://github.com/bd1976bris created https://github.com/llvm/llvm-project/pull/193931

Add a regression test exercising Distributed ThinLTO with Unicode characters and fix the code to allow this.

Fix:
IndexBitcodeWriter::writeModStrings() serializes module path strings into a SmallVector<unsigned> before emitting MST_CODE_ENTRY records. When a path contains UTF-8 bytes with the high bit set, appending from StringRef::begin()/end() can be incorrect depending on the signedness of char. Instead, append the module path through
bytes_begin()/bytes_end(), so the bitcode writer always serializes unsigned bytes.

>From 215c4dba4fad8e5bb3a48f48b305e8c036abf958 Mon Sep 17 00:00:00 2001
From: Ben <ben.dunbobbin at sony.com>
Date: Fri, 24 Apr 2026 10:51:10 +0100
Subject: [PATCH] [Distributed ThinLTO] Support Unicode characters

Add a regression test exercising Distributed ThinLTO with Unicode
characters and fix the code to allow this.

Fix:
IndexBitcodeWriter::writeModStrings() serializes module path strings
into a SmallVector<unsigned> before emitting MST_CODE_ENTRY records.
When a path contains UTF-8 bytes with the high bit set, appending from
StringRef::begin()/end() can be incorrect depending on the signedness
of char. Instead, append the module path through
bytes_begin()/bytes_end(), so the bitcode writer always serializes
unsigned bytes.
---
 clang/test/CodeGen/thinlto_unicode.test   | 35 +++++++++++++++++++++++
 llvm/lib/Bitcode/Writer/BitcodeWriter.cpp |  2 +-
 2 files changed, 36 insertions(+), 1 deletion(-)
 create mode 100644 clang/test/CodeGen/thinlto_unicode.test

diff --git a/clang/test/CodeGen/thinlto_unicode.test b/clang/test/CodeGen/thinlto_unicode.test
new file mode 100644
index 0000000000000..d8b05f0ce004b
--- /dev/null
+++ b/clang/test/CodeGen/thinlto_unicode.test
@@ -0,0 +1,35 @@
+# Test handling of files with unicode symbols and unicode names.
+
+# REQUIRES: x86-registered-target
+
+RUN: rm -rf %t && split-file %s %t && cd %t
+
+# Generate bitcode, a combined index, and distributed backend indexes.
+RUN: opt -module-summary -module-hash 💩.ll -o 💩.bc
+RUN: opt -module-summary -module-hash 😀.ll -o 😀.bc
+RUN: llvm-lto -thinlto-action=thinlink -o 🔗.bc 💩.bc 😀.bc
+RUN: llvm-lto -thinlto-action=distributedindexes -exported-symbol=😀 -thinlto-index=🔗.bc 💩.bc 😀.bc
+
+# This module will import 😀(). Check that this occured correctly.
+RUN: %clang -target x86_64-unknown-linux-gnu -O2 -o ☃.o -x ir 💩.bc -c -fthinlto-index=😀.bc.thinlto.bc -save-temps=obj
+RUN: llvm-dis 💩.s.3.import.bc -o - | FileCheck --check-prefix=IMPORT %s
+IMPORT: define available_externally i32 @"\F0\9F\98\80"()
+
+#--- 💩.ll
+source_filename = "💩.c"
+target triple = "x86_64-unknown-linux-gnu"
+
+define i32 @"💩"() {
+  %call = call i32 @"😀"()
+  ret i32 %call
+}
+
+declare i32 @"😀"()
+
+#--- 😀.ll
+source_filename = "😀.c"
+target triple = "x86_64-unknown-linux-gnu"
+
+define i32 @"😀"() {
+  ret i32 1
+}
diff --git a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
index 71d5a8bc98a4b..a49beaf885e0e 100644
--- a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
+++ b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
@@ -4263,7 +4263,7 @@ void IndexBitcodeWriter::writeModStrings() {
     auto ModuleId = ModuleIdMap.size();
     ModuleIdMap[Key] = ModuleId;
     Vals.push_back(ModuleId);
-    Vals.append(Key.begin(), Key.end());
+    Vals.append(Key.bytes_begin(), Key.bytes_end());
 
     // Emit the finished record.
     Stream.EmitRecord(bitc::MST_CODE_ENTRY, Vals, AbbrevToUse);