[clang] [llvm] WIP - [Distributed ThinLTO] Support Unicode characters (PR #193931)

[Ben Dunbobbin via cfe-commits]

https://github.com/bd1976bris updated https://github.com/llvm/llvm-project/pull/193931

>From 64c0a301b8691b678af609fa60df44aec0316be7 Mon Sep 17 00:00:00 2001
From: Ben <ben.dunbobbin at sony.com>
Date: Fri, 24 Apr 2026 10:51:10 +0100
Subject: [PATCH] [Distributed ThinLTO] Support Unicode characters

Add a regression test exercising Distributed ThinLTO with Unicode
characters and fix the code to allow this.

Fix:
IndexBitcodeWriter::writeModStrings() serializes module path strings
into a SmallVector<unsigned> before emitting MST_CODE_ENTRY records.
When a path contains UTF-8 bytes with the high bit set, appending from
StringRef::begin()/end() can be incorrect depending on the signedness
of char. Instead, append the module path through
bytes_begin()/bytes_end(), so the bitcode writer always serializes
unsigned bytes.
---
 clang/test/CodeGen/thinlto_unicode.test   | 36 +++++++++++++++++++++++
 llvm/lib/Bitcode/Writer/BitcodeWriter.cpp |  2 +-
 2 files changed, 37 insertions(+), 1 deletion(-)
 create mode 100644 clang/test/CodeGen/thinlto_unicode.test

diff --git a/clang/test/CodeGen/thinlto_unicode.test b/clang/test/CodeGen/thinlto_unicode.test
new file mode 100644
index 0000000000000..477e0a8870f1a
--- /dev/null
+++ b/clang/test/CodeGen/thinlto_unicode.test
@@ -0,0 +1,36 @@
+# Test handling of files with unicode symbols and unicode names.
+
+# REQUIRES: x86-registered-target
+
+RUN: rm -rf %t && split-file %s %t && cd %t
+
+# Generate bitcode, a combined index, and the importing module's distributed
+# backend index.
+RUN: opt -module-summary -module-hash 🥚.ll -o 🥚.bc
+RUN: opt -module-summary -module-hash 🐣.ll -o 🐣.bc
+RUN: llvm-lto -thinlto-action=thinlink -o 🔗.bc 🥚.bc 🐣.bc
+RUN: llvm-lto -thinlto-action=distributedindexes -exported-symbol=🥚 -thinlto-index=🔗.bc 🥚.bc
+
+# This module will import 🐣(). Check that this occured correctly.
+RUN: %clang -target x86_64-unknown-linux-gnu -O2 -o  ☃.o -x ir 🥚.bc -c -fthinlto-index=🥚.bc.thinlto.bc -save-temps=obj
+RUN: llvm-dis 🥚.s.3.import.bc -o - | FileCheck --check-prefix=IMPORT %s
+IMPORT: define available_externally i32 @"\F0\9F\90\A3"()
+
+#--- 🥚.ll
+source_filename = "🥚.c"
+target triple = "x86_64-unknown-linux-gnu"
+
+define i32 @"🥚"() {
+  %call = call i32 @"🐣"()
+  ret i32 %call
+}
+
+declare i32 @"🐣"()
+
+#--- 🐣.ll
+source_filename = "🐣.c"
+target triple = "x86_64-unknown-linux-gnu"
+
+define i32 @"🐣"() {
+  ret i32 1
+}
\ No newline at end of file
diff --git a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
index 71d5a8bc98a4b..a49beaf885e0e 100644
--- a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
+++ b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
@@ -4263,7 +4263,7 @@ void IndexBitcodeWriter::writeModStrings() {
     auto ModuleId = ModuleIdMap.size();
     ModuleIdMap[Key] = ModuleId;
     Vals.push_back(ModuleId);
-    Vals.append(Key.begin(), Key.end());
+    Vals.append(Key.bytes_begin(), Key.bytes_end());
 
     // Emit the finished record.
     Stream.EmitRecord(bitc::MST_CODE_ENTRY, Vals, AbbrevToUse);