[clang] [llvm] [llvm-strings] Add --encoding={s, S, utf8} option (PR #221794)

Harald van Dijk via cfe-commits cfe-commits at lists.llvm.org
Fri Nov 6 04:48:54 PST 2026


https://github.com/hvdijk updated https://github.com/llvm/llvm-project/pull/221794

>From b2334d9f932e6c6f8f7e106e4678c673a19ac944 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Mon, 7 Sep 2026 18:33:02 +0100
Subject: [PATCH 01/41] [llvm-strings] Add --encoding & --unicode options

The --encoding and --unicode options come from GNU strings, and are
mostly compatible with it, but intentionally not fully.

The --encoding option allows specifying the following encodings:
 s   (single 7-bit byte characters)
 S   (8-bit byte characters)
 u   (UTF-8 characters)
 b/l (UTF-16 characters, big/little endian)
 B/L (UTF-32 characters, big/little endian)

--encoding=s matches GNU strings and interprets 7-bit characters as
ASCII, rejecting any bytes with the high bit set.

--encoding=S interprets characters according to the current locale's
character set. In GNU strings, its meaning depends on --unicode and
either means all characters with the high bit set are considered
printable, or means characters are interpreted as UTF-8.

--encoding=u is an extension to GNU strings, which implicitly uses UTF-8
when --unicode options are used, but does not have a dedicated encoding
value for this and puts it under --encoding=S.

--encoding=b/l/B/L match GNU strings's documentation but deviate
slightly from their implementation: they interpret characters as
UTF-16/UTF-32. Although GNU strings's manual suggests it does the same,
it actually only accepts ASCII characters stored as 16-bit/32-bit units,
whereas this implementation accepts any printable Unicode character. To
reduce the number of false positives, unlike GNU strings, this
implementation requires strings to be aligned.

--unicode=default prints characters as UTF-8. It is the default output
for all UTF encodings.

--unicode=invalid rejects any non-ASCII characters. It is the default
output for --encoding=s.

--unicode=locale prints characters in the encoding specified by the
current locale. It is the default output for --encoding=S.

--unicode=escape/highlight print non-ASCII characters as \u0000 escape
sequences, optionally highlighted if output is to a terminal.

--unicode=hex prints non-ASCII characters as their UTF-8 byte
representation in hex.

The default encoding is --encoding=S, as specified by POSIX. Although
this deviates from GNU strings, this default is meant to be a good fit
for the common case where the user's character encoding (or, in Windows
terms, the code page) matches the binary expects. --encoding=s can be
used to restore the prior behaviour.

The fact that the -n option is aliased to --bytes is confusing, but GNU
strings already uses --bytes to specify the minimum character length,
not the minimum byte length, as well.
---
 llvm/test/tools/llvm-strings/length.test      |  57 +++-
 .../tools/llvm-strings/negative-char.test     |   7 +-
 llvm/tools/llvm-strings/Opts.td               |  16 +
 llvm/tools/llvm-strings/llvm-strings.cpp      | 297 +++++++++++++++++-
 4 files changed, 357 insertions(+), 20 deletions(-)

diff --git a/llvm/test/tools/llvm-strings/length.test b/llvm/test/tools/llvm-strings/length.test
index 896917d8db90dc..aaa3df198f6333 100644
--- a/llvm/test/tools/llvm-strings/length.test
+++ b/llvm/test/tools/llvm-strings/length.test
@@ -1,5 +1,5 @@
 ## Show that llvm-strings prints only strings with length of at least the
-## requested number of bytes.
+## requested number of characters.
 
 RUN: echo a > %t
 RUN: echo ab >> %t
@@ -44,3 +44,58 @@ CHECK-5:      abcde
 ## Show that a non-numeric argument is rejected.
 RUN: not llvm-strings -n foo %t 2>&1 | FileCheck %s --check-prefix=ERR
 ERR: error: expected a positive integer, but got 'foo'
+
+## Show that the count is number of characters, not number of bytes.
+RUN: printf '\302\241\n' > %t
+RUN: printf '\302\241\302\242\n' >> %t
+RUN: printf '\302\241\302\242\302\243\n' >> %t
+RUN: not llvm-strings --encoding u -n 0 2>&1 %t | FileCheck --check-prefix CHECK-0 %s
+RUN: llvm-strings --encoding u -n 1 %t | FileCheck --check-prefix UTF-8-1 %s --implicit-check-not={{.}}
+RUN: llvm-strings --encoding u -n 2 %t | FileCheck --check-prefix UTF-8-2 %s --implicit-check-not={{.}}
+RUN: llvm-strings --encoding u -n 3 %t | FileCheck --check-prefix UTF-8-3 %s --implicit-check-not={{.}}
+RUN: llvm-strings --encoding u -n 4 %t | FileCheck %s --implicit-check-not={{.}} --allow-empty
+
+RUN: printf '\241\0\n\0' > %t
+RUN: printf '\241\0\242\0\n\0' >> %t
+RUN: printf '\241\0\242\0\243\0\n\0' >> %t
+RUN: not llvm-strings --encoding l -n 0 2>&1 %t | FileCheck --check-prefix CHECK-0 %s
+RUN: llvm-strings --encoding l -n 1 %t | FileCheck --check-prefix UTF-8-1 %s --implicit-check-not={{.}}
+RUN: llvm-strings --encoding l -n 2 %t | FileCheck --check-prefix UTF-8-2 %s --implicit-check-not={{.}}
+RUN: llvm-strings --encoding l -n 3 %t | FileCheck --check-prefix UTF-8-3 %s --implicit-check-not={{.}}
+RUN: llvm-strings --encoding l -n 4 %t | FileCheck %s --implicit-check-not={{.}} --allow-empty
+
+RUN: printf '\0\241\0\n' > %t
+RUN: printf '\0\241\0\242\0\n' >> %t
+RUN: printf '\0\241\0\242\0\243\0\n' >> %t
+RUN: not llvm-strings --encoding b -n 0 2>&1 %t | FileCheck --check-prefix CHECK-0 %s
+RUN: llvm-strings --encoding b -n 1 %t | FileCheck --check-prefix UTF-8-1 %s --implicit-check-not={{.}}
+RUN: llvm-strings --encoding b -n 2 %t | FileCheck --check-prefix UTF-8-2 %s --implicit-check-not={{.}}
+RUN: llvm-strings --encoding b -n 3 %t | FileCheck --check-prefix UTF-8-3 %s --implicit-check-not={{.}}
+RUN: llvm-strings --encoding b -n 4 %t | FileCheck %s --implicit-check-not={{.}} --allow-empty
+
+RUN: printf '\241\0\0\0\n\0\0\0' > %t
+RUN: printf '\241\0\0\0\242\0\0\0\n\0\0\0' >> %t
+RUN: printf '\241\0\0\0\242\0\0\0\243\0\0\0\n\0\0\0' >> %t
+RUN: not llvm-strings --encoding L -n 0 2>&1 %t | FileCheck --check-prefix CHECK-0 %s
+RUN: llvm-strings --encoding L -n 1 %t | FileCheck --check-prefix UTF-8-1 %s --implicit-check-not={{.}}
+RUN: llvm-strings --encoding L -n 2 %t | FileCheck --check-prefix UTF-8-2 %s --implicit-check-not={{.}}
+RUN: llvm-strings --encoding L -n 3 %t | FileCheck --check-prefix UTF-8-3 %s --implicit-check-not={{.}}
+RUN: llvm-strings --encoding L -n 4 %t | FileCheck %s --implicit-check-not={{.}} --allow-empty
+
+RUN: printf '\0\0\0\241\0\0\0\n' > %t
+RUN: printf '\0\0\0\241\0\0\0\242\0\0\0\n' >> %t
+RUN: printf '\0\0\0\241\0\0\0\242\0\0\0\243\0\0\0\n' >> %t
+RUN: not llvm-strings --encoding B -n 0 2>&1 %t | FileCheck --check-prefix CHECK-0 %s
+RUN: llvm-strings --encoding B -n 1 %t | FileCheck --check-prefix UTF-8-1 %s --implicit-check-not={{.}}
+RUN: llvm-strings --encoding B -n 2 %t | FileCheck --check-prefix UTF-8-2 %s --implicit-check-not={{.}}
+RUN: llvm-strings --encoding B -n 3 %t | FileCheck --check-prefix UTF-8-3 %s --implicit-check-not={{.}}
+RUN: llvm-strings --encoding B -n 4 %t | FileCheck %s --implicit-check-not={{.}} --allow-empty
+
+UTF-8-1:      ¡
+UTF-8-1-NEXT: ¡¢
+UTF-8-1-NEXT: ¡¢£
+
+UTF-8-2:      ¡¢
+UTF-8-2-NEXT: ¡¢£
+
+UTF-8-3:      ¡¢£
diff --git a/llvm/test/tools/llvm-strings/negative-char.test b/llvm/test/tools/llvm-strings/negative-char.test
index 5154886f9200f0..3e661b04724706 100644
--- a/llvm/test/tools/llvm-strings/negative-char.test
+++ b/llvm/test/tools/llvm-strings/negative-char.test
@@ -1,6 +1,9 @@
 ## Show that llvm-strings can handle a negative signed char value (i.e. > 0x7f).
-## Such characters should form string delimiters like other unprintable ones.
+## Such characters should form string delimiters like other unprintable ones,
+## unless using an encoding where they form valid characters.
 
-# RUN: printf "z\0\200\0a\0" | llvm-strings --bytes 1 - | FileCheck %s
+# RUN: printf "z\0\200\0\302\241\0a\0" | llvm-strings --encoding=s --bytes 1 - | FileCheck %s
+# RUN: printf "z\0\200\0\302\241\0a\0" | llvm-strings --unicode=default --bytes 1 - | FileCheck %s --check-prefixes=CHECK,CHECK-UTF8
 # CHECK: z{{$}}
+# CHECK-UTF8-NEXT: {{^}}¡{{$}}
 # CHECK-NEXT: {{^}}a
diff --git a/llvm/tools/llvm-strings/Opts.td b/llvm/tools/llvm-strings/Opts.td
index 2ad77fae6c15ff..cf7a406fcec1f9 100644
--- a/llvm/tools/llvm-strings/Opts.td
+++ b/llvm/tools/llvm-strings/Opts.td
@@ -11,13 +11,29 @@ multiclass Eq<string name, string help> {
 
 def all : FF<"all", "Silently ignored. Present for GNU strings compatibility">;
 defm bytes : Eq<"bytes", "Print sequences of the specified length">;
+defm encoding : Eq<"encoding", [{Select the character encoding of the strings to find:
+s (7-bit characters)
+S (8-bit characters in the current locale)
+u (UTF-8)
+b (UTF-16 big endian)
+l (UTF-16 little endian)
+B (UTF-32 big endian)
+L (UTF-32 little endian)}]>;
 def help : FF<"help", "Display this help">;
 def print_file_name : Flag<["--"], "print-file-name">, HelpText<"Print the name of the file before each string">;
 defm radix : Eq<"radix", "Print the offset within the file with the specified radix: o (octal), d (decimal), x (hexadecimal)">, MetaVarName<"<radix>">;
+defm unicode : Eq<"unicode", [{Control the display of UTF-8 multibyte characters:
+default   (print UTF-8 characters)
+invalid   (reject multibyte characters)
+locale    (print characters in the current locale)
+escape    (\u0000 escape sequences)
+hex       (print UTF-8 encoding in hex)
+highlight (like escape, but highlighted)}]>;
 def version : FF<"version", "Display the version">;
 
 def : F<"a", "Alias for --all">, Alias<all>;
 def : F<"f", "Alias for --print-file-name">, Alias<print_file_name>;
 def : F<"h", "Alias for --help">, Alias<help>;
 def : JoinedOrSeparate<["-"], "n">, Alias<bytes_EQ>, HelpText<"Alias for --bytes">;
+def : JoinedOrSeparate<["-"], "e">, Alias<encoding_EQ>, HelpText<"Alias for --encoding">;
 def : JoinedOrSeparate<["-"], "t">, Alias<radix_EQ>, HelpText<"Alias for --radix">, MetaVarName<"<radix>">;
diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 9979b93de84270..655cca584437c2 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -18,13 +18,17 @@
 #include "llvm/Option/ArgList.h"
 #include "llvm/Option/Option.h"
 #include "llvm/Support/CommandLine.h"
+#include "llvm/Support/ConvertUTF.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/Format.h"
 #include "llvm/Support/InitLLVM.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Program.h"
+#include "llvm/Support/SwapByteOrder.h"
+#include "llvm/Support/Unicode.h"
 #include "llvm/Support/WithColor.h"
 #include <cctype>
+#include <locale>
 #include <string>
 
 using namespace llvm;
@@ -71,9 +75,15 @@ static cl::list<std::string> InputFileNames(cl::Positional,
 static int MinLength = 4;
 static bool PrintFileName;
 
-enum radix { none, octal, hexadecimal, decimal };
+enum class encoding { s, S, u, b, l, B, L };
+static encoding Encoding;
+
+enum class radix { none, octal, hexadecimal, decimal };
 static radix Radix;
 
+enum class unicode { default_, invalid, locale, escape, hex, highlight };
+static unicode Unicode;
+
 [[noreturn]] static void reportCmdLineError(const Twine &Message) {
   WithColor::error(errs(), ToolName) << Message << "\n";
   exit(1);
@@ -89,40 +99,224 @@ static void parseIntArg(const opt::InputArgList &Args, int ID, T &Value) {
 }
 
 static void strings(raw_ostream &OS, StringRef FileName, StringRef Contents) {
-  auto print = [&OS, FileName](unsigned Offset, StringRef L) {
-    if (L.size() < static_cast<size_t>(MinLength))
+  std::locale loc("");
+  auto &cvt = std::use_facet<std::codecvt<wchar_t, char, std::mbstate_t>>(loc);
+  auto &ctype = std::use_facet<std::ctype<wchar_t>>(loc);
+  std::mbstate_t mbs{};
+
+  const bool PrintRawBytes =
+      Encoding == encoding::s ||
+      (Encoding == encoding::S &&
+       (Unicode == unicode::invalid || Unicode == unicode::locale)) ||
+      (Encoding == encoding::u &&
+       (Unicode == unicode::default_ || Unicode == unicode::invalid));
+
+  auto read = [&cvt, &mbs](const char *&P, const char *E) -> UTF32 {
+    UTF32 Ch;
+
+    switch (Encoding) {
+    case encoding::s:
+      Ch = *P++;
+      break;
+
+    case encoding::S: {
+      const char *N;
+      wchar_t WCh;
+      wchar_t *WNext;
+      [[maybe_unused]] const auto res =
+          cvt.in(mbs, P, E, N, &WCh, &WCh + 1, WNext);
+      assert(res != std::codecvt_base::noconv);
+
+      // Only treat a non-null character as a successful conversion, as a null
+      // character may be the result of an incomplete multibyte character
+      // followed by a null byte. A null byte is safe to treat as an error, as
+      // a null byte is never printable in any locale.
+      if (WNext != &WCh && WCh) {
+        // Note: this assumes wchar_t is UCS2 or UTF32.
+        Ch = WCh;
+        P = N;
+      } else {
+        // If there was any error, skip the current byte and reset the state to
+        // allow the next byte to start a character.
+        Ch = 0;
+        mbs = {};
+        ++P;
+      }
+      break;
+    }
+
+    case encoding::u: {
+      const UTF8 *UP = reinterpret_cast<const UTF8 *>(P);
+      const UTF8 *UE = reinterpret_cast<const UTF8 *>(E);
+      UTF32 *Next = &Ch;
+      ConvertUTF8toUTF32(&UP, UE, &Next, &Ch + 1, strictConversion);
+      if (Next == &Ch || !Ch) {
+        // If there was any error, skip the current byte to allow the next to
+        // start a character.
+        Ch = 0;
+        UP = std::next(reinterpret_cast<const UTF8 *>(P));
+      }
+      P = reinterpret_cast<const char *>(UP);
+      break;
+    }
+
+    case encoding::b:
+    case encoding::l: {
+      const UTF16 *UP = reinterpret_cast<const UTF16 *>(P);
+      const UTF16 *UE = reinterpret_cast<const UTF16 *>(E);
+      const bool DoByteSwap =
+          (Encoding == encoding::b &&
+           endianness::native == endianness::little) ||
+          (Encoding == encoding::l && endianness::native == endianness::big);
+      UTF32 *Next = &Ch;
+      if (!DoByteSwap) {
+        ConvertUTF16toUTF32(&UP, UE, &Next, &Ch + 1, strictConversion);
+      } else {
+        // We never need more than two UTF16 words to make up one character.
+        const UTF16 Buf[2] = {byteswap(UP[0]),
+                              UP + 1 == UE ? UTF16(0) : byteswap(UP[1])};
+        const UTF16 *BufP = Buf;
+        ConvertUTF16toUTF32(&BufP, &Buf[2], &Next, &Ch + 1, strictConversion);
+        UP += BufP - Buf;
+      }
+      if (Next == &Ch || !Ch) {
+        // If there was any error, skip the current word to allow the next to
+        // start a character.
+        Ch = 0;
+        UP = std::next(reinterpret_cast<const UTF16 *>(P));
+      }
+      P = reinterpret_cast<const char *>(UP);
+      break;
+    }
+
+    case encoding::B:
+    case encoding::L: {
+      const UTF32 *UP = reinterpret_cast<const UTF32 *>(P);
+      const bool DoByteSwap =
+          (Encoding == encoding::B &&
+           endianness::native == endianness::little) ||
+          (Encoding == encoding::L && endianness::native == endianness::big);
+      Ch = DoByteSwap ? byteswap(*UP) : *UP;
+      ++UP;
+      P = reinterpret_cast<const char *>(UP);
+      break;
+    }
+
+    default:
+      llvm_unreachable("unhandled encoding");
+    }
+
+    return Ch;
+  };
+
+  auto print = [&OS, FileName, PrintRawBytes, &read,
+                &cvt](unsigned Offset, StringRef L, size_t N) {
+    if (N < static_cast<size_t>(MinLength))
       return;
     if (PrintFileName)
       OS << FileName << ": ";
     switch (Radix) {
-    case none:
+    case radix::none:
       break;
-    case octal:
+    case radix::octal:
       OS << format("%7o ", Offset);
       break;
-    case hexadecimal:
+    case radix::hexadecimal:
       OS << format("%7x ", Offset);
       break;
-    case decimal:
+    case radix::decimal:
       OS << format("%7u ", Offset);
       break;
     }
-    OS << L << '\n';
+
+    if (PrintRawBytes) {
+      OS << L << '\n';
+    } else {
+      mbstate_t mbs = {};
+
+      const char *P = L.begin();
+      const char *E = L.end();
+      while (P < E) {
+        const UTF32 Ch = read(P, E);
+        if (Unicode == unicode::invalid || Ch <= 0x7F) {
+          OS << (char)Ch;
+          continue;
+        }
+
+        if (Unicode == unicode::locale) {
+          // If we have a 16-bit wchar_t and the character does not fit, replace
+          // it with U+FFFD.
+          wchar_t WCh = Ch;
+          const wchar_t *WNext;
+          if ((UTF32)WCh != Ch)
+            WCh = 0xFFFD;
+          char mbstring[MB_LEN_MAX];
+          char *mbend;
+          [[maybe_unused]] const auto res =
+              cvt.out(mbs, &WCh, &WCh + 1, WNext, mbstring,
+                      &mbstring[MB_LEN_MAX], mbend);
+          assert(res == std::codecvt_base::ok);
+          OS << StringRef(mbstring, mbend - mbstring);
+          continue;
+        }
+
+        if (Unicode == unicode::escape || Unicode == unicode::highlight) {
+          WithColor COS(OS, raw_ostream::RED, false, false,
+                        Unicode == unicode::highlight ? ColorMode::Auto
+                                                      : ColorMode::Disable);
+          if (Ch <= 0xFFFF)
+            COS << "\\u" << utohexstr(Ch, false, 4);
+          else
+            COS << "\\U" << utohexstr(Ch, false, 8);
+          continue;
+        }
+
+        char UTF8[4];
+        char *UTF8end = UTF8;
+        ConvertCodePointToUTF8(Ch, UTF8end);
+        if (Unicode == unicode::hex) {
+          OS << "<0x";
+          for (unsigned char Byte : StringRef(UTF8, UTF8end - UTF8))
+            OS << utohexstr(Byte, true, 2);
+          OS << '>';
+          continue;
+        }
+
+        assert(Unicode == unicode::default_);
+        OS << StringRef(UTF8, UTF8end - UTF8);
+      }
+      OS << '\n';
+    }
   };
 
+  std::size_t NumPrintable = 0;
+
   const char *B = Contents.begin();
-  const char *P = nullptr, *E = nullptr, *S = nullptr;
-  for (P = Contents.begin(), E = Contents.end(); P < E; ++P) {
-    if (isPrint(*P) || *P == '\t') {
+  const char *P = Contents.begin(), *E = Contents.end(), *S = nullptr;
+  for (P = Contents.begin(), E = Contents.end(); P < E;) {
+    const char *N = P;
+    UTF32 Ch = read(N, E);
+
+    const bool Printable =
+        Ch == '\t' ||
+        (Unicode == unicode::invalid ? Ch <= 0x7f && isPrint(Ch)
+         : Encoding == encoding::S   ? ctype.is(std::ctype_base::print, Ch)
+                                     : sys::unicode::isPrintable(Ch));
+
+    if (Printable) {
       if (S == nullptr)
         S = P;
+      ++NumPrintable;
     } else if (S) {
-      print(S - B, StringRef(S, P - S));
+      print(S - B, StringRef(S, P - S), NumPrintable);
       S = nullptr;
+      NumPrintable = 0;
     }
+
+    P = N;
   }
   if (S)
-    print(S - B, StringRef(S, E - S));
+    print(S - B, StringRef(S, E - S), NumPrintable);
 }
 
 int main(int argc, char **argv) {
@@ -149,20 +343,89 @@ int main(int argc, char **argv) {
     return 0;
   }
 
+  const auto *EncodingArg = Args.getLastArg(OPT_encoding_EQ);
+  const auto *UnicodeArg = Args.getLastArg(OPT_unicode_EQ);
+
+  Encoding =
+      EncodingArg
+          ? llvm::StringSwitch<encoding>(EncodingArg->getValue())
+                .Case("s", encoding::s)
+                .Case("S", encoding::S)
+                .Case("u", encoding::u)
+                .Case("b", encoding::b)
+                .Case("l", encoding::l)
+                .Case("B", encoding::B)
+                .Case("L", encoding::L)
+                .Predicate(
+                    [](StringRef) -> bool {
+                      reportCmdLineError(
+                          "--encoding value should be one of: "
+                          "'s' (7-bit characters), "
+                          "'S' (8-bit characters), "
+                          "'u' (UTF-8 characters), "
+                          "'b', 'l' (16-bit big/little endian characters), "
+                          "'B', 'L' (32-bit big/little endian characters)");
+                    },
+                    encoding{})
+                .DefaultUnreachable()
+          : encoding::S;
   parseIntArg(Args, OPT_bytes_EQ, MinLength);
   PrintFileName = Args.hasArg(OPT_print_file_name);
   StringRef R = Args.getLastArgValue(OPT_radix_EQ);
   if (R.empty())
-    Radix = none;
+    Radix = radix::none;
   else if (R == "o")
-    Radix = octal;
+    Radix = radix::octal;
   else if (R == "d")
-    Radix = decimal;
+    Radix = radix::decimal;
   else if (R == "x")
-    Radix = hexadecimal;
+    Radix = radix::hexadecimal;
   else
     reportCmdLineError("--radix value should be one of: '' (no offset), 'o' "
                        "(octal), 'd' (decimal), 'x' (hexadecimal)");
+  Unicode = UnicodeArg ? llvm::StringSwitch<unicode>(UnicodeArg->getValue())
+                             .Case("default", unicode::default_)
+                             .Case("invalid", unicode::invalid)
+                             .Case("locale", unicode::locale)
+                             .Case("escape", unicode::escape)
+                             .Case("hex", unicode::hex)
+                             .Case("highlight", unicode::highlight)
+                             .Predicate(
+                                 [](StringRef) -> bool {
+                                   reportCmdLineError(
+                                       "--unicode value should be one of: "
+                                       "default, "
+                                       "invalid, "
+                                       "locale, "
+                                       "escape, "
+                                       "hex, "
+                                       "highlight");
+                                 },
+                                 unicode::default_)
+                             .DefaultUnreachable()
+                       : unicode::locale;
+
+  // The defaults are ugly to maintain compatibility with GNU strings in the
+  // common cases. The general idea is that --encoding specifies the input
+  // encoding, --unicode specifies the output encoding, but when one is
+  // specified, the other defaults to the best match with the limitation that
+  // the output encoding is never UTF-16 or UTF-32.
+  if (!EncodingArg && Unicode != unicode::invalid && Unicode != unicode::locale)
+    Encoding = encoding::u;
+
+  if (!UnicodeArg && Encoding != encoding::S)
+    Unicode = unicode::default_;
+
+  // With --encoding=s, all output encodings would give the same results. Use
+  // the simplest one.
+  if (Encoding == encoding::s)
+    Unicode = unicode::invalid;
+
+  // With --encoding=u --unicode=invalid, we do not need to spend time decoding
+  // UTF-8 only to reject whatever results we get, we can reject multibyte
+  // characters right away.
+  if (Encoding == encoding::u && Unicode == unicode::invalid)
+    Encoding = encoding::s;
 
   if (MinLength == 0) {
     errs() << "invalid minimum string length 0\n";

>From 11d4e557a1824d7cd71f720cbed1e9a162e69f50 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Mon, 7 Sep 2026 19:46:55 +0100
Subject: [PATCH 02/41] Remove llvm_unreachable to suppress compiler warning

---
 llvm/tools/llvm-strings/llvm-strings.cpp | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 655cca584437c2..79baa08cc6801f 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -201,9 +201,6 @@ static void strings(raw_ostream &OS, StringRef FileName, StringRef Contents) {
       P = reinterpret_cast<const char *>(UP);
       break;
     }
-
-    default:
-      llvm_unreachable("unhandled encoding");
     }
 
     return Ch;

>From d6cb511d4d1f18486f1221f7b57ea0cca7d7d026 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Tue, 8 Sep 2026 14:47:58 +0100
Subject: [PATCH 03/41] Update manpage, remove --encoding=b/l/B/L and
 --unicode, add test

---
 llvm/docs/CommandGuide/llvm-strings.md        |  15 +-
 llvm/test/tools/llvm-strings/encoding.test    |  28 +++
 llvm/test/tools/llvm-strings/length.test      |  36 ---
 .../tools/llvm-strings/negative-char.test     |   2 +-
 llvm/tools/llvm-strings/Opts.td               |  17 +-
 llvm/tools/llvm-strings/llvm-strings.cpp      | 208 ++----------------
 6 files changed, 67 insertions(+), 239 deletions(-)
 create mode 100644 llvm/test/tools/llvm-strings/encoding.test

diff --git a/llvm/docs/CommandGuide/llvm-strings.md b/llvm/docs/CommandGuide/llvm-strings.md
index da9d0a03d9fd55..6789a316a66849 100644
--- a/llvm/docs/CommandGuide/llvm-strings.md
+++ b/llvm/docs/CommandGuide/llvm-strings.md
@@ -12,8 +12,8 @@
 {program}`llvm-strings` is a tool intended as a drop-in replacement for GNU's
 {program}`strings`, which looks for printable strings in files and writes them
 to the standard output stream. A printable string is any sequence of four (by
-default) or more printable ASCII characters. The end of the file, or any other
-byte, terminates the current sequence.
+default) or more printable characters. The end of the file, or any other byte,
+terminates the current sequence.
 
 {program}`llvm-strings` looks for strings in each `input` file specified.
 Unlike GNU {program}`strings` it looks in the entire input file, regardless of
@@ -40,8 +40,15 @@ Silently ignored. Present for GNU {program}`strings` compatibility.
 :::
 
 :::{option} --bytes=<length>, -n
-Set the minimum number of printable ASCII characters required for a sequence of
-bytes to be considered a string. The default value is 4.
+Set the minimum number of printable characters required for a sequence to be
+considered a string. The default value is 4.
+:::
+
+:::{option} --encoding=<encoding>, -e
+Specifies the encoding of the input file. Valid arguments are:
+- s (ASCII characters)
+- S (characters in the system's or user's selected character set)
+- u (UTF-8 characters)
 :::
 
 :::{option} --help, -h
diff --git a/llvm/test/tools/llvm-strings/encoding.test b/llvm/test/tools/llvm-strings/encoding.test
new file mode 100644
index 00000000000000..7355516e693cfa
--- /dev/null
+++ b/llvm/test/tools/llvm-strings/encoding.test
@@ -0,0 +1,28 @@
+## Show that llvm-strings uses the specified encoding.
+
+RUN: echo a > %t
+RUN: echo ab >> %t
+RUN: echo abc >> %t
+RUN: echo abcd >> %t
+RUN: echo abcd€ >> %t
+
+# Check that long form options work.
+RUN: llvm-strings --encoding s 2>&1 %t | FileCheck --check-prefixes CHECK,CHECK-ASCII %s
+RUN: llvm-strings --encoding S 2>&1 %t | FileCheck                                    %s
+RUN: llvm-strings --encoding u 2>&1 %t | FileCheck --check-prefixes CHECK,CHECK-UTF8  %s
+RUN: not llvm-strings --encoding x 2>&1 %t | FileCheck --check-prefixes CHECK-ERROR       %s
+
+# Check that short form options work.
+RUN: llvm-strings -e s 2>&1 %t | FileCheck --check-prefixes CHECK,CHECK-ASCII %s
+RUN: llvm-strings -e S 2>&1 %t | FileCheck                                    %s
+RUN: llvm-strings -e u 2>&1 %t | FileCheck --check-prefixes CHECK,CHECK-UTF8  %s
+
+CHECK: abcd
+CHECK-ASCII-NOT: abcd€
+CHECK-UTF8: abcd€
+
+# Check that invalid encodings are rejected.
+RUN: not llvm-strings --encoding x 2>&1 %t | FileCheck --check-prefix CHECK-ERROR %s
+RUN: not llvm-strings -e x 2>&1 %t | FileCheck --check-prefix CHECK-ERROR %s
+
+CHECK-ERROR: error: --encoding value
diff --git a/llvm/test/tools/llvm-strings/length.test b/llvm/test/tools/llvm-strings/length.test
index aaa3df198f6333..5ec95baea317eb 100644
--- a/llvm/test/tools/llvm-strings/length.test
+++ b/llvm/test/tools/llvm-strings/length.test
@@ -55,42 +55,6 @@ RUN: llvm-strings --encoding u -n 2 %t | FileCheck --check-prefix UTF-8-2 %s --i
 RUN: llvm-strings --encoding u -n 3 %t | FileCheck --check-prefix UTF-8-3 %s --implicit-check-not={{.}}
 RUN: llvm-strings --encoding u -n 4 %t | FileCheck %s --implicit-check-not={{.}} --allow-empty
 
-RUN: printf '\241\0\n\0' > %t
-RUN: printf '\241\0\242\0\n\0' >> %t
-RUN: printf '\241\0\242\0\243\0\n\0' >> %t
-RUN: not llvm-strings --encoding l -n 0 2>&1 %t | FileCheck --check-prefix CHECK-0 %s
-RUN: llvm-strings --encoding l -n 1 %t | FileCheck --check-prefix UTF-8-1 %s --implicit-check-not={{.}}
-RUN: llvm-strings --encoding l -n 2 %t | FileCheck --check-prefix UTF-8-2 %s --implicit-check-not={{.}}
-RUN: llvm-strings --encoding l -n 3 %t | FileCheck --check-prefix UTF-8-3 %s --implicit-check-not={{.}}
-RUN: llvm-strings --encoding l -n 4 %t | FileCheck %s --implicit-check-not={{.}} --allow-empty
-
-RUN: printf '\0\241\0\n' > %t
-RUN: printf '\0\241\0\242\0\n' >> %t
-RUN: printf '\0\241\0\242\0\243\0\n' >> %t
-RUN: not llvm-strings --encoding b -n 0 2>&1 %t | FileCheck --check-prefix CHECK-0 %s
-RUN: llvm-strings --encoding b -n 1 %t | FileCheck --check-prefix UTF-8-1 %s --implicit-check-not={{.}}
-RUN: llvm-strings --encoding b -n 2 %t | FileCheck --check-prefix UTF-8-2 %s --implicit-check-not={{.}}
-RUN: llvm-strings --encoding b -n 3 %t | FileCheck --check-prefix UTF-8-3 %s --implicit-check-not={{.}}
-RUN: llvm-strings --encoding b -n 4 %t | FileCheck %s --implicit-check-not={{.}} --allow-empty
-
-RUN: printf '\241\0\0\0\n\0\0\0' > %t
-RUN: printf '\241\0\0\0\242\0\0\0\n\0\0\0' >> %t
-RUN: printf '\241\0\0\0\242\0\0\0\243\0\0\0\n\0\0\0' >> %t
-RUN: not llvm-strings --encoding L -n 0 2>&1 %t | FileCheck --check-prefix CHECK-0 %s
-RUN: llvm-strings --encoding L -n 1 %t | FileCheck --check-prefix UTF-8-1 %s --implicit-check-not={{.}}
-RUN: llvm-strings --encoding L -n 2 %t | FileCheck --check-prefix UTF-8-2 %s --implicit-check-not={{.}}
-RUN: llvm-strings --encoding L -n 3 %t | FileCheck --check-prefix UTF-8-3 %s --implicit-check-not={{.}}
-RUN: llvm-strings --encoding L -n 4 %t | FileCheck %s --implicit-check-not={{.}} --allow-empty
-
-RUN: printf '\0\0\0\241\0\0\0\n' > %t
-RUN: printf '\0\0\0\241\0\0\0\242\0\0\0\n' >> %t
-RUN: printf '\0\0\0\241\0\0\0\242\0\0\0\243\0\0\0\n' >> %t
-RUN: not llvm-strings --encoding B -n 0 2>&1 %t | FileCheck --check-prefix CHECK-0 %s
-RUN: llvm-strings --encoding B -n 1 %t | FileCheck --check-prefix UTF-8-1 %s --implicit-check-not={{.}}
-RUN: llvm-strings --encoding B -n 2 %t | FileCheck --check-prefix UTF-8-2 %s --implicit-check-not={{.}}
-RUN: llvm-strings --encoding B -n 3 %t | FileCheck --check-prefix UTF-8-3 %s --implicit-check-not={{.}}
-RUN: llvm-strings --encoding B -n 4 %t | FileCheck %s --implicit-check-not={{.}} --allow-empty
-
 UTF-8-1:      ¡
 UTF-8-1-NEXT: ¡¢
 UTF-8-1-NEXT: ¡¢£
diff --git a/llvm/test/tools/llvm-strings/negative-char.test b/llvm/test/tools/llvm-strings/negative-char.test
index 3e661b04724706..8e61700e07e8af 100644
--- a/llvm/test/tools/llvm-strings/negative-char.test
+++ b/llvm/test/tools/llvm-strings/negative-char.test
@@ -3,7 +3,7 @@
 ## unless using an encoding where they form valid characters.
 
 # RUN: printf "z\0\200\0\302\241\0a\0" | llvm-strings --encoding=s --bytes 1 - | FileCheck %s
-# RUN: printf "z\0\200\0\302\241\0a\0" | llvm-strings --unicode=default --bytes 1 - | FileCheck %s --check-prefixes=CHECK,CHECK-UTF8
+# RUN: printf "z\0\200\0\302\241\0a\0" | llvm-strings --encoding=u --bytes 1 - | FileCheck %s --check-prefixes=CHECK,CHECK-UTF8
 # CHECK: z{{$}}
 # CHECK-UTF8-NEXT: {{^}}¡{{$}}
 # CHECK-NEXT: {{^}}a
diff --git a/llvm/tools/llvm-strings/Opts.td b/llvm/tools/llvm-strings/Opts.td
index cf7a406fcec1f9..dd4f4349d7eed9 100644
--- a/llvm/tools/llvm-strings/Opts.td
+++ b/llvm/tools/llvm-strings/Opts.td
@@ -12,23 +12,12 @@ multiclass Eq<string name, string help> {
 def all : FF<"all", "Silently ignored. Present for GNU strings compatibility">;
 defm bytes : Eq<"bytes", "Print sequences of the specified length">;
 defm encoding : Eq<"encoding", [{Select the character encoding of the strings to find:
-s (7-bit characters)
-S (8-bit characters in the current locale)
-u (UTF-8)
-b (UTF-16 big endian)
-l (UTF-16 little endian)
-B (UTF-32 big endian)
-L (UTF-32 little endian)}]>;
+s (ASCII characters)
+S (characters in the system's or user's character set)
+u (UTF-8 characters)}]>;
 def help : FF<"help", "Display this help">;
 def print_file_name : Flag<["--"], "print-file-name">, HelpText<"Print the name of the file before each string">;
 defm radix : Eq<"radix", "Print the offset within the file with the specified radix: o (octal), d (decimal), x (hexadecimal)">, MetaVarName<"<radix>">;
-defm unicode : Eq<"unicode", [{Control the display of UTF-8 multibyte characters:
-default   (print UTF-8 characters)
-invalid   (reject multibyte characters)
-locale    (print characters in the current locale)
-escape    (\u0000 escape sequences)
-hex       (print UTF-8 encoding in hex)
-highlight (like escape, but highlighted)}]>;
 def version : FF<"version", "Display the version">;
 
 def : F<"a", "Alias for --all">, Alias<all>;
diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 79baa08cc6801f..bb61425f790d34 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -75,15 +75,12 @@ static cl::list<std::string> InputFileNames(cl::Positional,
 static int MinLength = 4;
 static bool PrintFileName;
 
-enum class encoding { s, S, u, b, l, B, L };
+enum class encoding { s, S, u };
 static encoding Encoding;
 
 enum class radix { none, octal, hexadecimal, decimal };
 static radix Radix;
 
-enum class unicode { default_, invalid, locale, escape, hex, highlight };
-static unicode Unicode;
-
 [[noreturn]] static void reportCmdLineError(const Twine &Message) {
   WithColor::error(errs(), ToolName) << Message << "\n";
   exit(1);
@@ -104,13 +101,6 @@ static void strings(raw_ostream &OS, StringRef FileName, StringRef Contents) {
   auto &ctype = std::use_facet<std::ctype<wchar_t>>(loc);
   std::mbstate_t mbs{};
 
-  const bool PrintRawBytes =
-      Encoding == encoding::s ||
-      (Encoding == encoding::S &&
-       (Unicode == unicode::invalid || Unicode == unicode::locale)) ||
-      (Encoding == encoding::u &&
-       (Unicode == unicode::default_ || Unicode == unicode::invalid));
-
   auto read = [&cvt, &mbs](const char *&P, const char *E) -> UTF32 {
     UTF32 Ch;
 
@@ -159,55 +149,13 @@ static void strings(raw_ostream &OS, StringRef FileName, StringRef Contents) {
       P = reinterpret_cast<const char *>(UP);
       break;
     }
-
-    case encoding::b:
-    case encoding::l: {
-      const UTF16 *UP = reinterpret_cast<const UTF16 *>(P);
-      const UTF16 *UE = reinterpret_cast<const UTF16 *>(E);
-      const bool DoByteSwap =
-          (Encoding == encoding::b &&
-           endianness::native == endianness::little) ||
-          (Encoding == encoding::l && endianness::native == endianness::big);
-      UTF32 *Next = &Ch;
-      if (!DoByteSwap) {
-        ConvertUTF16toUTF32(&UP, UE, &Next, &Ch + 1, strictConversion);
-      } else {
-        // We never need more than two UTF16 words to make up one character.
-        const UTF16 Buf[2] = {byteswap(UP[0]),
-                              UP + 1 == UE ? UTF16(0) : byteswap(UP[1])};
-        const UTF16 *BufP = Buf;
-        ConvertUTF16toUTF32(&BufP, &Buf[2], &Next, &Ch + 1, strictConversion);
-        UP += BufP - Buf;
-      }
-      if (Next == &Ch || !Ch) {
-        // If there was any error, skip the current word to allow the next to
-        // start a character.
-        Ch = 0;
-        UP = std::next(reinterpret_cast<const UTF16 *>(P));
-      }
-      P = reinterpret_cast<const char *>(UP);
-      break;
-    }
-
-    case encoding::B:
-    case encoding::L: {
-      const UTF32 *UP = reinterpret_cast<const UTF32 *>(P);
-      const bool DoByteSwap =
-          (Encoding == encoding::B &&
-           endianness::native == endianness::little) ||
-          (Encoding == encoding::L && endianness::native == endianness::big);
-      Ch = DoByteSwap ? byteswap(*UP) : *UP;
-      ++UP;
-      P = reinterpret_cast<const char *>(UP);
-      break;
-    }
     }
 
     return Ch;
   };
 
-  auto print = [&OS, FileName, PrintRawBytes, &read,
-                &cvt](unsigned Offset, StringRef L, size_t N) {
+  auto print = [&OS, FileName, &read, &cvt](unsigned Offset, StringRef L,
+                                            size_t N) {
     if (N < static_cast<size_t>(MinLength))
       return;
     if (PrintFileName)
@@ -226,64 +174,7 @@ static void strings(raw_ostream &OS, StringRef FileName, StringRef Contents) {
       break;
     }
 
-    if (PrintRawBytes) {
-      OS << L << '\n';
-    } else {
-      mbstate_t mbs = {};
-
-      const char *P = L.begin();
-      const char *E = L.end();
-      while (P < E) {
-        const UTF32 Ch = read(P, E);
-        if (Unicode == unicode::invalid || Ch <= 0x7F) {
-          OS << (char)Ch;
-          continue;
-        }
-
-        if (Unicode == unicode::locale) {
-          // If we have a 16-bit wchar_t and the character does not fit, replace
-          // it with U+FFFD.
-          wchar_t WCh = Ch;
-          const wchar_t *WNext;
-          if ((UTF32)WCh != Ch)
-            WCh = 0xFFFD;
-          char mbstring[MB_LEN_MAX];
-          char *mbend;
-          [[maybe_unused]] const auto res =
-              cvt.out(mbs, &WCh, &WCh + 1, WNext, mbstring,
-                      &mbstring[MB_LEN_MAX], mbend);
-          assert(res == std::codecvt_base::ok);
-          OS << StringRef(mbstring, mbend - mbstring);
-          continue;
-        }
-
-        if (Unicode == unicode::escape || Unicode == unicode::highlight) {
-          WithColor COS(OS, raw_ostream::RED, false, false,
-                        Unicode == unicode::highlight ? ColorMode::Auto
-                                                      : ColorMode::Disable);
-          if (Ch <= 0xFFFF)
-            COS << "\\u" << utohexstr(Ch, false, 4);
-          else
-            COS << "\\U" << utohexstr(Ch, false, 8);
-          continue;
-        }
-
-        char UTF8[4];
-        char *UTF8end = UTF8;
-        ConvertCodePointToUTF8(Ch, UTF8end);
-        if (Unicode == unicode::hex) {
-          OS << "<0x";
-          for (unsigned char Byte : StringRef(UTF8, UTF8end - UTF8))
-            OS << utohexstr(Byte, true, 2);
-          OS << '>';
-          continue;
-        }
-
-        assert(Unicode == unicode::default_);
-        OS << StringRef(UTF8, UTF8end - UTF8);
-      }
-      OS << '\n';
-    }
+    OS << L << '\n';
   };
 
   std::size_t NumPrintable = 0;
@@ -296,9 +187,9 @@ static void strings(raw_ostream &OS, StringRef FileName, StringRef Contents) {
 
     const bool Printable =
         Ch == '\t' ||
-        (Unicode == unicode::invalid ? Ch <= 0x7f && isPrint(Ch)
-         : Encoding == encoding::S   ? ctype.is(std::ctype_base::print, Ch)
-                                     : sys::unicode::isPrintable(Ch));
+        (Encoding == encoding::s   ? Ch <= 0x7f && isPrint(Ch)
+         : Encoding == encoding::S ? ctype.is(std::ctype_base::print, Ch)
+                                   : sys::unicode::isPrintable(Ch));
 
     if (Printable) {
       if (S == nullptr)
@@ -341,31 +232,23 @@ int main(int argc, char **argv) {
   }
 
   const auto *EncodingArg = Args.getLastArg(OPT_encoding_EQ);
-  const auto *UnicodeArg = Args.getLastArg(OPT_unicode_EQ);
-
-  Encoding =
-      EncodingArg
-          ? llvm::StringSwitch<encoding>(EncodingArg->getValue())
-                .Case("s", encoding::s)
-                .Case("S", encoding::S)
-                .Case("u", encoding::u)
-                .Case("b", encoding::b)
-                .Case("l", encoding::l)
-                .Case("B", encoding::B)
-                .Case("L", encoding::L)
-                .Predicate(
-                    [](StringRef) -> bool {
-                      reportCmdLineError(
-                          "--encoding value should be one of: "
-                          "'s' (7-bit characters), "
-                          "'S' (8-bit characters), "
-                          "'u' (UTF-8 characters), "
-                          "'b', 'l' (16-bit big/little endian characters), "
-                          "'B', 'L' (32-bit big/little endian characters)");
-                    },
-                    encoding{})
-                .DefaultUnreachable()
-          : encoding::S;
+
+  Encoding = EncodingArg ? llvm::StringSwitch<encoding>(EncodingArg->getValue())
+                               .Case("s", encoding::s)
+                               .Case("S", encoding::S)
+                               .Case("u", encoding::u)
+                               .Predicate(
+                                   [](StringRef) -> bool {
+                                     reportCmdLineError(
+                                         "--encoding value should be one of: "
+                                         "'s' (ASCII characters), "
+                                         "'S' (characters in the system's or "
+                                         "user's selected character set), "
+                                         "'u' (UTF-8 characters)");
+                                   },
+                                   encoding{})
+                               .DefaultUnreachable()
+                         : encoding::S;
   parseIntArg(Args, OPT_bytes_EQ, MinLength);
   PrintFileName = Args.hasArg(OPT_print_file_name);
   StringRef R = Args.getLastArgValue(OPT_radix_EQ);
@@ -380,49 +263,6 @@ int main(int argc, char **argv) {
   else
     reportCmdLineError("--radix value should be one of: '' (no offset), 'o' "
                        "(octal), 'd' (decimal), 'x' (hexadecimal)");
-  Unicode = UnicodeArg ? llvm::StringSwitch<unicode>(UnicodeArg->getValue())
-                             .Case("default", unicode::default_)
-                             .Case("invalid", unicode::invalid)
-                             .Case("locale", unicode::locale)
-                             .Case("escape", unicode::escape)
-                             .Case("hex", unicode::hex)
-                             .Case("highlight", unicode::highlight)
-                             .Predicate(
-                                 [](StringRef) -> bool {
-                                   reportCmdLineError(
-                                       "--unicode value should be one of: "
-                                       "default, "
-                                       "invalid, "
-                                       "locale, "
-                                       "escape, "
-                                       "hex, "
-                                       "highlight");
-                                 },
-                                 unicode::default_)
-                             .DefaultUnreachable()
-                       : unicode::locale;
-
-  // The defaults are ugly to maintain compatibility with GNU strings in the
-  // common cases. The general idea is that --encoding specifies the input
-  // encoding, --unicode specifies the output encoding, but when one is
-  // specified, the other defaults to the best match with the limitation that
-  // the output encoding is never UTF-16 or UTF-32.
-  if (!EncodingArg && Unicode != unicode::invalid && Unicode != unicode::locale)
-    Encoding = encoding::u;
-
-  if (!UnicodeArg && Encoding != encoding::S)
-    Unicode = unicode::default_;
-
-  // With --encoding=s, all output encodings would give the same results. Use
-  // the simplest one.
-  if (Encoding == encoding::s)
-    Unicode = unicode::invalid;
-
-  // With --encoding=u --unicode=invalid, we do not need to spend time decoding
-  // UTF-8 only to reject whatever results we get, we can reject multibyte
-  // characters right away.
-  if (Encoding == encoding::u && Unicode == unicode::invalid)
-    Encoding = encoding::s;
 
   if (MinLength == 0) {
     errs() << "invalid minimum string length 0\n";

>From 16213a08dc2fb154d0d3c105b611093f194c84ba Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Tue, 8 Sep 2026 16:04:04 +0100
Subject: [PATCH 04/41] Remove unused captures

---
 llvm/tools/llvm-strings/llvm-strings.cpp | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index bb61425f790d34..2444847ef812ca 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -154,8 +154,7 @@ static void strings(raw_ostream &OS, StringRef FileName, StringRef Contents) {
     return Ch;
   };
 
-  auto print = [&OS, FileName, &read, &cvt](unsigned Offset, StringRef L,
-                                            size_t N) {
+  auto print = [&OS, FileName](unsigned Offset, StringRef L, size_t N) {
     if (N < static_cast<size_t>(MinLength))
       return;
     if (PrintFileName)

>From 02654008d5f94523fa2fc8e8ac521af576b3653d Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Fri, 11 Sep 2026 14:17:34 +0100
Subject: [PATCH 05/41] Apply review comments to docs, encoding.test

---
 llvm/docs/CommandGuide/llvm-strings.md     |  6 +++---
 llvm/test/tools/llvm-strings/encoding.test | 17 +++++++++++------
 2 files changed, 14 insertions(+), 9 deletions(-)

diff --git a/llvm/docs/CommandGuide/llvm-strings.md b/llvm/docs/CommandGuide/llvm-strings.md
index e72cea195d6e5e..5a45dcd426c058 100644
--- a/llvm/docs/CommandGuide/llvm-strings.md
+++ b/llvm/docs/CommandGuide/llvm-strings.md
@@ -12,8 +12,8 @@
 {program}`llvm-strings` is a tool intended as a drop-in replacement for GNU's
 {program}`strings`, which looks for printable strings in files and writes them
 to the standard output stream. A printable string is any sequence of four (by
-default) or more printable characters. The end of the file, or any other byte,
-terminates the current sequence.
+default) or more printable characters. The end of the file, or any unprintable
+byte sequence, terminates the current sequence.
 
 {program}`llvm-strings` looks for strings in each `input` file specified.
 Unlike GNU {program}`strings` it looks in the entire input file, regardless of
@@ -51,7 +51,7 @@ characters.
 :::{option} --encoding=<encoding>, -e
 Specifies the encoding of the input file. Valid arguments are:
 - `s` (ASCII)
-- `S` (default character set)
+- `S` (current locale's character set)
 - `utf8` (UTF-8)
 :::
 
diff --git a/llvm/test/tools/llvm-strings/encoding.test b/llvm/test/tools/llvm-strings/encoding.test
index 46a58789c5bb27..b07df9209d5852 100644
--- a/llvm/test/tools/llvm-strings/encoding.test
+++ b/llvm/test/tools/llvm-strings/encoding.test
@@ -1,31 +1,36 @@
-## Show that llvm-strings uses the specified encoding.
+# Show that llvm-strings uses the specified encoding.
 RUN: echo a > %t
 RUN: echo ab >> %t
 RUN: echo abc >> %t
 RUN: echo abcd >> %t
 RUN: echo abcd€ >> %t
 
-## Check that long form options work.
+# Check that long form options work.
 RUN: llvm-strings --encoding s %t | FileCheck --check-prefixes CHECK,CHECK-ASCII --strict-whitespace --match-full-lines %s
 RUN: llvm-strings --encoding S %t | FileCheck --check-prefixes CHECK,CHECK-LOCALE --strict-whitespace --match-full-lines %s
 RUN: llvm-strings --encoding utf8 %t | FileCheck --check-prefixes CHECK,CHECK-UTF8 --strict-whitespace --match-full-lines %s
 
-## Check that short form options work.
+# Check that short form options work.
 RUN: llvm-strings -e s %t | FileCheck --check-prefixes CHECK,CHECK-ASCII --strict-whitespace --match-full-lines %s
 RUN: llvm-strings -e S %t | FileCheck --check-prefixes CHECK,CHECK-LOCALE --strict-whitespace --match-full-lines %s
 RUN: llvm-strings -e utf8 %t | FileCheck --check-prefixes CHECK,CHECK-UTF8 --strict-whitespace --match-full-lines %s
 
-## Check that the default output matches that of -e S.
+# Check that the default output matches that of -e S.
 RUN: llvm-strings -e S %t > %t.1
 RUN: llvm-strings %t > %t.2
 RUN: cmp %t.1 %t.2
 
+# The first line containing just abcd should always be matched.
 CHECK:abcd
-CHECK-ASCII-NOT:abcd€
+# In -e s, the bytes of € should not be included.
+CHECK-ASCII:abcd
+# In -e S, any bytes of € may or may not be included depending on the locale
+# in effect. The locale in effect may also be altered by llvm-lit.
 CHECK-LOCALE:abcd{{.*}}
+# In -e utf8, all bytes of € should be included.
 CHECK-UTF8:abcd€
 
-## Check that invalid --encoding values are rejected.
+# Check that invalid --encoding values are rejected.
 RUN: not llvm-strings --encoding x 2>&1 %t | FileCheck --check-prefix CHECK-ERROR %s
 RUN: not llvm-strings -e x 2>&1 %t | FileCheck --check-prefix CHECK-ERROR %s
 

>From 1210ded87928d89b63865cb4d22ee3ad07adc965 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Fri, 25 Sep 2026 17:42:42 +0100
Subject: [PATCH 06/41] Avoid raw UTF-8 in test

---
 llvm/test/tools/llvm-strings/encoding.test | 22 +++++++++++++++-------
 1 file changed, 15 insertions(+), 7 deletions(-)

diff --git a/llvm/test/tools/llvm-strings/encoding.test b/llvm/test/tools/llvm-strings/encoding.test
index b07df9209d5852..b0b1b8f3164584 100644
--- a/llvm/test/tools/llvm-strings/encoding.test
+++ b/llvm/test/tools/llvm-strings/encoding.test
@@ -3,7 +3,10 @@ RUN: echo a > %t
 RUN: echo ab >> %t
 RUN: echo abc >> %t
 RUN: echo abcd >> %t
-RUN: echo abcd€ >> %t
+# UTF-8 encoding of U+200A HAIR SPACE (printable)
+RUN: printf 'abcd\342\200\212\n' >> %t
+# UTF-8 encoding of U+200B ZERO WIDTH SPACE (non-printable)
+RUN: printf 'abcd\342\200\212\342\200\213\n' >> %t
 
 # Check that long form options work.
 RUN: llvm-strings --encoding s %t | FileCheck --check-prefixes CHECK,CHECK-ASCII --strict-whitespace --match-full-lines %s
@@ -22,13 +25,18 @@ RUN: cmp %t.1 %t.2
 
 # The first line containing just abcd should always be matched.
 CHECK:abcd
-# In -e s, the bytes of € should not be included.
+# In -e s, the bytes of U+200A and U+200B should not be included.
 CHECK-ASCII:abcd
-# In -e S, any bytes of € may or may not be included depending on the locale
-# in effect. The locale in effect may also be altered by llvm-lit.
-CHECK-LOCALE:abcd{{.*}}
-# In -e utf8, all bytes of € should be included.
-CHECK-UTF8:abcd€
+CHECK-ASCII:abcd
+# In -e S, any bytes of U+200A and U+200B may or may not be included depending
+# on the locale in effect. The locale in effect may also be altered by llvm-lit.
+# Make sure if changing the test that the byte representations should not allow
+# this to be split into multiple strings in non-UTF-8 locales.
+CHECK-LOCALE:abcd{{(\xe2(\x80\x8a?)?)?}}
+CHECK-LOCALE:abcd{{(\xe2(\x80(\x8a(\xe2(\x80\x8b?)?)?)?)?)?}}
+# In -e utf8, all bytes of U+200A should be included, but none of U+200B.
+CHECK-UTF8:abcd{{\xe2\x80\x8a}}
+CHECK-UTF8:abcd{{\xe2\x80\x8a}}
 
 # Check that invalid --encoding values are rejected.
 RUN: not llvm-strings --encoding x 2>&1 %t | FileCheck --check-prefix CHECK-ERROR %s

>From 61021570fd59da32bbf3df18d35a7b5b5147d215 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Fri, 25 Sep 2026 17:55:13 +0100
Subject: [PATCH 07/41] Change other tests too

---
 llvm/test/tools/llvm-strings/length.test        | 12 ++++++------
 llvm/test/tools/llvm-strings/negative-char.test |  2 +-
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/llvm/test/tools/llvm-strings/length.test b/llvm/test/tools/llvm-strings/length.test
index 01d64ee9cc71b2..79b7a324df6e9a 100644
--- a/llvm/test/tools/llvm-strings/length.test
+++ b/llvm/test/tools/llvm-strings/length.test
@@ -55,11 +55,11 @@ RUN: llvm-strings --encoding utf8 -n 2 %t | FileCheck --check-prefix UTF-8-2 %s
 RUN: llvm-strings --encoding utf8 -n 3 %t | FileCheck --check-prefix UTF-8-3 %s --implicit-check-not={{.}}
 RUN: llvm-strings --encoding utf8 -n 4 %t | FileCheck %s --implicit-check-not={{.}} --allow-empty
 
-UTF-8-1:      ¡
-UTF-8-1-NEXT: ¡¢
-UTF-8-1-NEXT: ¡¢£
+UTF-8-1:      {{\xc2\xa1}}
+UTF-8-1-NEXT: {{\xc2\xa1\xc2\xa2}}
+UTF-8-1-NEXT: {{\xc2\xa1\xc2\xa2\xc2\xa3}}
 
-UTF-8-2:      ¡¢
-UTF-8-2-NEXT: ¡¢£
+UTF-8-2:      {{\xc2\xa1\xc2\xa2}}
+UTF-8-2-NEXT: {{\xc2\xa1\xc2\xa2\xc2\xa3}}
 
-UTF-8-3:      ¡¢£
+UTF-8-3:      {{\xc2\xa1\xc2\xa2\xc2\xa3}}
diff --git a/llvm/test/tools/llvm-strings/negative-char.test b/llvm/test/tools/llvm-strings/negative-char.test
index b85dd133335e0a..d738f2fcb0b999 100644
--- a/llvm/test/tools/llvm-strings/negative-char.test
+++ b/llvm/test/tools/llvm-strings/negative-char.test
@@ -5,5 +5,5 @@
 # RUN: printf "z\0\200\0\302\241\0a\0" | llvm-strings --encoding=s --bytes 1 - | FileCheck %s
 # RUN: printf "z\0\200\0\302\241\0a\0" | llvm-strings --encoding=utf8 --bytes 1 - | FileCheck %s --check-prefixes=CHECK,CHECK-UTF8
 # CHECK: z{{$}}
-# CHECK-UTF8-NEXT: {{^}}¡{{$}}
+# CHECK-UTF8-NEXT: {{^\xc2\xa1$}}
 # CHECK-NEXT: {{^}}a

>From 85c6cf66e754250c85ae02e4517133414933565d Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Tue, 29 Sep 2026 15:19:03 +0100
Subject: [PATCH 08/41] Handle ok for partial multibyte characters, check EOF

---
 llvm/test/tools/llvm-strings/encoding.test | 2 +-
 llvm/tools/llvm-strings/llvm-strings.cpp   | 6 ++++--
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/llvm/test/tools/llvm-strings/encoding.test b/llvm/test/tools/llvm-strings/encoding.test
index 7b5931d774d32b..c2f014345e98c0 100644
--- a/llvm/test/tools/llvm-strings/encoding.test
+++ b/llvm/test/tools/llvm-strings/encoding.test
@@ -65,7 +65,7 @@ RUN: printf "%16377se\342\200\212e\342\200\213e\342\200\212e\342\200\212e" | \
 RUN: tr " " "\0" | llvm-strings -e utf8 | \
 RUN: FileCheck %s --check-prefix=CHECK-UTF8-CHUNK-B --match-full-lines --strict-whitespace --implicit-check-not={{.}}
 #                            v chunk boundary
-RUN: printf "%16373se\342\200\213e\342\200\212e\342\200\212e" | \
+RUN: printf "%16373se\342\200\213e\342\200\212e\342\200\212e\342" | \
 RUN: tr " " "\0" | llvm-strings -e utf8 | \
 RUN: FileCheck %s --check-prefix=CHECK-UTF8-CHUNK-B --match-full-lines --strict-whitespace --implicit-check-not={{.}}
 CHECK-UTF8-CHUNK-B:{{e\xe2\x80\x8ae\xe2\x80\x8ae}}
diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index b9661f54084578..9a56175a7b85c6 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -175,7 +175,9 @@ static void strings(raw_ostream &OS, StringRef FileName,
         }
         // Otherwise treat it as an error. A null byte is safe to treat as an
         // error, as a null byte is never printable in any locale.
-      } else if (Res == std::codecvt_base::partial && !AtEOF) {
+      } else if ((Res == std::codecvt_base::ok ||
+                  Res == std::codecvt_base::partial) &&
+                 !AtEOF) {
         // If we got a partial result but no character was written, we have an
         // incomplete multibyte character.  Do not treat this as an error,
         // instead reset the conversion state so that we can try again if/when
@@ -203,7 +205,7 @@ static void strings(raw_ostream &OS, StringRef FileName,
           P = reinterpret_cast<const char *>(UP);
           return true;
         }
-      } else if (Res == sourceExhausted) {
+      } else if (Res == sourceExhausted && !AtEOF) {
         Ch = 0;
         return false;
       }

>From 8ef0e08d724d30fed701771073b125b240da210b Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Sun, 4 Oct 2026 22:18:26 +0100
Subject: [PATCH 09/41] Drop C++ locale API, use C locale API directly. Be
 consistent in error handling.

---
 llvm/test/tools/llvm-strings/encoding.test |  56 +++--
 llvm/tools/llvm-strings/llvm-strings.cpp   | 255 ++++++++++-----------
 2 files changed, 161 insertions(+), 150 deletions(-)

diff --git a/llvm/test/tools/llvm-strings/encoding.test b/llvm/test/tools/llvm-strings/encoding.test
index c2f014345e98c0..00d1d7f7d93f5f 100644
--- a/llvm/test/tools/llvm-strings/encoding.test
+++ b/llvm/test/tools/llvm-strings/encoding.test
@@ -3,10 +3,15 @@ RUN: echo a > %t
 RUN: echo ab >> %t
 RUN: echo abc >> %t
 RUN: echo abcd >> %t
-# UTF-8 encoding of U+200A HAIR SPACE (printable)
-RUN: printf 'abcd\342\200\212\n' >> %t
+# UTF-8 encoding of U+2027 HYPHENATION POINT (printable)
+RUN: printf 'bcde\342\200\247\n' >> %t
 # UTF-8 encoding of U+200B ZERO WIDTH SPACE (non-printable)
-RUN: printf 'abcd\342\200\212\342\200\213\n' >> %t
+RUN: printf 'cdef\342\200\247\342\200\250\n' >> %t
+
+# Check that partial and invalid UTF-8 characters do not interfere with
+# preceding and subsequent strings.
+RUN: printf '\342defg\342\n' >> %t
+RUN: printf '\342\200efgh\342\200\n' >> %t
 
 # Check that long form options work.
 RUN: llvm-strings --encoding s %t | FileCheck --check-prefixes CHECK,CHECK-ASCII --strict-whitespace --match-full-lines %s
@@ -18,25 +23,38 @@ RUN: llvm-strings -e s %t | FileCheck --check-prefixes CHECK,CHECK-ASCII --stric
 RUN: llvm-strings -e S %t | FileCheck --check-prefixes CHECK,CHECK-LOCALE --strict-whitespace --match-full-lines %s
 RUN: llvm-strings -e utf8 %t | FileCheck --check-prefixes CHECK,CHECK-UTF8 --strict-whitespace --match-full-lines %s
 
-# Check that the default output matches that of -e S.
+# Check that the output of no options, -e S, and --encoding S all match.
 RUN: llvm-strings -e S %t > %t.1
-RUN: llvm-strings %t > %t.2
+RUN: llvm-strings --encoding S %t > %t.2
+RUN: llvm-strings %t > %t.3
 RUN: cmp %t.1 %t.2
+RUN: cmp %t.1 %t.3
 
 # The first line containing just abcd should always be matched.
 CHECK:abcd
-# In -e s, the bytes of U+200A and U+200B should not be included.
-CHECK-ASCII:abcd
-CHECK-ASCII:abcd
+
+# In -e s, non-ASCII bytes should not be included.
+CHECK-ASCII:bcde
+CHECK-ASCII:cdef
+CHECK-ASCII:defg
+CHECK-ASCII:efgh
+
 # In -e S, any bytes of U+200A and U+200B may or may not be included depending
 # on the locale in effect. The locale in effect may also be altered by llvm-lit.
 # Make sure if changing the test that the byte representations should not allow
 # this to be split into multiple strings in non-UTF-8 locales.
-CHECK-LOCALE:abcd{{(\xe2(\x80\x8a?)?)?}}
-CHECK-LOCALE:abcd{{(\xe2(\x80(\x8a(\xe2(\x80\x8b?)?)?)?)?)?}}
-# In -e utf8, all bytes of U+200A should be included, but none of U+200B.
-CHECK-UTF8:abcd{{\xe2\x80\x8a}}
-CHECK-UTF8:abcd{{\xe2\x80\x8a}}
+CHECK-LOCALE:bcde{{(\xe2(\x80\xa7?)?)?}}
+CHECK-LOCALE:cdef{{(\xe2(\x80(\xa7(\xe2(\x80\xa8?)?)?)?)?)?}}
+CHECK-LOCALE:{{\xe2?}}defg{{\xe2?}}
+CHECK-LOCALE:{{(\xe2?\x80)?}}efgh{{(\xe2\x80?)?}}
+
+# In -e utf8, all bytes of U+200A should be included, but none of U+200B. The
+# partial multibyte characters should not prevent abcd from being detected as a
+# string.
+CHECK-UTF8:bcde{{\xe2\x80\xa7}}
+CHECK-UTF8:cdef{{\xe2\x80\xa7}}
+CHECK-UTF8:defg
+CHECK-UTF8:efgh
 
 # Check that invalid --encoding values are rejected.
 RUN: not llvm-strings --encoding x 2>&1 %t | FileCheck --check-prefix CHECK-ERROR %s
@@ -51,21 +69,21 @@ CHECK-ERROR: error: 'x' is not a valid value for '--encoding='
 # the minimum length may or may not already be reached before the chunk
 # boundary, and the character that crosses the boundary may or may not be
 # printable.                             v chunk boundary
-RUN: printf "%16378s\342\200\212e\342\200\212e\342\200\212e\342\200\212" | \
+RUN: printf "%16378s\342\200\247e\342\200\247e\342\200\247e\342\200\247" | \
 RUN: tr " " "\0" | llvm-strings -e utf8 | \
 RUN: FileCheck %s --check-prefix=CHECK-UTF8-CHUNK-A --match-full-lines --strict-whitespace --implicit-check-not={{.}}
 #                           v chunk boundary
-RUN: printf "%16374s\342\200\212e\342\200\212e\342\200\212e\342\200\212" | \
+RUN: printf "%16374s\342\200\247e\342\200\247e\342\200\247e\342\200\247" | \
 RUN: tr " " "\0" | llvm-strings -e utf8 | \
 RUN: FileCheck %s --check-prefix=CHECK-UTF8-CHUNK-A --match-full-lines --strict-whitespace --implicit-check-not={{.}}
-CHECK-UTF8-CHUNK-A:{{\xe2\x80\x8ae\xe2\x80\x8ae\xe2\x80\x8ae\xe2\x80\x8a}}
+CHECK-UTF8-CHUNK-A:{{\xe2\x80\xa7e\xe2\x80\xa7e\xe2\x80\xa7e\xe2\x80\xa7}}
 
 #                                         v chunk boundary
-RUN: printf "%16377se\342\200\212e\342\200\213e\342\200\212e\342\200\212e" | \
+RUN: printf "%16377se\342\200\247e\342\200\250e\342\200\247e\342\200\247e" | \
 RUN: tr " " "\0" | llvm-strings -e utf8 | \
 RUN: FileCheck %s --check-prefix=CHECK-UTF8-CHUNK-B --match-full-lines --strict-whitespace --implicit-check-not={{.}}
 #                            v chunk boundary
-RUN: printf "%16373se\342\200\213e\342\200\212e\342\200\212e\342" | \
+RUN: printf "%16373se\342\200\250e\342\200\247e\342\200\247e\342" | \
 RUN: tr " " "\0" | llvm-strings -e utf8 | \
 RUN: FileCheck %s --check-prefix=CHECK-UTF8-CHUNK-B --match-full-lines --strict-whitespace --implicit-check-not={{.}}
-CHECK-UTF8-CHUNK-B:{{e\xe2\x80\x8ae\xe2\x80\x8ae}}
+CHECK-UTF8-CHUNK-B:{{e\xe2\x80\xa7e\xe2\x80\xa7e}}
diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 9a56175a7b85c6..13eaa0a359e31a 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -29,7 +29,7 @@
 #include "llvm/Support/Unicode.h"
 #include "llvm/Support/WithColor.h"
 #include <cctype>
-#include <locale>
+#include <cwctype>
 #include <string>
 
 using namespace llvm;
@@ -90,6 +90,118 @@ static void parseIntArg(const opt::InputArgList &Args, int ID, T &Value) {
   }
 }
 
+// Tries to read one character from the bytes P...E and store it in Ch.
+// Returns true if a (possibly invalid) character was read, false otherwise.
+//
+// If P...E starts with a valid complete character, P and MBState are updated
+// and true is returned.
+// If P...E is empty, or holds an incomplete character and AtEOF is false, P
+// and MBState are unchanged, Ch is set to 0, and false is returned. This is
+// intended to allow more bytes to be read and Read to be called again.
+// If P...E holds an incomplete character and AtEOF is true, or if P...E
+// starts with an invalid character, the first bytes are skipped and MBState is
+// reset to allow continuing from the next point, Ch is set to 0, and true is
+// returned.
+//
+// The number of skipped bytes for invalid characters follows the Unicode
+// definition of the maximal subpart of an ill-formed subsequence, applied to
+// arbitrary locales: it is the longest subsequence that could start a valid
+// multibyte character, or if the initial byte cannot start a valid multibyte
+// character, the initial byte. This allows consistent error recovery.
+static bool readChar(const char *&P, const char *E, std::mbstate_t &MBState,
+                     bool AtEOF, UTF32 &Ch) {
+  if (P == E)
+    return false;
+
+  switch (Encoding) {
+  case Encoding::Ascii:
+    Ch = *P++;
+    return true;
+
+  case Encoding::Locale: {
+    std::mbstate_t SaveMBState = MBState;
+    wchar_t WCh;
+    const size_t BytesRead = mbrtowc(&WCh, P, E - P, &MBState);
+    switch (BytesRead) {
+    default:
+      P += BytesRead;
+      Ch = WCh;
+      return true;
+    case 0:
+    case (size_t)-1: {
+      // We encountered a (possibly null) byte that cannot be part of a valid
+      // character from the previous state. Recover by retrying one byte at a
+      // time, processing as many bytes as possible. If this gives us a
+      // partial character, treat this partial character as a single invalid
+      // character, otherwise treat the first byte as a single invalid
+      // character.
+      const bool PartialChar = mbrtowc(&WCh, P, 1, &SaveMBState) == (size_t)-2;
+      do {
+        ++P;
+        assert(!PartialChar || P != E);
+      } while (PartialChar && mbrtowc(&WCh, P, 1, &SaveMBState) == (size_t)-2);
+      Ch = 0;
+      MBState = {};
+      return true;
+    }
+    case (size_t)-2:
+      MBState = SaveMBState;
+      return false;
+    }
+  }
+
+  case Encoding::Utf8: {
+    const UTF8 *UP = reinterpret_cast<const UTF8 *>(P);
+    const UTF8 *UE = reinterpret_cast<const UTF8 *>(E);
+    UTF32 *Next = &Ch;
+    const auto Res =
+        ConvertUTF8toUTF32(&UP, UE, &Next, &Ch + 1, strictConversion);
+    if (UP != reinterpret_cast<const UTF8 *>(P)) {
+      assert(Next != &Ch);
+    } else if (Res == sourceExhausted && !AtEOF) {
+      return false;
+    } else {
+      assert(Next == &Ch);
+      UP += findMaximalSubpartOfIllFormedUTF8Sequence(UP, UE);
+      *Next++ = 0;
+    }
+    P = reinterpret_cast<const char *>(UP);
+    return true;
+  }
+  }
+
+  llvm_unreachable("unhandled encoding");
+}
+
+static bool isStringChar(UTF32 Ch) {
+  if (Ch == '\t')
+    return true;
+
+  switch (Encoding) {
+  case Encoding::Ascii:
+    return isPrint(Ch);
+
+  case Encoding::Locale:
+    return iswprint(Ch);
+
+  case Encoding::Utf8:
+    return sys::unicode::isPrintable(Ch);
+  }
+
+  llvm_unreachable("unhandled encoding");
+}
+
+static void endString(raw_ostream &OS, std::mbstate_t &MBState) {
+  if (Encoding == Encoding::Locale) {
+    char Buf[MB_LEN_MAX];
+    // Note: This is only required for stateful encodings such as the
+    // ISO-2022 ones.
+    const size_t BytesWritten = wcrtomb(Buf, L'\0', &MBState);
+    OS << StringRef(Buf, BytesWritten - 1);
+  };
+  OS << '\n';
+}
+
 static void strings(raw_ostream &OS, StringRef FileName,
                     sys::fs::file_t Handle) {
   SmallString<sys::fs::DefaultReadChunkSize> Buffer;
@@ -111,127 +223,6 @@ static void strings(raw_ostream &OS, StringRef FileName,
     }
   };
 
-  std::locale Loc("");
-  auto &Cvt = std::use_facet<std::codecvt<wchar_t, char, std::mbstate_t>>(Loc);
-  auto &Ctype = std::use_facet<std::ctype<wchar_t>>(Loc);
-
-  auto IsStringChar = [&Ctype](UTF32 Ch) {
-    if (Ch == '\t')
-      return true;
-
-    switch (Encoding) {
-    case Encoding::Ascii:
-      return isPrint(Ch);
-
-    case Encoding::Locale:
-      return Ctype.is(std::ctype_base::print, Ch);
-
-    case Encoding::Utf8:
-      return sys::unicode::isPrintable(Ch);
-    }
-
-    llvm_unreachable("unhandled encoding");
-  };
-
-  // Tries to read one character from the bytes P...E and store it in Ch.
-  // Returns true if a (possibly invalid) character was read, false otherwise.
-  //
-  // If P...E starts with a valid complete character, P and MBState are updated
-  // and true is returned.
-  // If P...E is empty, or holds an incomplete character and AtEOF is false, P
-  // and MBState are unchanged, Ch is set to 0, and false is returned. This is
-  // intended to allow more bytes to be read and Read to be called again.
-  // If P...E holds an incomplete character and AtEOF is true, or if P...E
-  // starts with an invalid character, the first byte is skipped and MBState is
-  // reset to allow continuing from the next point, Ch is set to 0, and true is
-  // returned.
-  auto Read = [&Cvt](const char *&P, const char *E, std::mbstate_t &MBState,
-                     bool AtEOF, UTF32 &Ch) -> bool {
-    if (P == E)
-      return false;
-
-    switch (Encoding) {
-    case Encoding::Ascii:
-      Ch = *P++;
-      return true;
-
-    case Encoding::Locale: {
-      const char *N;
-      wchar_t WCh;
-      wchar_t *WNext;
-      std::mbstate_t SaveMBState = MBState;
-      const auto Res = Cvt.in(MBState, P, E, N, &WCh, &WCh + 1, WNext);
-      assert(Res != std::codecvt_base::noconv);
-
-      if (WNext != &WCh) {
-        // Only treat a non-null character as a successful conversion, as a
-        // null character may be the result of an incomplete multibyte
-        // character followed by a null byte.
-        if (WCh) {
-          // Note: this assumes wchar_t is UCS2 or UTF32.
-          Ch = WCh;
-          P = N;
-          return true;
-        }
-        // Otherwise treat it as an error. A null byte is safe to treat as an
-        // error, as a null byte is never printable in any locale.
-      } else if ((Res == std::codecvt_base::ok ||
-                  Res == std::codecvt_base::partial) &&
-                 !AtEOF) {
-        // If we got a partial result but no character was written, we have an
-        // incomplete multibyte character.  Do not treat this as an error,
-        // instead reset the conversion state so that we can try again if/when
-        // we have more characters, unless we know there are no more characters.
-        Ch = 0;
-        MBState = SaveMBState;
-        return false;
-      }
-      // If there was any error, reset the state to allow the next byte to
-      // start a character.
-      Ch = 0;
-      MBState = {};
-      ++P;
-      return true;
-    }
-
-    case Encoding::Utf8: {
-      const UTF8 *UP = reinterpret_cast<const UTF8 *>(P);
-      const UTF8 *UE = reinterpret_cast<const UTF8 *>(E);
-      UTF32 *Next = &Ch;
-      const auto Res =
-          ConvertUTF8toUTF32Partial(&UP, UE, &Next, &Ch + 1, strictConversion);
-      if (Next != &Ch) {
-        if (Ch) {
-          P = reinterpret_cast<const char *>(UP);
-          return true;
-        }
-      } else if (Res == sourceExhausted && !AtEOF) {
-        Ch = 0;
-        return false;
-      }
-      Ch = 0;
-      ++P;
-      return true;
-    }
-    }
-
-    llvm_unreachable("unhandled encoding");
-  };
-
-  auto EndString = [&OS, &Cvt](std::mbstate_t &MBState) {
-    if (Encoding == Encoding::Locale) {
-      char Buf[MB_LEN_MAX];
-      char *End;
-      auto Res = Cvt.unshift(MBState, Buf, &Buf[MB_LEN_MAX], End);
-      if (Res == std::codecvt_base::ok) {
-        // Note: This is only required for stateful encodings such as the
-        // ISO-2022 ones.
-        OS << StringRef(Buf, End - Buf);
-      }
-    };
-    OS << '\n';
-  };
-
   // To handle very large files without consuming excessive memory, we read the
   // file in a little at a time and process it then rather than reading the
   // entire file at once.
@@ -297,8 +288,8 @@ static void strings(raw_ostream &OS, StringRef FileName,
       for (;;) {
         StringEnd = Cur;
         PrevMBState = MBState;
-        EndOfChunk = !Read(Cur, End, MBState, AtEOF, Ch);
-        if (EndOfChunk || !IsStringChar(Ch))
+        EndOfChunk = !readChar(Cur, End, MBState, AtEOF, Ch);
+        if (EndOfChunk || !isStringChar(Ch))
           break;
         ++Len;
       }
@@ -333,7 +324,7 @@ static void strings(raw_ostream &OS, StringRef FileName,
         CandidateLength = 0;
       }
 
-      if (EndOfChunk || Cur == End) {
+      if (EndOfChunk) {
         // Finish handling the current chunk and update ChunkOffset.
         ChunkOffset += Cur - Begin;
         Buffer.erase(Buffer.begin(), Cur);
@@ -342,8 +333,8 @@ static void strings(raw_ostream &OS, StringRef FileName,
 
       if (InString) {
         // We haven't reached the end of the chunk, which means the string is
-        // terminated. Add a '\n' to start printing a new string.
-        EndString(PrevMBState);
+        // terminated.
+        endString(OS, PrevMBState);
         InString = false;
       }
     }
@@ -355,9 +346,9 @@ static void strings(raw_ostream &OS, StringRef FileName,
     for (;;) {
       const char *Prev = Cur;
       std::mbstate_t PrevMBState = MBState;
-      if (!Read(Cur, End, MBState, AtEOF, Ch))
+      if (!readChar(Cur, End, MBState, AtEOF, Ch))
         break;
-      if (IsStringChar(Ch)) {
+      if (isStringChar(Ch)) {
         // Find the start of the next string.
         if (!StrHead)
           StrHead = Prev;
@@ -368,7 +359,7 @@ static void strings(raw_ostream &OS, StringRef FileName,
         if (Len >= Min) {
           PrintHeader(ChunkOffset + (StrHead - Begin));
           OS << StringRef(StrHead, Prev - StrHead);
-          EndString(PrevMBState);
+          endString(OS, PrevMBState);
         }
         StrHead = nullptr;
         Len = 0;
@@ -398,10 +389,12 @@ static void strings(raw_ostream &OS, StringRef FileName,
   }
 
   if (InString)
-    EndString(MBState);
+    endString(OS, MBState);
 }
 
 int main(int argc, char **argv) {
+  setlocale(LC_ALL, "");
+
   InitLLVM X(argc, argv);
   BumpPtrAllocator A;
   StringSaver Saver(A);

>From 6fea204af11a1917d0cd202bfb91712d76ff930d Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Sun, 4 Oct 2026 23:25:53 +0100
Subject: [PATCH 10/41] Add missing #include

---
 llvm/tools/llvm-strings/llvm-strings.cpp | 1 +
 1 file changed, 1 insertion(+)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 13eaa0a359e31a..002c6f9b93a0a9 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -29,6 +29,7 @@
 #include "llvm/Support/Unicode.h"
 #include "llvm/Support/WithColor.h"
 #include <cctype>
+#include <clocale>
 #include <cwctype>
 #include <string>
 

>From cbb8e13345199dc50509a5eb085e0a229f47d236 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Tue, 6 Oct 2026 02:07:35 +0100
Subject: [PATCH 11/41] Optimise mbrtowc() for invalid characters, rename
 variables for clarity

---
 llvm/tools/llvm-strings/llvm-strings.cpp | 147 +++++++++++++----------
 1 file changed, 84 insertions(+), 63 deletions(-)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 002c6f9b93a0a9..3dd25be37fdf81 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -98,7 +98,7 @@ static void parseIntArg(const opt::InputArgList &Args, int ID, T &Value) {
 // and true is returned.
 // If P...E is empty, or holds an incomplete character and AtEOF is false, P
 // and MBState are unchanged, Ch is set to 0, and false is returned. This is
-// intended to allow more bytes to be read and Read to be called again.
+// intended to allow more bytes to be read and readChar to be called again.
 // If P...E holds an incomplete character and AtEOF is true, or if P...E
 // starts with an invalid character, the first bytes are skipped and MBState is
 // reset to allow continuing from the next point, Ch is set to 0, and true is
@@ -109,64 +109,85 @@ static void parseIntArg(const opt::InputArgList &Args, int ID, T &Value) {
 // arbitrary locales: it is the longest subsequence that could start a valid
 // multibyte character, or if the initial byte cannot start a valid multibyte
 // character, the initial byte. This allows consistent error recovery.
-static bool readChar(const char *&P, const char *E, std::mbstate_t &MBState,
+static bool readChar(const char *&Cur, const char *End, std::mbstate_t &MBState,
                      bool AtEOF, UTF32 &Ch) {
-  if (P == E)
+  if (Cur == End)
     return false;
 
   switch (Encoding) {
   case Encoding::Ascii:
-    Ch = *P++;
+    Ch = *Cur++;
     return true;
 
   case Encoding::Locale: {
-    std::mbstate_t SaveMBState = MBState;
+    const char *Next = Cur;
+    std::mbstate_t NextMBState = MBState;
     wchar_t WCh;
-    const size_t BytesRead = mbrtowc(&WCh, P, E - P, &MBState);
-    switch (BytesRead) {
-    default:
-      P += BytesRead;
-      Ch = WCh;
-      return true;
-    case 0:
-    case (size_t)-1: {
-      // We encountered a (possibly null) byte that cannot be part of a valid
-      // character from the previous state. Recover by retrying one byte at a
-      // time, processing as many bytes as possible. If this gives us a
-      // partial character, treat this partial character as a single invalid
-      // character, otherwise treat the first byte as a single invalid
-      // character.
-      const bool PartialChar = mbrtowc(&WCh, P, 1, &SaveMBState) == (size_t)-2;
-      do {
-        ++P;
-        assert(!PartialChar || P != E);
-      } while (PartialChar && mbrtowc(&WCh, P, 1, &SaveMBState) == (size_t)-2);
-      Ch = 0;
-      MBState = {};
-      return true;
-    }
-    case (size_t)-2:
-      MBState = SaveMBState;
-      return false;
+    for (;;) {
+      // Read one byte at a time. This is usually not the best way to use
+      // mbrtowc(), usually it would make more sense to pass the size of the
+      // buffer, but the strings utility is unusual in that it is expected to
+      // encounter many bytes that do not form valid characters and it is more
+      // useful to optimise for this case.
+      const size_t BytesRead = mbrtowc(&WCh, Next, 1, &NextMBState);
+      switch (BytesRead) {
+      case 1:
+        ++Next;
+        Cur = Next;
+        MBState = NextMBState;
+        Ch = WCh;
+        return true;
+
+      case (size_t)-2:
+        // We encountered a byte that is a valid start or continuation of a
+        // multibyte character. If we have more bytes, carry on. If we don't
+        // have more bytes yet, but we are not at the end of file, return false.
+        // If we don't have more bytes and we are at the end of file, fall
+        // through to treat it as an error.
+        ++Next;
+        if (Next != End)
+          continue;
+        if (AtEOF)
+          return false;
+        LLVM_FALLTHROUGH;
+
+      case 0:
+      case (size_t)-1:
+        // We cannot form a valid non-null character.
+        //
+        // If we processed any bytes already that formed an incomplete multibyte
+        // character, treat those bytes as a single null character, otherwise
+        // treat the current byte as a single null character.
+        if (Next == Cur)
+          ++Next;
+        Cur = Next;
+        MBState = {};
+        Ch = 0;
+        return true;
+
+      default:
+        llvm_unreachable("unexpected result from mbrtowc");
+      }
     }
   }
 
   case Encoding::Utf8: {
-    const UTF8 *UP = reinterpret_cast<const UTF8 *>(P);
-    const UTF8 *UE = reinterpret_cast<const UTF8 *>(E);
-    UTF32 *Next = &Ch;
-    const auto Res =
-        ConvertUTF8toUTF32(&UP, UE, &Next, &Ch + 1, strictConversion);
-    if (UP != reinterpret_cast<const UTF8 *>(P)) {
-      assert(Next != &Ch);
+    const UTF8 *UTF8Cur = reinterpret_cast<const UTF8 *>(Cur);
+    const UTF8 *UTF8Next = UTF8Cur;
+    const UTF8 *UTF8End = reinterpret_cast<const UTF8 *>(End);
+    UTF32 *UTF32Next = &Ch;
+    const auto Res = ConvertUTF8toUTF32(&UTF8Next, UTF8End, &UTF32Next, &Ch + 1,
+                                        strictConversion);
+    if (UTF8Next != UTF8Cur) {
+      assert(UTF32Next != &Ch);
     } else if (Res == sourceExhausted && !AtEOF) {
       return false;
     } else {
-      assert(Next == &Ch);
-      UP += findMaximalSubpartOfIllFormedUTF8Sequence(UP, UE);
-      *Next++ = 0;
+      assert(UTF32Next == &Ch);
+      UTF8Next += findMaximalSubpartOfIllFormedUTF8Sequence(UTF8Next, UTF8End);
+      Ch = 0;
     }
-    P = reinterpret_cast<const char *>(UP);
+    Cur = reinterpret_cast<const char *>(UTF8Next);
     return true;
   }
   }
@@ -233,9 +254,9 @@ static void strings(raw_ostream &OS, StringRef FileName,
   // never needs an arbitrarily large buffer. Candidate therefore only ever
   // holds a run that is shorter than MinLength and that was cut off by the end
   // of a chunk.
-  const size_t Min = MinLength;
+  const size_t MinLengthChars = MinLength;
   SmallString<DefaultMinLength> Candidate;
-  size_t CandidateLength = 0;
+  size_t CandidateLengthChars = 0;
   bool InString = false;
   std::mbstate_t MBState{};
   // Offset of the start of the current chunk within the file.
@@ -243,20 +264,20 @@ static void strings(raw_ostream &OS, StringRef FileName,
 
   while (true) {
     // Size of any partial character left over from the previous chunk.
-    size_t PartialCharSize = Buffer.size();
-    Buffer.resize_for_overwrite(PartialCharSize +
+    const size_t PendingCharSizeBytes = Buffer.size();
+    Buffer.resize_for_overwrite(PendingCharSizeBytes +
                                 sys::fs::DefaultReadChunkSize);
 
     Expected<size_t> ReadBytesOrErr = sys::fs::readNativeFile(
         Handle,
-        MutableArrayRef(Buffer.begin() + PartialCharSize, Buffer.end()));
+        MutableArrayRef(Buffer.begin() + PendingCharSizeBytes, Buffer.end()));
     if (!ReadBytesOrErr) {
       errs() << FileName << ": "
              << errorToErrorCode(ReadBytesOrErr.takeError()).message() << '\n';
       return;
     }
     const bool AtEOF = *ReadBytesOrErr == 0;
-    const size_t ChunkSize = PartialCharSize + *ReadBytesOrErr;
+    const size_t ChunkSize = PendingCharSizeBytes + *ReadBytesOrErr;
     if (ChunkSize == 0)
       break;
 
@@ -282,7 +303,7 @@ static void strings(raw_ostream &OS, StringRef FileName,
     if (InString || !Candidate.empty()) {
       const char *StringEnd;
       std::mbstate_t PrevMBState;
-      size_t Len = 0;
+      size_t LengthChars = 0;
       bool EndOfChunk;
 
       // Find the end of the current string.
@@ -292,37 +313,37 @@ static void strings(raw_ostream &OS, StringRef FileName,
         EndOfChunk = !readChar(Cur, End, MBState, AtEOF, Ch);
         if (EndOfChunk || !isStringChar(Ch))
           break;
-        ++Len;
+        ++LengthChars;
       }
 
-      size_t Size = StringEnd - Begin;
+      size_t SizeBytes = StringEnd - Begin;
       if (InString) {
         // Print the remaining part if the previous chunk has already printed
         // the header. E.g. header: aaaaa | bbbbb, where | is the chunk
         // boundary.
         // Output: Header: aaaaabbbbb, where bbbbb is printed in here.
-        OS << StringRef(Begin, Size);
-      } else if (CandidateLength + Len >= Min) {
+        OS << StringRef(Begin, SizeBytes);
+      } else if (CandidateLengthChars + LengthChars >= MinLengthChars) {
         // If the header hasn't been printed yet (i.e. the previous candidate
         // was smaller than Min), but we can print it now, print the header
         // first, followed by the candidate from the previous chunk and the
         // current string. E.g. aa | bbbbbb
         // Output Header: aabbbbbb, where aabbbbbb is printed in here.
         PrintHeader(ChunkOffset - Candidate.size());
-        OS << Candidate << StringRef(Begin, Size);
+        OS << Candidate << StringRef(Begin, SizeBytes);
         Candidate.clear();
-        CandidateLength = 0;
+        CandidateLengthChars = 0;
         InString = true;
       } else if (EndOfChunk) {
         // If the current chunk + previous candidate is still smaller than Min,
         // append it to Candidate.
         Candidate.append(Begin, StringEnd);
-        CandidateLength += Len;
+        CandidateLengthChars += LengthChars;
       } else {
         // If the string has terminated but is still smaller than Min, clear the
         // buffer since it is too short to print.
         Candidate.clear();
-        CandidateLength = 0;
+        CandidateLengthChars = 0;
       }
 
       if (EndOfChunk) {
@@ -343,7 +364,7 @@ static void strings(raw_ostream &OS, StringRef FileName,
     // At this point, we are always at the start of a new string because the
     // remaining part of the previous string has already been handled.
     const char *StrHead = nullptr;
-    size_t Len = 0;
+    size_t LengthChars = 0;
     for (;;) {
       const char *Prev = Cur;
       std::mbstate_t PrevMBState = MBState;
@@ -353,17 +374,17 @@ static void strings(raw_ostream &OS, StringRef FileName,
         // Find the start of the next string.
         if (!StrHead)
           StrHead = Prev;
-        ++Len;
+        ++LengthChars;
       } else if (StrHead) {
         // If it is not a printable character, we have reached the end of the
         // current string. Print it if long enough.
-        if (Len >= Min) {
+        if (LengthChars >= MinLengthChars) {
           PrintHeader(ChunkOffset + (StrHead - Begin));
           OS << StringRef(StrHead, Prev - StrHead);
           endString(OS, PrevMBState);
         }
         StrHead = nullptr;
-        Len = 0;
+        LengthChars = 0;
       }
     }
 
@@ -372,13 +393,13 @@ static void strings(raw_ostream &OS, StringRef FileName,
     // it again.
     if (StrHead) {
       // Print it, or append it to Candidate if it is too short.
-      if (Len >= Min) {
+      if (LengthChars >= MinLengthChars) {
         PrintHeader(ChunkOffset + (StrHead - Begin));
         OS << StringRef(StrHead, Cur - StrHead);
         InString = true;
       } else {
         Candidate.append(StrHead, Cur);
-        CandidateLength += Len;
+        CandidateLengthChars += LengthChars;
       }
     }
 

>From 03a6e3d753c1f54e777bbf75313b6847c945f9e8 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Wed, 7 Oct 2026 14:35:12 +0100
Subject: [PATCH 12/41] Move to a Struct class template to minimise runtime
 checks on Encoding

---
 llvm/tools/llvm-strings/llvm-strings.cpp | 274 +++++++++++++----------
 1 file changed, 155 insertions(+), 119 deletions(-)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 3dd25be37fdf81..6abb571ad825e5 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -70,7 +70,6 @@ static Encoding Encoding;
 
 enum class Radix { None, Octal, Hexadecimal, Decimal };
 static Radix Radix;
-} // namespace
 
 [[noreturn]] static void reportCmdLineError(const Twine &Message) {
   WithColor::error(errs(), ToolName) << Message << "\n";
@@ -91,141 +90,160 @@ static void parseIntArg(const opt::InputArgList &Args, int ID, T &Value) {
   }
 }
 
-// Tries to read one character from the bytes P...E and store it in Ch.
-// Returns true if a (possibly invalid) character was read, false otherwise.
-//
-// If P...E starts with a valid complete character, P and MBState are updated
-// and true is returned.
-// If P...E is empty, or holds an incomplete character and AtEOF is false, P
-// and MBState are unchanged, Ch is set to 0, and false is returned. This is
-// intended to allow more bytes to be read and readChar to be called again.
-// If P...E holds an incomplete character and AtEOF is true, or if P...E
-// starts with an invalid character, the first bytes are skipped and MBState is
-// reset to allow continuing from the next point, Ch is set to 0, and true is
-// returned.
-//
-// The number of skipped bytes for invalid characters follows the Unicode
-// definition of the maximal subpart of an ill-formed subsequence, applied to
-// arbitrary locales: it is the longest subsequence that could start a valid
-// multibyte character, or if the initial byte cannot start a valid multibyte
-// character, the initial byte. This allows consistent error recovery.
-static bool readChar(const char *&Cur, const char *End, std::mbstate_t &MBState,
-                     bool AtEOF, UTF32 &Ch) {
+template <enum Encoding> struct Strings {
+  Strings() = delete;
+
+  // Tries to read one character from the bytes P...E and store it in Ch.
+  // Returns true if a (possibly invalid) character was read, false otherwise.
+  //
+  // If P...E starts with a valid complete character, P and MBState are updated
+  // and true is returned.
+  // If P...E is empty, or holds an incomplete character and AtEOF is false, P
+  // and MBState are unchanged, Ch is set to 0, and false is returned. This is
+  // intended to allow more bytes to be read and readChar to be called again.
+  // If P...E holds an incomplete character and AtEOF is true, or if P...E
+  // starts with an invalid character, the first bytes are skipped and MBState
+  // is reset to allow continuing from the next point, Ch is set to 0, and true
+  // is returned.
+  //
+  // The number of skipped bytes for invalid characters follows the Unicode
+  // definition of the maximal subpart of an ill-formed subsequence, applied to
+  // arbitrary locales: it is the longest subsequence that could start a valid
+  // multibyte character, or if the initial byte cannot start a valid multibyte
+  // character, the initial byte. This allows consistent error recovery.
+  static bool readChar(const char *&Cur, const char *End,
+                       std::mbstate_t &MBState, bool AtEOF, UTF32 &Ch);
+
+  static bool isStringChar(UTF32 Ch);
+
+  static void endString(raw_ostream &OS, std::mbstate_t &MBState);
+
+  static void run(raw_ostream &OS, StringRef FileName, sys::fs::file_t Handle);
+};
+
+template <>
+bool Strings<Encoding::Ascii>::readChar(const char *&Cur, const char *End,
+                                        std::mbstate_t &MBState, bool AtEOF,
+                                        UTF32 &Ch) {
   if (Cur == End)
     return false;
 
-  switch (Encoding) {
-  case Encoding::Ascii:
-    Ch = *Cur++;
-    return true;
+  Ch = *Cur++;
+  return true;
+}
 
-  case Encoding::Locale: {
-    const char *Next = Cur;
-    std::mbstate_t NextMBState = MBState;
-    wchar_t WCh;
-    for (;;) {
-      // Read one byte at a time. This is usually not the best way to use
-      // mbrtowc(), usually it would make more sense to pass the size of the
-      // buffer, but the strings utility is unusual in that it is expected to
-      // encounter many bytes that do not form valid characters and it is more
-      // useful to optimise for this case.
-      const size_t BytesRead = mbrtowc(&WCh, Next, 1, &NextMBState);
-      switch (BytesRead) {
-      case 1:
-        ++Next;
-        Cur = Next;
-        MBState = NextMBState;
-        Ch = WCh;
-        return true;
-
-      case (size_t)-2:
-        // We encountered a byte that is a valid start or continuation of a
-        // multibyte character. If we have more bytes, carry on. If we don't
-        // have more bytes yet, but we are not at the end of file, return false.
-        // If we don't have more bytes and we are at the end of file, fall
-        // through to treat it as an error.
-        ++Next;
-        if (Next != End)
-          continue;
-        if (AtEOF)
-          return false;
-        LLVM_FALLTHROUGH;
-
-      case 0:
-      case (size_t)-1:
-        // We cannot form a valid non-null character.
-        //
-        // If we processed any bytes already that formed an incomplete multibyte
-        // character, treat those bytes as a single null character, otherwise
-        // treat the current byte as a single null character.
-        if (Next == Cur)
-          ++Next;
-        Cur = Next;
-        MBState = {};
-        Ch = 0;
-        return true;
-
-      default:
-        llvm_unreachable("unexpected result from mbrtowc");
-      }
-    }
-  }
+template <>
+bool Strings<Encoding::Locale>::readChar(const char *&Cur, const char *End,
+                                         std::mbstate_t &MBState, bool AtEOF,
+                                         UTF32 &Ch) {
+  if (Cur == End)
+    return false;
 
-  case Encoding::Utf8: {
-    const UTF8 *UTF8Cur = reinterpret_cast<const UTF8 *>(Cur);
-    const UTF8 *UTF8Next = UTF8Cur;
-    const UTF8 *UTF8End = reinterpret_cast<const UTF8 *>(End);
-    UTF32 *UTF32Next = &Ch;
-    const auto Res = ConvertUTF8toUTF32(&UTF8Next, UTF8End, &UTF32Next, &Ch + 1,
-                                        strictConversion);
-    if (UTF8Next != UTF8Cur) {
-      assert(UTF32Next != &Ch);
-    } else if (Res == sourceExhausted && !AtEOF) {
-      return false;
-    } else {
-      assert(UTF32Next == &Ch);
-      UTF8Next += findMaximalSubpartOfIllFormedUTF8Sequence(UTF8Next, UTF8End);
+  const char *Next = Cur;
+  std::mbstate_t NextMBState = MBState;
+  wchar_t WCh;
+  for (;;) {
+    // Read one byte at a time. This is usually not the best way to use
+    // mbrtowc(), usually it would make more sense to pass the size of the
+    // buffer, but the strings utility is unusual in that it is expected to
+    // encounter many bytes that do not form valid characters and it is more
+    // useful to optimise for this case.
+    const size_t BytesRead = mbrtowc(&WCh, Next, 1, &NextMBState);
+    switch (BytesRead) {
+    case 1:
+      ++Next;
+      Cur = Next;
+      MBState = NextMBState;
+      Ch = WCh;
+      return true;
+
+    case (size_t)-2:
+      // We encountered a byte that is a valid start or continuation of a
+      // multibyte character. If we have more bytes, carry on. If we don't
+      // have more bytes yet, but we are not at the end of file, return false.
+      // If we don't have more bytes and we are at the end of file, fall
+      // through to treat it as an error.
+      ++Next;
+      if (Next != End)
+        continue;
+      if (AtEOF)
+        return false;
+      LLVM_FALLTHROUGH;
+
+    case 0:
+    case (size_t)-1:
+      // We cannot form a valid non-null character.
+      //
+      // If we processed any bytes already that formed an incomplete multibyte
+      // character, treat those bytes as a single null character, otherwise
+      // treat the current byte as a single null character.
+      if (Next == Cur)
+        ++Next;
+      Cur = Next;
+      MBState = {};
       Ch = 0;
+      return true;
     }
-    Cur = reinterpret_cast<const char *>(UTF8Next);
-    return true;
-  }
   }
-
-  llvm_unreachable("unhandled encoding");
 }
 
-static bool isStringChar(UTF32 Ch) {
-  if (Ch == '\t')
-    return true;
+template <>
+bool Strings<Encoding::Utf8>::readChar(const char *&Cur, const char *End,
+                                       std::mbstate_t &MBState, bool AtEOF,
+                                       UTF32 &Ch) {
+  if (Cur == End)
+    return false;
 
-  switch (Encoding) {
-  case Encoding::Ascii:
-    return isPrint(Ch);
+  const UTF8 *UTF8Cur = reinterpret_cast<const UTF8 *>(Cur);
+  const UTF8 *UTF8Next = UTF8Cur;
+  const UTF8 *UTF8End = reinterpret_cast<const UTF8 *>(End);
+  UTF32 *UTF32Next = &Ch;
+  const auto Res = ConvertUTF8toUTF32(&UTF8Next, UTF8End, &UTF32Next, &Ch + 1,
+                                      strictConversion);
+  if (UTF8Next != UTF8Cur) {
+    assert(UTF32Next != &Ch);
+  } else if (Res == sourceExhausted && !AtEOF) {
+    return false;
+  } else {
+    assert(UTF32Next == &Ch);
+    UTF8Next += findMaximalSubpartOfIllFormedUTF8Sequence(UTF8Next, UTF8End);
+    Ch = 0;
+  }
+  Cur = reinterpret_cast<const char *>(UTF8Next);
+  return true;
+}
 
-  case Encoding::Locale:
-    return iswprint(Ch);
+template <> bool Strings<Encoding::Ascii>::isStringChar(UTF32 Ch) {
+  return Ch == '\t' || isPrint(Ch);
+}
 
-  case Encoding::Utf8:
-    return sys::unicode::isPrintable(Ch);
-  }
+template <> bool Strings<Encoding::Locale>::isStringChar(UTF32 Ch) {
+  return Ch == L'\t' || iswprint(Ch);
+}
 
-  llvm_unreachable("unhandled encoding");
+template <> bool Strings<Encoding::Utf8>::isStringChar(UTF32 Ch) {
+  return Ch == u'\t' || sys::unicode::isPrintable(Ch);
 }
 
-static void endString(raw_ostream &OS, std::mbstate_t &MBState) {
-  if (Encoding == Encoding::Locale) {
-    char Buf[MB_LEN_MAX];
-    // Note: This is only required for stateful encodings such as the
-    // ISO-2022 ones.
-    const size_t BytesWritten = wcrtomb(Buf, L'\0', &MBState);
-    OS << StringRef(Buf, BytesWritten - 1);
-  };
+template <enum Encoding Encoding>
+void Strings<Encoding>::endString(raw_ostream &OS, std::mbstate_t &MBState) {
   OS << '\n';
 }
 
-static void strings(raw_ostream &OS, StringRef FileName,
-                    sys::fs::file_t Handle) {
+template <>
+void Strings<Encoding::Locale>::endString(raw_ostream &OS,
+                                          std::mbstate_t &MBState) {
+  char Buf[MB_LEN_MAX];
+  // Note: This is only required for stateful encodings such as the
+  // ISO-2022 ones.
+  const size_t BytesWritten = wcrtomb(Buf, L'\0', &MBState);
+  Buf[BytesWritten - 1] = '\n';
+  OS << StringRef(Buf, BytesWritten);
+}
+
+template <enum Encoding Encoding>
+void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
+                            sys::fs::file_t Handle) {
   SmallString<sys::fs::DefaultReadChunkSize> Buffer;
   auto PrintHeader = [&OS, FileName](size_t StringStart) {
     if (PrintFileName)
@@ -413,6 +431,7 @@ static void strings(raw_ostream &OS, StringRef FileName,
   if (InString)
     endString(OS, MBState);
 }
+} // namespace
 
 int main(int argc, char **argv) {
   setlocale(LC_ALL, "");
@@ -474,13 +493,30 @@ int main(int argc, char **argv) {
     return EXIT_FAILURE;
   }
 
+  void (*const StringsImpl)(raw_ostream &OS, StringRef FileName,
+                            sys::fs::file_t Handle) = [] {
+    switch (Encoding) {
+    case Encoding::Ascii:
+      return Strings<Encoding::Ascii>::run;
+
+    case Encoding::Locale:
+      return Strings<Encoding::Locale>::run;
+
+    case Encoding::Utf8:
+      return Strings<Encoding::Utf8>::run;
+
+    default:
+      llvm_unreachable("unhandled encoding");
+    }
+  }();
+
   std::vector<std::string> InputFileNames = Args.getAllArgValues(OPT_INPUT);
   if (InputFileNames.empty())
     InputFileNames.push_back("-");
 
   for (const auto &File : InputFileNames) {
     if (File == "-") {
-      strings(llvm::outs(), "{standard input}", sys::fs::getStdinHandle());
+      StringsImpl(llvm::outs(), "{standard input}", sys::fs::getStdinHandle());
     } else {
       Expected<sys::fs::file_t> FDOrErr =
           sys::fs::openNativeFileForRead(File, sys::fs::OF_TextWithCRLF);
@@ -490,7 +526,7 @@ int main(int argc, char **argv) {
                << '\n';
         continue;
       }
-      strings(llvm::outs(), File, *FDOrErr);
+      StringsImpl(llvm::outs(), File, *FDOrErr);
     }
   }
 

>From 6be72df4d23b83d7c55bcd3ff731a97c42cc37e6 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Wed, 7 Oct 2026 18:17:58 +0100
Subject: [PATCH 13/41] Avoid compiler warning

---
 llvm/tools/llvm-strings/llvm-strings.cpp | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 6abb571ad825e5..94ea5cb9986c9b 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -504,10 +504,9 @@ int main(int argc, char **argv) {
 
     case Encoding::Utf8:
       return Strings<Encoding::Utf8>::run;
-
-    default:
-      llvm_unreachable("unhandled encoding");
     }
+
+    llvm_unreachable("unhandled encoding");
   }();
 
   std::vector<std::string> InputFileNames = Args.getAllArgValues(OPT_INPUT);

>From 2c19f00af54bb5f7257bb022e32d55d62c6a07a6 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Fri, 9 Oct 2026 15:00:22 +0100
Subject: [PATCH 14/41] Use std:: where appropriate, expand on comments

---
 llvm/test/tools/llvm-strings/encoding.test | 16 ++++---
 llvm/tools/llvm-strings/llvm-strings.cpp   | 55 ++++++++++++----------
 2 files changed, 38 insertions(+), 33 deletions(-)

diff --git a/llvm/test/tools/llvm-strings/encoding.test b/llvm/test/tools/llvm-strings/encoding.test
index 00d1d7f7d93f5f..4d15387814f562 100644
--- a/llvm/test/tools/llvm-strings/encoding.test
+++ b/llvm/test/tools/llvm-strings/encoding.test
@@ -62,13 +62,15 @@ RUN: not llvm-strings -e x 2>&1 %t | FileCheck --check-prefix CHECK-ERROR %s
 
 CHECK-ERROR: error: 'x' is not a valid value for '--encoding='
 
-# Encodings with multibyte characters may have characters cross chunk
-# boundaries. The string contains a full multibyte character before the
-# boundary, a multibyte character that crosses the boundary, and a full
-# multibyte character after the boundary. This test has four variants:
-# the minimum length may or may not already be reached before the chunk
-# boundary, and the character that crosses the boundary may or may not be
-# printable.                             v chunk boundary
+# llvm-strings reads the input in chunks of 16384 bytes. Encodings with
+# multibyte characters may have characters cross chunk boundaries. The string
+# contains a full multibyte character before the boundary, a multibyte
+# character that crosses the boundary, and a full multibyte character after the
+# boundary. This test has four variants: the minimum length may or may not
+# already be reached before the chunk boundary, and the character that crosses
+# the boundary may or may not be printable.
+#
+#                                        v chunk boundary
 RUN: printf "%16378s\342\200\247e\342\200\247e\342\200\247e\342\200\247" | \
 RUN: tr " " "\0" | llvm-strings -e utf8 | \
 RUN: FileCheck %s --check-prefix=CHECK-UTF8-CHUNK-A --match-full-lines --strict-whitespace --implicit-check-not={{.}}
diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 94ea5cb9986c9b..7f27012b4d8323 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -93,18 +93,21 @@ static void parseIntArg(const opt::InputArgList &Args, int ID, T &Value) {
 template <enum Encoding> struct Strings {
   Strings() = delete;
 
-  // Tries to read one character from the bytes P...E and store it in Ch.
+  // Tries to read one character from the bytes Cur...End and store it in Ch.
   // Returns true if a (possibly invalid) character was read, false otherwise.
   //
-  // If P...E starts with a valid complete character, P and MBState are updated
-  // and true is returned.
-  // If P...E is empty, or holds an incomplete character and AtEOF is false, P
-  // and MBState are unchanged, Ch is set to 0, and false is returned. This is
-  // intended to allow more bytes to be read and readChar to be called again.
-  // If P...E holds an incomplete character and AtEOF is true, or if P...E
-  // starts with an invalid character, the first bytes are skipped and MBState
-  // is reset to allow continuing from the next point, Ch is set to 0, and true
-  // is returned.
+  // If Cur...End starts with a valid complete character, Cur and MBState are
+  // updated and true is returned.
+  // If Cur...End is empty, or holds an incomplete character and AtEOF is false,
+  // Cur and MBState are unchanged, Ch is set to 0, and false is returned. This
+  // is intended to allow more bytes to be read and readChar to be called again.
+  // If Cur...End holds an incomplete character and AtEOF is true, or if
+  // Cur...End starts with an invalid character, the first bytes are skipped and
+  // MBState is reset to allow continuing from the next point, Ch is set to 0,
+  // and true is returned. This case is indistinguishable to the caller from a
+  // null byte being successfully converted to a null character, but a null
+  // character is never printable under any settings and that is how invalid
+  // bytes are treated as well.
   //
   // The number of skipped bytes for invalid characters follows the Unicode
   // definition of the maximal subpart of an ill-formed subsequence, applied to
@@ -148,7 +151,7 @@ bool Strings<Encoding::Locale>::readChar(const char *&Cur, const char *End,
     // buffer, but the strings utility is unusual in that it is expected to
     // encounter many bytes that do not form valid characters and it is more
     // useful to optimise for this case.
-    const size_t BytesRead = mbrtowc(&WCh, Next, 1, &NextMBState);
+    const std::size_t BytesRead = std::mbrtowc(&WCh, Next, 1, &NextMBState);
     switch (BytesRead) {
     case 1:
       ++Next;
@@ -157,7 +160,7 @@ bool Strings<Encoding::Locale>::readChar(const char *&Cur, const char *End,
       Ch = WCh;
       return true;
 
-    case (size_t)-2:
+    case (std::size_t)-2:
       // We encountered a byte that is a valid start or continuation of a
       // multibyte character. If we have more bytes, carry on. If we don't
       // have more bytes yet, but we are not at the end of file, return false.
@@ -171,7 +174,7 @@ bool Strings<Encoding::Locale>::readChar(const char *&Cur, const char *End,
       LLVM_FALLTHROUGH;
 
     case 0:
-    case (size_t)-1:
+    case (std::size_t)-1:
       // We cannot form a valid non-null character.
       //
       // If we processed any bytes already that formed an incomplete multibyte
@@ -218,7 +221,7 @@ template <> bool Strings<Encoding::Ascii>::isStringChar(UTF32 Ch) {
 }
 
 template <> bool Strings<Encoding::Locale>::isStringChar(UTF32 Ch) {
-  return Ch == L'\t' || iswprint(Ch);
+  return Ch == L'\t' || std::iswprint(Ch);
 }
 
 template <> bool Strings<Encoding::Utf8>::isStringChar(UTF32 Ch) {
@@ -236,7 +239,7 @@ void Strings<Encoding::Locale>::endString(raw_ostream &OS,
   char Buf[MB_LEN_MAX];
   // Note: This is only required for stateful encodings such as the
   // ISO-2022 ones.
-  const size_t BytesWritten = wcrtomb(Buf, L'\0', &MBState);
+  const std::size_t BytesWritten = std::wcrtomb(Buf, L'\0', &MBState);
   Buf[BytesWritten - 1] = '\n';
   OS << StringRef(Buf, BytesWritten);
 }
@@ -245,7 +248,7 @@ template <enum Encoding Encoding>
 void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
                             sys::fs::file_t Handle) {
   SmallString<sys::fs::DefaultReadChunkSize> Buffer;
-  auto PrintHeader = [&OS, FileName](size_t StringStart) {
+  auto PrintHeader = [&OS, FileName](std::size_t StringStart) {
     if (PrintFileName)
       OS << FileName << ": ";
     switch (Radix) {
@@ -272,21 +275,21 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
   // never needs an arbitrarily large buffer. Candidate therefore only ever
   // holds a run that is shorter than MinLength and that was cut off by the end
   // of a chunk.
-  const size_t MinLengthChars = MinLength;
+  const std::size_t MinLengthChars = MinLength;
   SmallString<DefaultMinLength> Candidate;
-  size_t CandidateLengthChars = 0;
+  std::size_t CandidateLengthChars = 0;
   bool InString = false;
   std::mbstate_t MBState{};
   // Offset of the start of the current chunk within the file.
-  size_t ChunkOffset = 0;
+  std::size_t ChunkOffset = 0;
 
   while (true) {
     // Size of any partial character left over from the previous chunk.
-    const size_t PendingCharSizeBytes = Buffer.size();
+    const std::size_t PendingCharSizeBytes = Buffer.size();
     Buffer.resize_for_overwrite(PendingCharSizeBytes +
                                 sys::fs::DefaultReadChunkSize);
 
-    Expected<size_t> ReadBytesOrErr = sys::fs::readNativeFile(
+    Expected<std::size_t> ReadBytesOrErr = sys::fs::readNativeFile(
         Handle,
         MutableArrayRef(Buffer.begin() + PendingCharSizeBytes, Buffer.end()));
     if (!ReadBytesOrErr) {
@@ -295,7 +298,7 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
       return;
     }
     const bool AtEOF = *ReadBytesOrErr == 0;
-    const size_t ChunkSize = PendingCharSizeBytes + *ReadBytesOrErr;
+    const std::size_t ChunkSize = PendingCharSizeBytes + *ReadBytesOrErr;
     if (ChunkSize == 0)
       break;
 
@@ -321,7 +324,7 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
     if (InString || !Candidate.empty()) {
       const char *StringEnd;
       std::mbstate_t PrevMBState;
-      size_t LengthChars = 0;
+      std::size_t LengthChars = 0;
       bool EndOfChunk;
 
       // Find the end of the current string.
@@ -334,7 +337,7 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
         ++LengthChars;
       }
 
-      size_t SizeBytes = StringEnd - Begin;
+      std::size_t SizeBytes = StringEnd - Begin;
       if (InString) {
         // Print the remaining part if the previous chunk has already printed
         // the header. E.g. header: aaaaa | bbbbb, where | is the chunk
@@ -382,7 +385,7 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
     // At this point, we are always at the start of a new string because the
     // remaining part of the previous string has already been handled.
     const char *StrHead = nullptr;
-    size_t LengthChars = 0;
+    std::size_t LengthChars = 0;
     for (;;) {
       const char *Prev = Cur;
       std::mbstate_t PrevMBState = MBState;
@@ -434,7 +437,7 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
 } // namespace
 
 int main(int argc, char **argv) {
-  setlocale(LC_ALL, "");
+  std::setlocale(LC_ALL, "");
 
   InitLLVM X(argc, argv);
   BumpPtrAllocator A;

>From d427e670738304efca81c8eeaf8d1ad88811a8de Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Fri, 16 Oct 2026 19:47:52 +0100
Subject: [PATCH 15/41] Use consistent definition of printable across platforms

---
 llvm/docs/CommandGuide/llvm-strings.md   |  5 +++--
 llvm/tools/llvm-strings/llvm-strings.cpp | 19 ++++++++++---------
 2 files changed, 13 insertions(+), 11 deletions(-)

diff --git a/llvm/docs/CommandGuide/llvm-strings.md b/llvm/docs/CommandGuide/llvm-strings.md
index 5a45dcd426c058..06be9b0cf6e13f 100644
--- a/llvm/docs/CommandGuide/llvm-strings.md
+++ b/llvm/docs/CommandGuide/llvm-strings.md
@@ -12,8 +12,9 @@
 {program}`llvm-strings` is a tool intended as a drop-in replacement for GNU's
 {program}`strings`, which looks for printable strings in files and writes them
 to the standard output stream. A printable string is any sequence of four (by
-default) or more printable characters. The end of the file, or any unprintable
-byte sequence, terminates the current sequence.
+default) or more printable characters. A printable character is tab, or any
+character that is not a control character. The end of the file, or any
+unprintable byte sequence, terminates the current sequence.
 
 {program}`llvm-strings` looks for strings in each `input` file specified.
 Unlike GNU {program}`strings` it looks in the entire input file, regardless of
diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 7f27012b4d8323..cb32fbef1aff88 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -28,9 +28,8 @@
 #include "llvm/Support/SwapByteOrder.h"
 #include "llvm/Support/Unicode.h"
 #include "llvm/Support/WithColor.h"
-#include <cctype>
 #include <clocale>
-#include <cwctype>
+#include <cuchar>
 #include <string>
 
 using namespace llvm;
@@ -144,20 +143,20 @@ bool Strings<Encoding::Locale>::readChar(const char *&Cur, const char *End,
 
   const char *Next = Cur;
   std::mbstate_t NextMBState = MBState;
-  wchar_t WCh;
+  char32_t UCh;
   for (;;) {
     // Read one byte at a time. This is usually not the best way to use
-    // mbrtowc(), usually it would make more sense to pass the size of the
+    // mbrtoc32(), usually it would make more sense to pass the size of the
     // buffer, but the strings utility is unusual in that it is expected to
     // encounter many bytes that do not form valid characters and it is more
     // useful to optimise for this case.
-    const std::size_t BytesRead = std::mbrtowc(&WCh, Next, 1, &NextMBState);
+    const std::size_t BytesRead = std::mbrtoc32(&UCh, Next, 1, &NextMBState);
     switch (BytesRead) {
     case 1:
       ++Next;
       Cur = Next;
       MBState = NextMBState;
-      Ch = WCh;
+      Ch = UCh;
       return true;
 
     case (std::size_t)-2:
@@ -221,11 +220,13 @@ template <> bool Strings<Encoding::Ascii>::isStringChar(UTF32 Ch) {
 }
 
 template <> bool Strings<Encoding::Locale>::isStringChar(UTF32 Ch) {
-  return Ch == L'\t' || std::iswprint(Ch);
+  return llvm::sys::unicode::isCharacter(Ch) &&
+         (Ch == U'\t' || !llvm::sys::unicode::isControl(Ch));
 }
 
 template <> bool Strings<Encoding::Utf8>::isStringChar(UTF32 Ch) {
-  return Ch == u'\t' || sys::unicode::isPrintable(Ch);
+  return llvm::sys::unicode::isCharacter(Ch) &&
+         (Ch == U'\t' || !llvm::sys::unicode::isControl(Ch));
 }
 
 template <enum Encoding Encoding>
@@ -239,7 +240,7 @@ void Strings<Encoding::Locale>::endString(raw_ostream &OS,
   char Buf[MB_LEN_MAX];
   // Note: This is only required for stateful encodings such as the
   // ISO-2022 ones.
-  const std::size_t BytesWritten = std::wcrtomb(Buf, L'\0', &MBState);
+  const std::size_t BytesWritten = std::c32rtomb(Buf, U'\0', &MBState);
   Buf[BytesWritten - 1] = '\n';
   OS << StringRef(Buf, BytesWritten);
 }

>From 82bc20fcba94c61ad30106a8583970fa58f5c178 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Sat, 17 Oct 2026 00:09:07 +0100
Subject: [PATCH 16/41] Update test

---
 llvm/test/tools/llvm-strings/encoding.test | 40 +++++++++++-----------
 1 file changed, 20 insertions(+), 20 deletions(-)

diff --git a/llvm/test/tools/llvm-strings/encoding.test b/llvm/test/tools/llvm-strings/encoding.test
index 4d15387814f562..20b7d6d8a30133 100644
--- a/llvm/test/tools/llvm-strings/encoding.test
+++ b/llvm/test/tools/llvm-strings/encoding.test
@@ -3,10 +3,10 @@ RUN: echo a > %t
 RUN: echo ab >> %t
 RUN: echo abc >> %t
 RUN: echo abcd >> %t
-# UTF-8 encoding of U+2027 HYPHENATION POINT (printable)
-RUN: printf 'bcde\342\200\247\n' >> %t
-# UTF-8 encoding of U+200B ZERO WIDTH SPACE (non-printable)
-RUN: printf 'cdef\342\200\247\342\200\250\n' >> %t
+# UTF-8 encoding of U+00A0 NON-BREAKING SPACE (which we treat as printable)
+RUN: printf 'bcde\302\240\n' >> %t
+# UTF-8 encoding of U+009F APPLICATION PROGRAM COMMAND (non-printable)
+RUN: printf 'cdef\302\240\302\237\n' >> %t
 
 # Check that partial and invalid UTF-8 characters do not interfere with
 # preceding and subsequent strings.
@@ -39,20 +39,20 @@ CHECK-ASCII:cdef
 CHECK-ASCII:defg
 CHECK-ASCII:efgh
 
-# In -e S, any bytes of U+200A and U+200B may or may not be included depending
+# In -e S, any bytes of U+00A0 and U+009F may or may not be included depending
 # on the locale in effect. The locale in effect may also be altered by llvm-lit.
 # Make sure if changing the test that the byte representations should not allow
 # this to be split into multiple strings in non-UTF-8 locales.
-CHECK-LOCALE:bcde{{(\xe2(\x80\xa7?)?)?}}
-CHECK-LOCALE:cdef{{(\xe2(\x80(\xa7(\xe2(\x80\xa8?)?)?)?)?)?}}
+CHECK-LOCALE:bcde{{(\xc2\xa0?)?}}
+CHECK-LOCALE:cdef{{(\xc2(\xa0(\xc2\x9f?)?)?)?}}
 CHECK-LOCALE:{{\xe2?}}defg{{\xe2?}}
 CHECK-LOCALE:{{(\xe2?\x80)?}}efgh{{(\xe2\x80?)?}}
 
-# In -e utf8, all bytes of U+200A should be included, but none of U+200B. The
+# In -e utf8, all bytes of U+00A0 should be included, but none of U+009F. The
 # partial multibyte characters should not prevent abcd from being detected as a
 # string.
-CHECK-UTF8:bcde{{\xe2\x80\xa7}}
-CHECK-UTF8:cdef{{\xe2\x80\xa7}}
+CHECK-UTF8:bcde{{\xc2\xa0}}
+CHECK-UTF8:cdef{{\xc2\xa0}}
 CHECK-UTF8:defg
 CHECK-UTF8:efgh
 
@@ -70,22 +70,22 @@ CHECK-ERROR: error: 'x' is not a valid value for '--encoding='
 # already be reached before the chunk boundary, and the character that crosses
 # the boundary may or may not be printable.
 #
-#                                        v chunk boundary
-RUN: printf "%16378s\342\200\247e\342\200\247e\342\200\247e\342\200\247" | \
+#                                v chunk boundary
+RUN: printf "%16380s\302\240e\302\240e\302\240e\302\240" | \
 RUN: tr " " "\0" | llvm-strings -e utf8 | \
 RUN: FileCheck %s --check-prefix=CHECK-UTF8-CHUNK-A --match-full-lines --strict-whitespace --implicit-check-not={{.}}
-#                           v chunk boundary
-RUN: printf "%16374s\342\200\247e\342\200\247e\342\200\247e\342\200\247" | \
+#                       v chunk boundary
+RUN: printf "%16383s\302\240e\302\240e\302\240e\302\240" | \
 RUN: tr " " "\0" | llvm-strings -e utf8 | \
 RUN: FileCheck %s --check-prefix=CHECK-UTF8-CHUNK-A --match-full-lines --strict-whitespace --implicit-check-not={{.}}
-CHECK-UTF8-CHUNK-A:{{\xe2\x80\xa7e\xe2\x80\xa7e\xe2\x80\xa7e\xe2\x80\xa7}}
+CHECK-UTF8-CHUNK-A:{{\xc2\xa0e\xc2\xa0e\xc2\xa0e\xc2\xa0}}
 
-#                                         v chunk boundary
-RUN: printf "%16377se\342\200\247e\342\200\250e\342\200\247e\342\200\247e" | \
+#                                 v chunk boundary
+RUN: printf "%16379se\302\240e\302\237e\302\240e\302\240e" | \
 RUN: tr " " "\0" | llvm-strings -e utf8 | \
 RUN: FileCheck %s --check-prefix=CHECK-UTF8-CHUNK-B --match-full-lines --strict-whitespace --implicit-check-not={{.}}
-#                            v chunk boundary
-RUN: printf "%16373se\342\200\250e\342\200\247e\342\200\247e\342" | \
+#                        v chunk boundary
+RUN: printf "%16382se\302\237e\302\240e\302\240e\342" | \
 RUN: tr " " "\0" | llvm-strings -e utf8 | \
 RUN: FileCheck %s --check-prefix=CHECK-UTF8-CHUNK-B --match-full-lines --strict-whitespace --implicit-check-not={{.}}
-CHECK-UTF8-CHUNK-B:{{e\xe2\x80\xa7e\xe2\x80\xa7e}}
+CHECK-UTF8-CHUNK-B:{{e\xc2\xa0e\xc2\xa0e}}

>From b46f1d1206cfbc1f5e6e7effb64b9504c38456d4 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Tue, 20 Oct 2026 23:01:52 +0100
Subject: [PATCH 17/41] Fix format specifiers

---
 llvm/tools/llvm-strings/llvm-strings.cpp | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index cb32fbef1aff88..4695ab01e9c4ba 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -256,13 +256,13 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
     case Radix::None:
       break;
     case Radix::Octal:
-      OS << format("%7o ", StringStart);
+      OS << format("%7zo ", StringStart);
       break;
     case Radix::Hexadecimal:
-      OS << format("%7x ", StringStart);
+      OS << format("%7zx ", StringStart);
       break;
     case Radix::Decimal:
-      OS << format("%7u ", StringStart);
+      OS << format("%7zu ", StringStart);
       break;
     }
   };

>From 4327bd5d21f6435291afe727b2b560c5fbf543ff Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Wed, 21 Oct 2026 00:51:06 +0100
Subject: [PATCH 18/41] Address review feedback.

* Document default --encoding=S in docs.
* Handle the fact that mbrtoc32() is non-conforming on all the major platforms.
  * On Windows, it ignores locale and always converts from UTF-8.
  * On macOS, it is not present.
  * On glibc, it returns the (size_t)-3 case as 0.
  On Windows and macOS, wchar_t is used instead, except that on Windows,
  if the active code page is UTF-8, the built-in UTF-8 decoder is used
  instead to handle characters outside the BMP.
* Do not try to read again after EOF.
---
 llvm/docs/CommandGuide/llvm-strings.md   |  1 +
 llvm/tools/llvm-strings/llvm-strings.cpp | 44 +++++++++++++++++++++++-
 2 files changed, 44 insertions(+), 1 deletion(-)

diff --git a/llvm/docs/CommandGuide/llvm-strings.md b/llvm/docs/CommandGuide/llvm-strings.md
index 06be9b0cf6e13f..1de4326d0fd00c 100644
--- a/llvm/docs/CommandGuide/llvm-strings.md
+++ b/llvm/docs/CommandGuide/llvm-strings.md
@@ -54,6 +54,7 @@ Specifies the encoding of the input file. Valid arguments are:
 - `s` (ASCII)
 - `S` (current locale's character set)
 - `utf8` (UTF-8)
+The default is `--encoding=S`.
 :::
 
 :::{option} --help, -h
diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 4695ab01e9c4ba..067e37057cc700 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -32,6 +32,10 @@
 #include <cuchar>
 #include <string>
 
+#ifdef _WIN32
+#include "llvm/Support/Windows/WindowsSupport.h"
+#endif
+
 using namespace llvm;
 using namespace llvm::object;
 
@@ -143,17 +147,42 @@ bool Strings<Encoding::Locale>::readChar(const char *&Cur, const char *End,
 
   const char *Next = Cur;
   std::mbstate_t NextMBState = MBState;
+#if defined(_WIN32) || defined(__APPLE__)
+  wchar_t UCh;
+#else
   char32_t UCh;
+#endif
   for (;;) {
     // Read one byte at a time. This is usually not the best way to use
     // mbrtoc32(), usually it would make more sense to pass the size of the
     // buffer, but the strings utility is unusual in that it is expected to
     // encounter many bytes that do not form valid characters and it is more
     // useful to optimise for this case.
-    const std::size_t BytesRead = std::mbrtoc32(&UCh, Next, 1, &NextMBState);
+#if defined(_WIN32) || defined(__APPLE__)
+    // Workarounds:
+    // On Windows, mbrtoc32() ignores the locale and only converts UTF-8, but
+    // wchar_t is UTF-16 and is close enough, as long as we make sure to avoid
+    // it for UTF-8 locales.
+    // On Apple, <uchar.h> is not available, but wchar_t is UTF-32 and is a full
+    // substitute.
+    std::size_t BytesRead = std::mbrtowc(&UCh, Next, 1, &NextMBState);
+#else
+    std::size_t BytesRead = std::mbrtoc32(&UCh, Next, 1, &NextMBState);
+#endif
+    // Workaround:
+    // BytesRead == 0 should mean we found a null byte and should always have
+    // UCh == 0, but on glibc, it also gets returned for the case that should
+    // be indicated by (std::size_t)-3, as indicating that 0 bytes were
+    // converted. It can be detected by UCh != 0.
+    if (BytesRead == 0 && UCh) {
+      BytesRead = (std::size_t)-3;
+    }
     switch (BytesRead) {
     case 1:
       ++Next;
+      LLVM_FALLTHROUGH;
+
+    case (std::size_t)-3:
       Cur = Next;
       MBState = NextMBState;
       Ch = UCh;
@@ -240,7 +269,13 @@ void Strings<Encoding::Locale>::endString(raw_ostream &OS,
   char Buf[MB_LEN_MAX];
   // Note: This is only required for stateful encodings such as the
   // ISO-2022 ones.
+#if defined(_WIN32) || defined(__APPLE__)
+  const std::size_t BytesWritten = std::wcrtomb(Buf, L'\0', &MBState);
+#else
   const std::size_t BytesWritten = std::c32rtomb(Buf, U'\0', &MBState);
+#endif
+  assert(BytesWritten < (std::size_t)-1 &&
+         "A null character must always be valid");
   Buf[BytesWritten - 1] = '\n';
   OS << StringRef(Buf, BytesWritten);
 }
@@ -369,6 +404,9 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
       }
 
       if (EndOfChunk) {
+        if (AtEOF)
+          break;
+
         // Finish handling the current chunk and update ChunkOffset.
         ChunkOffset += Cur - Begin;
         Buffer.erase(Buffer.begin(), Cur);
@@ -477,6 +515,10 @@ int main(int argc, char **argv) {
       invalidArgValue(EncodingArg);
     Encoding = *EncodingVal;
   }
+#ifdef _WIN32
+  if (Encoding == Encoding::Locale && GetACP() == CP_UTF8)
+    Encoding = Encoding::Utf8;
+#endif
   parseIntArg(Args, OPT_bytes_EQ, MinLength);
   PrintFileName = Args.hasArg(OPT_print_file_name);
   Arg *RadixArg = Args.getLastArg(OPT_radix_EQ);

>From b577caa8345adfa6f4277f3190bc0b5f8231e274 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Wed, 21 Oct 2026 13:25:36 +0100
Subject: [PATCH 19/41] Use wrapper functions for mbrto(wc/c32) and
 (wc/c32)rtomb

---
 llvm/tools/llvm-strings/llvm-strings.cpp | 67 ++++++++++++++----------
 1 file changed, 38 insertions(+), 29 deletions(-)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 067e37057cc700..2f6a21015da7fd 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -93,6 +93,41 @@ static void parseIntArg(const opt::InputArgList &Args, int ID, T &Value) {
   }
 }
 
+// Ideally we would use mbrtoc32() and c32rtomb() unconditionally but there are
+// problems on most platforms, thus we introduce wrappers. On Windows,
+// mbrtoc32() ignores the locale and only converts UTF-8, but wchar_t is UTF-16
+// and is close enough, as long as we make sure to avoid it for UTF-8 locales.
+// On Apple, <uchar.h> is not available, but wchar_t is UTF-32 and is a full
+// substitute.
+// On glibc, mbrtoc32() is a wrapper for mbrtowc() and never returns (size_t)-3,
+// instead it handles this case by returning 0 but setting Ch to a non-null
+// character.
+#if defined(_WIN32) || defined(__APPLE__)
+using char_t = wchar_t;
+static std::size_t mbrtoc(char_t *Ch, const char *S, size_t N,
+                          std::mbstate_t *MBState) {
+  const std::size_t Result = std::mbrtowc(Ch, S, N, MBState);
+  if (Result == 0 && *Ch)
+    Result = (std::size_t)-3;
+  return Result;
+}
+static std::size_t crtomb(char *S, char_t Ch, mbstate_t *MBState) {
+  return std::wcrtomb(S, Ch, MBState);
+}
+#else
+using char_t = char32_t;
+static std::size_t mbrtoc(char_t *Ch, const char *S, size_t N,
+                          std::mbstate_t *MBState) {
+  std::size_t Result = std::mbrtoc32(Ch, S, N, MBState);
+  if (Result == 0 && *Ch)
+    Result = (std::size_t)-3;
+  return Result;
+}
+static std::size_t crtomb(char *S, char_t Ch, mbstate_t *MBState) {
+  return std::c32rtomb(S, Ch, MBState);
+}
+#endif
+
 template <enum Encoding> struct Strings {
   Strings() = delete;
 
@@ -147,36 +182,14 @@ bool Strings<Encoding::Locale>::readChar(const char *&Cur, const char *End,
 
   const char *Next = Cur;
   std::mbstate_t NextMBState = MBState;
-#if defined(_WIN32) || defined(__APPLE__)
-  wchar_t UCh;
-#else
-  char32_t UCh;
-#endif
+  char_t UCh;
   for (;;) {
     // Read one byte at a time. This is usually not the best way to use
     // mbrtoc32(), usually it would make more sense to pass the size of the
     // buffer, but the strings utility is unusual in that it is expected to
     // encounter many bytes that do not form valid characters and it is more
     // useful to optimise for this case.
-#if defined(_WIN32) || defined(__APPLE__)
-    // Workarounds:
-    // On Windows, mbrtoc32() ignores the locale and only converts UTF-8, but
-    // wchar_t is UTF-16 and is close enough, as long as we make sure to avoid
-    // it for UTF-8 locales.
-    // On Apple, <uchar.h> is not available, but wchar_t is UTF-32 and is a full
-    // substitute.
-    std::size_t BytesRead = std::mbrtowc(&UCh, Next, 1, &NextMBState);
-#else
-    std::size_t BytesRead = std::mbrtoc32(&UCh, Next, 1, &NextMBState);
-#endif
-    // Workaround:
-    // BytesRead == 0 should mean we found a null byte and should always have
-    // UCh == 0, but on glibc, it also gets returned for the case that should
-    // be indicated by (std::size_t)-3, as indicating that 0 bytes were
-    // converted. It can be detected by UCh != 0.
-    if (BytesRead == 0 && UCh) {
-      BytesRead = (std::size_t)-3;
-    }
+    const std::size_t BytesRead = mbrtoc(&UCh, Next, 1, &NextMBState);
     switch (BytesRead) {
     case 1:
       ++Next;
@@ -269,11 +282,7 @@ void Strings<Encoding::Locale>::endString(raw_ostream &OS,
   char Buf[MB_LEN_MAX];
   // Note: This is only required for stateful encodings such as the
   // ISO-2022 ones.
-#if defined(_WIN32) || defined(__APPLE__)
-  const std::size_t BytesWritten = std::wcrtomb(Buf, L'\0', &MBState);
-#else
-  const std::size_t BytesWritten = std::c32rtomb(Buf, U'\0', &MBState);
-#endif
+  const std::size_t BytesWritten = crtomb(Buf, 0, &MBState);
   assert(BytesWritten < (std::size_t)-1 &&
          "A null character must always be valid");
   Buf[BytesWritten - 1] = '\n';

>From b428a50393d1d533f34592888be5a34b052a6d8c Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Wed, 21 Oct 2026 22:12:40 +0100
Subject: [PATCH 20/41] Remove leftover const, use size_t without std::

---
 llvm/tools/llvm-strings/llvm-strings.cpp | 54 ++++++++++++------------
 1 file changed, 27 insertions(+), 27 deletions(-)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 2f6a21015da7fd..0b697f44e2e400 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -30,6 +30,7 @@
 #include "llvm/Support/WithColor.h"
 #include <clocale>
 #include <cuchar>
+#include <stddef.h>
 #include <string>
 
 #ifdef _WIN32
@@ -104,26 +105,26 @@ static void parseIntArg(const opt::InputArgList &Args, int ID, T &Value) {
 // character.
 #if defined(_WIN32) || defined(__APPLE__)
 using char_t = wchar_t;
-static std::size_t mbrtoc(char_t *Ch, const char *S, size_t N,
-                          std::mbstate_t *MBState) {
-  const std::size_t Result = std::mbrtowc(Ch, S, N, MBState);
+static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
+                     std::mbstate_t *MBState) {
+  size_t Result = std::mbrtowc(Ch, S, N, MBState);
   if (Result == 0 && *Ch)
-    Result = (std::size_t)-3;
+    Result = (size_t)-3;
   return Result;
 }
-static std::size_t crtomb(char *S, char_t Ch, mbstate_t *MBState) {
+static size_t crtomb(char *S, char_t Ch, mbstate_t *MBState) {
   return std::wcrtomb(S, Ch, MBState);
 }
 #else
 using char_t = char32_t;
-static std::size_t mbrtoc(char_t *Ch, const char *S, size_t N,
-                          std::mbstate_t *MBState) {
-  std::size_t Result = std::mbrtoc32(Ch, S, N, MBState);
+static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
+                     std::mbstate_t *MBState) {
+  size_t Result = std::mbrtoc32(Ch, S, N, MBState);
   if (Result == 0 && *Ch)
-    Result = (std::size_t)-3;
+    Result = (size_t)-3;
   return Result;
 }
-static std::size_t crtomb(char *S, char_t Ch, mbstate_t *MBState) {
+static size_t crtomb(char *S, char_t Ch, mbstate_t *MBState) {
   return std::c32rtomb(S, Ch, MBState);
 }
 #endif
@@ -189,19 +190,19 @@ bool Strings<Encoding::Locale>::readChar(const char *&Cur, const char *End,
     // buffer, but the strings utility is unusual in that it is expected to
     // encounter many bytes that do not form valid characters and it is more
     // useful to optimise for this case.
-    const std::size_t BytesRead = mbrtoc(&UCh, Next, 1, &NextMBState);
+    const size_t BytesRead = mbrtoc(&UCh, Next, 1, &NextMBState);
     switch (BytesRead) {
     case 1:
       ++Next;
       LLVM_FALLTHROUGH;
 
-    case (std::size_t)-3:
+    case (size_t)-3:
       Cur = Next;
       MBState = NextMBState;
       Ch = UCh;
       return true;
 
-    case (std::size_t)-2:
+    case (size_t)-2:
       // We encountered a byte that is a valid start or continuation of a
       // multibyte character. If we have more bytes, carry on. If we don't
       // have more bytes yet, but we are not at the end of file, return false.
@@ -215,7 +216,7 @@ bool Strings<Encoding::Locale>::readChar(const char *&Cur, const char *End,
       LLVM_FALLTHROUGH;
 
     case 0:
-    case (std::size_t)-1:
+    case (size_t)-1:
       // We cannot form a valid non-null character.
       //
       // If we processed any bytes already that formed an incomplete multibyte
@@ -282,9 +283,8 @@ void Strings<Encoding::Locale>::endString(raw_ostream &OS,
   char Buf[MB_LEN_MAX];
   // Note: This is only required for stateful encodings such as the
   // ISO-2022 ones.
-  const std::size_t BytesWritten = crtomb(Buf, 0, &MBState);
-  assert(BytesWritten < (std::size_t)-1 &&
-         "A null character must always be valid");
+  const size_t BytesWritten = crtomb(Buf, 0, &MBState);
+  assert(BytesWritten < (size_t)-1 && "A null character must always be valid");
   Buf[BytesWritten - 1] = '\n';
   OS << StringRef(Buf, BytesWritten);
 }
@@ -293,7 +293,7 @@ template <enum Encoding Encoding>
 void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
                             sys::fs::file_t Handle) {
   SmallString<sys::fs::DefaultReadChunkSize> Buffer;
-  auto PrintHeader = [&OS, FileName](std::size_t StringStart) {
+  auto PrintHeader = [&OS, FileName](size_t StringStart) {
     if (PrintFileName)
       OS << FileName << ": ";
     switch (Radix) {
@@ -320,21 +320,21 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
   // never needs an arbitrarily large buffer. Candidate therefore only ever
   // holds a run that is shorter than MinLength and that was cut off by the end
   // of a chunk.
-  const std::size_t MinLengthChars = MinLength;
+  const size_t MinLengthChars = MinLength;
   SmallString<DefaultMinLength> Candidate;
-  std::size_t CandidateLengthChars = 0;
+  size_t CandidateLengthChars = 0;
   bool InString = false;
   std::mbstate_t MBState{};
   // Offset of the start of the current chunk within the file.
-  std::size_t ChunkOffset = 0;
+  size_t ChunkOffset = 0;
 
   while (true) {
     // Size of any partial character left over from the previous chunk.
-    const std::size_t PendingCharSizeBytes = Buffer.size();
+    const size_t PendingCharSizeBytes = Buffer.size();
     Buffer.resize_for_overwrite(PendingCharSizeBytes +
                                 sys::fs::DefaultReadChunkSize);
 
-    Expected<std::size_t> ReadBytesOrErr = sys::fs::readNativeFile(
+    Expected<size_t> ReadBytesOrErr = sys::fs::readNativeFile(
         Handle,
         MutableArrayRef(Buffer.begin() + PendingCharSizeBytes, Buffer.end()));
     if (!ReadBytesOrErr) {
@@ -343,7 +343,7 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
       return;
     }
     const bool AtEOF = *ReadBytesOrErr == 0;
-    const std::size_t ChunkSize = PendingCharSizeBytes + *ReadBytesOrErr;
+    const size_t ChunkSize = PendingCharSizeBytes + *ReadBytesOrErr;
     if (ChunkSize == 0)
       break;
 
@@ -369,7 +369,7 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
     if (InString || !Candidate.empty()) {
       const char *StringEnd;
       std::mbstate_t PrevMBState;
-      std::size_t LengthChars = 0;
+      size_t LengthChars = 0;
       bool EndOfChunk;
 
       // Find the end of the current string.
@@ -382,7 +382,7 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
         ++LengthChars;
       }
 
-      std::size_t SizeBytes = StringEnd - Begin;
+      size_t SizeBytes = StringEnd - Begin;
       if (InString) {
         // Print the remaining part if the previous chunk has already printed
         // the header. E.g. header: aaaaa | bbbbb, where | is the chunk
@@ -433,7 +433,7 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
     // At this point, we are always at the start of a new string because the
     // remaining part of the previous string has already been handled.
     const char *StrHead = nullptr;
-    std::size_t LengthChars = 0;
+    size_t LengthChars = 0;
     for (;;) {
       const char *Prev = Cur;
       std::mbstate_t PrevMBState = MBState;

>From 1b9ef79f804738e27e269e1622538a6fd108b0ad Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Thu, 22 Oct 2026 14:50:03 +0100
Subject: [PATCH 21/41] Add release note, remove include of <stddef.h> already
 indirectly included

---
 llvm/docs/ReleaseNotes.md                | 9 +++++++++
 llvm/tools/llvm-strings/llvm-strings.cpp | 1 -
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index 302aa57396105e..825f346621f03d 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -428,6 +428,15 @@ Makes programs 10x faster by doing Special New Thing.
 * llvm-rc now supports `/showIncludes` to report header and resource-file
   dependencies in a format compatible with Ninja's `deps = msvc` mode.
 
+* llvm-strings now supports the `-e` (`--encoding`) option and by default prints
+  non-ASCII strings it finds, using the system's configured character set. The
+  prior behavior can be restored by specifying `-es` (`--encoding=s`), which is
+  mostly useful on large files when the searched-for strings are known ahead of
+  time to be ASCII, as it searches faster and gives the user fewer results to
+  sift through. The `-eutf8` is also available to search for UTF-8 strings,
+  which is mostly useful on Windows where configuring the system to use UTF-8 is
+  more involved than setting an environment variable.
+
 ### Changes to LLDB
 
 * `platform.plugin.wasm.runtime-args` now precede the port argument on the Wasm
diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 0b697f44e2e400..6d06b2f49959a3 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -30,7 +30,6 @@
 #include "llvm/Support/WithColor.h"
 #include <clocale>
 #include <cuchar>
-#include <stddef.h>
 #include <string>
 
 #ifdef _WIN32

>From b465e8a3e15f29b37d4da1120ade6a411dfd468b Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Thu, 22 Oct 2026 15:09:39 +0100
Subject: [PATCH 22/41] Avoid isPrint

Strings<Encoding::Ascii>::isStringChar would implicitly convert its
argument to char in calling isPrint. It would not produce incorrect
results for how it is used, but it would produce incorrect results for
how a subsequent change would potentially use it, so just fix that now.
---
 llvm/tools/llvm-strings/llvm-strings.cpp | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 6d06b2f49959a3..c174335508c48b 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -258,7 +258,7 @@ bool Strings<Encoding::Utf8>::readChar(const char *&Cur, const char *End,
 }
 
 template <> bool Strings<Encoding::Ascii>::isStringChar(UTF32 Ch) {
-  return Ch == '\t' || isPrint(Ch);
+  return Ch == U'\t' || (Ch >= U' ' && Ch <= U'~');
 }
 
 template <> bool Strings<Encoding::Locale>::isStringChar(UTF32 Ch) {

>From 4fb09c36a5c9d3753105ea0264bdfd02fa36f271 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Fri, 23 Oct 2026 23:30:29 +0100
Subject: [PATCH 23/41] Optimise llvm-strings further

* For single-byte character sets (MB_CUR_MAX == 1), build a
  256-element lookup table rather than converting each character in the
  input to UTF-32.
* For multi-byte character sets, including UTF-8, build an
  0x110000-element lookup table rather than calling isCharacter and
  isControl for each converted character in the input.
* For UTF-8 locales, also switch to the internal decoder on Unix where
  we know it is safe.
---
 llvm/tools/llvm-strings/llvm-strings.cpp | 393 ++++++++++++++---------
 1 file changed, 233 insertions(+), 160 deletions(-)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index c174335508c48b..43025b5adf99c5 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -27,11 +27,17 @@
 #include "llvm/Support/Program.h"
 #include "llvm/Support/SwapByteOrder.h"
 #include "llvm/Support/Unicode.h"
+#include "llvm/Support/UnicodeCharRanges.h"
 #include "llvm/Support/WithColor.h"
+#include <bitset>
 #include <clocale>
 #include <cuchar>
 #include <string>
 
+#ifdef LLVM_ON_UNIX
+#include <langinfo.h>
+#endif
+
 #ifdef _WIN32
 #include "llvm/Support/Windows/WindowsSupport.h"
 #endif
@@ -39,6 +45,11 @@
 using namespace llvm;
 using namespace llvm::object;
 
+namespace llvm::sys::unicode {
+extern const UnicodeCharSet::CharRanges GeneratedCharacterRanges;
+extern const UnicodeCharSet::CharRanges GeneratedControlCharacterRanges;
+} // namespace llvm::sys::unicode
+
 namespace {
 enum ID {
   OPT_INVALID = 0, // This is not an option ID.
@@ -128,169 +139,216 @@ static size_t crtomb(char *S, char_t Ch, mbstate_t *MBState) {
 }
 #endif
 
-template <enum Encoding> struct Strings {
-  Strings() = delete;
+struct CharSet {
+  template <typename T> struct Base {
+    Base() = delete;
+
+    using CharT = T;
+
+    // Tries to read one character from the bytes Cur...End and store it in Ch.
+    // Returns true if a (possibly invalid) character was read, false otherwise.
+    //
+    // If Cur...End starts with a valid complete character, Cur and MBState are
+    // updated and true is returned.
+    // If Cur...End is empty, or holds an incomplete character and AtEOF is
+    // false, Cur and MBState are unchanged, Ch is set to 0, and false is
+    // returned. This is intended to allow more bytes to be read and readChar to
+    // be called again. If Cur...End holds an incomplete character and AtEOF is
+    // true, or if Cur...End starts with an invalid character, the first bytes
+    // are skipped and MBState is reset to allow continuing from the next point,
+    // Ch is set to 0, and true is returned. This case is indistinguishable to
+    // the caller from a null byte being successfully converted to a null
+    // character, but a null character is never printable under any settings and
+    // that is how invalid bytes are treated as well.
+    //
+    // The number of skipped bytes for invalid characters follows the Unicode
+    // definition of the maximal subpart of an ill-formed subsequence, applied
+    // to arbitrary locales: it is the longest subsequence that could start a
+    // valid multibyte character, or if the initial byte cannot start a valid
+    // multibyte character, the initial byte. This allows consistent error
+    // recovery.
+    static bool readChar(const char *&Cur, const char *End,
+                         std::mbstate_t &MBState, bool AtEOF, CharT &Ch);
+
+    static bool isStringChar(CharT Ch);
+
+    static void endString(raw_ostream &OS, std::mbstate_t &MBState) {
+      OS << '\n';
+    }
+  };
 
-  // Tries to read one character from the bytes Cur...End and store it in Ch.
-  // Returns true if a (possibly invalid) character was read, false otherwise.
-  //
-  // If Cur...End starts with a valid complete character, Cur and MBState are
-  // updated and true is returned.
-  // If Cur...End is empty, or holds an incomplete character and AtEOF is false,
-  // Cur and MBState are unchanged, Ch is set to 0, and false is returned. This
-  // is intended to allow more bytes to be read and readChar to be called again.
-  // If Cur...End holds an incomplete character and AtEOF is true, or if
-  // Cur...End starts with an invalid character, the first bytes are skipped and
-  // MBState is reset to allow continuing from the next point, Ch is set to 0,
-  // and true is returned. This case is indistinguishable to the caller from a
-  // null byte being successfully converted to a null character, but a null
-  // character is never printable under any settings and that is how invalid
-  // bytes are treated as well.
-  //
-  // The number of skipped bytes for invalid characters follows the Unicode
-  // definition of the maximal subpart of an ill-formed subsequence, applied to
-  // arbitrary locales: it is the longest subsequence that could start a valid
-  // multibyte character, or if the initial byte cannot start a valid multibyte
-  // character, the initial byte. This allows consistent error recovery.
-  static bool readChar(const char *&Cur, const char *End,
-                       std::mbstate_t &MBState, bool AtEOF, UTF32 &Ch);
+  struct MultiByte : Base<UTF32> {
+    static bool readChar(const char *&Cur, const char *End,
+                         std::mbstate_t &MBState, bool AtEOF, CharT &Ch) {
+      if (Cur == End)
+        return false;
 
-  static bool isStringChar(UTF32 Ch);
+      const char *Next = Cur;
+      std::mbstate_t NextMBState = MBState;
+      char_t UCh;
+      for (;;) {
+        // Read one byte at a time. This is usually not the best way to use
+        // mbrtoc32(), usually it would make more sense to pass the size of the
+        // buffer, but the strings utility is unusual in that it is expected to
+        // encounter many bytes that do not form valid characters and it is more
+        // useful to optimise for this case.
+        const size_t BytesRead = mbrtoc(&UCh, Next, 1, &NextMBState);
+        switch (BytesRead) {
+        case 1:
+          ++Next;
+          LLVM_FALLTHROUGH;
+
+        case (size_t)-3:
+          Cur = Next;
+          MBState = NextMBState;
+          Ch = std::min<UTF32>(UCh, 0x10FFFF);
+          return true;
+
+        case (size_t)-2:
+          // We encountered a byte that is a valid start or continuation of a
+          // multibyte character. If we have more bytes, carry on. If we don't
+          // have more bytes yet, but we are not at the end of file, return
+          // false. If we don't have more bytes and we are at the end of file,
+          // fall through to treat it as an error.
+          ++Next;
+          if (Next != End)
+            continue;
+          if (!AtEOF)
+            return false;
+          LLVM_FALLTHROUGH;
+
+        case 0:
+        case (size_t)-1:
+          // We cannot form a valid non-null character.
+          //
+          // If we processed any bytes already that formed an incomplete
+          // multibyte character, treat those bytes as a single null character,
+          // otherwise treat the current byte as a single null character.
+          if (Next == Cur)
+            ++Next;
+          Cur = Next;
+          MBState = {};
+          Ch = 0;
+          return true;
+        }
+      }
+    }
 
-  static void endString(raw_ostream &OS, std::mbstate_t &MBState);
+    static bool isStringChar(CharT Ch) {
+      using StringCharSet = std::bitset<0x110000>;
+      static const StringCharSet &StringChars = []() -> const StringCharSet & {
+        static StringCharSet StringChars;
 
-  static void run(raw_ostream &OS, StringRef FileName, sys::fs::file_t Handle);
-};
+        using llvm::sys::unicode::GeneratedCharacterRanges;
+        using llvm::sys::unicode::GeneratedControlCharacterRanges;
+        using llvm::sys::UnicodeCharRange;
 
-template <>
-bool Strings<Encoding::Ascii>::readChar(const char *&Cur, const char *End,
-                                        std::mbstate_t &MBState, bool AtEOF,
-                                        UTF32 &Ch) {
-  if (Cur == End)
-    return false;
+        for (const UnicodeCharRange &Range : GeneratedCharacterRanges)
+          for (UTF32 Ch = Range.Lower; Ch <= Range.Upper; ++Ch)
+            StringChars[Ch] = true;
+        for (const UnicodeCharRange &Range : GeneratedControlCharacterRanges)
+          for (UTF32 Ch = Range.Lower; Ch <= Range.Upper; ++Ch)
+            StringChars[Ch] = false;
+        StringChars[U'\t'] = true;
 
-  Ch = *Cur++;
-  return true;
-}
+        return StringChars;
+      }();
 
-template <>
-bool Strings<Encoding::Locale>::readChar(const char *&Cur, const char *End,
-                                         std::mbstate_t &MBState, bool AtEOF,
-                                         UTF32 &Ch) {
-  if (Cur == End)
-    return false;
-
-  const char *Next = Cur;
-  std::mbstate_t NextMBState = MBState;
-  char_t UCh;
-  for (;;) {
-    // Read one byte at a time. This is usually not the best way to use
-    // mbrtoc32(), usually it would make more sense to pass the size of the
-    // buffer, but the strings utility is unusual in that it is expected to
-    // encounter many bytes that do not form valid characters and it is more
-    // useful to optimise for this case.
-    const size_t BytesRead = mbrtoc(&UCh, Next, 1, &NextMBState);
-    switch (BytesRead) {
-    case 1:
-      ++Next;
-      LLVM_FALLTHROUGH;
-
-    case (size_t)-3:
-      Cur = Next;
-      MBState = NextMBState;
-      Ch = UCh;
-      return true;
+      return StringChars[Ch];
+    }
 
-    case (size_t)-2:
-      // We encountered a byte that is a valid start or continuation of a
-      // multibyte character. If we have more bytes, carry on. If we don't
-      // have more bytes yet, but we are not at the end of file, return false.
-      // If we don't have more bytes and we are at the end of file, fall
-      // through to treat it as an error.
-      ++Next;
-      if (Next != End)
-        continue;
-      if (AtEOF)
+    static void endString(raw_ostream &OS, std::mbstate_t &MBState) {
+      char Buf[MB_LEN_MAX];
+      // Note: This is only required for stateful encodings such as the
+      // ISO-2022 ones.
+      const size_t BytesWritten = crtomb(Buf, 0, &MBState);
+      assert(BytesWritten < (size_t)-1 &&
+             "A null character must always be valid");
+      Buf[BytesWritten - 1] = '\n';
+      OS << StringRef(Buf, BytesWritten);
+    }
+  };
+
+  struct Utf8 : MultiByte {
+    static bool readChar(const char *&Cur, const char *End,
+                         std::mbstate_t &MBState, bool AtEOF, CharT &Ch) {
+      if (Cur == End)
         return false;
-      LLVM_FALLTHROUGH;
-
-    case 0:
-    case (size_t)-1:
-      // We cannot form a valid non-null character.
-      //
-      // If we processed any bytes already that formed an incomplete multibyte
-      // character, treat those bytes as a single null character, otherwise
-      // treat the current byte as a single null character.
-      if (Next == Cur)
-        ++Next;
-      Cur = Next;
-      MBState = {};
-      Ch = 0;
+
+      const UTF8 *UTF8Cur = reinterpret_cast<const UTF8 *>(Cur);
+      const UTF8 *UTF8Next = UTF8Cur;
+      const UTF8 *UTF8End = reinterpret_cast<const UTF8 *>(End);
+      UTF32 *UTF32Next = &Ch;
+      const auto Res = ConvertUTF8toUTF32(&UTF8Next, UTF8End, &UTF32Next,
+                                          &Ch + 1, strictConversion);
+      if (UTF8Next != UTF8Cur) {
+        assert(UTF32Next != &Ch);
+        assert(Ch <= 0x10FFFF &&
+               "ConvertUTF8toUTF32 produced an out-of-range character");
+      } else if (Res == sourceExhausted && !AtEOF) {
+        return false;
+      } else {
+        assert(UTF32Next == &Ch);
+        UTF8Next +=
+            findMaximalSubpartOfIllFormedUTF8Sequence(UTF8Next, UTF8End);
+        Ch = 0;
+      }
+      Cur = reinterpret_cast<const char *>(UTF8Next);
       return true;
     }
-  }
-}
 
-template <>
-bool Strings<Encoding::Utf8>::readChar(const char *&Cur, const char *End,
-                                       std::mbstate_t &MBState, bool AtEOF,
-                                       UTF32 &Ch) {
-  if (Cur == End)
-    return false;
-
-  const UTF8 *UTF8Cur = reinterpret_cast<const UTF8 *>(Cur);
-  const UTF8 *UTF8Next = UTF8Cur;
-  const UTF8 *UTF8End = reinterpret_cast<const UTF8 *>(End);
-  UTF32 *UTF32Next = &Ch;
-  const auto Res = ConvertUTF8toUTF32(&UTF8Next, UTF8End, &UTF32Next, &Ch + 1,
-                                      strictConversion);
-  if (UTF8Next != UTF8Cur) {
-    assert(UTF32Next != &Ch);
-  } else if (Res == sourceExhausted && !AtEOF) {
-    return false;
-  } else {
-    assert(UTF32Next == &Ch);
-    UTF8Next += findMaximalSubpartOfIllFormedUTF8Sequence(UTF8Next, UTF8End);
-    Ch = 0;
-  }
-  Cur = reinterpret_cast<const char *>(UTF8Next);
-  return true;
-}
+    static void endString(raw_ostream &OS, std::mbstate_t &MBState) {
+      Base::endString(OS, MBState);
+    }
+  };
 
-template <> bool Strings<Encoding::Ascii>::isStringChar(UTF32 Ch) {
-  return Ch == U'\t' || (Ch >= U' ' && Ch <= U'~');
-}
+  struct SingleByte : Base<unsigned char> {
+    static bool readChar(const char *&Cur, const char *End,
+                         std::mbstate_t &MBState, bool AtEOF, CharT &Ch) {
+      if (Cur == End)
+        return false;
 
-template <> bool Strings<Encoding::Locale>::isStringChar(UTF32 Ch) {
-  return llvm::sys::unicode::isCharacter(Ch) &&
-         (Ch == U'\t' || !llvm::sys::unicode::isControl(Ch));
-}
+      Ch = *reinterpret_cast<const unsigned char *>(Cur++);
+      return true;
+    }
 
-template <> bool Strings<Encoding::Utf8>::isStringChar(UTF32 Ch) {
-  return llvm::sys::unicode::isCharacter(Ch) &&
-         (Ch == U'\t' || !llvm::sys::unicode::isControl(Ch));
-}
+    static bool isStringChar(CharT Ch) {
+      using StringCharSet = std::bitset<0x100>;
+      static const StringCharSet &StringChars = []() -> const StringCharSet & {
+        static StringCharSet StringChars;
+
+        using llvm::sys::unicode::isCharacter;
+        using llvm::sys::unicode::isControl;
+
+        for (int I = 0; I < 0x100; ++I) {
+          const unsigned char Ch = I;
+          const char *Cur = reinterpret_cast<const char *>(&Ch), *End = Cur + 1;
+          std::mbstate_t MBState = {};
+          UTF32 UCh;
+          if (MultiByte::readChar(Cur, End, MBState, /*AtEOF=*/true, UCh)) {
+            if (UCh == U'\t' || (isCharacter(UCh) && !isControl(UCh)))
+              StringChars[I] = true;
+          }
+        }
 
-template <enum Encoding Encoding>
-void Strings<Encoding>::endString(raw_ostream &OS, std::mbstate_t &MBState) {
-  OS << '\n';
-}
+        return StringChars;
+      }();
 
-template <>
-void Strings<Encoding::Locale>::endString(raw_ostream &OS,
-                                          std::mbstate_t &MBState) {
-  char Buf[MB_LEN_MAX];
-  // Note: This is only required for stateful encodings such as the
-  // ISO-2022 ones.
-  const size_t BytesWritten = crtomb(Buf, 0, &MBState);
-  assert(BytesWritten < (size_t)-1 && "A null character must always be valid");
-  Buf[BytesWritten - 1] = '\n';
-  OS << StringRef(Buf, BytesWritten);
-}
+      return StringChars[Ch];
+    }
+  };
+
+  struct Ascii : SingleByte {
+    static bool isStringChar(CharT Ch) {
+      return Ch == '\t' || (Ch >= ' ' && Ch <= '~');
+    }
+  };
+};
+
+template <typename CharSet>
+void strings(raw_ostream &OS, StringRef FileName, sys::fs::file_t Handle) {
+  using CharT = typename CharSet::CharT;
 
-template <enum Encoding Encoding>
-void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
-                            sys::fs::file_t Handle) {
   SmallString<sys::fs::DefaultReadChunkSize> Buffer;
   auto PrintHeader = [&OS, FileName](size_t StringStart) {
     if (PrintFileName)
@@ -355,7 +413,7 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
     const char *End = Begin + ChunkSize;
     const char *Cur = Begin;
 
-    UTF32 Ch;
+    CharT Ch;
 
     // Handle the remaining part from the previous chunk.
     // The previous chunk can be either shorter than MinSize or part of the
@@ -375,8 +433,8 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
       for (;;) {
         StringEnd = Cur;
         PrevMBState = MBState;
-        EndOfChunk = !readChar(Cur, End, MBState, AtEOF, Ch);
-        if (EndOfChunk || !isStringChar(Ch))
+        EndOfChunk = !CharSet::readChar(Cur, End, MBState, AtEOF, Ch);
+        if (EndOfChunk || !CharSet::isStringChar(Ch))
           break;
         ++LengthChars;
       }
@@ -424,7 +482,7 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
       if (InString) {
         // We haven't reached the end of the chunk, which means the string is
         // terminated.
-        endString(OS, PrevMBState);
+        CharSet::endString(OS, PrevMBState);
         InString = false;
       }
     }
@@ -436,9 +494,9 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
     for (;;) {
       const char *Prev = Cur;
       std::mbstate_t PrevMBState = MBState;
-      if (!readChar(Cur, End, MBState, AtEOF, Ch))
+      if (!CharSet::readChar(Cur, End, MBState, AtEOF, Ch))
         break;
-      if (isStringChar(Ch)) {
+      if (CharSet::isStringChar(Ch)) {
         // Find the start of the next string.
         if (!StrHead)
           StrHead = Prev;
@@ -449,7 +507,7 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
         if (LengthChars >= MinLengthChars) {
           PrintHeader(ChunkOffset + (StrHead - Begin));
           OS << StringRef(StrHead, Prev - StrHead);
-          endString(OS, PrevMBState);
+          CharSet::endString(OS, PrevMBState);
         }
         StrHead = nullptr;
         LengthChars = 0;
@@ -479,7 +537,7 @@ void Strings<Encoding>::run(raw_ostream &OS, StringRef FileName,
   }
 
   if (InString)
-    endString(OS, MBState);
+    CharSet::endString(OS, MBState);
 }
 } // namespace
 
@@ -523,10 +581,6 @@ int main(int argc, char **argv) {
       invalidArgValue(EncodingArg);
     Encoding = *EncodingVal;
   }
-#ifdef _WIN32
-  if (Encoding == Encoding::Locale && GetACP() == CP_UTF8)
-    Encoding = Encoding::Utf8;
-#endif
   parseIntArg(Args, OPT_bytes_EQ, MinLength);
   PrintFileName = Args.hasArg(OPT_print_file_name);
   Arg *RadixArg = Args.getLastArg(OPT_radix_EQ);
@@ -551,13 +605,32 @@ int main(int argc, char **argv) {
                             sys::fs::file_t Handle) = [] {
     switch (Encoding) {
     case Encoding::Ascii:
-      return Strings<Encoding::Ascii>::run;
+      return strings<CharSet::Ascii>;
 
     case Encoding::Locale:
-      return Strings<Encoding::Locale>::run;
+      if (MB_CUR_MAX == 1)
+        return strings<CharSet::SingleByte>;
+
+#ifdef LLVM_ON_UNIX
+      // Using LLVM's built-in decoder is faster than going through libc, but
+      // this doesn't affect correctness (assuming 32-bit wchar_t), so it's fine
+      // if this check has false negatives (e.g. if some OS returns "utf8"
+      // instead).
+      if (StringRef(nl_langinfo(CODESET)) == "UTF-8")
+        return strings<CharSet::Utf8>;
+#endif
+#ifdef _WIN32
+      // Using LLVM's built-in decoder is necessary for correctness as
+      // Strings<CharSet::MultiByte> on WIndows is implemented on top of 16-bit
+      // wchar_t which cannot represent all code points.
+      if (GetACP() == CP_UTF8)
+        return strings<CharSet::Utf8>;
+#endif
+
+      return strings<CharSet::MultiByte>;
 
     case Encoding::Utf8:
-      return Strings<Encoding::Utf8>::run;
+      return strings<CharSet::Utf8>;
     }
 
     llvm_unreachable("unhandled encoding");

>From fe46d2ee4d34ddba09d861f96fd4fca76a989c0c Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Mon, 26 Oct 2026 20:46:02 +0000
Subject: [PATCH 24/41] Fix up stateful encoding handling, avoid the complexity
 where not needed

---
 llvm/tools/llvm-strings/llvm-strings.cpp | 135 +++++++++++++----------
 1 file changed, 79 insertions(+), 56 deletions(-)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 43025b5adf99c5..8c26e5c06fb624 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -141,8 +141,6 @@ static size_t crtomb(char *S, char_t Ch, mbstate_t *MBState) {
 
 struct CharSet {
   template <typename T> struct Base {
-    Base() = delete;
-
     using CharT = T;
 
     // Tries to read one character from the bytes Cur...End and store it in Ch.
@@ -167,24 +165,27 @@ struct CharSet {
     // valid multibyte character, or if the initial byte cannot start a valid
     // multibyte character, the initial byte. This allows consistent error
     // recovery.
-    static bool readChar(const char *&Cur, const char *End,
-                         std::mbstate_t &MBState, bool AtEOF, CharT &Ch);
+    static bool readChar(const char *&Cur, const char *End, bool AtEOF,
+                         CharT &Ch);
 
     static bool isStringChar(CharT Ch);
 
-    static void endString(raw_ostream &OS, std::mbstate_t &MBState) {
-      OS << '\n';
-    }
+    static void startString() {}
+
+    static void writeString(raw_ostream &OS, StringRef Str) { OS << Str; }
+
+    static void endString(raw_ostream &OS) { OS << '\n'; }
   };
 
-  struct MultiByte : Base<UTF32> {
-    static bool readChar(const char *&Cur, const char *End,
-                         std::mbstate_t &MBState, bool AtEOF, CharT &Ch) {
+  struct MultiByteStateless : Base<UTF32> {
+    std::mbstate_t InMBState{};
+
+    bool readChar(const char *&Cur, const char *End, bool AtEOF, CharT &Ch) {
       if (Cur == End)
         return false;
 
       const char *Next = Cur;
-      std::mbstate_t NextMBState = MBState;
+      std::mbstate_t NextMBState = InMBState;
       char_t UCh;
       for (;;) {
         // Read one byte at a time. This is usually not the best way to use
@@ -200,7 +201,7 @@ struct CharSet {
 
         case (size_t)-3:
           Cur = Next;
-          MBState = NextMBState;
+          InMBState = NextMBState;
           Ch = std::min<UTF32>(UCh, 0x10FFFF);
           return true;
 
@@ -227,7 +228,7 @@ struct CharSet {
           if (Next == Cur)
             ++Next;
           Cur = Next;
-          MBState = {};
+          InMBState = {};
           Ch = 0;
           return true;
         }
@@ -256,22 +257,49 @@ struct CharSet {
 
       return StringChars[Ch];
     }
+  };
+
+  // Note: The complexity in writing is only required for stateful encodings
+  // such as the ISO-2022 ones.
+  struct MultiByteStateful : MultiByteStateless {
+    std::mbstate_t StrMBState{}, OutMBState{};
+
+    void startString() { StrMBState = InMBState; }
+
+    void writeString(raw_ostream &OS, StringRef Str) {
+      const char *Cur = Str.begin(), *End = Str.end();
+      while (Cur != End) {
+        char_t UCh;
+        const size_t BytesRead = mbrtoc(&UCh, Cur, End - Cur, &StrMBState);
+        assert(BytesRead != 0 && BytesRead != (size_t)-1 &&
+               BytesRead != (size_t)-2 &&
+               "If a string was initially valid, it cannot to become invalid "
+               "when re-reading");
+        if (BytesRead != (size_t)-3)
+          Cur += BytesRead;
+
+        char Buf[MB_LEN_MAX];
+        const size_t BytesWritten = crtomb(Buf, UCh, &OutMBState);
+        assert(BytesWritten != (size_t)-1 &&
+               "If we successfully read a character, we must also be able to "
+               "write it");
+        OS << StringRef(Buf, BytesWritten);
+      }
+    }
 
-    static void endString(raw_ostream &OS, std::mbstate_t &MBState) {
+    void endString(raw_ostream &OS) {
       char Buf[MB_LEN_MAX];
-      // Note: This is only required for stateful encodings such as the
-      // ISO-2022 ones.
-      const size_t BytesWritten = crtomb(Buf, 0, &MBState);
-      assert(BytesWritten < (size_t)-1 &&
+      const size_t BytesWritten = crtomb(Buf, 0, &OutMBState);
+      assert(BytesWritten != (size_t)-1 &&
              "A null character must always be valid");
       Buf[BytesWritten - 1] = '\n';
       OS << StringRef(Buf, BytesWritten);
     }
   };
 
-  struct Utf8 : MultiByte {
-    static bool readChar(const char *&Cur, const char *End,
-                         std::mbstate_t &MBState, bool AtEOF, CharT &Ch) {
+  struct Utf8 : MultiByteStateless {
+    static bool readChar(const char *&Cur, const char *End, bool AtEOF,
+                         CharT &Ch) {
       if (Cur == End)
         return false;
 
@@ -296,15 +324,11 @@ struct CharSet {
       Cur = reinterpret_cast<const char *>(UTF8Next);
       return true;
     }
-
-    static void endString(raw_ostream &OS, std::mbstate_t &MBState) {
-      Base::endString(OS, MBState);
-    }
   };
 
   struct SingleByte : Base<unsigned char> {
-    static bool readChar(const char *&Cur, const char *End,
-                         std::mbstate_t &MBState, bool AtEOF, CharT &Ch) {
+    static bool readChar(const char *&Cur, const char *End, bool AtEOF,
+                         CharT &Ch) {
       if (Cur == End)
         return false;
 
@@ -317,18 +341,14 @@ struct CharSet {
       static const StringCharSet &StringChars = []() -> const StringCharSet & {
         static StringCharSet StringChars;
 
-        using llvm::sys::unicode::isCharacter;
-        using llvm::sys::unicode::isControl;
-
+        MultiByteStateless MB;
         for (int I = 0; I < 0x100; ++I) {
           const unsigned char Ch = I;
           const char *Cur = reinterpret_cast<const char *>(&Ch), *End = Cur + 1;
-          std::mbstate_t MBState = {};
-          UTF32 UCh;
-          if (MultiByte::readChar(Cur, End, MBState, /*AtEOF=*/true, UCh)) {
-            if (UCh == U'\t' || (isCharacter(UCh) && !isControl(UCh)))
-              StringChars[I] = true;
-          }
+          MultiByteStateless::CharT UCh;
+          if (MB.readChar(Cur, End, /*AtEOF=*/true, UCh) &&
+              MB.isStringChar(UCh))
+            StringChars[I] = true;
         }
 
         return StringChars;
@@ -345,9 +365,10 @@ struct CharSet {
   };
 };
 
-template <typename CharSet>
+template <typename CharSetT>
 void strings(raw_ostream &OS, StringRef FileName, sys::fs::file_t Handle) {
-  using CharT = typename CharSet::CharT;
+  using CharT = typename CharSetT::CharT;
+  CharSetT CharSet;
 
   SmallString<sys::fs::DefaultReadChunkSize> Buffer;
   auto PrintHeader = [&OS, FileName](size_t StringStart) {
@@ -381,7 +402,6 @@ void strings(raw_ostream &OS, StringRef FileName, sys::fs::file_t Handle) {
   SmallString<DefaultMinLength> Candidate;
   size_t CandidateLengthChars = 0;
   bool InString = false;
-  std::mbstate_t MBState{};
   // Offset of the start of the current chunk within the file.
   size_t ChunkOffset = 0;
 
@@ -425,16 +445,14 @@ void strings(raw_ostream &OS, StringRef FileName, sys::fs::file_t Handle) {
     // enough data to decide whether to print it.
     if (InString || !Candidate.empty()) {
       const char *StringEnd;
-      std::mbstate_t PrevMBState;
       size_t LengthChars = 0;
       bool EndOfChunk;
 
       // Find the end of the current string.
       for (;;) {
         StringEnd = Cur;
-        PrevMBState = MBState;
-        EndOfChunk = !CharSet::readChar(Cur, End, MBState, AtEOF, Ch);
-        if (EndOfChunk || !CharSet::isStringChar(Ch))
+        EndOfChunk = !CharSet.readChar(Cur, End, AtEOF, Ch);
+        if (EndOfChunk || !CharSet.isStringChar(Ch))
           break;
         ++LengthChars;
       }
@@ -445,7 +463,7 @@ void strings(raw_ostream &OS, StringRef FileName, sys::fs::file_t Handle) {
         // the header. E.g. header: aaaaa | bbbbb, where | is the chunk
         // boundary.
         // Output: Header: aaaaabbbbb, where bbbbb is printed in here.
-        OS << StringRef(Begin, SizeBytes);
+        CharSet.writeString(OS, StringRef(Begin, SizeBytes));
       } else if (CandidateLengthChars + LengthChars >= MinLengthChars) {
         // If the header hasn't been printed yet (i.e. the previous candidate
         // was smaller than Min), but we can print it now, print the header
@@ -453,7 +471,8 @@ void strings(raw_ostream &OS, StringRef FileName, sys::fs::file_t Handle) {
         // current string. E.g. aa | bbbbbb
         // Output Header: aabbbbbb, where aabbbbbb is printed in here.
         PrintHeader(ChunkOffset - Candidate.size());
-        OS << Candidate << StringRef(Begin, SizeBytes);
+        CharSet.writeString(OS, Candidate);
+        CharSet.writeString(OS, StringRef(Begin, SizeBytes));
         Candidate.clear();
         CandidateLengthChars = 0;
         InString = true;
@@ -482,7 +501,7 @@ void strings(raw_ostream &OS, StringRef FileName, sys::fs::file_t Handle) {
       if (InString) {
         // We haven't reached the end of the chunk, which means the string is
         // terminated.
-        CharSet::endString(OS, PrevMBState);
+        CharSet.endString(OS);
         InString = false;
       }
     }
@@ -493,10 +512,11 @@ void strings(raw_ostream &OS, StringRef FileName, sys::fs::file_t Handle) {
     size_t LengthChars = 0;
     for (;;) {
       const char *Prev = Cur;
-      std::mbstate_t PrevMBState = MBState;
-      if (!CharSet::readChar(Cur, End, MBState, AtEOF, Ch))
+      if (!StrHead)
+        CharSet.startString();
+      if (!CharSet.readChar(Cur, End, AtEOF, Ch))
         break;
-      if (CharSet::isStringChar(Ch)) {
+      if (CharSet.isStringChar(Ch)) {
         // Find the start of the next string.
         if (!StrHead)
           StrHead = Prev;
@@ -506,8 +526,8 @@ void strings(raw_ostream &OS, StringRef FileName, sys::fs::file_t Handle) {
         // current string. Print it if long enough.
         if (LengthChars >= MinLengthChars) {
           PrintHeader(ChunkOffset + (StrHead - Begin));
-          OS << StringRef(StrHead, Prev - StrHead);
-          CharSet::endString(OS, PrevMBState);
+          CharSet.writeString(OS, StringRef(StrHead, Prev - StrHead));
+          CharSet.endString(OS);
         }
         StrHead = nullptr;
         LengthChars = 0;
@@ -521,7 +541,7 @@ void strings(raw_ostream &OS, StringRef FileName, sys::fs::file_t Handle) {
       // Print it, or append it to Candidate if it is too short.
       if (LengthChars >= MinLengthChars) {
         PrintHeader(ChunkOffset + (StrHead - Begin));
-        OS << StringRef(StrHead, Cur - StrHead);
+        CharSet.writeString(OS, StringRef(StrHead, Cur - StrHead));
         InString = true;
       } else {
         Candidate.append(StrHead, Cur);
@@ -537,7 +557,7 @@ void strings(raw_ostream &OS, StringRef FileName, sys::fs::file_t Handle) {
   }
 
   if (InString)
-    CharSet::endString(OS, MBState);
+    CharSet.endString(OS);
 }
 } // namespace
 
@@ -621,13 +641,16 @@ int main(int argc, char **argv) {
 #endif
 #ifdef _WIN32
       // Using LLVM's built-in decoder is necessary for correctness as
-      // Strings<CharSet::MultiByte> on WIndows is implemented on top of 16-bit
-      // wchar_t which cannot represent all code points.
+      // Strings<CharSet::MultiByteStateless> on WIndows is implemented on top
+      // of 16-bit wchar_t which cannot represent all code points.
       if (GetACP() == CP_UTF8)
         return strings<CharSet::Utf8>;
 #endif
 
-      return strings<CharSet::MultiByte>;
+      if (!mblen(NULL, 0))
+        return strings<CharSet::MultiByteStateless>;
+      else
+        return strings<CharSet::MultiByteStateful>;
 
     case Encoding::Utf8:
       return strings<CharSet::Utf8>;

>From 8e435d45978aead5b0cdd0ce5df4b4d1ff941abc Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Thu, 29 Oct 2026 01:09:00 +0000
Subject: [PATCH 25/41] Fix locale handling on macOS

Despite what I had read and assumed, wchar_t on macOS is not always
UTF-32. For some locales it is, for others it isn't. To convert to
UTF-32 reliably, we need to instead go through iconv(). This commit does
that, and adds a test for it.
---
 .../tools/llvm-strings/encoding-locale.test   | 14 ++++
 llvm/tools/llvm-strings/CMakeLists.txt        |  4 +
 llvm/tools/llvm-strings/llvm-strings.cpp      | 75 +++++++++++++++++--
 3 files changed, 88 insertions(+), 5 deletions(-)
 create mode 100644 llvm/test/tools/llvm-strings/encoding-locale.test

diff --git a/llvm/test/tools/llvm-strings/encoding-locale.test b/llvm/test/tools/llvm-strings/encoding-locale.test
new file mode 100644
index 00000000000000..cb5af1edfd6b7a
--- /dev/null
+++ b/llvm/test/tools/llvm-strings/encoding-locale.test
@@ -0,0 +1,14 @@
+# REQUIRES: system-darwin
+
+RUN: printf 'Hello\342\202\254\202\261\202\361\202\311\202\277\202\315' > %t
+
+RUN: env LANG=en_US.US-ASCII llvm-strings %t | FileCheck --check-prefixes CHECK-US-ASCII %s
+RUN: env LANG=en_US.ISO8859-1 llvm-strings %t | FileCheck --check-prefixes CHECK-ISO8859-1 %s
+RUN: env LANG=en_US.UTF-8 llvm-strings %t | FileCheck --check-prefixes CHECK-UTF-8 %s
+RUN: env LANG=ja_JP.SJIS llvm-strings %t | FileCheck --check-prefixes CHECK-SJIS %s
+
+CHECK-US-ASCII:Hello
+CHECK-ISO8859-1:Hello{{\xe2}}
+CHECK-UTF-8:Hello{{\xe2\x82\xac}}
+CHECK-SJIS:Hello
+CHECK-SJIS:{{\x82\xb1\x82\xf1\x82\xc9\x82\xbf\x82\xcd}}
diff --git a/llvm/tools/llvm-strings/CMakeLists.txt b/llvm/tools/llvm-strings/CMakeLists.txt
index 95dd69c778e477..f1725efd355d49 100644
--- a/llvm/tools/llvm-strings/CMakeLists.txt
+++ b/llvm/tools/llvm-strings/CMakeLists.txt
@@ -15,6 +15,10 @@ add_llvm_tool(llvm-strings
   StringsOptsTableGen
   )
 
+if(APPLE)
+  target_link_libraries(llvm-strings PRIVATE iconv)
+endif()
+
 if(LLVM_INSTALL_BINUTILS_SYMLINKS)
   add_llvm_tool_symlink(strings llvm-strings)
 endif()
diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 8c26e5c06fb624..7855f13f928e2f 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -42,6 +42,10 @@
 #include "llvm/Support/Windows/WindowsSupport.h"
 #endif
 
+#ifdef __APPLE__
+#include <iconv.h>
+#endif
+
 using namespace llvm;
 using namespace llvm::object;
 
@@ -108,12 +112,12 @@ static void parseIntArg(const opt::InputArgList &Args, int ID, T &Value) {
 // problems on most platforms, thus we introduce wrappers. On Windows,
 // mbrtoc32() ignores the locale and only converts UTF-8, but wchar_t is UTF-16
 // and is close enough, as long as we make sure to avoid it for UTF-8 locales.
-// On Apple, <uchar.h> is not available, but wchar_t is UTF-32 and is a full
-// substitute.
+// On Apple, <uchar.h> is not available, and wchar_t is not always UTF-32. We
+// implement mbrtoc() and crtomb() through iconv().
 // On glibc, mbrtoc32() is a wrapper for mbrtowc() and never returns (size_t)-3,
 // instead it handles this case by returning 0 but setting Ch to a non-null
 // character.
-#if defined(_WIN32) || defined(__APPLE__)
+#if defined(_WIN32)
 using char_t = wchar_t;
 static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
                      std::mbstate_t *MBState) {
@@ -122,9 +126,70 @@ static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
     Result = (size_t)-3;
   return Result;
 }
-static size_t crtomb(char *S, char_t Ch, mbstate_t *MBState) {
+static size_t crtomb(char *S, char_t Ch, std::mbstate_t *MBState) {
   return std::wcrtomb(S, Ch, MBState);
 }
+#elif defined(__APPLE__)
+using char_t = char32_t;
+static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
+                     std::mbstate_t *MBState) {
+  static struct IconvHandle {
+    IconvHandle() : Value(iconv_open("UCS-4-INTERNAL", nl_langinfo(CODESET))) {
+      if (Value == (iconv_t)-1)
+        report_fatal_error("failed to open iconv");
+    }
+    ~IconvHandle() { iconv_close(Value); }
+
+    iconv_t Value;
+  } CD;
+
+  wchar_t WCh;
+  size_t BytesRead = mbrtowc(&WCh, S, N, MBState);
+  if (BytesRead != (size_t)-2 && BytesRead != (size_t)-1) {
+    char Cvt[MB_LEN_MAX];
+    std::mbstate_t CvtMBState{};
+    const size_t BytesWritten = wcrtomb(Cvt, WCh, &CvtMBState);
+    char *PIn = Cvt;
+    size_t NIn = BytesWritten;
+    char *POut = reinterpret_cast<char *>(Ch);
+    size_t NOut = sizeof(char_t);
+    const size_t NumErrors = iconv(CD.Value, &PIn, &NIn, &POut, &NOut);
+    // mbrtowc() may be lenient and accept invalid characters that iconv()
+    // rejects.
+    if (NIn == BytesWritten)
+      return (size_t)-1;
+    if (NumErrors || NIn || NOut)
+      llvm_unreachable("internal error: iconv failed on valid character");
+  }
+  return BytesRead;
+}
+static size_t crtomb(char *S, char_t Ch, std::mbstate_t *MBState) {
+  static struct IconvHandle {
+    IconvHandle() : Value(iconv_open(nl_langinfo(CODESET), "UCS-4-INTERNAL")) {
+      if (Value == (iconv_t)-1)
+        report_fatal_error("failed to open iconv");
+    }
+    ~IconvHandle() { iconv_close(Value); }
+
+    iconv_t Value;
+  } CD;
+
+  char Cvt[MB_LEN_MAX];
+  std::mbstate_t CvtMBState{};
+  char *PIn = reinterpret_cast<char *>(Ch);
+  size_t NIn = sizeof(char_t);
+  char *POut = Cvt;
+  size_t NOut = MB_LEN_MAX;
+  const size_t NumErrors = iconv(CD.Value, &PIn, &NIn, &POut, &NOut);
+  if (NumErrors == (size_t)-1 || NIn)
+    llvm_unreachable("internal error: iconv failed on valid character");
+  if (NumErrors)
+    return (size_t)-1;
+  wchar_t WCh;
+  if (mbrtowc(&WCh, Cvt, PIn - Cvt, &CvtMBState) != size_t(PIn - Cvt))
+    llvm_unreachable("internal error: mbrtowc failed on valid character");
+  return wcrtomb(S, WCh, MBState);
+}
 #else
 using char_t = char32_t;
 static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
@@ -134,7 +199,7 @@ static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
     Result = (size_t)-3;
   return Result;
 }
-static size_t crtomb(char *S, char_t Ch, mbstate_t *MBState) {
+static size_t crtomb(char *S, char_t Ch, std::mbstate_t *MBState) {
   return std::c32rtomb(S, Ch, MBState);
 }
 #endif

>From db9481632558e78865dc32d41f6013922ec49c8b Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Fri, 30 Oct 2026 00:40:12 +0000
Subject: [PATCH 26/41] Further restrict tests, prepare for later update

---
 llvm/test/tools/llvm-strings/encoding-locale.test | 14 --------------
 llvm/test/tools/llvm-strings/locale.test          | 14 ++++++++++++++
 2 files changed, 14 insertions(+), 14 deletions(-)
 delete mode 100644 llvm/test/tools/llvm-strings/encoding-locale.test
 create mode 100644 llvm/test/tools/llvm-strings/locale.test

diff --git a/llvm/test/tools/llvm-strings/encoding-locale.test b/llvm/test/tools/llvm-strings/encoding-locale.test
deleted file mode 100644
index cb5af1edfd6b7a..00000000000000
--- a/llvm/test/tools/llvm-strings/encoding-locale.test
+++ /dev/null
@@ -1,14 +0,0 @@
-# REQUIRES: system-darwin
-
-RUN: printf 'Hello\342\202\254\202\261\202\361\202\311\202\277\202\315' > %t
-
-RUN: env LANG=en_US.US-ASCII llvm-strings %t | FileCheck --check-prefixes CHECK-US-ASCII %s
-RUN: env LANG=en_US.ISO8859-1 llvm-strings %t | FileCheck --check-prefixes CHECK-ISO8859-1 %s
-RUN: env LANG=en_US.UTF-8 llvm-strings %t | FileCheck --check-prefixes CHECK-UTF-8 %s
-RUN: env LANG=ja_JP.SJIS llvm-strings %t | FileCheck --check-prefixes CHECK-SJIS %s
-
-CHECK-US-ASCII:Hello
-CHECK-ISO8859-1:Hello{{\xe2}}
-CHECK-UTF-8:Hello{{\xe2\x82\xac}}
-CHECK-SJIS:Hello
-CHECK-SJIS:{{\x82\xb1\x82\xf1\x82\xc9\x82\xbf\x82\xcd}}
diff --git a/llvm/test/tools/llvm-strings/locale.test b/llvm/test/tools/llvm-strings/locale.test
new file mode 100644
index 00000000000000..3930576e54026c
--- /dev/null
+++ b/llvm/test/tools/llvm-strings/locale.test
@@ -0,0 +1,14 @@
+REQUIRES: system-darwin
+
+RUN: printf 'Hello\360\237\230\203\n\202\261\202\361\202\311\202\277\202\315\n' > %t
+
+RUN: env LANG=en_US.US-ASCII  llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-US-ASCII %s
+RUN: env LANG=en_US.ISO8859-1 llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-ISO8859-1 %s
+RUN: env LANG=en_US.UTF-8     llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-UTF-8 %s
+RUN: env LANG=ja_JP.SJIS      llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-SJIS %s
+
+CHECK-IN-US-ASCII:Hello
+CHECK-IN-ISO8859-1:Hello{{\xF0}}
+CHECK-IN-UTF-8:Hello{{\xF0\x9F\x98\x83}}
+CHECK-IN-SJIS:Hello
+CHECK-IN-SJIS:{{\x82\xB1\x82\xF1\x82\xC9\x82\xBF\x82\xCD}}

>From 7467f12e481ce57ce7e410fc533757e0ef96d13a Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Fri, 30 Oct 2026 18:33:50 +0000
Subject: [PATCH 27/41] Fix crtomb

---
 llvm/tools/llvm-strings/llvm-strings.cpp | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 7855f13f928e2f..12be17b88f456a 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -176,7 +176,7 @@ static size_t crtomb(char *S, char_t Ch, std::mbstate_t *MBState) {
 
   char Cvt[MB_LEN_MAX];
   std::mbstate_t CvtMBState{};
-  char *PIn = reinterpret_cast<char *>(Ch);
+  char *PIn = reinterpret_cast<char *>(&Ch);
   size_t NIn = sizeof(char_t);
   char *POut = Cvt;
   size_t NOut = MB_LEN_MAX;

>From 61b4265815be8cb3da319a15f4654916dff450e4 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Mon, 2 Nov 2026 14:28:32 +0000
Subject: [PATCH 28/41] Apply fix on FreeBSD too and enable tests there

---
 llvm/test/tools/llvm-strings/locale.test | 2 +-
 llvm/tools/llvm-strings/llvm-strings.cpp | 6 ++++--
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/llvm/test/tools/llvm-strings/locale.test b/llvm/test/tools/llvm-strings/locale.test
index 3930576e54026c..4eac28dc727fd0 100644
--- a/llvm/test/tools/llvm-strings/locale.test
+++ b/llvm/test/tools/llvm-strings/locale.test
@@ -1,4 +1,4 @@
-REQUIRES: system-darwin
+REQUIRES: system-darwin || system-freebsd
 
 RUN: printf 'Hello\360\237\230\203\n\202\261\202\361\202\311\202\277\202\315\n' > %t
 
diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 12be17b88f456a..f7fe6845353e49 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -42,7 +42,7 @@
 #include "llvm/Support/Windows/WindowsSupport.h"
 #endif
 
-#ifdef __APPLE__
+#if defined(__APPLE__) || defined(__FreeBSD__)
 #include <iconv.h>
 #endif
 
@@ -114,6 +114,8 @@ static void parseIntArg(const opt::InputArgList &Args, int ID, T &Value) {
 // and is close enough, as long as we make sure to avoid it for UTF-8 locales.
 // On Apple, <uchar.h> is not available, and wchar_t is not always UTF-32. We
 // implement mbrtoc() and crtomb() through iconv().
+// On FreeBSD, <uchar.h> is available but assumes wchar_t is always UTF-32, yet
+// on FreeBSD, wchar_t is not always UTF-32, so likewise use iconv().
 // On glibc, mbrtoc32() is a wrapper for mbrtowc() and never returns (size_t)-3,
 // instead it handles this case by returning 0 but setting Ch to a non-null
 // character.
@@ -129,7 +131,7 @@ static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
 static size_t crtomb(char *S, char_t Ch, std::mbstate_t *MBState) {
   return std::wcrtomb(S, Ch, MBState);
 }
-#elif defined(__APPLE__)
+#elif defined(__APPLE__) || defined(__FreeBSD__)
 using char_t = char32_t;
 static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
                      std::mbstate_t *MBState) {

>From 7011c5233831383e4b007fcab8aaef241e679664 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Mon, 2 Nov 2026 16:00:19 +0000
Subject: [PATCH 29/41] Correct FreeBSD description

---
 llvm/tools/llvm-strings/llvm-strings.cpp | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index f7fe6845353e49..c6ee9c2e68d174 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -114,8 +114,9 @@ static void parseIntArg(const opt::InputArgList &Args, int ID, T &Value) {
 // and is close enough, as long as we make sure to avoid it for UTF-8 locales.
 // On Apple, <uchar.h> is not available, and wchar_t is not always UTF-32. We
 // implement mbrtoc() and crtomb() through iconv().
-// On FreeBSD, <uchar.h> is available but assumes wchar_t is always UTF-32, yet
-// on FreeBSD, wchar_t is not always UTF-32, so likewise use iconv().
+// On FreeBSD, <uchar.h> is available but the result for encoding errors is
+// (size_t)-2 rather than (size_t) which we cannot recover from, so likewise use
+// iconv().
 // On glibc, mbrtoc32() is a wrapper for mbrtowc() and never returns (size_t)-3,
 // instead it handles this case by returning 0 but setting Ch to a non-null
 // character.

>From 1c71494c011c26b99e9b3436eaf1f1e303a60723 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Mon, 2 Nov 2026 18:08:06 +0000
Subject: [PATCH 30/41] Apply review feedback

---
 llvm/test/tools/llvm-strings/locale.test | 21 +++++++---
 llvm/tools/llvm-strings/llvm-strings.cpp | 53 ++++++++++++------------
 2 files changed, 42 insertions(+), 32 deletions(-)

diff --git a/llvm/test/tools/llvm-strings/locale.test b/llvm/test/tools/llvm-strings/locale.test
index 4eac28dc727fd0..f498e4b0a20646 100644
--- a/llvm/test/tools/llvm-strings/locale.test
+++ b/llvm/test/tools/llvm-strings/locale.test
@@ -1,11 +1,22 @@
-REQUIRES: system-darwin || system-freebsd
+UNSUPPORTED: system-windows
 
 RUN: printf 'Hello\360\237\230\203\n\202\261\202\361\202\311\202\277\202\315\n' > %t
 
-RUN: env LANG=en_US.US-ASCII  llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-US-ASCII %s
-RUN: env LANG=en_US.ISO8859-1 llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-ISO8859-1 %s
-RUN: env LANG=en_US.UTF-8     llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-UTF-8 %s
-RUN: env LANG=ja_JP.SJIS      llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-SJIS %s
+RUN: %if locale-en_US.US-ASCII %{ env LANG=en_US.US-ASCII \
+RUN: llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-US-ASCII %s %} \
+RUN: %else %{ echo en_US.US-ASCII locale not supported, skipping test %}
+
+RUN: %if locale-en_US.ISO8859-1 %{ env LANG=en_US.ISO8859-1 \
+RUN: llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-ISO8859-1 %s %} \
+RUN: %else %{ echo en_US.ISO8859-1 locale not supported, skipping test %}
+
+RUN: %if locale-en_US.UTF-8 %{ env LANG=en_US.UTF-8 \
+RUN: llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-UTF-8 %s %} \
+RUN: %else %{ echo en_US.UTF-8 locale not supported, skipping test %}
+
+RUN: %if locale-ja_JP.SJIS %{ env LANG=ja_JP.SJIS \
+RUN: llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-SJIS %s %} \
+RUN: %else %{ echo ja_JP.SJIS locale not supported, skipping test %}
 
 CHECK-IN-US-ASCII:Hello
 CHECK-IN-ISO8859-1:Hello{{\xF0}}
diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index c6ee9c2e68d174..3bcb3609218418 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -115,8 +115,8 @@ static void parseIntArg(const opt::InputArgList &Args, int ID, T &Value) {
 // On Apple, <uchar.h> is not available, and wchar_t is not always UTF-32. We
 // implement mbrtoc() and crtomb() through iconv().
 // On FreeBSD, <uchar.h> is available but the result for encoding errors is
-// (size_t)-2 rather than (size_t) which we cannot recover from, so likewise use
-// iconv().
+// (size_t)-2 rather than (size_t)-1. We cannot recover from this, so likewise
+// use iconv().
 // On glibc, mbrtoc32() is a wrapper for mbrtowc() and never returns (size_t)-3,
 // instead it handles this case by returning 0 but setting Ch to a non-null
 // character.
@@ -133,18 +133,24 @@ static size_t crtomb(char *S, char_t Ch, std::mbstate_t *MBState) {
   return std::wcrtomb(S, Ch, MBState);
 }
 #elif defined(__APPLE__) || defined(__FreeBSD__)
+struct IconvHandle {
+  IconvHandle(const char *DstName, const char *SrcName)
+      : Value(iconv_open(DstName, SrcName)) {
+    if (Value == (iconv_t)-1)
+      reportCmdLineError("failed to open iconv");
+  }
+  IconvHandle(IconvHandle &&) = delete;
+  ~IconvHandle() { iconv_close(Value); }
+  iconv_t value() const { return Value; }
+
+private:
+  iconv_t Value;
+};
+
 using char_t = char32_t;
 static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
                      std::mbstate_t *MBState) {
-  static struct IconvHandle {
-    IconvHandle() : Value(iconv_open("UCS-4-INTERNAL", nl_langinfo(CODESET))) {
-      if (Value == (iconv_t)-1)
-        report_fatal_error("failed to open iconv");
-    }
-    ~IconvHandle() { iconv_close(Value); }
-
-    iconv_t Value;
-  } CD;
+  static IconvHandle CD("UCS-4-INTERNAL", nl_langinfo(CODESET));
 
   wchar_t WCh;
   size_t BytesRead = mbrtowc(&WCh, S, N, MBState);
@@ -156,7 +162,7 @@ static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
     size_t NIn = BytesWritten;
     char *POut = reinterpret_cast<char *>(Ch);
     size_t NOut = sizeof(char_t);
-    const size_t NumErrors = iconv(CD.Value, &PIn, &NIn, &POut, &NOut);
+    const size_t NumErrors = iconv(CD.value(), &PIn, &NIn, &POut, &NOut);
     // mbrtowc() may be lenient and accept invalid characters that iconv()
     // rejects.
     if (NIn == BytesWritten)
@@ -167,15 +173,7 @@ static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
   return BytesRead;
 }
 static size_t crtomb(char *S, char_t Ch, std::mbstate_t *MBState) {
-  static struct IconvHandle {
-    IconvHandle() : Value(iconv_open(nl_langinfo(CODESET), "UCS-4-INTERNAL")) {
-      if (Value == (iconv_t)-1)
-        report_fatal_error("failed to open iconv");
-    }
-    ~IconvHandle() { iconv_close(Value); }
-
-    iconv_t Value;
-  } CD;
+  static IconvHandle CD(nl_langinfo(CODESET), "UCS-4-INTERNAL");
 
   char Cvt[MB_LEN_MAX];
   std::mbstate_t CvtMBState{};
@@ -183,7 +181,7 @@ static size_t crtomb(char *S, char_t Ch, std::mbstate_t *MBState) {
   size_t NIn = sizeof(char_t);
   char *POut = Cvt;
   size_t NOut = MB_LEN_MAX;
-  const size_t NumErrors = iconv(CD.Value, &PIn, &NIn, &POut, &NOut);
+  const size_t NumErrors = iconv(CD.value(), &PIn, &NIn, &POut, &NOut);
   if (NumErrors == (size_t)-1 || NIn)
     llvm_unreachable("internal error: iconv failed on valid character");
   if (NumErrors)
@@ -257,10 +255,11 @@ struct CharSet {
       char_t UCh;
       for (;;) {
         // Read one byte at a time. This is usually not the best way to use
-        // mbrtoc32(), usually it would make more sense to pass the size of the
-        // buffer, but the strings utility is unusual in that it is expected to
-        // encounter many bytes that do not form valid characters and it is more
-        // useful to optimise for this case.
+        // mbrtoc32(): usually passing the size of the buffer reduces the
+        // number of times it needs to be called. The strings utility is unusual
+        // in that it is expected to encounter many bytes that do not form valid
+        // characters and to recover from the subsequent decoding errors,
+        // however, so it is more useful to optimise for this case.
         const size_t BytesRead = mbrtoc(&UCh, Next, 1, &NextMBState);
         switch (BytesRead) {
         case 1:
@@ -709,7 +708,7 @@ int main(int argc, char **argv) {
 #endif
 #ifdef _WIN32
       // Using LLVM's built-in decoder is necessary for correctness as
-      // Strings<CharSet::MultiByteStateless> on WIndows is implemented on top
+      // Strings<CharSet::MultiByteStateless> on Windows is implemented on top
       // of 16-bit wchar_t which cannot represent all code points.
       if (GetACP() == CP_UTF8)
         return strings<CharSet::Utf8>;

>From b817d09a820a5b37851961a8fe50131108b06f5f Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Mon, 2 Nov 2026 23:30:41 +0000
Subject: [PATCH 31/41] Optimise for single-byte characters

---
 llvm/tools/llvm-strings/llvm-strings.cpp | 32 +++++++++++++++++++++++-
 1 file changed, 31 insertions(+), 1 deletion(-)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 3bcb3609218418..efbd201c579053 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -250,9 +250,34 @@ struct CharSet {
       if (Cur == End)
         return false;
 
+      static const UTF32(&LookupTable)[0x100] = [&]() -> const UTF32(&)[0x100] {
+        static UTF32 LookupTable[0x100];
+
+        for (int I = 0; I < 0x100; ++I) {
+          const unsigned char Ch = I;
+          const char *Cur = reinterpret_cast<const char *>(&Ch);
+          char_t UCh;
+          std::mbstate_t InitialMBState{};
+          if (mbrtoc(&UCh, Cur, 1, &InitialMBState) == 1)
+            LookupTable[Ch] = UCh;
+        }
+
+        return LookupTable;
+      }();
+
+      char_t UCh;
+
+      if (std::mbsinit(&InMBState)) {
+        UCh = LookupTable[static_cast<unsigned char>(*Cur)];
+        if (UCh || !*Cur) {
+          ++Cur;
+          Ch = UCh;
+          return true;
+        }
+      }
+
       const char *Next = Cur;
       std::mbstate_t NextMBState = InMBState;
-      char_t UCh;
       for (;;) {
         // Read one byte at a time. This is usually not the best way to use
         // mbrtoc32(): usually passing the size of the buffer reduces the
@@ -370,6 +395,11 @@ struct CharSet {
       if (Cur == End)
         return false;
 
+      if (static_cast<unsigned char>(*Cur) <= 0x7F) {
+        Ch = *Cur++;
+        return true;
+      }
+
       const UTF8 *UTF8Cur = reinterpret_cast<const UTF8 *>(Cur);
       const UTF8 *UTF8Next = UTF8Cur;
       const UTF8 *UTF8End = reinterpret_cast<const UTF8 *>(End);

>From dc68790c2b5a80690d6a01c5bfc40de5cd4d20a7 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Tue, 3 Nov 2026 12:25:19 +0000
Subject: [PATCH 32/41] Add missing file

---
 llvm/test/tools/llvm-strings/lit.local.cfg | 9 +++++++++
 1 file changed, 9 insertions(+)
 create mode 100644 llvm/test/tools/llvm-strings/lit.local.cfg

diff --git a/llvm/test/tools/llvm-strings/lit.local.cfg b/llvm/test/tools/llvm-strings/lit.local.cfg
new file mode 100644
index 00000000000000..62616b7347ce62
--- /dev/null
+++ b/llvm/test/tools/llvm-strings/lit.local.cfg
@@ -0,0 +1,9 @@
+import locale
+saved = locale.setlocale(locale.LC_ALL)
+for l in ('en_US.US-ASCII', 'en_US.ISO8859-1', 'en_US.UTF-8', 'ja_JP.SJIS'):
+  try:
+    old = locale.setlocale(locale.LC_ALL, l)
+    config.available_features.add('locale-' + l)
+  except locale.Error:
+    pass
+locale.setlocale(locale.LC_ALL, saved)

>From fc3ccccbd90965ba9ad3988a81df699dfd0b8945 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Tue, 3 Nov 2026 23:21:09 +0000
Subject: [PATCH 33/41] Add ISO2022-JP test, remove llvm_unreachable, fix
 mbrtowc call

---
 llvm/test/tools/llvm-strings/lit.local.cfg |  2 +-
 llvm/test/tools/llvm-strings/locale.test   | 10 +++++++++-
 llvm/tools/llvm-strings/llvm-strings.cpp   |  6 ++----
 3 files changed, 12 insertions(+), 6 deletions(-)

diff --git a/llvm/test/tools/llvm-strings/lit.local.cfg b/llvm/test/tools/llvm-strings/lit.local.cfg
index 62616b7347ce62..50709f7c8e4005 100644
--- a/llvm/test/tools/llvm-strings/lit.local.cfg
+++ b/llvm/test/tools/llvm-strings/lit.local.cfg
@@ -1,6 +1,6 @@
 import locale
 saved = locale.setlocale(locale.LC_ALL)
-for l in ('en_US.US-ASCII', 'en_US.ISO8859-1', 'en_US.UTF-8', 'ja_JP.SJIS'):
+for l in ('en_US.US-ASCII', 'en_US.ISO8859-1', 'en_US.UTF-8', 'ja_JP.SJIS', 'ja_JP.ISO2022-JP'):
   try:
     old = locale.setlocale(locale.LC_ALL, l)
     config.available_features.add('locale-' + l)
diff --git a/llvm/test/tools/llvm-strings/locale.test b/llvm/test/tools/llvm-strings/locale.test
index f498e4b0a20646..1439a5c8e9f5da 100644
--- a/llvm/test/tools/llvm-strings/locale.test
+++ b/llvm/test/tools/llvm-strings/locale.test
@@ -1,6 +1,8 @@
 UNSUPPORTED: system-windows
 
-RUN: printf 'Hello\360\237\230\203\n\202\261\202\361\202\311\202\277\202\315\n' > %t
+RUN: printf 'Hello\360\237\230\203\n' > %t
+RUN: printf '\202\261\202\361\202\311\202\277\202\315\n' >> %t
+RUN: printf '\033$B$3$s$K$A$O\033(B\n' >> %t
 
 RUN: %if locale-en_US.US-ASCII %{ env LANG=en_US.US-ASCII \
 RUN: llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-US-ASCII %s %} \
@@ -18,8 +20,14 @@ RUN: %if locale-ja_JP.SJIS %{ env LANG=ja_JP.SJIS \
 RUN: llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-SJIS %s %} \
 RUN: %else %{ echo ja_JP.SJIS locale not supported, skipping test %}
 
+RUN: %if locale-ja_JP.ISO2022-JP %{ env LANG=ja_JP.ISO2022-JP \
+RUN: llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-ISO2022-JP %s %} \
+RUN: %else %{ echo ja_JP.ISO2022-JP locale not supported, skipping test %}
+
 CHECK-IN-US-ASCII:Hello
 CHECK-IN-ISO8859-1:Hello{{\xF0}}
 CHECK-IN-UTF-8:Hello{{\xF0\x9F\x98\x83}}
 CHECK-IN-SJIS:Hello
 CHECK-IN-SJIS:{{\x82\xB1\x82\xF1\x82\xC9\x82\xBF\x82\xCD}}
+CHECK-IN-ISO2022-JP:Hello
+CHECK-IN-ISO2022-JP:{{\x1B}}$B$3$s$K$A$O{{\x1B}}(B
diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index efbd201c579053..ecd61f27ea5e52 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -165,10 +165,8 @@ static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
     const size_t NumErrors = iconv(CD.value(), &PIn, &NIn, &POut, &NOut);
     // mbrtowc() may be lenient and accept invalid characters that iconv()
     // rejects.
-    if (NIn == BytesWritten)
-      return (size_t)-1;
     if (NumErrors || NIn || NOut)
-      llvm_unreachable("internal error: iconv failed on valid character");
+      return (size_t)-1;
   }
   return BytesRead;
 }
@@ -187,7 +185,7 @@ static size_t crtomb(char *S, char_t Ch, std::mbstate_t *MBState) {
   if (NumErrors)
     return (size_t)-1;
   wchar_t WCh;
-  if (mbrtowc(&WCh, Cvt, PIn - Cvt, &CvtMBState) != size_t(PIn - Cvt))
+  if (mbrtowc(&WCh, Cvt, POut - Cvt, &CvtMBState) != size_t(POut - Cvt))
     llvm_unreachable("internal error: mbrtowc failed on valid character");
   return wcrtomb(S, WCh, MBState);
 }

>From 2adf6231123506485385aa3d73f86e01ab6e4c7f Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Wed, 4 Nov 2026 14:20:14 +0000
Subject: [PATCH 34/41] Comments, formatting

---
 llvm/test/tools/llvm-strings/lit.local.cfg | 19 +++++++++++++------
 llvm/tools/llvm-strings/llvm-strings.cpp   | 14 ++++++++++++--
 2 files changed, 25 insertions(+), 8 deletions(-)

diff --git a/llvm/test/tools/llvm-strings/lit.local.cfg b/llvm/test/tools/llvm-strings/lit.local.cfg
index 50709f7c8e4005..4bbad77f3c7e2d 100644
--- a/llvm/test/tools/llvm-strings/lit.local.cfg
+++ b/llvm/test/tools/llvm-strings/lit.local.cfg
@@ -1,9 +1,16 @@
 import locale
+
 saved = locale.setlocale(locale.LC_ALL)
-for l in ('en_US.US-ASCII', 'en_US.ISO8859-1', 'en_US.UTF-8', 'ja_JP.SJIS', 'ja_JP.ISO2022-JP'):
-  try:
-    old = locale.setlocale(locale.LC_ALL, l)
-    config.available_features.add('locale-' + l)
-  except locale.Error:
-    pass
+for l in (
+    "en_US.US-ASCII",
+    "en_US.ISO8859-1",
+    "en_US.UTF-8",
+    "ja_JP.SJIS",
+    "ja_JP.ISO2022-JP",
+):
+    try:
+        old = locale.setlocale(locale.LC_ALL, l)
+        config.available_features.add("locale-" + l)
+    except locale.Error:
+        pass
 locale.setlocale(locale.LC_ALL, saved)
diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index ecd61f27ea5e52..65f23c3660b037 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -122,6 +122,7 @@ static void parseIntArg(const opt::InputArgList &Args, int ID, T &Value) {
 // character.
 #if defined(_WIN32)
 using char_t = wchar_t;
+
 static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
                      std::mbstate_t *MBState) {
   size_t Result = std::mbrtowc(Ch, S, N, MBState);
@@ -129,6 +130,7 @@ static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
     Result = (size_t)-3;
   return Result;
 }
+
 static size_t crtomb(char *S, char_t Ch, std::mbstate_t *MBState) {
   return std::wcrtomb(S, Ch, MBState);
 }
@@ -148,6 +150,7 @@ struct IconvHandle {
 };
 
 using char_t = char32_t;
+
 static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
                      std::mbstate_t *MBState) {
   static IconvHandle CD("UCS-4-INTERNAL", nl_langinfo(CODESET));
@@ -162,6 +165,10 @@ static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
     size_t NIn = BytesWritten;
     char *POut = reinterpret_cast<char *>(Ch);
     size_t NOut = sizeof(char_t);
+    // This is not generally correct for all systems, but this is limited to
+    // Apple and FreeBSD. Neither supports stateful encodings in its locales, so
+    // we do not need to worry about them in the call to iconv(). Beware if this
+    // needs to be adapted to run on other systems.
     const size_t NumErrors = iconv(CD.value(), &PIn, &NIn, &POut, &NOut);
     // mbrtowc() may be lenient and accept invalid characters that iconv()
     // rejects.
@@ -170,6 +177,7 @@ static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
   }
   return BytesRead;
 }
+
 static size_t crtomb(char *S, char_t Ch, std::mbstate_t *MBState) {
   static IconvHandle CD(nl_langinfo(CODESET), "UCS-4-INTERNAL");
 
@@ -191,6 +199,7 @@ static size_t crtomb(char *S, char_t Ch, std::mbstate_t *MBState) {
 }
 #else
 using char_t = char32_t;
+
 static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
                      std::mbstate_t *MBState) {
   size_t Result = std::mbrtoc32(Ch, S, N, MBState);
@@ -198,6 +207,7 @@ static size_t mbrtoc(char_t *Ch, const char *S, size_t N,
     Result = (size_t)-3;
   return Result;
 }
+
 static size_t crtomb(char *S, char_t Ch, std::mbstate_t *MBState) {
   return std::c32rtomb(S, Ch, MBState);
 }
@@ -363,8 +373,8 @@ struct CharSet {
         const size_t BytesRead = mbrtoc(&UCh, Cur, End - Cur, &StrMBState);
         assert(BytesRead != 0 && BytesRead != (size_t)-1 &&
                BytesRead != (size_t)-2 &&
-               "If a string was initially valid, it cannot to become invalid "
-               "when re-reading");
+               "If a string was initially valid, it cannot become invalid when "
+               "re-reading");
         if (BytesRead != (size_t)-3)
           Cur += BytesRead;
 

>From 77e017931f0e6d32934a9948906ec3e0184d7b12 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Wed, 4 Nov 2026 15:35:13 +0000
Subject: [PATCH 35/41] Mark Generated*Ranges as LLVM_ABI to allow direct
 access

---
 clang/lib/Lex/UnicodeCharSetsGenerated.cpp        | 15 +++++++++------
 llvm/lib/Support/UnicodeCharSetsGenerated.cpp     | 14 ++++++++------
 llvm/tools/llvm-strings/llvm-strings.cpp          |  4 ++--
 .../UnicodeData/UnicodeCharSetsGenerator.cpp      |  3 ++-
 4 files changed, 21 insertions(+), 15 deletions(-)

diff --git a/clang/lib/Lex/UnicodeCharSetsGenerated.cpp b/clang/lib/Lex/UnicodeCharSetsGenerated.cpp
index f1967d496d9da6..b8e06083eb7145 100644
--- a/clang/lib/Lex/UnicodeCharSetsGenerated.cpp
+++ b/clang/lib/Lex/UnicodeCharSetsGenerated.cpp
@@ -255,7 +255,8 @@ static constexpr UnicodeCharRange GeneratedXIDStartRangesData[] = {
     {0x2CEB0, 0x2EBE0}, {0x2EBF0, 0x2EE5D}, {0x2F800, 0x2FA1D},
     {0x30000, 0x3134A}, {0x31350, 0x33479}, {0x3D000, 0x3FC3F},
 };
-extern const CharRanges GeneratedXIDStartRanges = GeneratedXIDStartRangesData;
+LLVM_ABI extern const CharRanges GeneratedXIDStartRanges =
+    GeneratedXIDStartRangesData;
 
 // XID_Continue, excluding XID_Start
 static constexpr UnicodeCharRange GeneratedXIDContinueRangesData[] = {
@@ -396,7 +397,7 @@ static constexpr UnicodeCharRange GeneratedXIDContinueRangesData[] = {
     {0x1E8D0, 0x1E8D6}, {0x1E944, 0x1E94A}, {0x1E950, 0x1E959},
     {0x1FBF0, 0x1FBF9}, {0xE0100, 0xE01EF},
 };
-extern const CharRanges GeneratedXIDContinueRanges =
+LLVM_ABI extern const CharRanges GeneratedXIDContinueRanges =
     GeneratedXIDContinueRangesData;
 
 // ID_Compat_Math_Start
@@ -408,8 +409,9 @@ static constexpr UnicodeCharRange
         {0x1D76F, 0x1D76F}, {0x1D789, 0x1D789}, {0x1D7A9, 0x1D7A9},
         {0x1D7C3, 0x1D7C3},
 };
-extern const CharRanges GeneratedMathematicalNotationProfileIDStartRanges =
-    GeneratedMathematicalNotationProfileIDStartRangesData;
+LLVM_ABI extern const CharRanges
+    GeneratedMathematicalNotationProfileIDStartRanges =
+        GeneratedMathematicalNotationProfileIDStartRangesData;
 
 // ID_Compat_Math_Continue, excluding ID_Compat_Math_Start
 static constexpr UnicodeCharRange
@@ -417,7 +419,8 @@ static constexpr UnicodeCharRange
         {0x00B2, 0x00B3}, {0x00B9, 0x00B9}, {0x2070, 0x2070},
         {0x2074, 0x207E}, {0x2080, 0x208E},
 };
-extern const CharRanges GeneratedMathematicalNotationProfileIDContinueRanges =
-    GeneratedMathematicalNotationProfileIDContinueRangesData;
+LLVM_ABI extern const CharRanges
+    GeneratedMathematicalNotationProfileIDContinueRanges =
+        GeneratedMathematicalNotationProfileIDContinueRangesData;
 
 } // namespace clang
diff --git a/llvm/lib/Support/UnicodeCharSetsGenerated.cpp b/llvm/lib/Support/UnicodeCharSetsGenerated.cpp
index cba25f21eacb93..51f58b5eda81ab 100644
--- a/llvm/lib/Support/UnicodeCharSetsGenerated.cpp
+++ b/llvm/lib/Support/UnicodeCharSetsGenerated.cpp
@@ -266,7 +266,8 @@ static constexpr UnicodeCharRange GeneratedCharacterRangesData[] = {
     {0x3D000, 0x3FC3F}, {0xE0001, 0xE0001}, {0xE0020, 0xE007F},
     {0xE0100, 0xE01EF}, {0xF0000, 0xFFFFD}, {0x100000, 0x10FFFD},
 };
-extern const CharRanges GeneratedCharacterRanges = GeneratedCharacterRangesData;
+LLVM_ABI extern const CharRanges GeneratedCharacterRanges =
+    GeneratedCharacterRangesData;
 
 // General_Category=L|M|N|P|S|Zs
 static constexpr UnicodeCharRange GeneratedPrintableRangesData[] = {
@@ -518,14 +519,15 @@ static constexpr UnicodeCharRange GeneratedPrintableRangesData[] = {
     {0x2F800, 0x2FA1D}, {0x30000, 0x3134A}, {0x31350, 0x33479},
     {0x3D000, 0x3FC3F}, {0xE0100, 0xE01EF},
 };
-extern const CharRanges GeneratedPrintableRanges = GeneratedPrintableRangesData;
+LLVM_ABI extern const CharRanges GeneratedPrintableRanges =
+    GeneratedPrintableRangesData;
 
 // General_Category=Cc
 static constexpr UnicodeCharRange GeneratedControlCharacterRangesData[] = {
     {0x0000, 0x001F},
     {0x007F, 0x009F},
 };
-extern const CharRanges GeneratedControlCharacterRanges =
+LLVM_ABI extern const CharRanges GeneratedControlCharacterRanges =
     GeneratedControlCharacterRangesData;
 
 // General_Category=Cf
@@ -538,7 +540,7 @@ static constexpr UnicodeCharRange GeneratedFormatCharacterRangesData[] = {
     {0x110CD, 0x110CD}, {0x13430, 0x1343F}, {0x1BCA0, 0x1BCA3},
     {0x1D173, 0x1D17A}, {0xE0001, 0xE0001}, {0xE0020, 0xE007F},
 };
-extern const CharRanges GeneratedFormatCharacterRanges =
+LLVM_ABI extern const CharRanges GeneratedFormatCharacterRanges =
     GeneratedFormatCharacterRangesData;
 
 // General_Category=Mn|Me
@@ -666,7 +668,7 @@ static constexpr UnicodeCharRange GeneratedCombiningCharacterRangesData[] = {
     {0x1E6EE, 0x1E6EF}, {0x1E6F5, 0x1E6F5}, {0x1E8D0, 0x1E8D6},
     {0x1E944, 0x1E94A}, {0xE0100, 0xE01EF},
 };
-extern const CharRanges GeneratedCombiningCharacterRanges =
+LLVM_ABI extern const CharRanges GeneratedCombiningCharacterRanges =
     GeneratedCombiningCharacterRangesData;
 
 // East_Asian_Width=F|W
@@ -714,7 +716,7 @@ static constexpr UnicodeCharRange GeneratedDoubleWidthCharacterRangesData[] = {
     {0x1FAC8, 0x1FAC8}, {0x1FACC, 0x1FADD}, {0x1FADF, 0x1FAEB},
     {0x1FAEF, 0x1FAFA}, {0x20000, 0x2FFFD}, {0x30000, 0x3FFFD},
 };
-extern const CharRanges GeneratedDoubleWidthCharacterRanges =
+LLVM_ABI extern const CharRanges GeneratedDoubleWidthCharacterRanges =
     GeneratedDoubleWidthCharacterRangesData;
 
 } // namespace llvm::sys::unicode
diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 65f23c3660b037..75343c98ce40b5 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -50,8 +50,8 @@ using namespace llvm;
 using namespace llvm::object;
 
 namespace llvm::sys::unicode {
-extern const UnicodeCharSet::CharRanges GeneratedCharacterRanges;
-extern const UnicodeCharSet::CharRanges GeneratedControlCharacterRanges;
+LLVM_ABI extern const UnicodeCharSet::CharRanges GeneratedCharacterRanges;
+LLVM_ABI extern const UnicodeCharSet::CharRanges GeneratedControlCharacterRanges;
 } // namespace llvm::sys::unicode
 
 namespace {
diff --git a/llvm/utils/UnicodeData/UnicodeCharSetsGenerator.cpp b/llvm/utils/UnicodeData/UnicodeCharSetsGenerator.cpp
index 58b305a4fa32ad..6a8c859a739a48 100644
--- a/llvm/utils/UnicodeData/UnicodeCharSetsGenerator.cpp
+++ b/llvm/utils/UnicodeData/UnicodeCharSetsGenerator.cpp
@@ -201,7 +201,8 @@ static void emitTable(raw_ostream &OS, StringRef Name, StringRef Description,
      << "Data[] = {\n";
   for (sys::UnicodeCharRange Range : Ranges)
     OS << formatv("    {{{0:X4}, {1:X4}},\n", Range.Lower, Range.Upper);
-  OS << "};\nextern const CharRanges " << Name << " = " << Name << "Data;\n";
+  OS << "};\nLLVM_ABI extern const CharRanges " << Name << " = " << Name
+     << "Data;\n";
 }
 
 static void writeFile(StringRef Path, StringRef Namespace, StringRef Version,

>From 3f86134c0c718268677c5bc525620df6c3515c9d Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Wed, 4 Nov 2026 15:39:22 +0000
Subject: [PATCH 36/41] clang-format a second time

---
 llvm/tools/llvm-strings/llvm-strings.cpp | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/llvm/tools/llvm-strings/llvm-strings.cpp b/llvm/tools/llvm-strings/llvm-strings.cpp
index 75343c98ce40b5..95d66b503cde83 100644
--- a/llvm/tools/llvm-strings/llvm-strings.cpp
+++ b/llvm/tools/llvm-strings/llvm-strings.cpp
@@ -51,7 +51,8 @@ using namespace llvm::object;
 
 namespace llvm::sys::unicode {
 LLVM_ABI extern const UnicodeCharSet::CharRanges GeneratedCharacterRanges;
-LLVM_ABI extern const UnicodeCharSet::CharRanges GeneratedControlCharacterRanges;
+LLVM_ABI extern const UnicodeCharSet::CharRanges
+    GeneratedControlCharacterRanges;
 } // namespace llvm::sys::unicode
 
 namespace {

>From 1513c85fce4f493e1d5cef80c61e5242c7bba6f1 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Wed, 4 Nov 2026 22:14:40 +0000
Subject: [PATCH 37/41] Keep LLVM_ABI out of the Clang files

---
 clang/lib/Lex/UnicodeCharSetsGenerated.cpp    | 15 ++---
 .../UnicodeData/UnicodeCharSetsGenerator.cpp  | 67 ++++++++++---------
 2 files changed, 43 insertions(+), 39 deletions(-)

diff --git a/clang/lib/Lex/UnicodeCharSetsGenerated.cpp b/clang/lib/Lex/UnicodeCharSetsGenerated.cpp
index b8e06083eb7145..f1967d496d9da6 100644
--- a/clang/lib/Lex/UnicodeCharSetsGenerated.cpp
+++ b/clang/lib/Lex/UnicodeCharSetsGenerated.cpp
@@ -255,8 +255,7 @@ static constexpr UnicodeCharRange GeneratedXIDStartRangesData[] = {
     {0x2CEB0, 0x2EBE0}, {0x2EBF0, 0x2EE5D}, {0x2F800, 0x2FA1D},
     {0x30000, 0x3134A}, {0x31350, 0x33479}, {0x3D000, 0x3FC3F},
 };
-LLVM_ABI extern const CharRanges GeneratedXIDStartRanges =
-    GeneratedXIDStartRangesData;
+extern const CharRanges GeneratedXIDStartRanges = GeneratedXIDStartRangesData;
 
 // XID_Continue, excluding XID_Start
 static constexpr UnicodeCharRange GeneratedXIDContinueRangesData[] = {
@@ -397,7 +396,7 @@ static constexpr UnicodeCharRange GeneratedXIDContinueRangesData[] = {
     {0x1E8D0, 0x1E8D6}, {0x1E944, 0x1E94A}, {0x1E950, 0x1E959},
     {0x1FBF0, 0x1FBF9}, {0xE0100, 0xE01EF},
 };
-LLVM_ABI extern const CharRanges GeneratedXIDContinueRanges =
+extern const CharRanges GeneratedXIDContinueRanges =
     GeneratedXIDContinueRangesData;
 
 // ID_Compat_Math_Start
@@ -409,9 +408,8 @@ static constexpr UnicodeCharRange
         {0x1D76F, 0x1D76F}, {0x1D789, 0x1D789}, {0x1D7A9, 0x1D7A9},
         {0x1D7C3, 0x1D7C3},
 };
-LLVM_ABI extern const CharRanges
-    GeneratedMathematicalNotationProfileIDStartRanges =
-        GeneratedMathematicalNotationProfileIDStartRangesData;
+extern const CharRanges GeneratedMathematicalNotationProfileIDStartRanges =
+    GeneratedMathematicalNotationProfileIDStartRangesData;
 
 // ID_Compat_Math_Continue, excluding ID_Compat_Math_Start
 static constexpr UnicodeCharRange
@@ -419,8 +417,7 @@ static constexpr UnicodeCharRange
         {0x00B2, 0x00B3}, {0x00B9, 0x00B9}, {0x2070, 0x2070},
         {0x2074, 0x207E}, {0x2080, 0x208E},
 };
-LLVM_ABI extern const CharRanges
-    GeneratedMathematicalNotationProfileIDContinueRanges =
-        GeneratedMathematicalNotationProfileIDContinueRangesData;
+extern const CharRanges GeneratedMathematicalNotationProfileIDContinueRanges =
+    GeneratedMathematicalNotationProfileIDContinueRangesData;
 
 } // namespace clang
diff --git a/llvm/utils/UnicodeData/UnicodeCharSetsGenerator.cpp b/llvm/utils/UnicodeData/UnicodeCharSetsGenerator.cpp
index 6a8c859a739a48..b5777ff06b6df7 100644
--- a/llvm/utils/UnicodeData/UnicodeCharSetsGenerator.cpp
+++ b/llvm/utils/UnicodeData/UnicodeCharSetsGenerator.cpp
@@ -186,7 +186,8 @@ static CharacterDatabase loadDatabase(StringRef Path) {
 
 template <typename Predicate>
 static void emitTable(raw_ostream &OS, StringRef Name, StringRef Description,
-                      ArrayRef<Properties> Chars, Predicate Pred) {
+                      ArrayRef<Properties> Chars, Predicate Pred,
+                      bool LlvmAbi) {
   SmallVector<sys::UnicodeCharRange> Ranges;
   for (unsigned CodePoint = 0, E = Chars.size(); CodePoint != E; ++CodePoint) {
     if (!Pred(Chars[CodePoint]))
@@ -201,19 +202,25 @@ static void emitTable(raw_ostream &OS, StringRef Name, StringRef Description,
      << "Data[] = {\n";
   for (sys::UnicodeCharRange Range : Ranges)
     OS << formatv("    {{{0:X4}, {1:X4}},\n", Range.Lower, Range.Upper);
-  OS << "};\nLLVM_ABI extern const CharRanges " << Name << " = " << Name
-     << "Data;\n";
+  OS << "};\n";
+  if (LlvmAbi)
+    OS << "LLVM_ABI ";
+  OS << "extern const CharRanges " << Name << " = " << Name << "Data;\n";
 }
 
+template <typename EmitTablesT>
 static void writeFile(StringRef Path, StringRef Namespace, StringRef Version,
-                      function_ref<void(raw_ostream &)> EmitTables) {
+                      ArrayRef<Properties> Chars, bool LlvmAbi,
+                      EmitTablesT &&EmitTables) {
   std::error_code EC;
   ToolOutputFile Out(Path, EC, sys::fs::OF_Text);
   if (EC)
     error(Path + ": " + EC.message());
 
   Out.os() << formatv(FileHeader.drop_front().data(), Version, Namespace);
-  EmitTables(Out.os());
+  EmitTables([&](StringRef Name, StringRef Description, auto &&Pred) {
+    emitTable(Out.os(), Name, Description, Chars, Pred, LlvmAbi);
+  });
   Out.os() << "\n} // namespace " << Namespace << '\n';
   Out.keep();
 }
@@ -232,35 +239,35 @@ int main(int argc, char **argv) {
   CharacterDatabase UCD = loadDatabase(UCDFile);
   ArrayRef<Properties> Chars = UCD.Chars;
 
-  writeFile(ClangOutput, "clang", UCD.Version, [&](raw_ostream &OS) {
-    emitTable(OS, "GeneratedXIDStartRanges", "XID_Start", Chars,
-              [](Properties P) { return P.XIDStart; });
-    emitTable(OS, "GeneratedXIDContinueRanges",
-              "XID_Continue, excluding XID_Start", Chars,
-              [](Properties P) { return P.XIDContinue && !P.XIDStart; });
-    emitTable(OS, "GeneratedMathematicalNotationProfileIDStartRanges",
-              "ID_Compat_Math_Start", Chars,
-              [](Properties P) { return P.MathStart; });
-    emitTable(OS, "GeneratedMathematicalNotationProfileIDContinueRanges",
-              "ID_Compat_Math_Continue, excluding ID_Compat_Math_Start", Chars,
-              [](Properties P) { return P.MathContinue && !P.MathStart; });
-  });
   writeFile(
-      LLVMOutput, "llvm::sys::unicode", UCD.Version, [&](raw_ostream &OS) {
-        emitTable(OS, "GeneratedCharacterRanges", "Characters", Chars,
+      ClangOutput, "clang", UCD.Version, Chars, /*LlvmAbi=*/false,
+      [&](auto &&EmitTable) {
+        EmitTable("GeneratedXIDStartRanges", "XID_Start",
+                  [](Properties P) { return P.XIDStart; });
+        EmitTable("GeneratedXIDContinueRanges",
+                  "XID_Continue, excluding XID_Start",
+                  [](Properties P) { return P.XIDContinue && !P.XIDStart; });
+        EmitTable("GeneratedMathematicalNotationProfileIDStartRanges",
+                  "ID_Compat_Math_Start",
+                  [](Properties P) { return P.MathStart; });
+        EmitTable("GeneratedMathematicalNotationProfileIDContinueRanges",
+                  "ID_Compat_Math_Continue, excluding ID_Compat_Math_Start",
+                  [](Properties P) { return P.MathContinue && !P.MathStart; });
+      });
+  writeFile(
+      LLVMOutput, "llvm::sys::unicode", UCD.Version, Chars, /*LlvmAbi=*/true,
+      [&](auto &&EmitTable) {
+        EmitTable("GeneratedCharacterRanges", "Characters",
                   [](Properties P) { return P.Char; });
-        emitTable(OS, "GeneratedPrintableRanges",
-                  "General_Category=L|M|N|P|S|Zs", Chars,
+        EmitTable("GeneratedPrintableRanges", "General_Category=L|M|N|P|S|Zs",
                   [](Properties P) { return P.Printable; });
-        emitTable(OS, "GeneratedControlCharacterRanges", "General_Category=Cc",
-                  Chars, [](Properties P) { return P.Control; });
-        emitTable(OS, "GeneratedFormatCharacterRanges", "General_Category=Cf",
-                  Chars, [](Properties P) { return P.Format; });
-        emitTable(OS, "GeneratedCombiningCharacterRanges",
-                  "General_Category=Mn|Me", Chars,
+        EmitTable("GeneratedControlCharacterRanges", "General_Category=Cc",
+                  [](Properties P) { return P.Control; });
+        EmitTable("GeneratedFormatCharacterRanges", "General_Category=Cf",
+                  [](Properties P) { return P.Format; });
+        EmitTable("GeneratedCombiningCharacterRanges", "General_Category=Mn|Me",
                   [](Properties P) { return P.Combining; });
-        emitTable(OS, "GeneratedDoubleWidthCharacterRanges",
-                  "East_Asian_Width=F|W", Chars,
+        EmitTable("GeneratedDoubleWidthCharacterRanges", "East_Asian_Width=F|W",
                   [](Properties P) { return P.DoubleWidth; });
       });
 }

>From 75f5c4bb461721d4d459a394ac6a1084e83a74e3 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Thu, 5 Nov 2026 20:34:34 +0000
Subject: [PATCH 38/41] More echo

---
 llvm/test/tools/llvm-strings/negative-char.test | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/llvm/test/tools/llvm-strings/negative-char.test b/llvm/test/tools/llvm-strings/negative-char.test
index d738f2fcb0b999..71f9a860ca9e41 100644
--- a/llvm/test/tools/llvm-strings/negative-char.test
+++ b/llvm/test/tools/llvm-strings/negative-char.test
@@ -2,8 +2,9 @@
 ## Such characters should form string delimiters like other unprintable ones,
 ## unless using an encoding where they form valid characters.
 
-# RUN: printf "z\0\200\0\302\241\0a\0" | llvm-strings --encoding=s --bytes 1 - | FileCheck %s
-# RUN: printf "z\0\200\0\302\241\0a\0" | llvm-strings --encoding=utf8 --bytes 1 - | FileCheck %s --check-prefixes=CHECK,CHECK-UTF8
+# RUN: echo -n -e "z\0\200\0\302\241\0a\0" > %t
+# RUN: llvm-strings --encoding=s --bytes 1 %t | FileCheck %s
+# RUN: llvm-strings --encoding=utf8 --bytes 1 %t | FileCheck %s --check-prefixes=CHECK,CHECK-UTF8
 # CHECK: z{{$}}
 # CHECK-UTF8-NEXT: {{^\xc2\xa1$}}
 # CHECK-NEXT: {{^}}a

>From e145f17c41b7a09b4cbabb34d31c167a9feb0dd3 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Thu, 5 Nov 2026 21:36:30 +0000
Subject: [PATCH 39/41] Even more echo

---
 llvm/test/tools/llvm-strings/length.test | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/llvm/test/tools/llvm-strings/length.test b/llvm/test/tools/llvm-strings/length.test
index 79b7a324df6e9a..f17a304b89a0fc 100644
--- a/llvm/test/tools/llvm-strings/length.test
+++ b/llvm/test/tools/llvm-strings/length.test
@@ -46,9 +46,9 @@ RUN: not llvm-strings -n foo %t 2>&1 | FileCheck %s --check-prefix=ERR
 ERR: error: expected a positive integer, but got 'foo'
 
 ## Show that the count is number of characters, not number of bytes.
-RUN: printf '\302\241\n' > %t
-RUN: printf '\302\241\302\242\n' >> %t
-RUN: printf '\302\241\302\242\302\243\n' >> %t
+RUN: echo -e '\302\241' > %t
+RUN: echo -e '\302\241\302\242' >> %t
+RUN: echo -e '\302\241\302\242\302\243' >> %t
 RUN: not llvm-strings --encoding utf8 -n 0 2>&1 %t | FileCheck --check-prefix CHECK-0 %s
 RUN: llvm-strings --encoding utf8 -n 1 %t | FileCheck --check-prefix UTF-8-1 %s --implicit-check-not={{.}}
 RUN: llvm-strings --encoding utf8 -n 2 %t | FileCheck --check-prefix UTF-8-2 %s --implicit-check-not={{.}}

>From 74929ae079ca51b6f34acc298850f90c4cc258af Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Fri, 6 Nov 2026 12:05:34 +0000
Subject: [PATCH 40/41] Add AIX name for SJIS to test too

---
 llvm/test/tools/llvm-strings/lit.local.cfg | 1 +
 llvm/test/tools/llvm-strings/locale.test   | 4 ++++
 2 files changed, 5 insertions(+)

diff --git a/llvm/test/tools/llvm-strings/lit.local.cfg b/llvm/test/tools/llvm-strings/lit.local.cfg
index 4bbad77f3c7e2d..2f63209db7bb6b 100644
--- a/llvm/test/tools/llvm-strings/lit.local.cfg
+++ b/llvm/test/tools/llvm-strings/lit.local.cfg
@@ -6,6 +6,7 @@ for l in (
     "en_US.ISO8859-1",
     "en_US.UTF-8",
     "ja_JP.SJIS",
+    "Ja_JP.IBM-943",
     "ja_JP.ISO2022-JP",
 ):
     try:
diff --git a/llvm/test/tools/llvm-strings/locale.test b/llvm/test/tools/llvm-strings/locale.test
index cc75b3d470abde..2f8c92857d3e60 100644
--- a/llvm/test/tools/llvm-strings/locale.test
+++ b/llvm/test/tools/llvm-strings/locale.test
@@ -20,6 +20,10 @@ RUN: %if locale-ja_JP.SJIS %{ env LANG=ja_JP.SJIS \
 RUN: llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-SJIS %s %} \
 RUN: %else %{ echo ja_JP.SJIS locale not supported, skipping test %}
 
+RUN: %if locale-Ja_JP.IBM-943 %{ env LANG=Ja_JP.IBM-943 \
+RUN: llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-SJIS %s %} \
+RUN: %else %{ echo Ja_JP.IBM-943 locale not supported, skipping test %}
+
 RUN: %if locale-ja_JP.ISO2022-JP %{ env LANG=ja_JP.ISO2022-JP \
 RUN: llvm-strings -eS %t | FileCheck --match-full-lines --strict-whitespace --check-prefixes CHECK-IN-ISO2022-JP %s %} \
 RUN: %else %{ echo ja_JP.ISO2022-JP locale not supported, skipping test %}

>From b2b5c02ba7b094859e8c95d8ae4ba5dcc05200a9 Mon Sep 17 00:00:00 2001
From: Harald van Dijk <hdijk at accesssoftek.com>
Date: Fri, 6 Nov 2026 12:48:17 +0000
Subject: [PATCH 41/41] Add comment explaining IBM-943

---
 llvm/test/tools/llvm-strings/lit.local.cfg | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/llvm/test/tools/llvm-strings/lit.local.cfg b/llvm/test/tools/llvm-strings/lit.local.cfg
index 2f63209db7bb6b..c37e131b9c4edc 100644
--- a/llvm/test/tools/llvm-strings/lit.local.cfg
+++ b/llvm/test/tools/llvm-strings/lit.local.cfg
@@ -6,7 +6,7 @@ for l in (
     "en_US.ISO8859-1",
     "en_US.UTF-8",
     "ja_JP.SJIS",
-    "Ja_JP.IBM-943",
+    "Ja_JP.IBM-943",  # AIX name for SJIS
     "ja_JP.ISO2022-JP",
 ):
     try:



More information about the cfe-commits mailing list