[clang] f385542 - [Tooling/Inclusion] Modify the Python script to open the C++ reference with UTF-8 encoding. (#121341)

[via cfe-commits]

Author: c8ef
Date: 2024-12-31T09:28:34+08:00
New Revision: f385542f62fa1f57001c95c476165e1618cb54ba

URL: https://github.com/llvm/llvm-project/commit/f385542f62fa1f57001c95c476165e1618cb54ba
DIFF: https://github.com/llvm/llvm-project/commit/f385542f62fa1f57001c95c476165e1618cb54ba.diff

LOG: [Tooling/Inclusion] Modify the Python script to open the C++ reference with UTF-8 encoding. (#121341)

This will prevent the error on systems with a default encoding other
than utf-8.

```
UnicodeDecodeError: 'gbk' codec can't decode byte 0xb6 in position 12958: illegal multibyte sequence
```

Added: 
    

Modified: 
    clang/tools/include-mapping/cppreference_parser.py

Removed: 
    


################################################################################
diff  --git a/clang/tools/include-mapping/cppreference_parser.py b/clang/tools/include-mapping/cppreference_parser.py
index 9101f3dbff0f94..f7da2ba8bb6d84 100644
--- a/clang/tools/include-mapping/cppreference_parser.py
+++ b/clang/tools/include-mapping/cppreference_parser.py
@@ -139,7 +139,7 @@ def _ParseIndexPage(index_page_html):
 
 
 def _ReadSymbolPage(path, name, qual_name):
-    with open(path) as f:
+    with open(path, encoding="utf-8") as f:
         return _ParseSymbolPage(f.read(), name, qual_name)
 
 
@@ -156,7 +156,7 @@ def _GetSymbols(pool, root_dir, index_page_name, namespace, variants_to_accept):
     #      contains the defined header.
     #   2. Parse the symbol page to get the defined header.
     index_page_path = os.path.join(root_dir, index_page_name)
-    with open(index_page_path, "r") as f:
+    with open(index_page_path, "r", encoding="utf-8") as f:
         # Read each symbol page in parallel.
         results = []  # (symbol_name, promise of [header...])
         for symbol_name, symbol_page_path, variant in _ParseIndexPage(f.read()):