[libcxx-commits] [libcxx] [libc++] Specify `encoding='utf-8'` in `conformance` script (PR #189534)

[via libcxx-commits]

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-libcxx

Author: A. Jiang (frederick-vs-ja)

<details>
<summary>Changes</summary>

In some environments, especially Windows with active code page other than 65001, some characters in the CSV files cause parse errors without explicitly specifying the UTF-8 encoding.

Explicitly using `encoding='utf-8'` in `click.File` fixes such errors.

---
Full diff: https://github.com/llvm/llvm-project/pull/189534.diff


1 Files Affected:

- (modified) libcxx/utils/conformance (+9-9) 


``````````diff
diff --git a/libcxx/utils/conformance b/libcxx/utils/conformance
index 20750204fce50..ae8093c0dd7ce 100755
--- a/libcxx/utils/conformance
+++ b/libcxx/utils/conformance
@@ -496,8 +496,8 @@ def github_link(issues, meeting_voted, status):
         link_github_issue(issue, meeting_voted, status)
 
 @github_cmd.command('create')
- at click.argument('csvfiles', nargs=-1, type=click.File('r'))
- at click.option('--load-github-from', type=click.File('r'), required=False,
+ at click.argument('csvfiles', nargs=-1, type=click.File('r', encoding='utf-8'))
+ at click.option('--load-github-from', type=click.File('r', encoding='utf-8'), required=False,
     help='Optionally load the Github Conformance tracking data from the given file. Useful to avoid '
          'running into API call limits when testing locally. See `conformance github download`.')
 @click.option('--labels', type=str, multiple=True,
@@ -565,7 +565,7 @@ def github_create(csvfiles, load_github_from, labels):
 
 
 @github_cmd.command('download')
- at click.option('--output', '-o', type=click.File('w'), default='-',
+ at click.option('--output', '-o', type=click.File('w', encoding='utf-8'), default='-',
     help='Save the data to the provided file. Default is stdout.')
 def github_download(output):
     """Download libc++ conformance tracking data from Github issues and save it locally."""
@@ -580,7 +580,7 @@ def csv_cmd():
 
 
 @csv_cmd.command('validate')
- at click.argument('files', nargs=-1, type=click.File('r'))
+ at click.argument('files', nargs=-1, type=click.File('r', encoding='utf-8'))
 def csv_validate(files):
     """Validate the format of the provided CSV files."""
     for file in files:
@@ -591,10 +591,10 @@ def csv_validate(files):
 
 
 @csv_cmd.command('synchronize')
- at click.argument('files', nargs=-1, type=click.File('r'))
- at click.option('--output', '-o', type=click.File('w'), default='-',
+ at click.argument('files', nargs=-1, type=click.File('r', encoding='utf-8'))
+ at click.option('--output', '-o', type=click.File('w', encoding='utf-8'), default='-',
     help='Write the generated CSV to the specified file. Defaults to stdout.')
- at click.option('--load-github-from', type=click.File('r'), required=False,
+ at click.option('--load-github-from', type=click.File('r', encoding='utf-8'), required=False,
     help='Optionally load the Github Conformance tracking data from the given file. Useful to avoid '
          'running into API call limits when testing locally. See `conformance github download`.')
 def csv_synchronize(files, output, load_github_from):
@@ -615,9 +615,9 @@ def csv_synchronize(files, output, load_github_from):
 
 @csv_cmd.command('create')
 @click.argument('issues', nargs=-1)
- at click.option('--output', '-o', type=click.File('w'), default='-',
+ at click.option('--output', '-o', type=click.File('w', encoding='utf-8'), default='-',
     help='Write the generated CSV to the specified file. Defaults to stdout.')
- at click.option('--load-github-from', type=click.File('r'), required=False,
+ at click.option('--load-github-from', type=click.File('r', encoding='utf-8'), required=False,
     help='Optionally load the Github Conformance tracking data from the given file. Useful to avoid '
          'running into API call limits when testing locally. See `conformance github download`.')
 def csv_create(issues, output, load_github_from):

``````````

</details>


https://github.com/llvm/llvm-project/pull/189534