[llvm] r329022 - [lit] Fix problem in how Python versions open files with different encodings
via llvm-commits
llvm-commits at lists.llvm.org
Mon Apr 2 17:19:49 PDT 2018
Hi Aaron,
I don't know if you are still working on this, but the PS4 Windows bot still seems to be failing.
http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/builds/16215
UNRESOLVED: Clang :: Modules/Werror-Wsystem-headers.m (8206 of 39290)
******************** TEST 'Clang :: Modules/Werror-Wsystem-headers.m' FAILED ********************
Exception during script execution:
Traceback (most recent call last):
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\run.py", line 202, in _execute_test_impl
result = test.config.test_format.execute(test, lit_config)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\formats\shtest.py", line 25, in execute
self.execute_external)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 1537, in executeShTest
res = _runShTest(test, litConfig, useExternalSh, script, tmpBase)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 1485, in _runShTest
res = executeScriptInternal(test, litConfig, tmpBase, script, execdir)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 997, in executeScriptInternal
exitCode, timeoutInfo = executeShCmd(cmd, shenv, results, timeout=litConfig.maxIndividualTestTime)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 152, in executeShCmd
finalExitCode = _executeShCmd(cmd, shenv, results, timeoutHelper)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 720, in _executeShCmd
res = _executeShCmd(cmd.lhs, shenv, results, timeoutHelper)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 720, in _executeShCmd
res = _executeShCmd(cmd.lhs, shenv, results, timeoutHelper)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 725, in _executeShCmd
res = _executeShCmd(cmd.rhs, shenv, results, timeoutHelper)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 778, in _executeShCmd
cmdResult = executeBuiltinDiff(cmd.commands[0], shenv)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 548, in executeBuiltinDiff
exitCode = compareTwoFiles(filepaths)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 411, in compareTwoFiles
return compareTwoTextFiles(filepaths, encoding)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 438, in compareTwoTextFiles
filelines.append(f.readlines())
File "C:\Python27\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 44: character maps to <undefined>
Can you take a look?
Douglas Yung
> -----Original Message-----
> From: llvm-commits [mailto:llvm-commits-bounces at lists.llvm.org] On Behalf Of
> Aaron Smith via llvm-commits
> Sent: Monday, April 02, 2018 15:09
> To: llvm-commits at lists.llvm.org
> Subject: [llvm] r329022 - [lit] Fix problem in how Python versions open files
> with different encodings
>
> Author: asmith
> Date: Mon Apr 2 15:08:56 2018
> New Revision: 329022
>
> URL: http://llvm.org/viewvc/llvm-project?rev=329022&view=rev
> Log:
> [lit] Fix problem in how Python versions open files with different encodings
>
> Reapply D43165 which was reverted because of different versions of python
> failing.
> The one line fix for the different python versions was commited at the same
> time that D43165 was reverted. If this change is giving you issues then get in
> touch with your python version and we will fix it.
>
> Modified:
> llvm/trunk/utils/lit/lit/TestRunner.py
>
> Modified: llvm/trunk/utils/lit/lit/TestRunner.py
> URL: http://llvm.org/viewvc/llvm-
> project/llvm/trunk/utils/lit/lit/TestRunner.py?rev=329022&r1=329021&r2=329022&
> view=diff
> ==============================================================================
> --- llvm/trunk/utils/lit/lit/TestRunner.py (original)
> +++ llvm/trunk/utils/lit/lit/TestRunner.py Mon Apr 2 15:08:56 2018
> @@ -388,7 +388,47 @@ def executeBuiltinDiff(cmd, cmd_shenv):
> def compareTwoFiles(filepaths):
> filelines = []
> for file in filepaths:
> - with io.open(file, 'r') as f:
> + try:
> + with open(file, 'r') as f:
> + filelines.append(f.readlines())
> + except UnicodeDecodeError:
> + try:
> + with open(file, 'r', encoding="utf-8") as f:
> + filelines.append(f.readlines())
> + encoding = "utf-8"
> + except:
> + compare_bytes = True
> +
> + if compare_bytes:
> + return compareTwoBinaryFiles(filepaths)
> + else:
> + return compareTwoTextFiles(filepaths, encoding)
> +
> + def compareTwoBinaryFiles(filepaths):
> + filelines = []
> + for file in filepaths:
> + with open(file, 'rb') as f:
> + filelines.append(f.readlines())
> +
> + exitCode = 0
> + if hasattr(difflib, 'diff_bytes'):
> + # python 3.5 or newer
> + diffs = difflib.diff_bytes(difflib.unified_diff, filelines[0],
> filelines[1], filepaths[0].encode(), filepaths[1].encode())
> + diffs = [diff.decode() for diff in diffs]
> + else:
> + # python 2.7
> + func = difflib.unified_diff if unified_diff else
> difflib.context_diff
> + diffs = func(filelines[0], filelines[1], filepaths[0],
> + filepaths[1])
> +
> + for diff in diffs:
> + stdout.write(diff)
> + exitCode = 1
> + return exitCode
> +
> + def compareTwoTextFiles(filepaths, encoding):
> + filelines = []
> + for file in filepaths:
> + with io.open(file, 'r', encoding=encoding) as f:
> filelines.append(f.readlines())
>
> exitCode = 0
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
More information about the llvm-commits
mailing list