[llvm] r329022 - [lit] Fix problem in how Python versions open files with different encodings
Aaron Smith via llvm-commits
llvm-commits at lists.llvm.org
Mon Apr 2 18:21:15 PDT 2018
Your Windows bot is good now.
http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/builds/16216
On 4/2/18, 5:19 PM, "douglas.yung at sony.com" <douglas.yung at sony.com> wrote:
Hi Aaron,
I don't know if you are still working on this, but the PS4 Windows bot still seems to be failing.
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flab.llvm.org%3A8011%2Fbuilders%2Fllvm-clang-lld-x86_64-scei-ps4-windows10pro-fast%2Fbuilds%2F16215&data=02%7C01%7Caaron.smith%40microsoft.com%7C8410e12b0f3f49cf7d1308d598f89f6d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636583115959611850&sdata=F1k%2FZszjahxmMLOvWADr0xultK%2FSUChzl1IWjv6Waew%3D&reserved=0
UNRESOLVED: Clang :: Modules/Werror-Wsystem-headers.m (8206 of 39290)
******************** TEST 'Clang :: Modules/Werror-Wsystem-headers.m' FAILED ********************
Exception during script execution:
Traceback (most recent call last):
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\run.py", line 202, in _execute_test_impl
result = test.config.test_format.execute(test, lit_config)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\formats\shtest.py", line 25, in execute
self.execute_external)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 1537, in executeShTest
res = _runShTest(test, litConfig, useExternalSh, script, tmpBase)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 1485, in _runShTest
res = executeScriptInternal(test, litConfig, tmpBase, script, execdir)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 997, in executeScriptInternal
exitCode, timeoutInfo = executeShCmd(cmd, shenv, results, timeout=litConfig.maxIndividualTestTime)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 152, in executeShCmd
finalExitCode = _executeShCmd(cmd, shenv, results, timeoutHelper)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 720, in _executeShCmd
res = _executeShCmd(cmd.lhs, shenv, results, timeoutHelper)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 720, in _executeShCmd
res = _executeShCmd(cmd.lhs, shenv, results, timeoutHelper)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 725, in _executeShCmd
res = _executeShCmd(cmd.rhs, shenv, results, timeoutHelper)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 778, in _executeShCmd
cmdResult = executeBuiltinDiff(cmd.commands[0], shenv)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 548, in executeBuiltinDiff
exitCode = compareTwoFiles(filepaths)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 411, in compareTwoFiles
return compareTwoTextFiles(filepaths, encoding)
File "C:/ps4-buildslave2/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/llvm.src\utils\lit\lit\TestRunner.py", line 438, in compareTwoTextFiles
filelines.append(f.readlines())
File "C:\Python27\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 44: character maps to <undefined>
Can you take a look?
Douglas Yung
> -----Original Message-----
> From: llvm-commits [mailto:llvm-commits-bounces at lists.llvm.org] On Behalf Of
> Aaron Smith via llvm-commits
> Sent: Monday, April 02, 2018 15:09
> To: llvm-commits at lists.llvm.org
> Subject: [llvm] r329022 - [lit] Fix problem in how Python versions open files
> with different encodings
>
> Author: asmith
> Date: Mon Apr 2 15:08:56 2018
> New Revision: 329022
>
> URL: https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fllvm.org%2Fviewvc%2Fllvm-project%3Frev%3D329022%26view%3Drev&data=02%7C01%7Caaron.smith%40microsoft.com%7C8410e12b0f3f49cf7d1308d598f89f6d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636583115959611850&sdata=48Ap866YXiWKlO0Ytc4jgZYVh9kN3V633J375vymhUc%3D&reserved=0
> Log:
> [lit] Fix problem in how Python versions open files with different encodings
>
> Reapply D43165 which was reverted because of different versions of python
> failing.
> The one line fix for the different python versions was commited at the same
> time that D43165 was reverted. If this change is giving you issues then get in
> touch with your python version and we will fix it.
>
> Modified:
> llvm/trunk/utils/lit/lit/TestRunner.py
>
> Modified: llvm/trunk/utils/lit/lit/TestRunner.py
> URL: https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fllvm.org%2Fviewvc%2Fllvm-&data=02%7C01%7Caaron.smith%40microsoft.com%7C8410e12b0f3f49cf7d1308d598f89f6d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636583115959611850&sdata=uFndL6LctMk4SGrgR3qllhb8%2BShSgC3syYfxpn60yNQ%3D&reserved=0
> project/llvm/trunk/utils/lit/lit/TestRunner.py?rev=329022&r1=329021&r2=329022&
> view=diff
> ==============================================================================
> --- llvm/trunk/utils/lit/lit/TestRunner.py (original)
> +++ llvm/trunk/utils/lit/lit/TestRunner.py Mon Apr 2 15:08:56 2018
> @@ -388,7 +388,47 @@ def executeBuiltinDiff(cmd, cmd_shenv):
> def compareTwoFiles(filepaths):
> filelines = []
> for file in filepaths:
> - with io.open(file, 'r') as f:
> + try:
> + with open(file, 'r') as f:
> + filelines.append(f.readlines())
> + except UnicodeDecodeError:
> + try:
> + with open(file, 'r', encoding="utf-8") as f:
> + filelines.append(f.readlines())
> + encoding = "utf-8"
> + except:
> + compare_bytes = True
> +
> + if compare_bytes:
> + return compareTwoBinaryFiles(filepaths)
> + else:
> + return compareTwoTextFiles(filepaths, encoding)
> +
> + def compareTwoBinaryFiles(filepaths):
> + filelines = []
> + for file in filepaths:
> + with open(file, 'rb') as f:
> + filelines.append(f.readlines())
> +
> + exitCode = 0
> + if hasattr(difflib, 'diff_bytes'):
> + # python 3.5 or newer
> + diffs = difflib.diff_bytes(difflib.unified_diff, filelines[0],
> filelines[1], filepaths[0].encode(), filepaths[1].encode())
> + diffs = [diff.decode() for diff in diffs]
> + else:
> + # python 2.7
> + func = difflib.unified_diff if unified_diff else
> difflib.context_diff
> + diffs = func(filelines[0], filelines[1], filepaths[0],
> + filepaths[1])
> +
> + for diff in diffs:
> + stdout.write(diff)
> + exitCode = 1
> + return exitCode
> +
> + def compareTwoTextFiles(filepaths, encoding):
> + filelines = []
> + for file in filepaths:
> + with io.open(file, 'r', encoding=encoding) as f:
> filelines.append(f.readlines())
>
> exitCode = 0
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.llvm.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fllvm-commits&data=02%7C01%7Caaron.smith%40microsoft.com%7C8410e12b0f3f49cf7d1308d598f89f6d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636583115959611850&sdata=SZQhyk5vxFV8EmVqWTdamdZKGKYww7RyrNL8mpjSHXA%3D&reserved=0
More information about the llvm-commits
mailing list