[PATCH] D76768: [analyzer] Added support of scan-build and exploded-graph-rewriter regression tests for Windows
Artem Dergachev via Phabricator via cfe-commits
cfe-commits at lists.llvm.org
Tue Apr 7 07:00:56 PDT 2020
NoQ added a comment.
In D76768#1966813 <https://reviews.llvm.org/D76768#1966813>, @thakis wrote:
> This seems to break tests on Windows: http://45.33.8.238/win/12320/step_7.txt
Thank you! Reverted.
================
Comment at: clang/test/Analysis/scan-build/exclude_directories.test:1-2
-// FIXME: Actually, "perl".
-REQUIRES: shell
-
----------------
ASDenysPetrov wrote:
> Charusso wrote:
> > NoQ wrote:
> > > Well, let's hope this actually works :)
> > +1 beware.
> As I figured out this might protect us from crashes (from lit.config.cfg):
>
> ```
> use_lit_shell = os.environ.get("LIT_USE_INTERNAL_SHELL")
> config.test_format = lit.formats.ShTest(use_lit_shell == "0")
> ```
Ok, so i only reverted the part where `REQUIRES: shell` is removed (rGcfd388d344ab) because that's when the buildbot problems started. The part that removes `UNSUPPORTED: system-windows` (rG2ddd3325c4d7) is still in. So i guess `REQUIRES: shell` can't be //entirely// removed. The error doesn't look like it's about not being able to run scan-build though; scan-build started just fine and produced some output. So it's likely a smaller bug.
@ASDenysPetrov any immediate thoughts? Do tests actually run on your machine when `REQUIRES: shell` is still in? I.e., do we get at least some testing on user's machines on the current trunk after my latest revert?
Here's a snapshot of the failure:
```lines=20
ninja: Entering directory `out/gn'
[0/2] ACTION //clang/test:check-clang(//llvm/utils/gn/build/toolchain:win)
llvm-lit.py: C:/src/llvm-project/llvm\utils\lit\lit\llvm\config.py:343: note: using clang: c:\src\llvm-project\out\gn\bin\clang.exe
-- Testing: 17064 tests, 32 workers --
Testing: 0
FAIL: Clang :: Analysis/scan-build/html_output.test (974 of 17064)
******************** TEST 'Clang :: Analysis/scan-build/html_output.test' FAILED ********************
Script:
--
: 'RUN: at line 1'; rm -rf C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\html_output.test.tmp.output_dir && mkdir C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\html_output.test.tmp.output_dir
: 'RUN: at line 2'; 'c:\src\llvm-project\clang\tools\scan-build\bin\scan-build.bat' --use-analyzer=c:\src\llvm-project\out\gn\bin\clang.exe -o C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\html_output.test.tmp.output_dir c:\src\llvm-project\out\gn\bin\clang.exe -S C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/single_null_dereference.c | c:\src\llvm-project\out\gn\bin\filecheck.exe C:\src\llvm-project\clang\test\Analysis\scan-build\html_output.test -check-prefix CHECK-STDOUT
: 'RUN: at line 12'; ls C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\html_output.test.tmp.output_dir/*/ | c:\src\llvm-project\out\gn\bin\filecheck.exe C:\src\llvm-project\clang\test\Analysis\scan-build\html_output.test -check-prefix CHECK-FILENAMES
: 'RUN: at line 21'; cat C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\html_output.test.tmp.output_dir/*/index.html | c:\src\llvm-project\out\gn\bin\filecheck.exe C:\src\llvm-project\clang\test\Analysis\scan-build\html_output.test -check-prefix CHECK-INDEX-HTML
: 'RUN: at line 27'; cat C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\html_output.test.tmp.output_dir/*/report-*.html | c:\src\llvm-project\out\gn\bin\filecheck.exe C:\src\llvm-project\clang\test\Analysis\scan-build\html_output.test -check-prefix CHECK-REPORT-HTML
--
Exit Code: 1
Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "rm" "-rf" "C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\html_output.test.tmp.output_dir"
$ "mkdir" "C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\html_output.test.tmp.output_dir"
$ ":" "RUN: at line 2"
$ "c:\src\llvm-project\clang\tools\scan-build\bin\scan-build.bat" "--use-analyzer=c:\src\llvm-project\out\gn\bin\clang.exe" "-o" "C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\html_output.test.tmp.output_dir" "c:\src\llvm-project\out\gn\bin\clang.exe" "-S" "C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/single_null_dereference.c"
$ "c:\src\llvm-project\out\gn\bin\filecheck.exe" "C:\src\llvm-project\clang\test\Analysis\scan-build\html_output.test" "-check-prefix" "CHECK-STDOUT"
# command stderr:
C:\src\llvm-project\clang\test\Analysis\scan-build\html_output.test:8:15: error: CHECK-STDOUT: expected string not found in input
CHECK-STDOUT: scan-build: 1 bug found.
^
<stdin>:4:1: note: scanning from here
scan-build: Removing directory '/c/src/llvm-project/out/gn/obj/clang/test/Analysis/scan-build/Output/html_output.test.tmp.output_dir/2020-04-07-090114-54704-1' because it contains no reports.
^
<stdin>:5:1: note: possible intended match here
scan-build: No bugs found.
^
error: command failed with exit status: 1
--
********************
Testing: 0
FAIL: Clang :: Analysis/scan-build/plist_html_output.test (977 of 17064)
******************** TEST 'Clang :: Analysis/scan-build/plist_html_output.test' FAILED ********************
Script:
--
: 'RUN: at line 1'; rm -rf C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_html_output.test.tmp.output_dir && mkdir C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_html_output.test.tmp.output_dir
: 'RUN: at line 2'; 'c:\src\llvm-project\clang\tools\scan-build\bin\scan-build.bat' --use-analyzer=c:\src\llvm-project\out\gn\bin\clang.exe -plist-html -o C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_html_output.test.tmp.output_dir c:\src\llvm-project\out\gn\bin\clang.exe -S C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/single_null_dereference.c | c:\src\llvm-project\out\gn\bin\filecheck.exe C:\src\llvm-project\clang\test\Analysis\scan-build\plist_html_output.test -check-prefix CHECK-STDOUT
: 'RUN: at line 14'; ls C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_html_output.test.tmp.output_dir/*/ | c:\src\llvm-project\out\gn\bin\filecheck.exe C:\src\llvm-project\clang\test\Analysis\scan-build\plist_html_output.test -check-prefix CHECK-FILENAMES
--
Exit Code: 1
Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "rm" "-rf" "C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_html_output.test.tmp.output_dir"
$ "mkdir" "C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_html_output.test.tmp.output_dir"
$ ":" "RUN: at line 2"
$ "c:\src\llvm-project\clang\tools\scan-build\bin\scan-build.bat" "--use-analyzer=c:\src\llvm-project\out\gn\bin\clang.exe" "-plist-html" "-o" "C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_html_output.test.tmp.output_dir" "c:\src\llvm-project\out\gn\bin\clang.exe" "-S" "C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/single_null_dereference.c"
$ "c:\src\llvm-project\out\gn\bin\filecheck.exe" "C:\src\llvm-project\clang\test\Analysis\scan-build\plist_html_output.test" "-check-prefix" "CHECK-STDOUT"
# command stderr:
C:\src\llvm-project\clang\test\Analysis\scan-build\plist_html_output.test:10:15: error: CHECK-STDOUT: expected string not found in input
CHECK-STDOUT: scan-build: 1 bug found.
^
<stdin>:6:1: note: scanning from here
scan-build: Removing directory '/c/src/llvm-project/out/gn/obj/clang/test/Analysis/scan-build/Output/plist_html_output.test.tmp.output_dir/2020-04-07-090114-54706-1' because it contains no reports.
^
<stdin>:7:1: note: possible intended match here
scan-build: No bugs found.
^
error: command failed with exit status: 1
--
********************
Testing: 0
FAIL: Clang :: Analysis/scan-build/exclude_directories.test (1014 of 17064)
******************** TEST 'Clang :: Analysis/scan-build/exclude_directories.test' FAILED ********************
Script:
--
: 'RUN: at line 1'; rm -rf C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\exclude_directories.test.tmp.output_dir && mkdir C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\exclude_directories.test.tmp.output_dir
: 'RUN: at line 2'; 'c:\src\llvm-project\clang\tools\scan-build\bin\scan-build.bat' --use-analyzer=c:\src\llvm-project\out\gn\bin\clang.exe -o C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\exclude_directories.test.tmp.output_dir c:\src\llvm-project\out\gn\bin\clang.exe -S C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/multidirectory_project/directory1/file1.c C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/multidirectory_project/directory2/file2.c | c:\src\llvm-project\out\gn\bin\filecheck.exe C:\src\llvm-project\clang\test\Analysis\scan-build\exclude_directories.test -check-prefix CHECK-NO-EXCLUDE
: 'RUN: at line 17'; rm -rf C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\exclude_directories.test.tmp.output_dir && mkdir C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\exclude_directories.test.tmp.output_dir
: 'RUN: at line 18'; 'c:\src\llvm-project\clang\tools\scan-build\bin\scan-build.bat' --use-analyzer=c:\src\llvm-project\out\gn\bin\clang.exe -o C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\exclude_directories.test.tmp.output_dir --exclude directory1 c:\src\llvm-project\out\gn\bin\clang.exe -S C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/multidirectory_project/directory1/file1.c C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/multidirectory_project/directory2/file2.c | c:\src\llvm-project\out\gn\bin\filecheck.exe C:\src\llvm-project\clang\test\Analysis\scan-build\exclude_directories.test -check-prefix CHECK-EXCLUDE1
: 'RUN: at line 27'; rm -rf C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\exclude_directories.test.tmp.output_dir && mkdir C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\exclude_directories.test.tmp.output_dir
: 'RUN: at line 28'; 'c:\src\llvm-project\clang\tools\scan-build\bin\scan-build.bat' --use-analyzer=c:\src\llvm-project\out\gn\bin\clang.exe -o C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\exclude_directories.test.tmp.output_dir --exclude directory1 --exclude directory2 c:\src\llvm-project\out\gn\bin\clang.exe -S C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/multidirectory_project/directory1/file1.c C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/multidirectory_project/directory2/file2.c | c:\src\llvm-project\out\gn\bin\filecheck.exe C:\src\llvm-project\clang\test\Analysis\scan-build\exclude_directories.test -check-prefix CHECK-EXCLUDE-BOTH
--
Exit Code: 1
Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "rm" "-rf" "C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\exclude_directories.test.tmp.output_dir"
$ "mkdir" "C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\exclude_directories.test.tmp.output_dir"
$ ":" "RUN: at line 2"
$ "c:\src\llvm-project\clang\tools\scan-build\bin\scan-build.bat" "--use-analyzer=c:\src\llvm-project\out\gn\bin\clang.exe" "-o" "C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\exclude_directories.test.tmp.output_dir" "c:\src\llvm-project\out\gn\bin\clang.exe" "-S" "C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/multidirectory_project/directory1/file1.c" "C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/multidirectory_project/directory2/file2.c"
$ "c:\src\llvm-project\out\gn\bin\filecheck.exe" "C:\src\llvm-project\clang\test\Analysis\scan-build\exclude_directories.test" "-check-prefix" "CHECK-NO-EXCLUDE"
# command stderr:
C:\src\llvm-project\clang\test\Analysis\scan-build\exclude_directories.test:13:19: error: CHECK-NO-EXCLUDE: expected string not found in input
CHECK-NO-EXCLUDE: scan-build: 2 bugs found.
^
<stdin>:2:1: note: scanning from here
C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build>perl -S scan-build --use-analyzer=c:\src\llvm-project\out\gn\bin\clang.exe -o C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\exclude_directories.test.tmp.output_dir c:\src\llvm-project\out\gn\bin\clang.exe -S C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/multidirectory_project/directory1/file1.c C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/multidirectory_project/directory2/file2.c
^
<stdin>:5:1: note: possible intended match here
scan-build: No bugs found.
^
error: command failed with exit status: 1
--
********************
Testing: 0.
FAIL: Clang :: Analysis/scan-build/plist_output.test (1116 of 17064)
******************** TEST 'Clang :: Analysis/scan-build/plist_output.test' FAILED ********************
Script:
--
: 'RUN: at line 1'; rm -rf C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_output.test.tmp.output_dir && mkdir C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_output.test.tmp.output_dir
: 'RUN: at line 2'; 'c:\src\llvm-project\clang\tools\scan-build\bin\scan-build.bat' --use-analyzer=c:\src\llvm-project\out\gn\bin\clang.exe -plist -o C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_output.test.tmp.output_dir c:\src\llvm-project\out\gn\bin\clang.exe -S C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/single_null_dereference.c | c:\src\llvm-project\out\gn\bin\filecheck.exe C:\src\llvm-project\clang\test\Analysis\scan-build\plist_output.test -check-prefix CHECK-STDOUT
: 'RUN: at line 12'; ls C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_output.test.tmp.output_dir/*/ | c:\src\llvm-project\out\gn\bin\filecheck.exe C:\src\llvm-project\clang\test\Analysis\scan-build\plist_output.test -check-prefix CHECK-FILENAMES
: 'RUN: at line 17'; cat C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_output.test.tmp.output_dir/*/report-*.plist | c:\src\llvm-project\out\gn\bin\filecheck.exe C:\src\llvm-project\clang\test\Analysis\scan-build\plist_output.test -check-prefix CHECK-REPORT-PLIST-CONTENTS
--
Exit Code: 2
Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "rm" "-rf" "C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_output.test.tmp.output_dir"
$ "mkdir" "C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_output.test.tmp.output_dir"
$ ":" "RUN: at line 2"
$ "c:\src\llvm-project\clang\tools\scan-build\bin\scan-build.bat" "--use-analyzer=c:\src\llvm-project\out\gn\bin\clang.exe" "-plist" "-o" "C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_output.test.tmp.output_dir" "c:\src\llvm-project\out\gn\bin\clang.exe" "-S" "C:\src\llvm-project\clang\test\Analysis\scan-build/Inputs/single_null_dereference.c"
$ "c:\src\llvm-project\out\gn\bin\filecheck.exe" "C:\src\llvm-project\clang\test\Analysis\scan-build\plist_output.test" "-check-prefix" "CHECK-STDOUT"
$ ":" "RUN: at line 12"
$ "ls" "C:\src\llvm-project\out\gn\obj\clang\test\Analysis\scan-build\Output\plist_output.test.tmp.output_dir/*/"
$ "c:\src\llvm-project\out\gn\bin\filecheck.exe" "C:\src\llvm-project\clang\test\Analysis\scan-build\plist_output.test" "-check-prefix" "CHECK-FILENAMES"
# command stderr:
FileCheck error: '<stdin>' is empty.
FileCheck command line: c:\src\llvm-project\out\gn\bin\filecheck.exe C:\src\llvm-project\clang\test\Analysis\scan-build\plist_output.test -check-prefix CHECK-FILENAMES
error: command failed with exit status: 2
--
********************
Testing: 0.. 10.. 20.. 30.. 40.. 50.. 60
FAIL: Clang :: SemaObjC/parameterized_classes_subst.m (11256 of 17064)
******************** TEST 'Clang :: SemaObjC/parameterized_classes_subst.m' FAILED ********************
Script:
--
: 'RUN: at line 1'; c:\src\llvm-project\out\gn\bin\clang.exe -cc1 -internal-isystem c:\src\llvm-project\out\gn\lib\clang\11.0.0\include -nostdsysteminc -fblocks -fsyntax-only -Wnullable-to-nonnull-conversion C:\src\llvm-project\clang\test\SemaObjC\parameterized_classes_subst.m -verify
--
Exit Code: 1
Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "c:\src\llvm-project\out\gn\bin\clang.exe" "-cc1" "-internal-isystem" "c:\src\llvm-project\out\gn\lib\clang\11.0.0\include" "-nostdsysteminc" "-fblocks" "-fsyntax-only" "-Wnullable-to-nonnull-conversion" "C:\src\llvm-project\clang\test\SemaObjC\parameterized_classes_subst.m" "-verify"
# command stderr:
error: 'error' diagnostics seen but not expected:
File C:\src\llvm-project\clang\test\SemaObjC\parameterized_classes_subst.m Line 479: type argument 'T' (aka 'id') does not satisfy the bound ('id<NSCopying>') of type parameter 'T'
error: 'note' diagnostics seen but not expected:
File C:\src\llvm-project\clang\test\SemaObjC\parameterized_classes_subst.m Line 475: type parameter 'T' declared here
2 errors generated.
error: command failed with exit status: 1
--
********************
Testing: 0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
Testing Time: 173.99s
********************
Failing Tests (5):
Clang :: Analysis/scan-build/exclude_directories.test
Clang :: Analysis/scan-build/html_output.test
Clang :: Analysis/scan-build/plist_html_output.test
Clang :: Analysis/scan-build/plist_output.test
Clang :: SemaObjC/parameterized_classes_subst.m
Expected Passes : 16885
Expected Failures : 28
Unsupported Tests : 146
Unexpected Failures: 5
FAILED: gen/clang/test/run-lit
C:/src/depot_tools/bootstrap-3_8_0_chromium_8_bin/python/bin/python.exe bin/llvm-lit.py -sv ../../clang/test
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "../hack/llvmgnsyncbot/syncbot.py", line 233, in <module>
sys.exit(main())
File "../hack/llvmgnsyncbot/syncbot.py", line 228, in main
run(args.last_exit)
File "../hack/llvmgnsyncbot/syncbot.py", line 191, in run
subprocess.check_call(['ninja', '-C', 'out/gn', test])
File "C:\python_27_amd64\files\lib\subprocess.py", line 190, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ninja', '-C', 'out/gn', 'check-clang']' returned non-zero exit status 1
```
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76768/new/
https://reviews.llvm.org/D76768
More information about the cfe-commits
mailing list