[llvm-dev] Longstanding failing tests - clang-tidy, MachO, Polly

Wed Jan 22 06:33:20 PST 2020

I'm seeing this on two different machines, with completely different hardware. The git checkout is clean.
I've tried again, with the exact cmake cmd-line used on the MSVC bot, it fails on the same tests.

The only difference I see is that the bot uses MSVC 16.3.7, whereas I use the latest 16.4.3. The first 16.4 was released at the beginning of December, and it's about the time where I started seeing the issues. The tests might fail because of the new MSVC optimizer, this doesn't happen on Debug builds.

I'm also occasionally seeing crashes in opt, in LLVM :: Transforms/InstCombine/strcpy-2.ll, but this is not 100% repro (using the exact cmd-line as on the bot):

FAIL: LLVM :: Transforms/InstCombine/strcpy-2.ll (35388 of 35388)
******************** TEST 'LLVM :: Transforms/InstCombine/strcpy-2.ll' FAILED ********************
Script:
--
: 'RUN: at line 2';   d:\llvm-project\buildninjarelmsvc\bin\opt.exe < D:\llvm-project\llvm\test\Transforms\InstCombine\strcpy-2.ll -instcombine -S | d:\llvm-project\buildninjarelmsvc\bin\filecheck.exe D:\llvm-project\llvm\test\Transforms\InstCombine\strcpy-2.ll
--
Exit Code: 2

Command Output (stdout):
--
$ ":" "RUN: at line 2"
$ "d:\llvm-project\buildninjarelmsvc\bin\opt.exe" "-instcombine" "-S"
note: command had no output on stdout or stderr
error: command failed with exit status: 3221225477
$ "d:\llvm-project\buildninjarelmsvc\bin\filecheck.exe" "D:\llvm-project\llvm\test\Transforms\InstCombine\strcpy-2.ll"
# command stderr:
FileCheck error: '-' is empty.
FileCheck command line:  d:\llvm-project\buildninjarelmsvc\bin\filecheck.exe D:\llvm-project\llvm\test\Transforms\InstCombine\strcpy-2.ll

error: command failed with exit status: 2

Reid, do you think it would be possible to upgrade windows-gcebot2<http://lab.llvm.org:8011/buildslaves/windows-gcebot2> to latest MSVC 16.4?

De : Eli Friedman <efriedma at quicinc.com>
Envoyé : January 21, 2020 8:46 PM
À : Alexandre Ganea <alexandre.ganea at ubisoft.com>; llvm-dev <llvm-dev at lists.llvm.org>
Objet : RE: [llvm-dev] Longstanding failing tests - clang-tidy, MachO, Polly

I just XFAIL'ed the polly test; it was also failing on the polly bots.

The other tests you mentioned are not failing on any of the bots on lab.llvm.org, as far as I can tell, including http://lab.llvm.org:8011/builders/clang-x64-windows-msvc , which runs a similar configuration to yours.  So it's going to be hard for anyone without your exact environment to reproduce the issues.

-Eli

From: llvm-dev <llvm-dev-bounces at lists.llvm.org<mailto:llvm-dev-bounces at lists.llvm.org>> On Behalf Of Alexandre Ganea via llvm-dev
Sent: Tuesday, January 21, 2020 4:39 PM
To: llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>>
Subject: [EXT] [llvm-dev] Longstanding failing tests - clang-tidy, MachO, Polly

Hi,

A few tests seem broken for a long time, some for more than a month. Would it possible for respective owners to take a look please?

I'm at checkout 133a7e631cee97965e310f0d110739217427fd3d, compiling on Windows 10.

These tests fail with Visual Studio 2019:

Failing Tests (7):
    Clang Tools :: clang-tidy/checkers/cert-mem57-cpp-cpp17.cpp
    Clang Tools :: clang-tidy/checkers/performance-noexcept-move-constructor-fix.cpp
    LLVM :: MC/MachO/gen-dwarf-cpp.s
    LLVM :: MC/MachO/gen-dwarf-macro-cpp.s
    LLVM :: MC/MachO/gen-dwarf-producer.s
    LLVM :: MC/MachO/gen-dwarf.s
    Polly :: ScopInfo/memset_null.ll

  Expected Passes    : 57692
  Expected Failures  : 283
  Unsupported Tests  : 1941
  Unexpected Failures: 7

The clang-tidy tests fail since end of November-December.
The MachO tests fail since about the beginning of December.
The Polly test fails since a few days ago.
(see below for more info)

I'm using the following cmake cmd-line:

cmake -GNinja %ROOT%/llvm -DCMAKE_BUILD_TYPE=Release -DLLVM_OPTIMIZED_TABLEGEN=ON -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_LIBXML2=OFF -DCMAKE_C_COMPILER="%VS2019%/bin/HostX64/x64/cl.EXE" -DCMAKE_CXX_COMPILER="%VS2019%/bin/HostX64/x64/cl.EXE" -DCMAKE_LINKER="%VS2019%/bin/HostX64/x64/link.EXE" -DLLVM_ENABLE_PROJECTS="llvm;clang;lld;clang-tools-extra;compiler-rt;mlir;polly" -DLLVM_ENABLE_PDB=ON -DLLVM_POLLY_LINK_INTO_TOOLS=ON

These tests fail with Clang 9.0.1:

Failing Tests (3):
    Clang Tools :: clang-tidy/checkers/cert-mem57-cpp-cpp17.cpp
    Clang Tools :: clang-tidy/checkers/performance-noexcept-move-constructor-fix.cpp
    Polly :: ScopInfo/memset_null.ll

  Expected Passes    : 57727
  Expected Failures  : 283
  Unsupported Tests  : 1911
  Unexpected Failures: 3

1 warning(s) in tests

I'm using the following cmake cmd-line:

cmake -GNinja %ROOT%/llvm -DCMAKE_BUILD_TYPE=Release -DLLVM_OPTIMIZED_TABLEGEN=ON -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_LIBXML2=OFF -DCMAKE_C_COMPILER="%LLVM%/bin/clang-cl.EXE" -DCMAKE_CXX_COMPILER="%LLVM%/bin/clang-cl.EXE" -DCMAKE_LINKER="%LLVM%/bin/lld-link.EXE" -DLLVM_ENABLE_PROJECTS="llvm;clang;lld;clang-tools-extra;compiler-rt;mlir;polly" -DLLVM_ENABLE_PDB=ON -DLLVM_POLLY_LINK_INTO_TOOLS=ON -DLLVM_ENABLE_LLD=ON

Thanks in advance!
Alex.

________________________________

The clang-tidy cert-mem57 tests seem to be related to varying messages (warning|error) being emitted.

All three MachO/gen-dwarf test are crashing at about the same place:
Stack dump:
0.      Program arguments: d:\llvm-project\buildninjarelmsvc\bin\llvm-dwarfdump.exe -all D:\llvm-project\buildninjaRelMSVC\test\MC\MachO\Output\gen-dwarf.s.tmp
#0 0x00007ff6c125dfb8 llvm::DWARFCompileUnit::dump(class llvm::raw_ostream &, struct llvm::DIDumpOptions) D:\llvm-project\llvm\lib\DebugInfo\DWARF\DWARFCompileUnit.cpp:23:0
#1 0x00007ff6c1236e18 <lambda_25ce692e137c1a998cd3557ebf29fa33>::operator() D:\llvm-project\llvm\lib\DebugInfo\DWARF\DWARFContext.cpp:367:0
#2 0x00007ff6c123968b llvm::DWARFContext::dump(class llvm::raw_ostream &, struct llvm::DIDumpOptions, class std::array<class llvm::Optional<unsigned __int64>, 28>) D:\llvm-project\llvm\lib\DebugInfo\DWARF\DWARFContext.cpp:372:0
#3 0x00007ff6c12197f8 dumpObjectFile D:\llvm-project\llvm\tools\llvm-dwarfdump\llvm-dwarfdump.cpp:448:0
#4 0x00007ff6c1218486 std::_Func_impl_no_alloc<bool (__cdecl *)(class llvm::object::ObjectFile &, class llvm::DWARFContext &, class llvm::Twine, class llvm::raw_ostream &), bool, class llvm::object::ObjectFile &, class llvm::DWARFContext &, class llvm::Twine, class llvm::raw_ostream &>::_Do_call(class llvm::object::ObjectFile &, class llvm::DWARFContext &, class llvm::Twine &&, class llvm::raw_ostream &) C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.24.28314\include\functional:927:0
#5 0x00007ff6c121c519 handleBuffer D:\llvm-project\llvm\tools\llvm-dwarfdump\llvm-dwarfdump.cpp:495:0
#6 0x00007ff6c121d16a handleFile D:\llvm-project\llvm\tools\llvm-dwarfdump\llvm-dwarfdump.cpp:528:0
#7 0x00007ff6c121eb87 main D:\llvm-project\llvm\tools\llvm-dwarfdump\llvm-dwarfdump.cpp:643:0
#8 0x00007ff6c16b6420 __scrt_common_main_seh d:\agent\_work\5\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288:0
#9 0x00007ffb52557bd4 (C:\WINDOWS\System32\KERNEL32.DLL+0x17bd4)
#10 0x00007ffb5278ced1 (C:\WINDOWS\SYSTEM32\ntdll.dll+0x6ced1)

error: command failed with exit status: 3221225477

Polly :: ScopInfo/memset_null.ll fails because the second opt fails:
D:\llvm-project>buildninjarelmsvc\bin\opt  -polly-process-unprofitable  -polly-remarks-minimal  -polly-use-llvm-names  -polly-import-jscop-dir=D:\llvm-project\polly\test\ScopInfo  -polly-codegen-verify  -polly-allow-modref-calls -polly-scops -analyze < D:\llvm-project\polly\test\ScopInfo\memset_null.ll
Printing analysis 'Polly - Create polyhedral description of Scops' for region: 'for.cond5.preheader.us221 => for.end68' in function 'test':
Invalid Scop!
Printing analysis 'Polly - Create polyhedral description of Scops' for region: 'entry => for.end68' in function 'test':
Invalid Scop!
Printing analysis 'Polly - Create polyhedral description of Scops' for region: 'entry => <Function Return>' in function 'test':
Invalid Scop!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200122/e89bd0ec/attachment-0001.html>