[libcxx-commits] [libcxx] [libc++] Run the Lit test suite against an installed version of the library (PR #96910)

[Louis Dionne via libcxx-commits]

ldionne wrote:

@mstorsjo We are still seeing strange failures on Windows and MinGW after fixing the obvious. Windows and MinGW both fail with different modes. The MinGW failures look like (from https://github.com/llvm/llvm-project/actions/runs/9779406851/job/26999519602?pr=96910):

```
********************
FAIL: llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/monadic/transform.pass.cpp (8399 of 9666)
******************** TEST 'llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/monadic/transform.pass.cpp' FAILED ********************
Exit Code: 4294967295

Command Output (stdout):
--
# COMPILED WITH
C:/llvm-mingw/bin/c++.exe D:\a\llvm-project\llvm-project\libcxx\test\std\utilities\expected\expected.expected\monadic\transform.pass.cpp  --target=x86_64-w64-windows-gnu -nostdinc++ -I D:/a/llvm-project/llvm-project/build/mingw-dll/libcxx/test-suite-install/include/c++/v1 -I D:/a/llvm-project/llvm-project/build/mingw-dll/libcxx/test-suite-install/include/c++/v1 -I D:/a/llvm-project/llvm-project/libcxx/test/support -std=c++26 -Werror -Wall -Wctad-maybe-unsupported -Wextra -Wshadow -Wundef -Wunused-template -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-noexcept-type -Wno-atomic-alignment -Wno-reserved-module-identifier -Wdeprecated-copy -Wdeprecated-copy-dtor -Wno-user-defined-literals -Wno-tautological-compare -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -Wno-local-type-template-args -Wno-c++11-extensions -Wno-unknown-pragmas -Wno-pass-failed -Wno-mismatched-new-delete -Wno-redundant-move -Wno-self-move -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBCPP_ENABLE_EXPERIMENTAL -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_NONE -Werror=thread-safety -Wuser-defined-warnings -DTEST_WINDOWS_DLL  -lc++experimental -nostdlib++ -L D:/a/llvm-project/llvm-project/build/mingw-dll/libcxx/test-suite-install/lib -lc++ -o D:\a\llvm-project\llvm-project\build\mingw-dll\test\std\utilities\expected\expected.expected\monadic\Output\transform.pass.cpp.dir\t.tmp.exe
# note: command had no output on stdout or stderr
# EXECUTED AS
'C:\hostedtoolcache\windows\Python\3.9.13\x64\python3.exe' 'D:\a\llvm-project\llvm-project\libcxx\utils\run.py' --execdir D:\a\llvm-project\llvm-project\build\mingw-dll\test\std\utilities\expected\expected.expected\monadic\Output\transform.pass.cpp.dir --prepend_env PATH=D:/a/llvm-project/llvm-project/build/mingw-dll/libcxx/test-suite-install/lib --  D:\a\llvm-project\llvm-project\build\mingw-dll\test\std\utilities\expected\expected.expected\monadic\Output\transform.pass.cpp.dir\t.tmp.exe
# note: command had no output on stdout or stderr
# error: command failed with exit status: 0xffffffff
```

On MinGW we see only 33 failures:

```
  llvm-libc++-mingw.cfg.in :: libcxx/input.output/iostreams.base/ios.base/ios.base.cons/dtor.uninitialized.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/input.output/iostream.format/ext.manip/get_money.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/input.output/iostream.format/ext.manip/put_money.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/input.output/iostream.format/input.streams/istream.unformatted/sync.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/input.output/iostream.format/output.streams/ostream.formatted/ostream.inserters.arithmetic/long_double.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/localization/locale.categories/category.numeric/locale.nm.put/facet.num.put.members/put_double.hex.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/localization/locale.categories/category.numeric/locale.nm.put/facet.num.put.members/put_long_double.hex.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/localization/locale.categories/category.numeric/locale.nm.put/facet.num.put.members/put_long_double.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/strings/string.conversions/to_string.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/strings/string.conversions/to_wstring.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/time/time.duration/time.duration.nonmember/ostream.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.bad/ctor.error.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.bad/error.member.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.bad/void-specialization.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.bad/what.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/assign/assign.U.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/assign/assign.copy.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/assign/assign.move.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/assign/emplace.intializer_list.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/assign/emplace.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/ctor/ctor.convert.copy.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/ctor/ctor.convert.move.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/ctor/ctor.copy.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/ctor/ctor.default.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/ctor/ctor.inplace.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/ctor/ctor.inplace_init_list.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/ctor/ctor.move.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/ctor/ctor.u.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/monadic/transform.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/observers/arrow.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/observers/deref.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.expected/observers/value.pass.cpp
  llvm-libc++-mingw.cfg.in :: std/utilities/expected/expected.void/observers/value.pass.cpp
```

Can you think about an obvious reason why these tests would fail when we start running against the installed library?

https://github.com/llvm/llvm-project/pull/96910