[libcxx-commits] [libcxx] Improve CI output. (PR #70236)

[via libcxx-commits]

https://github.com/EricWF created https://github.com/llvm/llvm-project/pull/70236

There is currently a major problem with the CI output:

The information you need to see never appears in the visible log.

This is because our logs are very verbose, and list (A) every test as they run it, and (B) every unsupported tests. This can be thousands of lines.

(A) was introduced by me when I disabled the PTY progress bar, which doesn't play nice with log files. That change was an improvement, but I have now disabled the PTY on the builders, so we can go back to passing `-s`. When `-s` is passed but no PTY is available, it prints a long friendly progress indicator.

(B) is solved here by disabling the printing of unsupported tests at the end of the test suite. While it can be useful on occasion to audit the list of unsupported tests, it's far from a common operation. Instead people want to see the log of their failure. We should upload the xml results, so if auditing is needed, it can be done using that.

Hopefully this change will make it so that the test failures appear in the actual log output

>From 0e87e133f9b81017715e593d2ef2c88c8bcd0c1f Mon Sep 17 00:00:00 2001
From: eric <eric at efcs.ca>
Date: Wed, 25 Oct 2023 13:50:02 -0400
Subject: [PATCH] Improve CI output.

There is currently a major problem with the CI output:

The information you need to see never appears in the visible log.

This is because our logs are very verbose, and list (A) every test as
they run it, and (B) every unsupported tests. This can be thousands of
lines.

(A) was introduced by me when I disabled the PTY progress bar, which
doesn't play nice with log files. That change was an improvement, but I
have now disabled the PTY on the builders, so we can go back to passing
`-s`. When `-s` is passed but no PTY is available, it prints a long
friendly progress indicator.

(B) is solved here by disabling the printing of unsupported tests at the
end of the test suite. While it can be useful on occasion to audit the
list of unsupported tests, it's far from a common operation. Instead
people want to see the log of their failure. We should upload the xml
results, so if auditing is needed, it can be done using that.

Hopefully this change will make it so that the test failures
appear in the actual log output
---
 libcxx/utils/ci/run-buildbot | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libcxx/utils/ci/run-buildbot b/libcxx/utils/ci/run-buildbot
index 89910738f52d040..ebb255243ba9c15 100755
--- a/libcxx/utils/ci/run-buildbot
+++ b/libcxx/utils/ci/run-buildbot
@@ -137,7 +137,7 @@ function generate-cmake-base() {
           -DLIBUNWIND_ENABLE_WERROR=YES \
           -DLIBCXX_ENABLE_CLANG_TIDY=${ENABLE_CLANG_TIDY} \
           ${ENABLE_STD_MODULES} \
-          -DLLVM_LIT_ARGS="-v --show-unsupported --xunit-xml-output test-results.xml --timeout=1500 --time-tests" \
+          -DLLVM_LIT_ARGS="-sv --xunit-xml-output test-results.xml --timeout=1500 --time-tests" \
           "${@}"
 }
 
@@ -372,7 +372,7 @@ bootstrapping-build)
           -DLLVM_TARGETS_TO_BUILD="host" \
           -DRUNTIMES_BUILD_ALLOW_DARWIN=ON \
           -DLLVM_ENABLE_ASSERTIONS=ON \
-          -DLLVM_LIT_ARGS="-v --show-unsupported --xunit-xml-output test-results.xml --timeout=1500 --time-tests"
+          -DLLVM_LIT_ARGS="-sv --xunit-xml-output test-results.xml --timeout=1500 --time-tests"
 
     echo "+++ Running the libc++ and libc++abi tests"
     ${NINJA} -vC "${BUILD_DIR}" check-runtimes