[all-commits] [llvm/llvm-project] 8507db: [llvm][llvm-lit] Add option to create unique resul...

[David Spickett via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 8507dbaec3f644b8a0c6291f097800d82a4f4b16
      https://github.com/llvm/llvm-project/commit/8507dbaec3f644b8a0c6291f097800d82a4f4b16
  Author: David Spickett <david.spickett at linaro.org>
  Date:   2024-10-21 (Mon, 21 Oct 2024)

  Changed paths:
    M llvm/utils/lit/lit/cl_arguments.py
    M llvm/utils/lit/lit/reports.py
    A llvm/utils/lit/tests/unique-output-file.py

  Log Message:
  -----------
  [llvm][llvm-lit] Add option to create unique result file names if results already exist (#112729)

When running a build like:
```
ninja check-clang check-llvm
```
Prior to my changes you ended up with one results file, in this specific case Junit XML:
```
results.xml
```
This would only include the last set of tests lit ran, which were for
llvm. To get around this, many CI systems will run one check target,
move the file away, then run another, somehow propgating the return code
as well.
```
rectode=0
for target in targets:
  ninja target
  retcode=$?
  mv results.xml results-${target}.xml
<report the overall return code>
```
I want to use something like this Buildkite reporting plugin in CI, which needs to have all the results available:
https://buildkite.com/docs/agent/v3/cli-annotate#using-annotations-to-report-test-results

Modifying CI's build scripts for Windows and Linux is a lot of work. So
my changes instead make lit detect an existing result file and modify
the file name to find a new file to write to. Now you will get:
```
results.xml results.<tempfile generated value>.xml
```
This will work for all result file types since I'm doing it in the base
Report class. Now you've got separate files, it's easy to collect them
with `<path>/*.xml`.

Note that the `<tempfile generated value>` is not ordered.



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications