[PATCH] D143519: [lit] [PATCH 2/2] Add "--reduced-xunit-report" option

[Joel E. Denny via Phabricator via llvm-commits]

jdenny added a comment.

@gbedwell, thanks for working on this.  I have also noticed that gitlab CI's web UI for browsing test results can be tough to navigate for LLVM because of the massive number of tests.  This could help.

As you mentioned in the review summary, it is misleading that non-failing tests appear not to have run at all.  I'm imagining a manager or some external person looking at CI results and thinking nothing ever passes.  To address that issue, one possibility is to add a fake test result per non-failing test code that perhaps sums up its number of tests and elapsed times.  I'm not requesting this improvement before this lands.  It's just a thought.

In D143519#4111328 <https://reviews.llvm.org/D143519#4111328>, @yln wrote:

> Initially, my main question was: would this functionality make sense for reports other than the xml one too?

It might.  For lit's own test suite, I noticed the `--time-trace-output` output is about twice as big as the `--xunit-xml-output` output when all tests pass.  Perhaps the filter can just be moved up a level where `write_results` is called.  It seems like that would also shrink this patch.

> @jdenny: Comments on the larger trade-off of having this feature?

Thanks for pinging me on this.  Especially if the implementation is moved out of the individual reports, it's a small amount of code, and I can see how it would be useful.



================
Comment at: llvm/utils/lit/lit/reports.py:22-23
                               lit.Test.SKIPPED, lit.Test.UNSUPPORTED}
+        self.failed_codes = {lit.Test.UNRESOLVED, lit.Test.TIMEOUT,
+                             lit.Test.FAIL, lit.Test.XPASS}
 
----------------
This list is already in `lit/Test.py`.  It can be accessed with something like `test.result.code.isFailure`.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143519/new/

https://reviews.llvm.org/D143519