[PATCH] D143519: [lit] [PATCH 2/2] Add "--reduced-xunit-report" option

[Greg Bedwell via Phabricator via llvm-commits]

gbedwell created this revision.
gbedwell added reviewers: yln, haowei.
Herald added a subscriber: delcypher.
Herald added a project: All.
gbedwell requested review of this revision.
Herald added a project: LLVM.

This option is designed to save on disk space in cases where a CI system
is running dozens of builds each day and is using the information in the
XUnit report to display information about the tests. Due to the sheer
number of tests in the LLVM repo each XUnit report is multiple MB in
size and when multiplied by large numbers of builds these reports can
consume swathes of disk space when each report is archived by CI unless
an aggressive expiration strategy is in place, meaning that historical
data about failures is lost.

In general, the most interesting results are those were a test has
failed (UNRESOLVED/TIMEOUT/FAIL/XPASS) so this new option limits the
report to only include these tests. This comes at the expense of
continuity of test data (a new failure will show as a new test for
example, and no duration data will be recorded for passing tests), but
for some users this may be a trade-off worth making.

I'm not sure whether this would be considered controversial or not as it somewhat breaks the xunit format (although only when the new flag is enabled), so if we ultimately decide not to go with this it's okay. We would definitely find it useful internally though. I've tested it with Jenkins' xunit reporting and it works as I'd expect.  I can see all the failures in any given build - just not the passes, and therefore the history isn't as useful.  It's an absolute fraction of the size though. This is a trade-off that seems worth it for us.


https://reviews.llvm.org/D143519

Files:
  llvm/utils/lit/lit/cl_arguments.py
  llvm/utils/lit/lit/main.py
  llvm/utils/lit/lit/reports.py

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D143519.495583.patch
Type: text/x-patch
Size: 4434 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20230207/65de00c7/attachment.bin>