[all-commits] [llvm/llvm-project] 383df1: [llvm][llvm-lit] Add total time for each testsuite...

[David Spickett via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 383df16317eec3b29b93025e2a86ea024b3f59c7
      https://github.com/llvm/llvm-project/commit/383df16317eec3b29b93025e2a86ea024b3f59c7
  Author: David Spickett <david.spickett at linaro.org>
  Date:   2024-10-16 (Wed, 16 Oct 2024)

  Changed paths:
    M llvm/utils/lit/lit/reports.py
    M llvm/utils/lit/tests/shtest-format.py
    M llvm/utils/lit/tests/xunit-output.py

  Log Message:
  -----------
  [llvm][llvm-lit] Add total time for each testsuite in JUnit XML output (#112230)

Currently we write out a time taken to run all test suites:
```
<testsuites time="8.28">
```
And one for each test:
```
<testcase classname="lldb-shell.Breakpoint" name="breakpoint-command.test" time="2.38"/>
```
However, the schema says there should be one for each suite and test,
but none for testsuites:

https://github.com/windyroad/JUnit-Schema/blob/cfa434d4b8e102a8f55b8727b552a0063ee9044e/JUnit.xsd#L216

I'm leaving the `testsuites` time in though because no one has
complained so far, and someone out there probably has a script relying
on it by now. Most XML tools handle unknown attributes quite well
anyway.

I'm adding a per testsuite time to comply with the schema and maybe be
more compatible with other JUnit tools.
```
<testsuite name="lldb-shell" ... time="12.34">
```

The test suite time is the sum of the time taken for all tests in the
suite. This will ignore some overhead in setting up the suite, and means
that the sum of the times for all individual suites may not equal the
`testsuites` time.

As we're usually focusing on the execution time of particular tests, not
lit's book keeping, I think this is a reasonable choice.



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications