[PATCH] D122251: [lit] Use sharding for GoogleTest format

[Yuanfang Chen via Phabricator via llvm-commits]

ychen added a comment.

In D122251#3490585 <https://reviews.llvm.org/D122251#3490585>, @nigelp-xmos wrote:

> This has changed test totals reported on the buildbot dashboard, so I would be interested in a comment on that.
>
> As others have mentioned, some tests are now reported as a kind of summary. For example I used to get 1332 lines of "ADTTests" like:
>
> LLVM-Unit :: ADT/./ADTTests/AnyTest.BadAnyCast
>
> and now get three lines in total:
>
> LLVM-Unit :: ADT/./ADTTests/0/3
> LLVM-Unit :: ADT/./ADTTests/1/3
> LLVM-Unit :: ADT/./ADTTests/2/3
> (3-core VM)
>
> Before this change, the XCore buildbot totals on the buildbot staging dashboard matched the totals at the end of the "check-all" log:
>
> https://lab.llvm.org/staging/#/builders/145/builds/5438
>
> But after this change, the dashboard totals do not match the totals at the end of the log. The passed total is down by 20,000 and "skipped" (skipped Google unit tests) is not reported at all:
>
> https://lab.llvm.org/staging/#/builders/145/builds/5439
>
> I think the buildbot dashboard report counts log lines, rather than just displaying the totals at the end of the log.
>
> A green buildbot with the change is clang-ppc64le-linux-multistage on main buildbot master:
>
> https://lab.llvm.org/buildbot/#/builders/121/builds/18347
> https://lab.llvm.org/buildbot/#/builders/121/builds/18350
>
> Again the number of "expected passes" does not match the end of the log, "skipped" is not on the dashboard, and the log has numbers instead of test names.
>
> (I tried the follow-up commit but it does not give test names of passing tests.)
>
> Is this all intended and/or expected? What do you think about the buildbot totals?
>
> Thanks,
> Nigel

This is all intended. The buildbot totals are counting the number of shards across all unit tests. The summary at the end of the log is counting the number of test cases like before. I think it is possible to let the buildbot show the summary but I'd understand the use case before doing that. Are you blocked in some way due to the mismatch?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D122251/new/

https://reviews.llvm.org/D122251