[PATCH] D122251: [lit] Use sharding for GoogleTest format

[Yuanfang Chen via Phabricator via llvm-commits]

ychen added a comment.

In D122251#3428173 <https://reviews.llvm.org/D122251#3428173>, @thakis wrote:

>> http://45.33.8.238/win/55614/summary.html did not regress which is expected since these runs do not include unit tests.
>
> `check-llvm` includes llvm's unit tests, `check-lld` includes lld's unit tests, `check-clang` includes clang's unit tests. That build does include unit tests.
>
> If you compare the stdout:
> http://45.33.8.238/linux/72772/step_7.txt
> http://45.33.8.238/linux/72807/step_7.txt
>
> then one says "30362 tests" while the other says "16250 tests" (which I think is expected since we now count number of unit test processes instead of number of tests?), so it does seem to have an effect. It's just not (much) faster.
>
>> before (~270) https://lab.llvm.org/buildbot/#/builders/216/builds/2323/steps/7/logs/stdio
>> after (~240) https://lab.llvm.org/buildbot/#/builders/216/builds/2356/steps/7/logs/stdio
>>
>> The time fluctuates to various degrees for bots. bot 216 is relatively stable and the speedup is what I would expect for a windows machine.
>
> Even on that it it's a 30s improvement on a bit with a 9-15 min cycle time, so a 3-5% speedup. Is that worth giving up the property that we're guaranteed we don't have ordering dependencies between tests?

https://github.com/google/googletest/blob/main/docs/advanced.md says that `Remember that the test order is undefined, so your code can't depend on a test preceding or following another.`. And https://github.com/google/googletest/blob/main/docs/advanced.md#shuffling-the-tests seems to suggest random order is that default. If we're paranoid, I could pass in `GTEST_SHUFFLE` all the time. WDYT?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D122251/new/

https://reviews.llvm.org/D122251

STAMPS
actor(@ychen) application(Differential) author(@ychen) herald(H223) herald(H297) herald(H428) herald(H576) herald(H615) herald(H864) mention(@thakis) monogram(D122251) object-type(DREV) phid(PHID-DREV-ovmkwmtnl2c5hwyfa3ye) reviewer(@compnerd) reviewer(@ddunbar) reviewer(@delcypher) reviewer(@jdenny) reviewer(@jhenderson) reviewer(@mstorsjo) reviewer(@rnk) reviewer(@yln) revision-repository(rG) revision-status(accepted) subscriber(@abrachet) subscriber(@browneee) subscriber(@compnerd) subscriber(@fhahn) subscriber(@llvm-commits) subscriber(@mstorsjo) subscriber(@probinson) subscriber(@rnk) subscriber(@russell.gallop) subscriber(@thakis) subscriber(@vitalybuka) tag(#all) tag(#llvm) via(web)