[Lldb-commits] [PATCH] D46005: [test] Add a utility for dumping all tests in the dotest suite
Pavel Labath via Phabricator via lldb-commits
lldb-commits at lists.llvm.org
Wed Apr 25 00:31:13 PDT 2018
labath added a comment.
In https://reviews.llvm.org/D46005#1077016, @zturner wrote:
> Note that there's currently no precedent (that i'm aware of anwyay) in LLVM or any of its subprojects for splitting the running of a single file into multiple parallel jobs. All of LLVM's other projects parallelize at file granularity, and so it's up to to the person writing tests to ensure that the file granularity matches that with which they want tests to be parallelized at.
There's always gtest. Regardless of whether you consider a .cpp file or the built exectable to be a "test file", it will generally contain a number of tests. And arguably, our test suite is more similar to the gtest suite than the traditional lit (ShTest) suites (for one, it hijacks a third party unit testing library, and then tries to make it run under lit). And we run all gtest tests individually (I've often wondered what kind of performance impact that has, but they seem to be running fast enough anyway...).
For what it's worth, paralelization is not my motivation here. There some tests which run disproportionately long, and this will speed them up, but that may be offset by the fact we need to start more work this way (if anyone is interested I can try to do some measurements). My main reason for doing this is so that we can have better mapping of test result states. As it stands now, we only have two possible results for a test: passed or failed. Lit has more results than that, and they roughly match the existing dotest states (the different treatment of XFAILs is the biggest difference), so it should be possible to represent them with more fidelity. However, right now it's not possible to translate them reasonably, as a "single test" can result in any number of fails/xfails/skips/... (or no results at all).
> That doesn't mean it's not possible (as you've shown here), but it adds some additional complexity and I'm not sure it's worth it.
It adds complexity, but not much imho. I was actually pleasantly surprised at how easy it was to pull this off. The entire implementation is 78 LOC right now. The changes to the test format will probably push it over a 100, but not by much.
That said, I am not interested in forcing this onto everyone. I did it because it seemed like a nice thing to have and I was curious if it is doable (easily). However, if there's no interest for it, then I can just abandon it..
More information about the lldb-commits