[Openmp-commits] [libcxx] [llvm] [openmp] [lit] Add a flag to disable lit time tests (PR #98270)

[James Henderson via Openmp-commits]

jh7370 wrote:

Thanks for the example, that helped me confirm my thoughts. Orthogonal to the real issue here, but have you considered the additional overhead of spinning up lit per test? I would expect (but it's always possible I'm wrong) that spinning up lit 10 times to run 10 tests individually would be slower than spinning it up once to run them all.

> > arguably the external build systems aren't even that relevant here: I've been known in the past to run lit in two different LLVM test directories myself at the same time for various reasons. Thoughts?
>
> IIUC, you're suggesting to implement some sharding mechanism within a test-suite? If so, that would also work, but here I'm choosing the path of least resistance and disabling it since I don't see any reason for needing it at the moment.

Not exactly. The issue I'm more illustrating is that the test times file is already unsafe in the face of concurrent usage of the same test suite (even when the runs don't overlap), so perhaps we need to revisit how that file is written, rather than work around it. One random idea that I haven't really given much thought to how effective it would be (or even whether it would actually work) would be to avoid a single centralised file and instead have a file per test, that are read in following test discovery to form a combined database of sorts for determining order within that set of tests. In this way, there's only a risk of concurrency issues if the same test is run multiple times simultaneously, but that's already a limitation in lit anyway. If I'm not mistaken, this solves the issue both for your use case and for the more general case I've highlighted. Does this make sense? Of course, other implementations might be better.

https://github.com/llvm/llvm-project/pull/98270