[PATCH] D88807: [lit] Try to remove the flakeyness of `shtest-timeout.py` and `googletest-timeout.py`.

[Dan Liew via Phabricator via llvm-commits]

delcypher created this revision.
delcypher added reviewers: dblaikie, yln, jroelofs, cmatthews, MatzeB, mehdi_amini.
Herald added a project: LLVM.
delcypher requested review of this revision.

The tests previously relied on the `short.py` and `FirstTest.subTestA`
script being executed on a machine within a short time window (1 or 2
seconds). While this "seems to work" it can fail on resource constrained
machines. We could bump the timeout a little bit (bumping it too
much would mean the test would take a long time to execute) but it wouldn't
really solve the problem of the test being prone to failures.

Instead this patch tries to make the tests less flakeys by no longer
running `short.py` and `FirstTest.subTestA`. The trade-off here is that
this means we no longer test if it is possible for a test to complete
execution when a timeout is set.

This seems like the right trade-off right now because debugging this
flakey test is not a good use of engineering time.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D88807

Files:
  llvm/utils/lit/tests/Inputs/googletest-timeout/DummySubDir/OneTest.py
  llvm/utils/lit/tests/Inputs/shtest-timeout/short.py
  llvm/utils/lit/tests/googletest-timeout.py
  llvm/utils/lit/tests/shtest-timeout.py

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D88807.296080.patch
Type: text/x-patch
Size: 5682 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20201004/97421d25/attachment.bin>