[llvm-dev] Timeout tests timing out

[Dan Liew via llvm-dev]

On Wed, 16 Sep 2020 at 22:32, Mehdi AMINI <joker.eph at gmail.com> wrote:
>
> Hi,
>
> Depending on timing in a test is quite brittle in general: can we mock the timeout instead and make this fully deterministic somehow?
>
> The test has this comment:
>
> > FIXME: This test is fragile because it relies on time which can
> > be affected by system performance. In particular we are currently
> > assuming that `short.py` can be successfully executed within 2
> > seconds of wallclock time.
>
> Maybe "short.py" can be replaced by adding into lit itself a "no op" which would just not really spawn a process and instead mark the task as completed immediately internally?

I think it's probably easier to just remove running "short.py" in the
test. We'd lose some test coverage because we no longer test that a
program can run to completion when a timeout is set but maybe this is
an acceptable trade-off to avoid the test being flakey?