[PATCH] D77819: [lit] Add SKIPPED test result category

[Julian Lettner via Phabricator via llvm-commits]

yln marked an inline comment as done.
yln added inline comments.


================
Comment at: llvm/utils/lit/tests/max-time.py:3
+#
+# RUN: %{lit} %{inputs}/max-time --max-time=1 2>&1  |  FileCheck %s
+
----------------
jdenny wrote:
> yln wrote:
> > jdenny wrote:
> > > yln wrote:
> > > > jdenny wrote:
> > > > > On heavily loaded test systems, is there a chance of a race here?
> > > > Yes, race is between `--max-time=1` and `sleep 5` in [slow.txt].  I will increase 5 to 60 when landing.  That should work for all practical purposes.
> > > Isn't there theoretically a race between `--max-time=1` and fast.txt as well?  I'm not saying you need to change it, but I at least want to be sure I'm not misunderstanding something.
> > Yes, you are right! I forgot to consider this.  I would like to keep this at 1 for now (because it increases the actual running time of the test) and only increase it if we discover that it is a problem in practice.
> I think that's fine.
At least one bot actually hit this:

http://lab.llvm.org:8011/builders/clang-cmake-armv7-quick/builds/15079/steps/ninja%20check%201/logs/FAIL%3A%20lit%3A%3A%20max-time.py

I will adapt the timeouts:
fast.txt: true (0)
--max-time: 5  <-- this is how long the test runs (bot reported test time: 1.58s, so it should be good enough)
slow.txt: sleep 60

One more slow test in the lit test suite :(



Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77819/new/

https://reviews.llvm.org/D77819