[PATCH] D107695: [llvm] [lit] Support forcing lexical test order

[Joel E. Denny via Phabricator via llvm-commits]

jdenny added reviewers: davezarzycki, yln, jhenderson, thopre.
jdenny added a comment.

Thanks for working on this!

The patch doesn't spell it consistently: sometimes `--lexical` and sometimes `--lexical-order`.

Shouldn't we put `--lexical-order` directly into lit's test suite's `%{lit}` substitution?  That way, if another of lit's current or future tests is susceptible to this problem, no one has to waste time trying to reproduce the race and figure out what's happening.  Blindly using `%{lit}` in lit's test suite would just work.

As far as I know, `reorder.py` is the only test in lit's test suite that requires the reordering.  Either it can use a `%{lit-reordered}` substitution that drops `--lexical-order`, or it can use a new command-line option that selects the default order and overrides any previous `--lexical-order` or `--shuffle`.



================
Comment at: llvm/utils/lit/tests/shtest-shell.py:12
 # FIXME: lit's testing sets source_root == exec_root which complicates running lit more than once per test.
 # RUN: rm -f %{inputs}/shtest-shell/.lit_test_times.txt
 # RUN: env PYTHONIOENCODING=ascii not %{lit} -j 1 -a %{inputs}/shtest-shell > %t.ascii.out
----------------
Any reason to keep this?  It might be worthwhile to grep for `lit_test_times`, if you didn't already.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D107695/new/

https://reviews.llvm.org/D107695