[llvm-dev] Marking lit::shtest-format.py unsupported on PS4?, Re: buildbot failure in LLVM on llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast

[Jeremy Morse via llvm-dev]

Hi,

We've given this an eyeball, and reckon there's interference between the
lit "shtest-format" and "shtest-xunit-output" tests -- the following two
run commands:

    # RUN: not %{lit} -j 1 -v %{inputs}/shtest-format > %t.out
    # RUN: not %{lit} -j 1 -v %{inputs}/shtest-format --xunit-xml-output
%t.xml

appear in the tests respectively, using the
utils/lit/tests/Inputs/shtest-format directory as the target test suite.
For external-shell tests in that suite however, lit produces a
'.script.bat' file in the build directory to be run by the external shell:

    $ pwd

/cygdrive/d/build/utils/lit/tests/Inputs/shtest-format/external_shell/Output

    $ ls
    fail.txt.script.bat  fail_with_bad_encoding.txt.script.bat
pass.txt.script.bat

Two concurrent lit invocations running on the same test suite will both
write to those files, creating racy filesystem behaviour and likely the
errors that have been seen. I've found at least one shtest-xunit-output
test run with a "Permission denied" error when writing to the script file
(on Windows). The race is still present for non-windows systems but
presumably much rarer.

I'm not very familiar with lit, and presumably making the external-shell
scripts temporarily named files would fix this, but there may be other
assumptions lit makes about a test suite not being operated on concurrently.

--
Thanks,
Jeremy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180723/15a30d2e/attachment.html>