[LLVMbugs] [Bug 19790] New: Running lit tests under windows can fail due to race conditions

[bugzilla-daemon at llvm.org]

http://llvm.org/bugs/show_bug.cgi?id=19790

            Bug ID: 19790
           Summary: Running lit tests under windows can fail due to race
                    conditions
           Product: Test Suite
           Version: trunk
          Hardware: PC
                OS: other
            Status: NEW
          Severity: normal
          Priority: P
         Component: lit
          Assignee: unassignedbugs at nondot.org
          Reporter: dan at su-root.co.uk
                CC: daniel at zuster.org, llvmbugs at cs.uiuc.edu
    Classification: Unclassified

Created attachment 12540
  --> http://llvm.org/bugs/attachment.cgi?id=12540&action=edit
Example output from lit under Windows

OS: Windows 8.1
Python: Python 3.4

It seems that when running with the default ``--use-threads`` under Windows 8
with multiple threads tests can randomly fail. Which looks like a race
condition.

What it looks like is happening is that commands in the test are being executed
on files before those files are closed.

Steps to reproduce

Here is a randomly generated test suite that uses lit and which appears on
Windows to randomly fail (the number of tests and which tests fail is not
consistent) when using --use-threads and thread count is greater than 1.

> git clone https://github.com/delcypher/lit-concurrency-windows-bug.git
> cd lit-concurrency-windows-bug
> lit --use-threads -vs .

The tests have the following format

# RUN: python %s > %t
# RUN: %diff %s.expect %t

``%s.expect`` is a text file containing the expected output of running the
first command.

When I run I randomly see messages like this...
```
Command 1: "fc" "/W"
"C:\Users\Daniel\Desktop\lit-windows-concurrency-bug\testdi
r-1\test-11.py.expect"
"C:\Users\Daniel\Desktop\lit-windows-concurrency-bug\test
dir-1\Output\test-11.py.tmp"
Command 1 Result: 1
Command 1 Output:
Comparing files
C:\USERS\DANIEL\DESKTOP\LIT-WINDOWS-CONCURRENCY-BUG\TESTDIR-1\te
st-11.py.expect and
C:\USERS\DANIEL\DESKTOP\LIT-WINDOWS-CONCURRENCY-BUG\TESTDIR-
1\OUTPUT\TEST-11.PY.TMP
Resync Failed.  Files are too different.
*****
C:\USERS\DANIEL\DESKTOP\LIT-WINDOWS-CONCURRENCY-BUG\TESTDIR-1\test-11.py.e
xpect
FOO 0x44dcda6a797d76de
FOO 0x598b88dbaa99e079
FOO 0x61b339ff248174e5
FOO 0x7b87a9e25fefe911
...
...
FOO 0x4bca2f3e6eb5568b
FOO 0x258dc101eb3a8082
*****
C:\USERS\DANIEL\DESKTOP\LIT-WINDOWS-CONCURRENCY-BUG\TESTDIR-1\OUTPUT\TEST-
11.PY.TMP
*****



Command 1 Stderr:



--

********************
```

The output shown above (truncated) from the ``fc`` tool (the Windows version of
the diff tool) seems to suggest that the temporary file created by the first
command in the tests (%t) is empty when ``fc`` was executed which suggests to
me that the temporary file isn't closed properly before executing the next
command.

It appears if I use the ``--use-processes`` flag instead then these random test
failures do not occur. Attached is an example of the output seen when executing
under Windows.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20140519/bde5a973/attachment.html>