[llvm-bugs] [Bug 30288] New: Consider using a separate working dir per test suite .py file to decouple serialization from test directory

[via llvm-bugs]

https://llvm.org/bugs/show_bug.cgi?id=30288

            Bug ID: 30288
           Summary: Consider using a separate working dir per test suite
                    .py file to decouple serialization from test directory
           Product: lldb
           Version: unspecified
          Hardware: PC
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P
         Component: All Bugs
          Assignee: lldb-dev at lists.llvm.org
          Reporter: todd.fiala at gmail.com
                CC: llvm-bugs at lists.llvm.org
    Classification: Unclassified

Currently, the LLDB test suite uses the test suite directory as the unit of
serialization.  This is primarily due to the test suite building the test
inferior build artifacts into the test directory itself.  This implementation
detail then causes us to only allow one python file to run at a time within
that directory, and only one test method within that file.

The only way to increase throughput with our current system is to break larger
test files into smaller files (fewer test methods per file), and to break those
python files into separate directories.  That then allows all of those
broken-down test .py files to run concurrently.

What we really want here is, at a minimum:

* Do what our remote test suite does, which is to create a separate working
directory per .py test file, thus allowing multiple .py files in the same
directory to be executed independent of one another.

For bonus points, we could:

* Allow each test method to run in a separate working dir.

The latter would allow us to (1) eliminate serialization within a given .py
file, and (2) probably could be done in tandem with enabling rerunning of
individual failed test methods.  Currently, we are forced to do a rerun at the
level of a .py file (i.e. potentially rerunning many more tests than we
actually need to rerun), solely because we only support running all the methods
in a test .py file.  There are some failure scenarios where being able to run
all test methods in a .py file *except* for the one that timed out might be
helpful.  Currently a stuck test method takes out all the test methods in a
test .py file that come after the one that times out.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20160906/27335da4/attachment.html>