[llvm-bugs] [Bug 30295] New: test infra: consider storing and re-using test inferior build artifacts for a given test directory

via llvm-bugs llvm-bugs at lists.llvm.org
Tue Sep 6 09:38:48 PDT 2016


https://llvm.org/bugs/show_bug.cgi?id=30295

            Bug ID: 30295
           Summary: test infra: consider storing and re-using test
                    inferior build artifacts for a given test directory
           Product: lldb
           Version: unspecified
          Hardware: PC
                OS: All
            Status: NEW
          Severity: normal
          Priority: P
         Component: All Bugs
          Assignee: lldb-dev at lists.llvm.org
          Reporter: todd.fiala at gmail.com
                CC: llvm-bugs at lists.llvm.org
    Classification: Unclassified

As brought up in llvm.org/pr30271
(https://llvm.org/bugs/show_bug.cgi?id=30271):

Pavel had the idea of re-using build artifacts across test methods and test
files in a given test directory.  This likely would increase the speed with
which we could run the test suite as it eliminates the need to rebuild test
inferiors for any test that includes more than one test method.

My further comments from pr30271:

====8<====

> We could achieve a big speedup by just not re-building the binaries over and over again.

I could see that being a really big win.

We'd have to parameterize the memoization of the built artifacts by anything
that could change.  These would include:
* make command line
* environment variables

I think that already covers the compilers, as they would be specified by CC/CXX
env vars or command line parameters to make.

We'd need a concept of the build artifacts for a given set of build settings.

We could conceivably allow these to be stored in a specified directory, which
could allow builders to preserve them across clean builds of the lldb product
if we so desired.  (We might not want that, so that a clean build is really a
clean build of the test artifacts as well).  We could figure out that later.

I think this is a very worthwhile item to investigate.

I don't know if/how this impacts what we would do if we used lit to run our
tests.  (As I mentioned on a different thread, I have not yet put any effort
into looking at what our test suite would look like running existing tests in
that test runner.  We currently have a number of assumptions about how our
build process interacts with test method runs that may or may not fit into the
lit expectations as they currently stand.  Zach mentioned we could write some
kind of test runner that plugs into lit, which maybe can handle these details. 
In that case, the effort for your suggestion could be easily lifted and used in
both scenarios, and therefore wouldn't be a waste of effort if we switched test
runners).

====8<====

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20160906/6663e889/attachment-0001.html>


More information about the llvm-bugs mailing list