[lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Zachary Turner via lldb-dev]

If we're going to be making any significant changes to the way inferiors
are compiled, why not use cmake?  Make clean is already not implemented
correctly in many places, leading to lots of remnants left over in the
source tree after test runs.  Furthermore, make is run every single time
currently, leading to hundreds (if not thousands) of unnecessary
compilations.  Seems to me like all the inferiors should be compiled one
time, up front, as part of the configure step, and into the build
directory.  This is nice because it already integrates perfectly into the
existing LLVM "way" of building things.

On Wed, Jan 17, 2018 at 12:32 PM Adrian Prantl via lldb-dev <
lldb-dev at lists.llvm.org> wrote:

> Hi lldb-dev!
>
> I've been investigating some spurious LLDB test suite failures on
> http://green.lab.llvm.org/green/ that had to do with build artifacts from
> previous runs lying around in the test directories and this prompted me to
> ask a couple of general noob questions about the LLDB testsuite.
>
> My understanding is that all execution tests are compiled using using
> `make` in-tree. I.e.: the test driver (dotest.py) effectively executes
> something equivalent to `cd $srcdir/packages/.../mytest && make`. And it
> does this in a serial fashion for all configurations (dwarf, dSYM, dwo,
> ...) and relies on the `clean` target to be implemented correctly.
>
> I don't understand all the design decisions that went into the LLDB
> testsuite, but my naive intuition tells me that this is sub-optimal
> (because of the serialization of the configurations) and dangerous (because
> it relies on make clean being implemented correctly). It seems to me that a
> better approach would be to create a separate build directory for each test
> variant and then invoke something like `cd $builddir/test/mytest.dwarf &&
> make -C $srcdir/packages/.../mytest`. This way all configurations can build
> in parallel, and we can simply nuke the build directory afterwards and this
> way get rid of all custom implementations of the `clean` target.
>
> - Is this already possible, and/or am I misunderstanding how it works?
> - Would this be a goal that is worthwhile to pursue?
> - Is there a good reason why we are not already doing it this way?
>
> thanks,
> adrian
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20180117/8aae8258/attachment.html>