[lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Jim Ingham via lldb-dev]

> On Jan 17, 2018, at 3:25 PM, Zachary Turner <zturner at google.com> wrote:
> 
> I don't know what would be involved in getting the tests building out of tree with Make.  But I do know it would be simple with CMake.  I'm sure it's probably not terrible with Make either, I just don't know enough about it to say.
> 
> One thing that I do like about CMake is that it can be integrated into the existing LLDB build configuration step, which already uses CMake, to build inferiors up front.  This has the potential to speed up the test suite by an order of magnitude.
> 
> Can we get that same effect with a Make-based solution?

You are going to muck with a bunch of tests to get this to work.  For instance, only dotest currently knows what debug variants we are building for (they are often specified in the test file itself.)  Also a number of tests - for things like rebuild & rerun and others build several times in the test.  The test for basic types has a single source file with the basic type in a define that runs the build & debug once for each basic type, supplying the appropriate define each time.

So for reasons not related to make vrs. cmake I think this is a harder thing than you are thinking it is.

Jim


> 
> On Wed, Jan 17, 2018 at 3:18 PM Jim Ingham <jingham at apple.com> wrote:
> Yeah, w.r.t. the actual builder part, it seems to me any option is going to be sufficiently simple to use that it would be hard for the incremental benefits to lldb developers to ever amortize the cost of switching.  The only compelling reason to me is if one or the other tool made it much easier to get building the test cases out of tree working, but that seems unlikely.
> 
> Jim
> 
> 
> > On Jan 17, 2018, at 3:07 PM, Zachary Turner <zturner at google.com> wrote:
> >
> >
> >
> > On Wed, Jan 17, 2018 at 3:04 PM Adrian Prantl <aprantl at apple.com> wrote:
> >
> > On the other hand:
> > - everybody already knows make
> >
> > I'm not sold on this particular reason.  Make is not the LLVM build system, CMake is.  "I don't know the build system of the project I actually work on, but I do know this other build system" is not a compelling argument.
> >
> > (As an aside, not every knows Make that well, but it doesn't actually matter because the amount of actual Make code is negligibly small, i.e. 1-2 lines per test in a vast majority of cases)
>