[lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Jim Ingham via lldb-dev]

> On Jan 17, 2018, at 3:32 PM, Zachary Turner <zturner at google.com> wrote:
> 
> On Wed, Jan 17, 2018 at 3:26 PM Greg Clayton <clayborg at gmail.com> wrote:
> I don't see why we would go with a lit based system that manually specifies the arguments. Seems like a pain to get the right compiler flags for creating shared libs on different systems (or executables, frameworks, etc). Seems like cmake is good at that and very simple.
> 
> One reason it's nice is because you can specify more than just a compiler command line.  You can take your input from an assembly file, for example and compile it with llc
> 
>  You can mess around with files and stick them into an archive or you can compile a dll, then run a tool on it to strip something out of it.  You can copy files around to set up a build directory a certain way.  Only limited by your imagination.
> 
> When the compiler invocation is "just another command", you can easily create test cases that are a lot more interesting than just "make an executable from this source code, and debug it".
> 
> I think it can be done, and would be valuable if done right, but I do think getting it right would take some care.

I don't see any of these operations that can't be done in a make file, after all you can run arbitrary commands there.  We do make directories, dylibs, move and strip files, etc in some of the makefiles in the test case.

OTOH, it is pretty common to have a test directory that has a Test.py with a bunch of test cases that all build the same thing.  If we use a command-style driver for building, each of these tests will do a full rebuild, whereas now make figures out what needs to be done, and the build only actually happens once.  If we think avoiding extra compiles is important then you do want the build tool to be able to compute dependencies.

Jim