[lldb-dev] Questions about the LLDB testsuite and improving its reliability

[Adrian Prantl via lldb-dev]

> On Jan 17, 2018, at 2:56 PM, Jim Ingham <jingham at apple.com> wrote:
> 
>> 
>> On Jan 17, 2018, at 2:55 PM, Adrian Prantl <aprantl at apple.com> wrote:
>> 
>> 
>> 
>>> On Jan 17, 2018, at 2:50 PM, Jim Ingham <jingham at apple.com> wrote:
>>> 
>>> 
>>> 
>>>> On Jan 17, 2018, at 2:49 PM, Adrian Prantl <aprantl at apple.com> wrote:
>>>> 
>>>> Question: how would you feel about converting the Makefiles to LIT-style .test files with very explicit RUN-lines?
>>> 
>>> I'm not sure what you mean by this.
>> 
>> Instead of using a build system at all to build the tests, we would hard-code the compiler and linker invocations without encoding any dependencies. Because we still need this to be configurable, it would probably look something like this:
>> 
>> RUN: %CXX test.cpp -O0 %CXXFLAGS -o test.exe
>> RUN: %test_driver test.exe mytest.py
> 
> I'm worried we'd back into building another make system over time.  What advantage would we get from this.

(It's possible that this isn't the right trade-off, I'm just exploring ideas here)

Some advantages would be:
- remove the dependency on Make
- possibly easier top debug testcases because all actions are explicit
- potentially slightly faster build times because we don't need to spawn make, but note that it also means that these actions will always be unconditionally executed
- uniformity with other LLVM projects and thus a smaller cognitive burden for developers that touch both clang and lldb

On the other hand:
- everybody already knows make
- maybe we do want a full build system to allow incremental builds of testcases

-- adrian