[Lldb-commits] [PATCH] D54914: Add a generic build script for building test inferiors

[Zachary Turner via Phabricator via lldb-commits]

zturner added a comment.

In D54914#1309901 <https://reviews.llvm.org/D54914#1309901>, @aprantl wrote:

> I would like to ask a general question that I (indirectly) also asked in D54731 <https://reviews.llvm.org/D54731>: Why do we want to implement support for building inferiors in LIT-based tests?


I think this is not much different than asking "Why do we want LIT-based tests?".  If you can't build an inferior, then you can't really do anything with LLDB at all, in which case there's nothing to test.  So without support for building inferiors, there is no such thing as lldb lit tests.

> IMHO, if we need to support for dealing with specific compilers, we should implement that once in `Makefile.rules` (which is in a declarative domain-specific-language for describing build logic) and write a `dotest.py`-style test instead. I'm assuming here that we need the support in `Makefile.rules` anyway to support the bulk of the tests. Having this support in two places increases the maintenance effort and cognitive load.

At least for the time being, I think of dotest.py style tests as being synonymous with "SB API test".  I think there is a need for a testing framework that is more familiar to LLVM developers with a lower barrier to entry as it encourages people to write more tests, which is always a good thing.  FWIW, I think the results so far have been positive.  I did a search over a 3-month period on the repository.  During that time, 37 new lit tests have been added, and 32 dotest tests have been added while 9 have been removed.

Your point about having the support in two places increasing the maintenance effort is reasonable, but if the build script actually does support everything that the Makefile does, then perhaps we could port the existing dotest tests over to use the build script.  Make is notoriously finnicky and many people don't like it or understand it well and it's very easy to run into portability problems when using it.  So if we can eventually do that, that would actually *decrease* the cognitive load and maintenance burden.  I'm not proposing that at this point and I think the benefits of a system such as the one in this patch stand on their own regardless of whether we port dotest builders over to use this script, but it's something to think about regardless.

Note that the Makefiles do not currently support clang-cl since it uses a totally different command line syntax.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D54914/new/

https://reviews.llvm.org/D54914