[PATCH] Teach CMake build system to run lit's own test suite

[Jonathan Roelofs via llvm-commits]

On 12/28/15 2:45 AM, Dan Liew wrote:
> Hi,
>
> Jonathan Roelofs recently remarked that we aren't running lit's (aka
> llvm-lit) own test suite as part of the ``check-all`` target.
>
> The attached patch is a first attempt at fixing this by
>
> * Adding a ``check-lit`` target which runs lit's test suite.
> * Due to the implementation the tests are implicitly run as part of
> ``make check-all``.
>
> Two issues may need addressing:
>
> 1. Is this the right implementation? One could argue that lit's own
> test suite should be run before testing any of LLVM with lit. This
> implementation seemed simpler though. If we do decide to implement
> this differently we need to decide if running ``check-lit`` is a
> prerequisite of only ``check-all`` or all of the other ``check-*``
> targets (this might be annoying when developing because the lit test
> suite is slow due to running the timeout tests).

Make them faster? Timeout tests don't have to be slow. Just set them up 
as infinite loops that get killed by a very short timeout. If the 
timeout is working, the test runs quick. If the timeout is not working, 
then the test runs slowly (or never finishes), indicating that the test 
is broken.

>
> 2. I'm concerned that this may break several build bots. Although the
> lit test suite is supposed to be portable I'm not sure if it has been
> run on all the platforms that we build LLVM on. I'm wondering if we
> should give owners of the buildbot slaves a heads up before this gets
> committed. Also if we want all features of lit to be tested the owners
> will need to install the ``psutil`` python module which is currently
> required to use the newly added ``--timeout=`` feature.

If there are buildbots that would fail this testsuite, we've got bigger 
problems.

One more issue:

The lit testsuite writes temporaries to the src dir, which is 
inappropriate for out-of-source builds. I implemented this in 
http://reviews.llvm.org/D14722, and promptly forgot about it.


Jon

>
> Thanks,
> Dan.
>

-- 
Jon Roelofs
jonathan at codesourcery.com
CodeSourcery / Mentor Embedded