LIT-ify the test-suite

[James Molloy]

Hi all,

We've been trying to improve the LNT test driver to be a bit more clever
with reruns, and have run into the problem that it is very tightly coupled
to the Makefile system in the test-suite.

The Makefile system was obviously designed some time ago before Clang was
production ready, and it has a bunch of features to support running through
LLC and suchlike that add serious complexity. It's nontrivial to just run a
test and get a result!

So I got to thinking, why don't we just use LIT, the really well written
test driver that our regression tests use? That would allow us to simplify
the interface, reduce the number of test drivers to 1, have pretty progress
bars, export the results in JSON, shuffle the test order...

So I attach an alpha version of a lit.cfg for the test-suite. Some caveats
with this that make it not quite production-ready:
  * It doesn't use perf for timing yet, it just uses python's time()
interface.
  * It doesn't yet report compile time, but that's actually a one line
change.
  * The biggest problem: A bunch of the test suites require lit.local.cfg
files in them to tell lit how to build and run them. However, many of the
test suite directories are under blanket non-LLVM licenses which makes for
problems with our legal permissions.

So attached are two patches: 0001 adds the lit.cfg - just apply it to the
test-suite directory, and 0002 adds a selection of lit.local.cfgs - for all
the subfolders that aren't covered by blanket licenses.

Because it excludes some lit.local.cfgs, some tests won't run. However, all
the unit tests do, so that's an easy proof of concept:

$ cd test-suite
$ CC=clang llvm-lit -sv ./MultiSource/UnitTests

Please give it a try and let us know your thoughts, so I know how hard I
should push this.

Cheers,

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20141024/de3af042/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Add-lit.cfg.patch
Type: application/octet-stream
Size: 19384 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20141024/de3af042/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-Add-lit.local.cfgs.patch
Type: application/octet-stream
Size: 110856 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20141024/de3af042/attachment-0001.obj>