[LLVMdev] RFC: Dropping DejaGNU

[Daniel Dunbar]

Hi all,

As you may or may not have noticed, I have gradually been replacing
the testing infrastructure in LLVM. Most of this work is moving LLVM
and Clang to share a single testing tool, 'lit'. See:
  http://llvm.org/cmds/lit.html
for documentation on the tool itself.

If you haven't already tried it, please consider switch to 'make
check-lit' as an alternative to 'make check'. If it doesn't work for
you, or you find it doesn't do something DejaGNU did and you like,
please let me know. My eventual plan is to move to lit entirely and
drop DejaGNU support, so consider yourself warned.

A couple key points about 'make check-lit':
 1. It runs the unittests as well, there is no separate 'make
unittests' step. If you like, there is also a 'make check-all' which
will also run the Clang tests at the same time if it happens to be
checked out in the standard location (tools/clang).
 2. The public buildbots have switched to it. It shouldn't happen, but
if you do find a discrepancy between DejaGNU and lit, let me know.

If you aren't already familiar with it, here are some of the advantages of lit:
 1. It is portable. It already is being used on the Windows buildbot
for Clang, and almost all of our tests "just work".
 2. It uses multiple threads to test. While it doesn't yet scale quite
as well as I would like, switching the Linux x86_64 buildbot to it
dropped its cycle time by over 30% (just because running the tests got
about 3x faster).
 3. I've tried hard to make it easy to use, and will keep working in
this direction. Eventually I want to replace the TestRunner scripts as
well so that all that is needed to run a single test is 'lit
path/to/the/test'. I also would like to improve the output for
failures to be more informative (particularly when run via buildbot).

See http://llvm.org/PR5217 for my TODO list before dropping DejaGNU,
feel free to add comments, requests, or blocking bugs to that one.

 - Daniel