[LLVMdev] RFC: Dropping DejaGNU

[Daniel Dunbar]

On Sun, Oct 18, 2009 at 9:00 PM, Chris Lattner <clattner at apple.com> wrote:
>
> On Oct 18, 2009, at 7:49 PM, Daniel Dunbar wrote:
>
>> Hi all,
>>
>> As you may or may not have noticed, I have gradually been replacing
>> the testing infrastructure in LLVM. Most of this work is moving LLVM
>> and Clang to share a single testing tool, 'lit'. See:
>>  http://llvm.org/cmds/lit.html
>> for documentation on the tool itself.
>>
>> If you haven't already tried it, please consider switch to 'make
>> check-lit' as an alternative to 'make check'. If it doesn't work for
>> you, or you find it doesn't do something DejaGNU did and you like,
>> please let me know. My eventual plan is to move to lit entirely and
>> drop DejaGNU support, so consider yourself warned.
>
> This is great work Daniel,
>
> One question: Why not switch 'make check' to default to lit and offer 'make
> check-dejagnu' as a workaround.  That forces adoption :)

I intend to as soon as it won't break anything. The sole blocker for
doing that is making sure it doesn't break the nightly testing
infrastructure.

 - Daniel

> -Chris
>
>>
>> A couple key points about 'make check-lit':
>> 1. It runs the unittests as well, there is no separate 'make
>> unittests' step. If you like, there is also a 'make check-all' which
>> will also run the Clang tests at the same time if it happens to be
>> checked out in the standard location (tools/clang).
>> 2. The public buildbots have switched to it. It shouldn't happen, but
>> if you do find a discrepancy between DejaGNU and lit, let me know.
>>
>> If you aren't already familiar with it, here are some of the advantages of
>> lit:
>> 1. It is portable. It already is being used on the Windows buildbot
>> for Clang, and almost all of our tests "just work".
>> 2. It uses multiple threads to test. While it doesn't yet scale quite
>> as well as I would like, switching the Linux x86_64 buildbot to it
>> dropped its cycle time by over 30% (just because running the tests got
>> about 3x faster).
>> 3. I've tried hard to make it easy to use, and will keep working in
>> this direction. Eventually I want to replace the TestRunner scripts as
>> well so that all that is needed to run a single test is 'lit
>> path/to/the/test'. I also would like to improve the output for
>> failures to be more informative (particularly when run via buildbot).
>>
>> See http://llvm.org/PR5217 for my TODO list before dropping DejaGNU,
>> feel free to add comments, requests, or blocking bugs to that one.
>>
>> - Daniel
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>