[LLVMdev] Validating LLVM
dag at cray.com
Wed Nov 12 15:03:19 PST 2008
On Wednesday 12 November 2008 00:33, Bill Wendling wrote:
> > Please take a look and send feedback toi the list. I'd like to get
> > the
> > process moving early in the 2.4 cycle.
> Hi Dave,
> Here are my opinions:
> I like the idea of regular validation tagging. However, I think that
> it should be as automated as possible. I'm worried that validation
> testing will be pushed off of people's plates indefinitely – even for
> people who care deeply about a particular platform.
Yes, automation is key. I think it is very possible to do with this proposal.
> Here are the minimal set of tests that should go into a validation
> test. (All of these should be done in "Release" mode.)
> * Regression Tests - This catches obvious errors, but not major ones.
For clarity, you mean "make check" on the LLVM tools, right?
> * Full Bootstrap of LLVM-GCC - LLVM-GCC is a complex program. It's our
> second indication if something has gone horribly awry.
> * Nightly Testsuite - A very good suite of tests; much more extensive
> than a simple bootstrap.
How does this differ from "make check" or llvm-test?
> * LLVM-GCC Testsuite - Many thousands of great tests to test the many
> facets of the compiler. WAY too few people run these.
By this do you mean llvm-test or the testsuite that ships with gcc? To
my knowledge, LLVM has never passed the gcc testsuite ("make check"
> As far as I know, Dale's the only one who's been slogging through the
> LLVM-GCC testsuite, finding and fixing errors. I think that there are
> still many failures that should be addressed.
Depending on how you're defining LLVM-GCC I may also be running those tests
> All four of the above should be run on at least a nightly basis (more
> frequently for some, like the regression tests). Each of these are
> automated, making that easy. If there are no regressions from the
> above four, we could tag that revision as being potentially "valid".
Right. I would add one thing. We want to run these suites with Debug,
Release, Release+Asserts and Debug+ExpensiveChecks builds. No one
but me seems to run Debug+ExpensiveChecks tests because I see things
break regularly. It's a valuable tool to find subtle C++ errors.
More information about the llvm-dev