[LLVMdev] Validating LLVM
isanbard at gmail.com
Tue Nov 11 22:33:55 PST 2008
On Nov 10, 2008, at 12:59 PM, David Greene wrote:
> Back during the LLVM developer's meeting, I talked with some of you
> about a
> proposal to "validate" llvm. Now that 2.4 is almost out the door,
> it seems a
> good time to start that discussion.
> I've written up a detailed proposal and attached it to this
> message. The goal
> is to ease LLVM use by third parties. We've got consideral
> experience with
> LLVM and the community development model here and see its benefits
> as well as
> challenges. This proposal attempts to address what I feel is the main
> challenge: testing and stability of LLVM between releases.
> Please take a look and send feedback toi the list. I'd like to get
> process moving early in the 2.4 cycle.
Here are my opinions:
I like the idea of regular validation tagging. However, I think that
it should be as automated as possible. I'm worried that validation
testing will be pushed off of people's plates indefinitely – even for
people who care deeply about a particular platform.
Here are the minimal set of tests that should go into a validation
test. (All of these should be done in "Release" mode.)
* Regression Tests - This catches obvious errors, but not major ones.
* Full Bootstrap of LLVM-GCC - LLVM-GCC is a complex program. It's our
second indication if something has gone horribly awry.
* Nightly Testsuite - A very good suite of tests; much more extensive
than a simple bootstrap.
* LLVM-GCC Testsuite - Many thousands of great tests to test the many
facets of the compiler. WAY too few people run these.
As far as I know, Dale's the only one who's been slogging through the
LLVM-GCC testsuite, finding and fixing errors. I think that there are
still many failures that should be addressed.
All four of the above should be run on at least a nightly basis (more
frequently for some, like the regression tests). Each of these are
automated, making that easy. If there are no regressions from the
above four, we could tag that revision as being potentially "valid".
More information about the llvm-dev