[LLVMdev] Validating LLVM
regehr at cs.utah.edu
Mon Nov 10 20:17:53 PST 2008
Lately our random C program generator has seemed quite successful at
catching regressions in llvm-gcc that the test suite misses. I'd suggest
running some fixed number of random programs as part of the validation
suite. On a fastish quad core I can test about 25,000 programs in 24
hours. Our hacked valgrind (which looks for volatile miscompilations) is
a bottleneck, leaving this out would speed up the process considerably.
We've never tested llvm-gcc for x64 using random testing, doing this would
likely turn up a nice crop of bugs.
I just started a random test run of llvm-gcc 2.0-2.4 that should provide
some interesting quantitative results comparing these compilers in terms
of crashes, volatile miscompilations, and regular miscompilations.
However it may take a month or so to get statistical significance since
2.3 and 2.4 have quite low failure rates.
More information about the llvm-dev