[llvm-dev] How LLVM guarantee the qualify of the product from the limited test suit?

[Alex Bradbury via llvm-dev]

On 9 November 2015 at 09:44, Renato Golin via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> The GCC test-suite, AFAIK, has very poor quality on what's considered
> a pass or a failure, and it's common to release GCC with thousands of
> failures on those tests. Some people may run it, but I honestly don't
> trust it myself, nor have the time to sift through every single test
> to make sure my errors are compiler errors or test errors. You can't
> assume that just because GCC runs *more* tests, that what they're
> testing is more *thorough*. There are also lots of tests that have
> erratic behaviour, which only adds noise to the process.

I've found the gcc test suite pretty useful for my out-of-tree
research backend, though it does require some initial work in
disabling tests that exercise GCC features unsupported by Clang. I
haven't yet switched to using it, but Ed Jones at Embecosm did some
work on making the gcc test suite easier to use with Clang
http://www.embecosm.com/2015/04/21/flexible-runtime-testing-of-llvm-on-embedded-systems/

Alex