[LLVMdev] Bad test health

[David Blaikie]

On Wed, Feb 12, 2014 at 9:21 PM, Owen Anderson <resistor at mac.com> wrote:

>
> On Feb 12, 2014, at 7:19 PM, Nico Rieck <nico.rieck at gmail.com> wrote:
>
> Oh, and some tests don't even call FileCheck in the first
> place!
>
>
> That's probably because it's a regression test for a crasher.  It's
> checking that the compiler exited successfully.
>

I assume Nico means it's a test with CHECK lines but doesn't invoke
FileCheck.

Though personally I object to tests that don't check anything other than
"doesn't crash" - if a certain example previously crashed and now doesn't,
there was some codepath that was untested (its output unconstrained). The
crash just revealed the untested codepath, testing that it doesn't crash
still leaves the behavior mostly unconstrained to "do anything other than
crashing" which seems insufficient. We should take the opportunity to test
the untested behavior that's been discovered.

- David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140213/74200409/attachment.html>