[cfe-dev] Rotten Green Tests project

[via cfe-dev]

Thanks, James!

> When writing googletest unit tests, I almost always run the test
> executable directly. This is because it's by far the easiest way
> to run the test and debug in Visual Studio ("Set startup project"
> -> F5).

This is exactly the use-case I had in mind.

> I wouldn't be happy if this started showing up false test
> failures in some form or other, unless someone can point at an
> equally simple way of doing the same thing.

None of them show up as individual *test* failures; what happens
is you get a report at the end with a list of rotten assertions.
The test *program* will have a failure exit status, but the list
of passing/failing tests is unaffected.

At the moment, the number of false positives appears to be small-ish.

A DISABLED test will show up as rotten, unless you run the test with
--gtest_also_run_disabled_tests; LLVM has 3 test programs like this,
out of 56, and Clang has two out of 22.

There are also some cases where EXPECT_NONFATAL_FAILURE has another 
EXPECT as an argument, and that doesn't seem to work with RGT (but I 
haven't figured out whether this is a case of Don't Do That, or 
something else). There are two LLVM unittest programs like this.

There might be more, I haven't analyzed all the failure reports to
track down exactly what's going on.  Most of the rest are likely
issues with the test programs themselves.

--paulr