[cfe-dev] Rotten Green Tests project
cfe-dev at lists.llvm.org
Mon Mar 1 07:25:51 PST 2021
> When writing googletest unit tests, I almost always run the test
> executable directly. This is because it's by far the easiest way
> to run the test and debug in Visual Studio ("Set startup project"
> -> F5).
This is exactly the use-case I had in mind.
> I wouldn't be happy if this started showing up false test
> failures in some form or other, unless someone can point at an
> equally simple way of doing the same thing.
None of them show up as individual *test* failures; what happens
is you get a report at the end with a list of rotten assertions.
The test *program* will have a failure exit status, but the list
of passing/failing tests is unaffected.
At the moment, the number of false positives appears to be small-ish.
A DISABLED test will show up as rotten, unless you run the test with
--gtest_also_run_disabled_tests; LLVM has 3 test programs like this,
out of 56, and Clang has two out of 22.
There are also some cases where EXPECT_NONFATAL_FAILURE has another
EXPECT as an argument, and that doesn't seem to work with RGT (but I
haven't figured out whether this is a case of Don't Do That, or
something else). There are two LLVM unittest programs like this.
There might be more, I haven't analyzed all the failure reports to
track down exactly what's going on. Most of the rest are likely
issues with the test programs themselves.
More information about the cfe-dev