[cfe-dev] RFC: Clang test runner changes

[Eli Friedman]

On Wed, Jul 29, 2009 at 5:13 PM, John
Thompson<john.thompson.jtsoftware at gmail.com> wrote:
> Just out of curiosity, what kind of program with the test runner be?  C++ or
> a script of some sort?

I'm assuming it'll stay in Python unless there's some reason why that
isn't sufficient.

> Question: I just ran the tests on Ubuntu Linux, and it gave a message
> saying: "Expected Failures (17):" followed by a list of 17 files, presumably
> the tests that had failures.  So basically this means that as long as the
> number of files listed matches the number given in the message, the overall
> test is considered successful, right?  I know this is probably obvious, but
> I want to be sure.  Perhaps a note about this in the "Testing" section on
> the hacking page would be a good idea.

You're okay as long as there isn't a section called "Failing Tests".
Perhaps the testrunner should be tweaked not to print expected
failures, though, since the information there can be easily gathered
using grep.

-Eli