[cfe-dev] RFC: Clang test runner changes

[Daniel Dunbar]

On Wed, Jul 29, 2009 at 5:13 PM, John
Thompson<john.thompson.jtsoftware at gmail.com> wrote:
> Daniel,
>
> I'm so glad you're taking on this work.
>
> Any idea when it might be ready?
> Just out of curiosity, what kind of program with the test runner be?  C++ or
> a script of some sort?
>
> It seems like a one-man job, but if you want any help, please let me know.

FYI, this has progressed far enough that -- with local patches, and
some setup -- it is possible to get interesting test results on
Windows.

Currently on my machine with an assortment of patches I'm down to 121
failures, almost all of which are failures not related to the testing
infrastructure. If you are interested in getting things going further,
a good place would be to start looking at those failures and how to
fix them. If you want to take this one, the first step is to get the
test runner working in your setup from the clang-test project (this is
probably only a matter of passing --no-sh in the custom build step,
and maybe tweaking some paths). If that works for you I can pass along
my extra patches to cut down on the spurious failures (adding
count.bat and not.bat in llvm/test/Scripts is the most major one).

 - Daniel

> Meanwhile, I've got a VMWare virtual machine set up to do Linux, such that
> I'm thinking I'll develop on Windows, but copy over the changes and run the
> tests on Linux, at least until your test runner is ready on Windows.  I like
> the Visual Studio debugger too much...
>
> Question: I just ran the tests on Ubuntu Linux, and it gave a message
> saying: "Expected Failures (17):" followed by a list of 17 files, presumably
> the tests that had failures.  So basically this means that as long as the
> number of files listed matches the number given in the message, the overall
> test is considered successful, right?  I know this is probably obvious, but
> I want to be sure.  Perhaps a note about this in the "Testing" section on
> the hacking page would be a good idea.
>
> -John
> --
> John Thompson
> John.Thompson.JTSoftware at gmail.com
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>