[llvm-commits] [patch][zorg/clang-tests] Process expected failures within the clang-test execution, instead of on the buildmaster

[Duncan Sands]

Hi David,

> Here's a rather cheesy attempt to move the logic that ignores expected
> failures for the clang test suite (specifically for the gdb portion)
> from the buildbot config (executing on the buildmaster) down into the
> test suite Makefile (on the slave). This way the ignore list is read
> live on every execution, instead of just when the buildmaster is
> reconfigured, and the buildmaster machine doesn't need an extra
> private checkout of the clang test suite to read the ignore list from.

doesn't the list depend on what kind of machine the build is running on and
on how LLVM etc was configured?  In theory at least.  Also, if you want to
go this way how about factoring out some helper functions so it is easy to
do this for any kind of test, for example by having a routine that lets you
grab the list for a build when declaring a particular build.

Ciao, Duncan.

>
> The fix consists of two parts:
> 1) zorg.diff, which should be able to be committed separately/ahead of
> time with no impact: this modifies the builder test running command to
> be able to handle logs that already have some IGNORE/FLAKY prefixes on
> test results.
> 2) clang-tests.diff, which adds a python script (open to other ideas
> if anyone has them - but this only runs once at the end of the suite
> execution, so it has a pretty small impact & it's /just/ a bit more
> complicated than really seemed suitable for sed/awk/bash hackery - at
> least to me) & a small modification to the GDB suite Makefile to
> execute the script after runtests is executed & filter the log file
> in-place, adding the desired "IGNORE" prefixes to any ignored tests.
>
> How's this look? reasonable? If so, once this goes in I'll work on
> refactoring it further to account for the GCC portion of the suite as
> well.
>
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>