[LLVMdev] Test Suite - Livermore Loops

[David Blaikie]

On Tue, Jan 8, 2013 at 1:40 AM, Renato Golin <renato.golin at linaro.org> wrote:
> On 8 January 2013 04:49, Duncan Sands <baldrick at free.fr> wrote:
>>
>> While this should be investigated,
>> I'm tempted to just move everything over to LNT instead...
>
>
> That's the latent bugs that David mentioned. I agree we should have LNT and
> LNT+LTO and possibly other configurations in the future.
>
> Regarding your buildbots, gcc12 is easy to replace by LNT, because the
> default, AFAICT, is to run only clang tests (check-all I presume)

I'm confused. If it only runs check-all, why would we be migrating it to LNT?

By the looks of it, the gcc12 slave is assigned to two builders.
Picking a recent build from each:

http://lab.llvm.org:8011/builders/dragonegg-x86_64-linux-gcc-4.6-fnt/builds/2147
http://lab.llvm.org:8011/builders/clang-x86_64-debian/builds/5723

The second is just a clang check-all on debian. That should probably
remain untouched.

The first is dragonegg 'fnt' AKA the test-suite or "nightly tests".
This could be migrated to LNT.

(FWIW: I tend to say "regression tests" or "make check" or something
else when I'm talking about the stuff you get when you run "make
check-all" in LLVM/clang, and reserve the "test suite" term to reply
to the test-suite repository containing a grab bag of application
source code, AKA the "nightly tests". I don't know whether this is
consistent/useful terminology & I'm open to other ideas)

>, but gcc20
> is running the nightly tests, and I'm not sure I should change them to
> "simple" or keep as "nightly" but do it via LNT.

I don't think LNT is able to run the "nightly" version. I believe it
only runs the "simple" version.

> When I added Livermore, I presumed I was running LNT with the nightly
> variant (nt):

Strangely enough, I believe LNT's "nt" corresponds to the test-suite's
"TEST=simple". I could be wrong, though.

>
> ./sandbox/bin/python sandbox/bin/lnt runtest \
>        --submit ~/devel/llvm/test/perfdb \
>        nt \
>        -j8 \
>        --sandbox sandbox \
>        --test-suite ~/devel/llvm/test/test-suite \
>        --cc ~/devel/llvm/build/bin/clang \
>        --cxx ~/devel/llvm/build/bin/clang++ \
>        --only-test SingleSource/Benchmarks/LivermoreLoops
>
> Is that right?
>
> cheers,
> --renato