[LLVMdev] Test Suite - Livermore Loops

[Daniel Dunbar]

On Tue, Jan 8, 2013 at 8:39 AM, David Blaikie <dblaikie at gmail.com> wrote:

> On Tue, Jan 8, 2013 at 1:40 AM, Renato Golin <renato.golin at linaro.org>
> wrote:
> > On 8 January 2013 04:49, Duncan Sands <baldrick at free.fr> wrote:
> >>
> >> While this should be investigated,
> >> I'm tempted to just move everything over to LNT instead...
> >
> >
> > That's the latent bugs that David mentioned. I agree we should have LNT
> and
> > LNT+LTO and possibly other configurations in the future.
> >
> > Regarding your buildbots, gcc12 is easy to replace by LNT, because the
> > default, AFAICT, is to run only clang tests (check-all I presume)
>
> I'm confused. If it only runs check-all, why would we be migrating it to
> LNT?
>
> By the looks of it, the gcc12 slave is assigned to two builders.
> Picking a recent build from each:
>
>
> http://lab.llvm.org:8011/builders/dragonegg-x86_64-linux-gcc-4.6-fnt/builds/2147
> http://lab.llvm.org:8011/builders/clang-x86_64-debian/builds/5723
>
> The second is just a clang check-all on debian. That should probably
> remain untouched.
>
> The first is dragonegg 'fnt' AKA the test-suite or "nightly tests".
> This could be migrated to LNT.
>
> (FWIW: I tend to say "regression tests" or "make check" or something
> else when I'm talking about the stuff you get when you run "make
> check-all" in LLVM/clang, and reserve the "test suite" term to reply
> to the test-suite repository containing a grab bag of application
> source code, AKA the "nightly tests". I don't know whether this is
> consistent/useful terminology & I'm open to other ideas)
>

I also use "regression tests" to refer to the LLVM and clang tests.


>
> >, but gcc20
> > is running the nightly tests, and I'm not sure I should change them to
> > "simple" or keep as "nightly" but do it via LNT.
>
> I don't think LNT is able to run the "nightly" version. I believe it
> only runs the "simple" version.
>

See previous reply, --test-style=nightly does it.


>
> > When I added Livermore, I presumed I was running LNT with the nightly
> > variant (nt):
>
> Strangely enough, I believe LNT's "nt" corresponds to the test-suite's
> "TEST=simple". I could be wrong, though.
>

The "nt" is a historical accident, we didn't have a good name for the
"test-suite" other than "nightly tests" (and calling it the "test suite"
has obvious problems), so I just started using "nt" to refer to it despite
the broken connotations.

 - Daniel


> >
> > ./sandbox/bin/python sandbox/bin/lnt runtest \
> >        --submit ~/devel/llvm/test/perfdb \
> >        nt \
> >        -j8 \
> >        --sandbox sandbox \
> >        --test-suite ~/devel/llvm/test/test-suite \
> >        --cc ~/devel/llvm/build/bin/clang \
> >        --cxx ~/devel/llvm/build/bin/clang++ \
> >        --only-test SingleSource/Benchmarks/LivermoreLoops
> >
> > Is that right?
> >
> > cheers,
> > --renato
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130109/e6645cdc/attachment.html>