[LNT][PATCH] extract configuration from nightly tests

Renato Golin renato.golin at linaro.org
Fri Aug 9 04:51:10 PDT 2013


On 8 August 2013 21:55, Chris Matthews <chris.matthews at apple.com> wrote:

> This is the first of several patches to refactor the LNT nt test, to make
> it more amenable to single test reruns.
>
> This patch extracts common configuration options into a separate stateless
> object so that they can be accessed again for rerunning.  Common
> functionality from the run is also extracted so that it can be run more
> than once.
>

Hi Chris,

My Python skills are seriously lacking, so I'll only comment on the
description of the functionality.

Is the idea to save this context with with the runs themselves, so that you
can de-serialize it back to object and run "the same configuration" back
again? I'm assuming you're controlling everything on the LLVM side, but not
so much on the system side (library and platform compiler versions,
architecture, etc).

In the case you could save the system information, maybe you could warn the
user that something had changed, but you can't install/change it yourself.

For instance, if the users tries to compare runs A and B, both on
x86_64+Linux, but A has a different glibc version than B, you could warn
saying that some results might be different due to library version mismatch.

Another usage would be to compare the relative performance difference
between two different architectures from the same change in library.

So, if in the first case you found a difference in x86_64, but got a
warning due to libraries, you can go and check other architectures that had
the same library change, to see if there was the same relative difference
in performance *just* due to library change.

cheers,
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20130809/8aa9f907/attachment.html>


More information about the llvm-commits mailing list