[LLVMdev] Dev Meeting BOF: Performance Tracking

[Yi Kong]

On 2 August 2014 00:40, Renato Golin <renato.golin at linaro.org> wrote:
> If memory serves me well (it doesn't), these are the list of things we
> agreed on making, and their progress:
>
> 1. Performance-specific test-suite: a group of specific benchmarks
> that should be tracked with the LNT infrastructure. Hal proposed to
> look at this, but other people helped implement it. Last I heard there
> was some way of running it but I'm not sure how to do it. I'd love to
> have this as a buildbot, though, so we can track its progress.

We have this in LNT now which can be activated using `--benchmarking-only`.
It's about 50% faster than a full run and massively reduces the number of
false positives.

Chris has also posted a patch to rerun tests which the server said changed.
Haven't tried it yet but it looks really promising.

> 2. Statistical analysis of the LNT data. A lot of work has been put
> into this and I believe it's a lot better. Anton, Yi and others have
> been discussing and submitting many patches to make the LNT reporting
> infrastructure more stable, less prone to noise and more useful all
> round. It's not perfect yet, but a lot better than last year's.

There's definitely lots of room for improvement. I'm going to propose some
more once we've solved the LNT stability issues.

> 3. LNT website got really unstable (Internal Server Error every other
> day). This is the reason I stopped submitting results to it, since it
> would make my bot fail. And because I still don't have a performance
> test-suite bot, I don't really care much for the results. But with the
> noise reduction, it'd be really interesting to monitor the progress,
> even of the full test-suite, but right now, I can't afford to have
> random failures. This seriously needs looking into and would be good
> to have that as a topic in this BoF.

We are now testing PostgreSQL as database backend on the public perf
server, replacing the SQLite db. Hopefully this can improve the stability
and system performance.

Also being discussing is to move the LNT server to a PaaS service, as it
has higher availability and saves a lot of maintenance work. However this
will need community to provide or fund the hosting service.

-Yi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140802/e9716261/attachment.html>