[PATCH][LNT] Add support for more detailed profiling to lnt.

[Kristof Beyls]

Yes, absolutely!

 

Right now, we’re slowly getting to the point where we’re seeing the significant performance changes on http://llvm.org/perf.
I’m currently focussing mainly on improving the daily report page (http://llvm.org/perf/db_default/v4/nts/daily_report) as that
seems the most useful report to get an overview of the significant changes.

 

But once that is working well, the next step is to make it easy to understand what caused the significant performance change.

Indeed, something visual that helps to intuitively understand the main changes in the profile + a way to also see the corresponding
changes in assembly code.

 

Measuring the profile + recording the assembly code for all programs at every run will likely be prohibitively expensive – both
due to the performance tracking bots slowing down and the storage capacity needed to store the information for all runs on the server.
But maybe the rerun-functionality could be extended so that for the significant changes, the bot is asked to do extra profiling runs
of the programs with significant performance deltas and is asked to also submit the assembly produced. That will need to be done
for both the current version of clang under test + the previous version. How to make sure the bot caches a number of previous versions
(or rebuilds it) is probably not entirely trivial. We’ll also need to make sure that the profiles and the assembly code also don’t
massively increase the storage needs on the server side. But all-in-all, off the top of my head, this seems to be the most straightforward
way to get to a system like you describe below.

 

So, yes, absolutely, being able to detect significant performance deltas is not the end goal, but rather a system like you describe below.

 

Kristof

 

From: Sean Silva [mailto:chisophugis at gmail.com] 
Sent: 25 June 2015 03:23
To: Kristof Beyls
Cc: llvm-commits; Tobias Grosser
Subject: Re: [PATCH][LNT] Add support for more detailed profiling to lnt.

 

It would be awesome if when we detect a performance change of one of the benchmarks (not the LNT server), we did a similar analysis across the two versions and presented a graph like this, but where the weight is the performance diff of the function. Bonus points for making the graph nodes clickable and showing an annotated assembly diff.

 

-- Sean Silva

 

On Wed, Jun 24, 2015 at 10:15 AM, Kristof Beyls <kristof.beyls at arm.com> wrote:

Hi,

 

The attached patch adds support for getting call-graph profiles from lnt runserver. It does this by adding
a new –profiler-dir command line option, which results in a pstat profile to be stored to a separate file
in that sub directory, per web request. I find the extra call graph info very
useful when trying to understand what the bottlenecks are in the rendering of pages by the
lnt webserver.

 

The pstat files can easily be converted into something graphical by a command like:

$ gprof2dot -f pstats profile_dir/POST.submitRun.000591ms.1435075964.prof |  dot -Tsvg > submitRun.svg

 

I’ve attached such an svg graph for a request of the daily report page to give an idea of
the kind of information this more detailed profiling can give.

 

Does the patch look OK to commit?

 

Thanks,

 

Kristof


_______________________________________________
llvm-commits mailing list
llvm-commits at cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20150625/a701db7e/attachment.html>