[llvm-dev] Performance metrics with LLVM

[Kristof Beyls via llvm-dev]

Hi Tobias,

The metrics that you can collect in LNT are fixed per "test suite".
There are 2 such "test suite"s defined in LNT at the moment: nts and compile.
For more details on this, see http://llvm.org/docs/lnt/concepts.html#test-suites.

AFAIK, If you need to collect different metrics, you'll need to define a new "test suite". I'm afraid I don't really know what is needed for that.
I'm guessing you may need to write some LNT code to do so, but I'm not sure. Hopefully Matthias or Chris will be able to explain how to do that.

We probably should investigate how to make it easier to define new "test-suite"s more easily. Or at least make it easier to record different sets of metrics more easily, without having to change the LNT code or a running LNT server instance.
The question on recording a different set of metrics has come up on this list before, so it seems like it's an issue people do run into from time to time.

Thanks,

Kristof


On 4 Jul 2017, at 08:27, Tobias Grosser <tobias.grosser at inf.ethz.ch<mailto:tobias.grosser at inf.ethz.ch>> wrote:

Dear all,

I wanted to gather LLVM statistics with lnt and found a nice flag, but
am unsure how such statistics can be made available in the LNT web
interface.

--cmake-define=TEST_SUITE_COLLECT_STATS=ON

which allows me to gather all the LLVM "-stats" output. On top of this I
see that the LNT cmake test-suite also dumps code-size statistics when
running, that look as follows:

size: 10848
size..bss: 48
size..comment: 218
size..ctors: 16
size..data: 4
size..dtors: 16
size..dynamic: 416
size..dynsym: 168
size..eh_frame: 172
size..eh_frame_hdr: 44

I can find all these statistics in a file called:

/scratch/leone/grosser/base/sandbox/test-2017-07-04_06-14-43/outputTd2xPU.json

but they do not appear in:

/scratch/leone/grosser/base/sandbox/test-2017-07-04_06-14-43/report.json

and in fact do not seem to be submitted to the LNT server.

Matthias added support for TEST_SUITE_COLLECT_STATS a while ago, but I
am unsure how it is expected to be used. A google search did not find
any relevant documentation. Is anybody using this feature today?

Best,
Tobias

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170704/99ba4270/attachment-0001.html>