[llvm-dev] Benchmarks for LLVM-generated Binaries

[Renato Golin via llvm-dev]

On 1 September 2016 at 07:45, Dean Michael Berris via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> I've lately been wondering where benchmarks for LLVM-generated binaries are hosted, and whether they're tracked over time.

Hi Dean,

Do you mean Perf?

http://llvm.org/perf/

Example, ARM and AArch64 tracking performance at:

http://llvm.org/perf/db_default/v4/nts/machine/41
http://llvm.org/perf/db_default/v4/nts/machine/46


> - Is the test-suite repository the right place to put these generated-code benchmarks?

I believe there would be the best place, yes.


> - Are there any objections to using a later version of the Google benchmarking library [0] in the test-suite?

While this looks like a very nice tool set, I wonder how we're going
to integrate it.

Checking it out in the test-suite wouldn't be the best option (version
rot), but neither would be requiring people to install it before
running the test-suite, especially if the installation process isn't
as easy as "apt-get install", like all the other dependencies.


> - Are the docs on the Testing Infrastructure Guide still relevant and up-to-date, and is that a good starting point for exploration here?

Unfortunately, that's mostly for the "make check" tests, not for the
test-suite. The test-suite execution is covered by LNT's doc
(http://llvm.org/docs/lnt), but it's mostly about LNT internals and
not the test-suite itself.

However, it's not that hard to understand the test-suite structure. To
add new tests, you just need to find a suitable place { (SingleSource
/ MultiSource) / Benchmarks / YourBench } and copy ( CMakeFiles.txt,
Makefile, lit.local.cfg ), change to your needs, and it should be
done.

cheers,
--renato