[llvm-dev] Adding a new External Suite to test-suite

[Fernando Magno Quintao Pereira via llvm-dev]

Hi Johannes,

> I'd also like to know what the intention here is. What is tested and how?

    we have a few uses for these benchmarks in the technical report:
http://lac.dcc.ufmg.br/pubs/TechReports/LaC_TechReport012020.pdf, but
since then, we came up with other applications. All these programs
produce object files without external dependencies. We have been using
them to train a predictive compiler that reduces code size (the
technical report has more about that). In addition, you can use them
to compare compilation time, for instance, as Michael had asked. We
have also used these benchmarks in two studies:

1) http://cuda.dcc.ufmg.br/angha/chordAnalysis
2) http://cuda.dcc.ufmg.br/angha/staticProperties

A few other applications that I know about (outside our research
group), include:

* Comparing the size of code produced by three HLS tools: Intel HLS,
Vivado and LegUp.
* Testing the Ultimate Buchi Automizer, to see which kind of C
constructs it handles
* Comparing compilation time of gcc vs clang

A few other studies that I would like to carry out:

* Checking the runtime of different C parsers that we have.
* Trying to infer, empirically, the complexity of compiler analyses
and optimizations.

> Looking at a few of these it seems there is not much you can do as it is little code with a lot of unknown function calls and global symbols.

Most of the programs are small (avg 63 bytecodes, std 97); however,
among these 1M C functions, we have a few large ones, with more than
40K bytecodes.

Regards,

Fernando