[llvm-dev] Adding a new External Suite to test-suite

[Michael Kruse via llvm-dev]

I suggest to have a look into the CMakeLists.txt of the test-suite.
These are mostly self-explanatory.

Am Mo., 6. Apr. 2020 um 04:25 Uhr schrieb Bruno Conde Kind via
llvm-dev <llvm-dev at lists.llvm.org>:
> Q1: What do I have to do to start integrating my collection of .c files as an
> External Suite? They are all single source, single function,
> compilable headerless files extracted from open source repositories.
> They are grouped in folders, in a similar way the original project
> was.

External tests are in
https://github.com/llvm/llvm-test-suite/tree/master/External.
Tests where each source file is one executable are in
https://github.com/llvm/llvm-test-suite/tree/master/SingleSource
Tests where each source file in a directory compiles to object files
that is linked to an executable are in
https://github.com/llvm/llvm-test-suite/tree/master/MultiSource


> Q2: I read that using glob might not be a good idea, even though there is a
> partial workaround. However, when trying to understand what the other External
> suites were doing, I noticed several of them were using it.
> May I ignore the note at
> https://cmake.org/cmake/help/latest/command/file.html#glob ?

The typical use case of the test-suite is to configure and build it
from scratch every time with a different compiler executable. There is
no typical edit-compile-debug cycle.

> Q3: The C files were not extracted/reconstructed with execution in mind.
> For now, we're mainly interested in collecting stats and testing the
> compiler. Since we don't generate executables, should I call cmake's
> add_library() for each .c file?

See compile time benchmarks in
https://github.com/llvm/llvm-test-suite/tree/master/CTMark

Are the C files intended to compile-time benchmarking as well? If not,
what are they useful for?

Michael