[llvm-dev] Adding a new External Suite to test-suite

[Johannes Doerfert via llvm-dev]

On 4/6/20 8:46 AM, Michael Kruse via llvm-dev wrote:
> I suggest to have a look into the CMakeLists.txt of the test-suite.
> These are mostly self-explanatory.
>
> Am Mo., 6. Apr. 2020 um 04:25 Uhr schrieb Bruno Conde Kind via
> llvm-dev <llvm-dev at lists.llvm.org>:
>> Q1: What do I have to do to start integrating my collection of .c files as an
>> External Suite? They are all single source, single function,
>> compilable headerless files extracted from open source repositories.
>> They are grouped in folders, in a similar way the original project
>> was.
> External tests are in
> https://github.com/llvm/llvm-test-suite/tree/master/External.
> Tests where each source file is one executable are in
> https://github.com/llvm/llvm-test-suite/tree/master/SingleSource
> Tests where each source file in a directory compiles to object files
> that is linked to an executable are in
> https://github.com/llvm/llvm-test-suite/tree/master/MultiSource
>
>
>> Q2: I read that using glob might not be a good idea, even though there is a
>> partial workaround. However, when trying to understand what the other External
>> suites were doing, I noticed several of them were using it.
>> May I ignore the note at
>> https://cmake.org/cmake/help/latest/command/file.html#glob ?
> The typical use case of the test-suite is to configure and build it
> from scratch every time with a different compiler executable. There is
> no typical edit-compile-debug cycle.
>
>> Q3: The C files were not extracted/reconstructed with execution in mind.
>> For now, we're mainly interested in collecting stats and testing the
>> compiler. Since we don't generate executables, should I call cmake's
>> add_library() for each .c file?
> See compile time benchmarks in
> https://github.com/llvm/llvm-test-suite/tree/master/CTMark
>
> Are the C files intended to compile-time benchmarking as well? If not,
> what are they useful for?

I'd also like to know what the intention here is. What is tested and how?

Looking at a few of these it seems there is not much you can do as it is

little code with a lot of unknown function calls and global symbols.


Cheers,

   Johannes



> Michael
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev