[llvm-dev] Separate test harness for LLVM test-suite

[Matthias Braun via llvm-dev]

If this is mainly about size then I would rather propose to split the "ABI-Testsuite" folder into a separate repository. With 1.5G it takes up more space than the rest of the repository combined yet it is coneptually independent of the rest of the testsuite and most people are not running it.

- Matthias

> On Jul 9, 2018, at 9:11 AM, Michael Kruse via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> 
> If I understand correctly, you wish to put the the test-suite
> framework (cmake/, litsupport/, litsupport-tests/, tools/, utils/)
> into a different repository than the test sources (MultiSource/,
> SingleSource, etc)?
> 
> Conceptually, the test framework (llvm-lit) is already separated,
> living in the main LLVM repository. The build instructions (cmake
> files) are hard to separate from the source files to compile.
> 
> I don't find that the External/ part works well. It was created for
> licensing reasons, for proprietary source that cannot be included in
> the repository. I have yet to successfully make any non-SPEC program
> work. For instance, I cannot find the source of Povray that works with
> test-suite's makefile/CMakeLists.txt
> 
> Michael
> 
> 
> 
> 
> 2018-07-09 2:26 GMT-05:00 Elena Lepilkina via llvm-dev
> <llvm-dev at lists.llvm.org>:
>> Dear community,
>> 
>> 
>> 
>> LLVM test-suite includes both default applications (benchmarks, tests) and
>> test harness (CMake modules, lit-support, some scripts, etc.). Test suite
>> provides extension interface to add “external” tests suite and folders. This
>> worked well, we enhanced basic harness and integrated large set of internal
>> benchmarks and applications.
>> 
>> 
>> 
>> However, we have users interested in LLVM test harness to build, profile,
>> test applications and experiment with tools/compilers without using
>> applications from full LLVM test-suite (~2,5 Gb).
>> 
>> 
>> 
>> Are there reasons why separation of test-suite into 1) basic test-harness
>> and 2) applications is undesirable or impossible?
>> 
>> 
>> 
>> We will be glad to hear opinions of community.
>> 
>> 
>> 
>> Best regards,
>> 
>> Elena Lepilkina,
>> 
>> Synopsys Corp, ARC processors
>> 
>> 
>> 
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev