[llvm-dev] Separate test harness for LLVM test-suite

[Hal Finkel via llvm-dev]

On 07/09/2018 12:32 PM, Matthias Braun via llvm-dev wrote:
> If this is mainly about size then I would rather propose to split the "ABI-Testsuite" folder into a separate repository. With 1.5G it takes up more space than the rest of the repository combined yet it is coneptually independent of the rest of the testsuite and most people are not running it.

I think is a good idea regardless.

Actually, do we have any buildbots running the ABI testsuite?

 -Hal

>
> - Matthias
>
>> On Jul 9, 2018, at 9:11 AM, Michael Kruse via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>
>> If I understand correctly, you wish to put the the test-suite
>> framework (cmake/, litsupport/, litsupport-tests/, tools/, utils/)
>> into a different repository than the test sources (MultiSource/,
>> SingleSource, etc)?
>>
>> Conceptually, the test framework (llvm-lit) is already separated,
>> living in the main LLVM repository. The build instructions (cmake
>> files) are hard to separate from the source files to compile.
>>
>> I don't find that the External/ part works well. It was created for
>> licensing reasons, for proprietary source that cannot be included in
>> the repository. I have yet to successfully make any non-SPEC program
>> work. For instance, I cannot find the source of Povray that works with
>> test-suite's makefile/CMakeLists.txt
>>
>> Michael
>>
>>
>>
>>
>> 2018-07-09 2:26 GMT-05:00 Elena Lepilkina via llvm-dev
>> <llvm-dev at lists.llvm.org>:
>>> Dear community,
>>>
>>>
>>>
>>> LLVM test-suite includes both default applications (benchmarks, tests) and
>>> test harness (CMake modules, lit-support, some scripts, etc.). Test suite
>>> provides extension interface to add “external” tests suite and folders. This
>>> worked well, we enhanced basic harness and integrated large set of internal
>>> benchmarks and applications.
>>>
>>>
>>>
>>> However, we have users interested in LLVM test harness to build, profile,
>>> test applications and experiment with tools/compilers without using
>>> applications from full LLVM test-suite (~2,5 Gb).
>>>
>>>
>>>
>>> Are there reasons why separation of test-suite into 1) basic test-harness
>>> and 2) applications is undesirable or impossible?
>>>
>>>
>>>
>>> We will be glad to hear opinions of community.
>>>
>>>
>>>
>>> Best regards,
>>>
>>> Elena Lepilkina,
>>>
>>> Synopsys Corp, ARC processors
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory