[llvm-dev] Separate test harness for LLVM test-suite

[Hal Finkel via llvm-dev]

On 07/09/2018 01:53 PM, Matthias Braun wrote:
> I personally was hoping that maybe when we move to github and have to
> reorganize the repositories anyway, we can split out "ABI-Testsuite"
> and "ClangAnalyzer" from the llvm-testsuite repository, given that
> they pretty orthogonal to the rest.

Sounds good to me.

>
>
> Judging from builder names we one ABI-Testuite
> builder: http://lab.llvm.org:8011/builders/clang-x86_64-linux-abi-test

Should we, in principle, have these for other architectures?

> (and BTW: I am very happy that we have it!)

(Certainly +1 to that).

>
> - Matthias
>
>> On Jul 9, 2018, at 10:48 AM, Hal Finkel <hfinkel at anl.gov
>> <mailto:hfinkel at anl.gov>> wrote:
>>
>>
>> On 07/09/2018 12:32 PM, Matthias Braun via llvm-dev wrote:
>>> If this is mainly about size then I would rather propose to split
>>> the "ABI-Testsuite" folder into a separate repository. With 1.5G it
>>> takes up more space than the rest of the repository combined yet it
>>> is coneptually independent of the rest of the testsuite and most
>>> people are not running it.
>>
>> I think is a good idea regardless.
>>
>> Actually, do we have any buildbots running the ABI testsuite?
>>
>>  -Hal
>>
>>>
>>> - Matthias
>>>
>>>> On Jul 9, 2018, at 9:11 AM, Michael Kruse via llvm-dev
>>>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>>>
>>>> If I understand correctly, you wish to put the the test-suite
>>>> framework (cmake/, litsupport/, litsupport-tests/, tools/, utils/)
>>>> into a different repository than the test sources (MultiSource/,
>>>> SingleSource, etc)?
>>>>
>>>> Conceptually, the test framework (llvm-lit) is already separated,
>>>> living in the main LLVM repository. The build instructions (cmake
>>>> files) are hard to separate from the source files to compile.
>>>>
>>>> I don't find that the External/ part works well. It was created for
>>>> licensing reasons, for proprietary source that cannot be included in
>>>> the repository. I have yet to successfully make any non-SPEC program
>>>> work. For instance, I cannot find the source of Povray that works with
>>>> test-suite's makefile/CMakeLists.txt
>>>>
>>>> Michael
>>>>
>>>>
>>>>
>>>>
>>>> 2018-07-09 2:26 GMT-05:00 Elena Lepilkina via llvm-dev
>>>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>:
>>>>> Dear community,
>>>>>
>>>>>
>>>>>
>>>>> LLVM test-suite includes both default applications (benchmarks,
>>>>> tests) and
>>>>> test harness (CMake modules, lit-support, some scripts, etc.).
>>>>> Test suite
>>>>> provides extension interface to add “external” tests suite and
>>>>> folders. This
>>>>> worked well, we enhanced basic harness and integrated large set of
>>>>> internal
>>>>> benchmarks and applications.
>>>>>
>>>>>
>>>>>
>>>>> However, we have users interested in LLVM test harness to build,
>>>>> profile,
>>>>> test applications and experiment with tools/compilers without using
>>>>> applications from full LLVM test-suite (~2,5 Gb).
>>>>>
>>>>>
>>>>>
>>>>> Are there reasons why separation of test-suite into 1) basic
>>>>> test-harness
>>>>> and 2) applications is undesirable or impossible?
>>>>>
>>>>>
>>>>>
>>>>> We will be glad to hear opinions of community.
>>>>>
>>>>>
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Elena Lepilkina,
>>>>>
>>>>> Synopsys Corp, ARC processors
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> -- 
>> Hal Finkel
>> Lead, Compiler Technology and Programming Languages
>> Leadership Computing Facility
>> Argonne National Laboratory
>

-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180709/209fb1cb/attachment.html>