[llvm-dev] Running lit tests with pre-built binaries

[Tom Stellard via llvm-dev]

On 12/17/20 1:28 PM, Kaylor, Andrew wrote:
> Hi Tom,
> 
> Thanks for the tip. It sounds very promising. It looks like that does essentially what I need to do. Now I just need to figure out the mechanics of how it works.
> 
> How is the package created? Do you make it directly from the test directories in the source tree or is there an install target for the tests? How is the lit configuration handled?
> 

The process is not all that straight-forward.  You can look at the 
Fedora spec file[1] to see exactly how it works.  At a high-level what 
it does is manually install (no install target)l the test directory 
along with a fedora specific lit config file[2].  The LLVM lit config 
file is patched[3] to load the fedora config.  The fedora lit config has 
some parameters that can be overridden at runtime, which is what the 
run-lit-tests script does.

Do you want to try to recreate this package or do you want to try to use 
the Fedora package directly.  I have never tried this, but it may work 
to point the tests at a non-system install just by updating the 
config.llvm_tools_dir in lit.fedora.cfg.py.

-Tom

[1] https://src.fedoraproject.org/rpms/llvm/blob/master/f/llvm.spec
[2] https://src.fedoraproject.org/rpms/llvm/blob/master/f/lit.fedora.cfg.py
[3]  https://src.fedoraproject.org/rpms/llvm/blob/master/f/llvm.spec#_335
> Thanks,
> Andy
> 
> -----Original Message-----
> From: Tom Stellard <tstellar at redhat.com>
> Sent: Wednesday, December 16, 2020 8:12 PM
> To: Kaylor, Andrew <andrew.kaylor at intel.com>; llvm-dev at lists.llvm.org
> Subject: Re: [llvm-dev] Running lit tests with pre-built binaries
> 
> On 12/16/20 4:44 PM, Kaylor, Andrew via llvm-dev wrote:
>> Hi everyone,
>>
>> Can anyone tell me if there is a way to run in-tree lit tests using
>> pre-built binaries rather than building everything in the current workspace?
>>
> 
> On Fedora, we package the lit tests into an RPM that can be installed and run against the system LLVM:
> 
> dnf install llvm-test
> /usr/libexec/tests/llvm/run-lit-tests
> 
> I'm not sure if it will work for your use case, but it might be a good starting point.
> 
> -Tom
> 
>> The specific problem I’m trying to solve is testing a heterogeneous
>> runtime library with multiple offload configurations. The host system
>> architecture is the same for all the cases I want to test, so I’d
>> like to be able to build the compiler once and then deploy it to
>> different systems and run the runtime library’s LIT tests on each of
>> those systems. Some of the systems I need to test don’t have a lot
>> of host compute power, so I’d like to avoid rebuilding the project if possible.
>>
>> It seems like this should be possible, but I’m not sure of the best
>> way to do it.
>>
>> I suppose I could copy the entire build directory from my build system
>> into an appropriate location on the test system to make it look like
>> it was built there, but that feels like a very fragile solution.
>>
>> I could also duplicate the tests into a configuration like the
>> llvm-test-suite that is designed to be run with a pre-built compiler,
>> but that would introduce problems with keeping the tests in synch with
>> the source.
>>
>> Would it be reasonable to modify the runtime library’s lit config
>> files to provide an in-tree option for them to use an external
>> compiler and library path?
>>
>> Thanks in advance for any suggestions.
>>
>> -Andy
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>