[libcxx-dev] How to execute cross-compiled libcxx tests on a remote target?

[Dominik Montada via libcxx-dev]

Hi Louis,

thanks for the response and the commit to support this use case!

In the meantime I have implemented a feature to load an additional 
configuration file from the auto generated one, which solves the problem 
for me so far. Especially since I also need to set some additional flags 
(depending on the configuration) which get passed to my execution 
script, which seems like something which is not supported by your last 
commit?

By the way, if you're interested I'd be happy to create patch from my 
extension.

While we're at it, is there a way to declare some tests as unsupported? 
We have a couple which we do not support on our target and would like to 
exclude them so they don't show up as failures at all.

Cheers,

Dominik

Am 11.06.20 um 22:30 schrieb Louis Dionne:
> BCC: LLVM Dev
>
> Hi,
>
>> On May 28, 2020, at 06:19, Dominik Montada <dominik.montada at hightec-rt.com> wrote:
>>
>> Hi Louis,
>>
>> I saw that you were the one who implemented the new testing system for libcxx, so I figured that you might be able to help me out with my problem.
>>
>> I'm working on a downstream target and we're now in the process of adding support for libcxx. We would like to leverage the existing libcxx test suite to make sure that the implementation is working correctly. The problem is that our target is a bare-metal target and we need to cross-compile the tests and execute them remotely. There isn't really any documentation for how to do it, but I came across an old email thread from 2015 which said to implement an executor and use that. However, it seems like those executors are not used with the new test format any longer (except for a check whether to use `run.py` or `ssh.py`).
>>
>> Also, in the original Phabricator review where you first introduced the new format, you said:
>>
>>> As a side effect of this design, configuration files for the test suite can be as simple as:
>>>
>>> config.substitutions.append(('%{cxx}', '<path-to-compiler>'))
>>> config.substitutions.append(('%{compile_flags}', '<flags>'))
>>> config.substitutions.append(('%{link_flags}', '<flags>'))
>>> config.substitutions.append(('%{exec}', '<script-to-execute>'))
>>>
>>> This should allow storing lit.cfg files for various configurations directly in the repository
>> However I'm not sure what exactly I need to do here. Do I need to write my own `lit.site.cfg` and point CMake to it? If possible I'd like to reuse the default- and auto-generated config as much as possible.
>>
>> We have a script which can execute a cross-compiled binary on our remote target. I just want the test suite to call this script. Could you give me some help on what I need to do to make this work?
> As of 96e6cbbf941d0f937b7e823433d4c222967a1817 (committed today), you should be able to use:
>
>      cmake [...] -DLIBCXX_EXECUTOR="<path-to-script>"
>
> and that script will be used to run executables in the test suite. Look at how ssh.py and run.py handle their arguments for the arguments that should be accepted by your script, however it doesn't really differ much from old style executors. I'll work on documenting those in the libcxx docs.
>
> Please reach out again if you run into issues with the test suite, I'll help you resolve them. I'd like everyone to move to the new format, which is simpler and more flexible.
>
> Cheers,
> Louis
>
>
-- 
----------------------------------------------------------------------
Dominik Montada                   Email: dominik.montada at hightec-rt.com
HighTec EDV-Systeme GmbH          Phone: +49 681 92613 19
Europaallee 19                    Fax:   +49-681-92613-26
D-66113 Saarbrücken               WWW: http://www.hightec-rt.com

Managing Director: Vera Strothmann
Register Court: Saarbrücken, HRB 10445, VAT ID: DE 138344222

This e-mail may contain confidential and/or privileged information. If
you are not the intended recipient please notify the sender immediately
and destroy this e-mail. Any unauthorised copying, disclosure or
distribution of the material in this e-mail is strictly forbidden.
---

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 6822 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/libcxx-dev/attachments/20200612/9b1192e8/attachment-0001.bin>