[llvm-dev] How to execute cross-compiled libcxx tests on a remote target?

[Louis Dionne via llvm-dev]

BCC: LLVM Dev

Hi,

> On May 28, 2020, at 06:19, Dominik Montada <dominik.montada at hightec-rt.com> wrote:
> 
> Hi Louis,
> 
> I saw that you were the one who implemented the new testing system for libcxx, so I figured that you might be able to help me out with my problem.
> 
> I'm working on a downstream target and we're now in the process of adding support for libcxx. We would like to leverage the existing libcxx test suite to make sure that the implementation is working correctly. The problem is that our target is a bare-metal target and we need to cross-compile the tests and execute them remotely. There isn't really any documentation for how to do it, but I came across an old email thread from 2015 which said to implement an executor and use that. However, it seems like those executors are not used with the new test format any longer (except for a check whether to use `run.py` or `ssh.py`).
> 
> Also, in the original Phabricator review where you first introduced the new format, you said:
> 
> >As a side effect of this design, configuration files for the test suite can be as simple as:
> >
> >config.substitutions.append(('%{cxx}', '<path-to-compiler>'))
> >config.substitutions.append(('%{compile_flags}', '<flags>'))
> >config.substitutions.append(('%{link_flags}', '<flags>'))
> >config.substitutions.append(('%{exec}', '<script-to-execute>'))
> >
> >This should allow storing lit.cfg files for various configurations directly in the repository
> 
> However I'm not sure what exactly I need to do here. Do I need to write my own `lit.site.cfg` and point CMake to it? If possible I'd like to reuse the default- and auto-generated config as much as possible.
> 
> We have a script which can execute a cross-compiled binary on our remote target. I just want the test suite to call this script. Could you give me some help on what I need to do to make this work?

As of 96e6cbbf941d0f937b7e823433d4c222967a1817 (committed today), you should be able to use:

    cmake [...] -DLIBCXX_EXECUTOR="<path-to-script>"

and that script will be used to run executables in the test suite. Look at how ssh.py and run.py handle their arguments for the arguments that should be accepted by your script, however it doesn't really differ much from old style executors. I'll work on documenting those in the libcxx docs.

Please reach out again if you run into issues with the test suite, I'll help you resolve them. I'd like everyone to move to the new format, which is simpler and more flexible.

Cheers,
Louis