[LLVMdev] QEMU testing for LIT execution tests

[Greg Fitzgerald]

Jon,

> One issue to start with is that none of the libcxx/libcxxabi tests have RUN lines.

Sorry, I didn't mean to imply that 'RUN:' lines should be added to
libcxx tests.  It's fine that every one is implicitly:

    // RUN: %clang %s -o %t && %run %t


> I think that this necessitates inventing a new LIT syntax, something like '// DEPENDS-ON: filename.dat'

If you didn't want to change lit, this would be a good place to add a
'RUN:' line.  Something like:

    // RUN: %clang %s -o %t && %run %t filename.dat

and then your %run implementation would see the dependency on the
local file and ship it over to TARGET.


> A third issue is that I don't think the current '%run' solution works for
> full Canadian cross testing (i.e. run lit on BUILD, clang/llvm/whatever
> on HOST, and the tests themselves on TARGET, where {BUILD,
> HOST,TARGET} are all different machines).

In compiler-rt, I'd configure such a build with the following CMake variables:

LLVM_LIT=$BUILD/lit
CMAKE_C_COMPILER=$HOST/clang
COMPILER_RT_EMULATOR=$TARGET/qemu

For libcxx, how about adding LIBCXX_EMULATOR, and then changing
lit.cfg to acknowledge it when invoking executables?  You'd then put
your ssh functionality in a standalone script.  If a command-line
interface is too inflexible (or painful), perhaps add a
LIBCXX_PY_EMULATOR variable that points to a Python module.

-Greg


On Mon, Sep 8, 2014 at 12:49 PM, Jonathan Roelofs
<jonathan at codesourcery.com> wrote:
>
>
> On 9/8/14, 12:39 PM, Greg Fitzgerald wrote:
>>
>> Hi Jon,
>>
>> Compiler-rt, libcxx, libcxxabi should all use the same
>> cross-compilation testing strategy.  Compiler-rt already has a working
>> solution.  If it is inadequate for libcxx or libcxxabi, can we start
>> by addressing those deficiencies?
>
> One issue to start with is that none of the libcxx/libcxxabi tests have RUN
> lines. All of the tests are either: *.fail.cpp ones that fail to compile, or
> *.pass.cpp ones that are expected to exit with a return status of 0 without
> failing an asserts. Granted, this is more of a test semantic issue, but
> there are a *lot* of tests to update in order to converge on compiler_rt's
> 'RUN: ' format.
>
> [Sidenote: I don't see any 'RUN:' lines on the builtins unit tests. How does
> it work for those ones?]
>
> Another issue, which Dan Albert recently pointed out to me, is that there
> are several libcxx tests which would require copying in of test data (in
> addition to test executables).  I think that this necessitates inventing a
> new LIT syntax, something like '// DEPENDS-ON: filename.dat' in order to
> inform the lit.cfg that the dependent file also needs to be copied to the
> remote target.
>
> A third issue is that I don't think the current '%run' solution works for
> full Canadian cross testing (i.e. run lit on BUILD, clang/llvm/whatever on
> HOST, and the tests themselves on TARGET, where {BUILD,HOST,TARGET} are all
> different machines).  This use case is of particular interest to me. I
> suppose %run could be renamed %run_target, add %run_host, and then have an
> implied no-op %run_build on commands that don't have %run_target or
> %run_host. Perhaps someone might want to have BUILD be a different machine
> than where LIT runs, in which case %run_build would have to be explicit (but
> I'm not really interested in that use case).
>
>
> Cheers,
> Jon
>>
>>
>> Thanks,
>> Greg
>>
>>
>
> --
> Jon Roelofs
> jonathan at codesourcery.com
> CodeSourcery / Mentor Embedded