[lldb-dev] Best way to support multiple compilers from lit tests

[Zachary Turner via lldb-dev]

One of the issues we've faced a couple of times (and will continue to face)
is that lit is generally built around substitutions and command lines, but
different compilers will generally have different substitutions, and worse
-- different command line syntaxes.

The most recent example of this is the stop-hooks test that was added which
uses the %cc and %cxx substitutions.  This will work fine with GCC or
clang, but not with clang-cl, where we need an entirely different command
line.

If we're going to grow this test suite, I think we're going to need a
solution to this.

The main issue I want to address here is that we need a way to abstract
over the difference between command line syntaxes.  gcc and clang are
pretty similar, but they can occasionally differ in minor ways.  But
clang-cl will always be different.

One idea I've had here is to extend lit with the ability to create our own
custom prefix commands (similar to RUN lines), but where we can provide lit
with the prefix and a function to call to execute it.  The thinking being
that we can add something like a COMPILE and LINK command, which you could
invoke like this:

// COMPILE: source=%p/Inputs/foo.cpp \
// COMPILE:    opt=none \
// COMPILE:    compiler=default \
// COMPILE:    out=%t.obj \
// COMPILE:    link=no
// LINK:         obj=%t.obj \
// LINK:            linker=default \
// LINK:            nodefaultlib \
// LINK:            entry=main \
// LINK:            out=%t.exe

Here "default" means whatever lit decides, which would usually depend on
how you configured your CMake.  But for some tests you could specify an
explicit compiler, for example, you could say compiler=gcc or
compiler=msvc, and the test would fail if those compilers are not
configured.

This is actually very similar in spirit to how dotest.py's "builders" work,
but extended to lit.

If we go this route, the first step would be to extend lit with a general
notion of pluggable commands, independently of LLDB.

After that, we could implement the COMPILE and LINK commands in LLDB,
perhaps even reusing much of builder.py from the dotest sutite.


Note that I'm not attempting to address the idea of running the test suite
with different compilers in the same run.  I have further ideas for how to
address that, but I don't think we need to do that right now as it's not an
immediate need.


Does anyone have any thoughts on this?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20181118/9dce299b/attachment.html>