[LLVMdev] Target-specific defaults for LLVM tools?

Tue Mar 17 11:13:53 PDT 2015

Hello, LLVM.

We'd like to start a discussion about what would be the best way of
supporting target-specific defaults for options in the LLVM tools.

Problem description
-------------------

LLVM strives to be a generic compiler and it has a lot of code generation
options which are (rightly) not linked to the target triple.
However, some target triples do impose hard requirements for such options.
For
example, PS4 requires the relocation model to be PIC. Cross-compilers for
specific target systems are likely to have similar restrictions.

One very effective way to ensure that users of Clang/LLVM don't misuse the
tools by passing the wrong options is to set the default values for those
options in the cross-compiler once the target triple is known. This has the
following rationale:

- Correctness: the user must be able to generate correct code for the target
  in the default workflow (with or without optimizations).
- Performance: the user should be able to generate good code for the target
  in the default workflow with optimizations enabled.
- Usability: although the default workflow could potentially be defined as
  "sorry, but you must pass options X, Y and Z to the tools", obviously it
would
  be easier if it did not require such esoteric options.

And of course, from a company's point of view:

- Commercial reasons: if a tool is released commercially, the user expects
such
  commercial product to work "out of the box" for the target system.

This is very easy to do in Clang, as the driver does the job of
"dispatching"
the correct command line options to subsequent invocations of tools.
Because of
the driver being the "middle man", it can parse the options, understand
what the
target triple is, and invoke other tools accordingly (i.e. actively adding
an
option, or erroring out upon encountering an invalid set of options).

A vendor can set the default target triple for Clang even at build time, and
this approach seems to not cause any trouble in the Clang tests (which
don't get
as far as checking the output assembly). So for Clang the problem is solved
to a
certain degree.

However, issues start to pop up when we consider all the other tools (opt,
llc,
and so on) and the complications of having for example a "cross-llc". Usage
of
such tools is becoming increasingly likely thanks to the existence of other
tools and frontends which can generate LLVM IR. Some crazy people might
even try
to write IR instead of C++! :-)

First of all, overriding the defaults based on the triple is harder in LLVM
tools, because of how the cl::opt options work. If you consider llc, it
determines the triple in this sequence:

- Parse the IR and extract the triple if present.
- If there's a triple on the command line, override the IR triple.
- If no triple was found by either method, use the default triple.

This process is performed in the compileModule() function, which happens way
after the cl::init initializers for the cl::opt option defaults, and shortly
after the cl::ParseCommandLineOptions() call. If the value of some option
needs
to be changed _after_ the triple has been determined, additional work has
to be
performed to tweak the cl::opt options on the fly, but only if they had not
already been set on the command line (which would mean the user explicitly
wanted a non-default value). It would be quite ugly, although doable.

The real problem, however, is that tools are used not only by the end user,
but
also extensively by the LLVM tests that check the output assembly.
Currently, there is a non-trivial number of LLVM tests that do not set the
full
triple, but only parts of it, and then rely on specific default code
generation
options. In the past, some defaults have been locked down as opposed to
being
auto-detected (for example, -mcpu=generic) specifically for the purpose of
making tests stable and correct for cross-compilers.

Changing the code to override the defaults for opt, llc, and so on based on
the
target triple can make some tests fail, because it would reinstate the same
problems we had before locking down such defaults. Of course, such a change
could be kept private and out of the open-source tree, and the tests could
be
modified accordingly. However, it is clear that it could become a large
burden
to support these private changes when new tests are constantly added to the
codebase.

We would like to start a discussion with the community about how best to
support
the use case of "target-specific defaults for LLVM tools", especially in the
case of cross-compiling tools. Maybe somebody else is running into our same
use-case.
And this could even prove to be tied to the cl::opt reworking that was
going on
a while ago, so new requirements / limitations / ideas might emerge.

We have identified a few approaches that could serve as the starting point
of a
discussion. I will briefly outline them below. We don't want to push for
any of
them, we just want to discuss them. There could even be more effective ways
of
solving the problem, which is why we are interested in having this
discussion
here.

A - Fixed option defaults, irrelevant default triple logic
----------------------------------------------------------

I believe this is the current situation, possibly caused by how cl::opt
works.
This alternative assumes that:

- There is no functionality in the open-source tree to change the defaults
based
  on the target triple.
- Vendors may or may not change the default triple in their cross-compilers.

If a vendor wants to both change a tool to have a specific default triple,
and
cause option defaults to depend on the triple, it is the vendor's
responsibility
to adapt failing tests, and changes will have to be kept private. If a
vendor
tried to push them onto the community ("yes, this -mcpu=generic in the test
is
totally unnecessary, but can we have it in tree?"), such a change would
never
pass a review.

If this approach is chosen, then vendors will have to maintain private
changes,
but the state of the LLVM tests in the open-source tree will be consistent
and
there will be no target-specific code to override option defaults.

B - Triple-based option defaults, variable default triple logic
---------------------------------------------------------------

This alternative is at the other end of the spectrum and it assumes that:

- There is some functionality (a new API in llvm::cl?) in the open-source
tree
  to override the defaults based on the target triple.
- The default target triple can be freely changed in cross-compilers, with
  impunity.

As described above, this would cause problems to some existing tests. It
could
be argued though that, if a test is unstable with respect to code generation
options, then it should specify all options that it actually requires in
order
to run properly.
This would mean that tests must specify a target triple in the RUN lines or
in
the IR, otherwise they cannot assume that the code will be generated as
expected.

This puts a burden on the test implementors, but could it be argued that
this is
technically correct? Shouldn't tests be robust? On the other hand, if tests
had
to specify a target triple, wouldn't this reduce coverage (as they'd be
testing
a single target as opposed to many)?

If this approach was chosen, then a good number of LLVM tests in the
open-source
tree would need to be adapted, but vendors could be free to implement their
own
defaults and set the default triple without significant consequences.

C - Triple-based option defaults, fixed default triple logic
------------------------------------------------------------

This alternative assumes that:

- There is some functionality (a new API in llvm::cl?) in the open-source
tree
  to override the defaults based on the target triple.
- Vendors are NOT free to change the logic determining the default triple.
- The default triple is inferred by some fixed logic, which is
test-friendly.

There is code in the Clang driver that parses the program name to identify a
prefix that is used to determine the target triple. For example, I seem to
understand that "x86_64-scei-ps4-clang" would indicate a Clang which has a
PS4
target triple by default.
Opt, llc, and similar tools do not support anything of the sort, so at the
moment the default target triple cannot be selected with the same approach.

Anyway, consider the case where option defaults are set based on the target
triple, but the default triple can be inferred from the name prefix, just as
Clang does; if no such prefix is found, the defaulting falls back to what
we do
currently. This allows tests with RUN lines running "llc" to just use the
triple
that we currently use, and therefore all existing open-source tests will
continue to work.

At the same time, a vendor could release a tool with a name prefix and reap
the
benefits of having the desired defaults; and to make it convenient for the
user,
a symlink (or something more complex on Windows) could easily be used.

If this approach was chosen, vendor-specific tests which expect the vendor
defaults would have to somehow rename the tool on the fly before running
it; or
maybe they could change the build process to produce both an "llc" and a
"<triple>-llc" executables. Also vendors would need to release the renamed
tools, but this often happens anyway with cross-compilers. This pushes some
burden on the vendor, but it is much less than maintaining a lot of private
changes to the open-source code and tests.

What does the community think?
Discuss. :-)

Cheers,
    Dario Domizioli
    SN Systems - Sony Computer Entertainment Group
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150317/ba6339b4/attachment.html>