<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p><br>
</p>
<div class="moz-cite-prefix">On 03/20/2017 07:19 AM, Serge Pavlov
wrote:<br>
</div>
<blockquote
cite="mid:CACOhrX5jf39uGf9A3_58n-Zvs-eXp+NsnvfcVWxOWtD8HqN_vw@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">2017-03-20 12:48 GMT+07:00 James Y
Knight <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:jyknight@google.com" target="_blank">jyknight@google.com</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span class="gmail-">On Thu,
Mar 16, 2017 at 9:25 AM, Serge Pavlov <span
dir="ltr"><<a moz-do-not-send="true"
href="mailto:sepavloff@gmail.com"
target="_blank">sepavloff@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span
class="gmail-m_6136006684703735351gmail-m_4370221758571261446m_8457082739418504159gmail-">2017-03-16
9:46 GMT+07:00 James Y Knight <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:jyknight@google.com"
target="_blank">jyknight@google.com</a>></span>:<br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span
class="gmail-m_6136006684703735351gmail-m_4370221758571261446m_8457082739418504159gmail-m_7067103267148724478gmail-">
<div> </div>
</span>
<div>I'd really like to at least
have a *design* for how this
can eventually incorporate
target-specific options before
moving forward with adding a
--config option, even if the
initial commit won't implement
the full design.</div>
<div><br>
</div>
<div>I don't believe hand-wavy
"maybe we'll add syntax that
looks kinda like a comment so
older versions will ignore it"
is good enough there.</div>
<div><br>
</div>
<div>I'd like to again keep in
mind the use-case I mentioned
a while ago. Approximately
every linux distro configures
GCC to set their default
target cpu levels. E.g.,
Debian seems to set the
following:</div>
<div>- On x86, the default CPU
should be i686.</div>
<div>- But on x86-64, the
default CPU isn't changed (aka
it's left as "x86-64").</div>
<div>- For ppc32/ppc64, the
default CPU should be power7
but tune for power8.</div>
<div>- For ppc64le, the default
CPU should be power8.</div>
<div>- On ARM (hf), armv7-a
should be the default cpu,
vfpv3-d16 the default fpu, and
it should default to thumb
mode.</div>
<div>etc...</div>
<div><br>
</div>
<div>Note that those defaults
are different on different
releases of the distro.</div>
<div><br>
</div>
<div>The way you do this with
GCC is via options passed to
the configure script:
--with-arch-32=
--with-arch-64= --with-fpu=
--with-mode= etc. which turn
into values used in the
target-specific
OPTION_DEFAULT_SPECS macro.
Since GCC only builds for one
target at a time (or two, if
you count 32/64 separately),
and you're expected to need to
build a new gcc any time you
want to cross-compile, that's
sufficient.</div>
<div><br>
</div>
<div>Clang is intrinsically a
cross-compiler, so gcc's
solution isn't good enough for
clang (nor is clang's current
behavior enough). So, what's
the plan to actually solve
this?</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>Initial versions of this proposal
defined two kinds of config files:</div>
<div><span class="gmail-m_6136006684703735351gmail-m_4370221758571261446m_8457082739418504159gmail-m_7067103267148724478gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>-
named, which should be explicitly
specified by a user by option --config
or be encoded into executable name as
`armv7l-clang`.</div>
<div><span class="gmail-m_6136006684703735351gmail-m_4370221758571261446m_8457082739418504159gmail-m_7067103267148724478gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>-
default, which is loaded always much
like `.bashrc` or any similar file.</div>
<div>These two kinds of config file shared
implementation but addressed different
use cases, which made confusion during
discussion. To facilitate review process
the support of default config files was
removed from the proposal. The issues
you mention should mostly be solved by
the default config files.</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>OK -- focusing on cross-compilation for a
minute. Clang and LLD have the potential to make
cross-compilation *MUCH* simpler than it's
traditionally been, and I'd like to be careful to
think about how to take the best advantage of the
fact that we're getting really close to having a
single toolchain which is able to easily build
programs for any target architecture/environment.</div>
<div><br>
</div>
<div>Here's the scenario I'm using in my mind:<br>
</div>
<div><br>
</div>
<div>Let's say I've got a super-fast RISC-V machine
running OpenBSD, and want to cross-build from that
to all of the various supported Debian
architectures. I want to use my existing
OpenBSD-supplied clang+lld toolchain, because that
toolchain can natively cross-compile to
everything, if given appropriate command-line
arguments. So, I don't need to make a special
build of the compiler or linker. Woohoo!</div>
<div><br>
</div>
<div>I have copied over the root partition of the
target Debian system into a directory on this
machine, so I have headers and libraries to link
against.</div>
<div><br>
</div>
<div>
<div>(I also need the crtbegin/end and libgcc or
compiler-rt. For this thread, let's assume I
copied those over as well, even though they're
typically "part of" the compiler toolchain
suite. A separate discussion should be had about
what to do about making THAT part of the
cross-compile story easier -- it's almost the
only tricky part remaining!)</div>
</div>
<div><br>
</div>
<div>Now, given that scenario --</div>
<div><br>
</div>
<div>How do I specify that I want to cross-build to
that debian system?<br>
</div>
<div><br>
</div>
<div>I need to know the proper configuration, and I
need to know how I'm supposed to invoke clang to
use the configuration.</div>
<div><br>
</div>
<div>At the moment, you might run something like:</div>
<div> "clang --target x86_64-linux-gnu
--sysroot=/path/to/debian-<wbr>installation
--other-super-important-cpu/<wbr>abi-flags"</div>
<div><br>
</div>
<div>With your proposed patch, I would create a
config file, "x86_64-debian.cfg", containing the
above flags. I could put it in /etc/something or
~/.something, and run:</div>
<div>$ clang -config x86_64-debian</div>
<div><br>
</div>
<div>Alternatively, I could put it in
~/bin/x86_64-debian.cfg and</div>
<div>$ ln -s /usr/bin/clang
~/bin/x86_64-debian-clang</div>
<div>$ x86_64-debian-clang</div>
<span class="gmail-">
<div><br>
</div>
</span></div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>In general you are right I think.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span class="gmail-">
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>If the default config files were
implemented in clang, driver would
search binary directory for default
configuration description. If compiler
is named as `armv7l-clang`, driver first
tries file `armv7l.cfg` then
`clang.cfg`. If config file is found,
options listed there are put into the
set of driver arguments before any
option specified in command line.</div>
</div>
</div>
</div>
</blockquote>
<div> <br>
</div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>With this feature a distribution or
SDK can supply set of config files as a
part of clang package and tunes compiler
appropriately.</div>
</div>
</div>
</div>
</blockquote>
<div> <br>
</div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span
class="gmail-m_6136006684703735351gmail-m_4370221758571261446m_8457082739418504159gmail-">
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>1. There needs to a way to
be able to configure the
defaults for all supported
architectures of the platform
(either in a single config, or
in multiple that clang knows
how to select).<br>
</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>Each supported target can have
separate config file.<br>
</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>
<div>But, despite the target being x86_64, some
software builds 32-bit x86 code, and will run
"x86_64-debian-clang -m32" to do so. In that
case, where does clang get the appropriate
default x86 cpu from? (Note again that GCC
compiles-in separate defaults for the 32 and
64-bit architecture variants to handle this).</div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>This is a real problem. In fact -m32 convert the
specified target `x86_64-debian-clang` into
`i686-debian-clang`. We cannot rely on the specified
target as effective target can be different. This is the
root of evil.</div>
<div><br>
</div>
<div>Simple solution as config file cannot solve this
problem. We could:</div>
<div>- extend syntax of config file by allowing some sort of
conditional directives, or</div>
<div>- add additional logic (target specific) that reloads
config file if effective target changes. For instance,
invocation of `x86_64-debian-clang -m32` first loads
`x86_64-debian.cfg` which contains -target=..., which sets
defaults for 64-bit target, then command line is scanned
to calculate effective target, the option `-m32` changes
the effective target to `i686-debian-clang`. Driver then
removes all options read from config file and puts
defaults from `i686-debian.cfg`. Only then the command
line options are processed by regular logic. It sounds
complex, it fact it can be easily implemented.</div>
</div>
</div>
</div>
</blockquote>
<br>
I agree. This is an unfortunate special case. I also agree that
proposed solution (gather command line arguments including from
config files, check for -m32/64, discard as necessary and reprocess)
is a relatively clean solution. I'll note that our triple class has
get32BitArchVariant/get64BitArchVariant helpers which make adjusting
the triple relatively easy.<br>
<br>
<blockquote
cite="mid:CACOhrX5jf39uGf9A3_58n-Zvs-eXp+NsnvfcVWxOWtD8HqN_vw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>More complex solution such as spec file does not suffer
this problem, it can check presence of options like `m32`
and conditionally choose appropriate defaults.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>If it's the case that each supported target has
its own config file, clang will somehow need to
know how to choose the "i386-debian.cfg" file when
invoked as "x86_64-debian-clang -m32"....or
something like that.</div>
<span class="gmail-">
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span
class="gmail-m_6136006684703735351gmail-m_4370221758571261446m_8457082739418504159gmail-">
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>2. Those platform defaults
should, somehow, avoid
interfering with the use of
clang to cross-compile TO a
different platform, and be
easy to use FROM a different
host platform. (find default
config via sysroot, maybe?)</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>Default config in sysroot could be
included by default clang config,
however driver must know where the
sysroot is. We could support a set of
macros, for instance expand `$TARGET` in
config file to target name as
`armv7l-clang`. This topic is not
elaborated yet.</div>
<span
class="gmail-m_6136006684703735351gmail-m_4370221758571261446m_8457082739418504159gmail-"></span></div>
</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>So -- I should've separated out the two points
in my comment above.</div>
<div><br>
</div>
<div>2.1 (TO different): What happens if you do
something like "x86_64-debian-clang -target
armv7a-apple-darwin". Is that an error? Or, if
not, what does it do? Does it still use the
x86_64-debian-clang config file? Or, does it
completely ignore the config file?</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I think multiple options `-target` must produce error.
We now can call clang in such way: `clang -target
x86_64-linux -target armv7-apple-ios6.0.0 abc.c` and clang
successfully executes. Is there any use case for such
target override?</div>
</div>
</div>
</div>
</blockquote>
<br>
I agree. I think we should handle this in a manner consistent with
our current processing (we can already take a triple from the
executable prefix and a target command-line argument).<br>
<br>
<blockquote
cite="mid:CACOhrX5jf39uGf9A3_58n-Zvs-eXp+NsnvfcVWxOWtD8HqN_vw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>This problem isn't so major for
explicitly-specified configs ("Don't do that!"),
but becomes a larger issue if there's a default
config file.</div>
<div><br>
</div>
<div>This is why I was pondering if -target perhaps
should not be the recommended way for end-users to
select an architecture. The "-target" flag could
be made to mean "ignore the platform
configuration, just use this target with clang
defaults". And, the "-arch" flag could mean:
"within the current configuration, select this
named architecture configuration".</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>It would mean that clang must somehow calculate other
things as ABI.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>2.2 (FROM different): Default config files
could be searched for inside the specified
sysroot. That is, "clang --sysroot=/wherever/"
would find the proper configs for the platform
automatically, if they've been distributed in the
sysroot. The canonical way to cross-build with
clang would be to use your existing compiler, and
specify two additional arguments: --sysroot and
-target (or -arch?).</div>
<span class="gmail-">
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span
class="gmail-m_6136006684703735351gmail-m_4370221758571261446m_8457082739418504159gmail-">
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>3. How do we recommend
users select a target?<br>
</div>
<div>Do we need to look for the
proper defaults when the user
specifies "--target $TARGET"
to clang?</div>
<div>Or maybe we favor of the
"$TARGET-clang" symlink
method?</div>
<div>Or maybe "--target" is a
low-level feature not
recommended for end-users, and
we should steer people to
using something like "-arch",
to select a named architecture
within the current platform
configuration, like apple does
with darwin-based platforms
now?<br>
</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>To specify a target looks like a more
flexible solution. "$TARGET-clang"
symlink method was already implemented
in early versions of <a
moz-do-not-send="true"
href="https://reviews.llvm.org/D24933"
target="_blank">https://reviews.llvm.org/D2493<wbr>3</a>.
It is possible also to extend treatment
of `--target` so that it acted similar
to `--config`.</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>
<div>I note that Apple has hardcoded behavior in
clang for their targets, that lets it do
something sensible when specifying the sysroot:</div>
<div><br>
</div>
<div>That is, you can do:<br>
$ clang -isysroot /path/to/iPhoneOS9.3.sdk -arch
arm64<br>
</div>
<div>[Note, the "i" in the "isysroot" in that
commandline there is correct, but a
presumably-historical bogosity. It actually
finds libraries in the specified "i"sysroot,
too.]<br>
</div>
<div><br>
</div>
<div>Given the above, it:</div>
<div>1. Sets the target platform to ios9.3
(*-apple-ios9.3.0), based on matching the
pathname specified by isysroot.<br>
</div>
<div>2. Sets the target to aarch64 (printed
arm64-apple-watchos2.2.0), based on mapping the
-arch name back to a triple.</div>
<div>3. Also sets the default CPU to "cyclone".
(-target-cpu cyclone), based on hardcoded
behavior for "-arch arm64".<br>
</div>
</div>
<div><br>
</div>
<div>That seems a pretty good example of what I'm
talking about, except it's been implemented with
driver hacks instead of a generic config file
support.</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
<br>
James, what exactly are you proposing in this regard? Do you want us
to factor out the driver hack so that it can also be used by the
logic which looks for config files? Or you're trying to replace that
hack altogether with some kind of config-file-based setup?<br>
<br>
-Hal <br>
<br>
<blockquote
cite="mid:CACOhrX5jf39uGf9A3_58n-Zvs-eXp+NsnvfcVWxOWtD8HqN_vw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
</blockquote>
</div>
<br>
</div>
<div class="gmail_extra">That would be one more way to specify
config file, in addition to option `--config` and symlink
`x86_64-debian-clang`. Does such way have advantage over these
ways?<br>
$ clang --config /path/to/iPhoneOS9.3.sdk.cfg<br>
</div>
<div class="gmail_extra">or</div>
<div class="gmail_extra">
<div class="gmail_extra">$ clang --config iPhoneOS9.3.sdk.cfg<br>
</div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">Sysroot directories could contain
config files that set defaults appropriate for this SDK. For
instance, directory /path/to/iPhoneOS9.3.sdk contains file
platform.cfg. Config file loaded by clang could contain
command `@/path/to/iPhoneOS9.3.sdk/platform.cfg`. That could
be used to separate per-SDK defaults from other defaults.</div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">I don't object to such use case, we
just need to find balance between solution complexity, user
convenience and maintainability.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>--Serge</div>
</div>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory</pre>
</body>
</html>