[cfe-dev] Configuration files
Hal Finkel via cfe-dev
cfe-dev at lists.llvm.org
Mon Mar 20 17:09:59 PDT 2017
On 03/20/2017 07:19 AM, Serge Pavlov wrote:
> 2017-03-20 12:48 GMT+07:00 James Y Knight <jyknight at google.com
> <mailto:jyknight at google.com>>:
>
> On Thu, Mar 16, 2017 at 9:25 AM, Serge Pavlov <sepavloff at gmail.com
> <mailto:sepavloff at gmail.com>> wrote:
>
> 2017-03-16 9:46 GMT+07:00 James Y Knight <jyknight at google.com
> <mailto:jyknight at google.com>>:
>
> I'd really like to at least have a *design* for how this
> can eventually incorporate target-specific options before
> moving forward with adding a --config option, even if the
> initial commit won't implement the full design.
>
> I don't believe hand-wavy "maybe we'll add syntax that
> looks kinda like a comment so older versions will ignore
> it" is good enough there.
>
> I'd like to again keep in mind the use-case I mentioned a
> while ago. Approximately every linux distro configures GCC
> to set their default target cpu levels. E.g., Debian seems
> to set the following:
> - On x86, the default CPU should be i686.
> - But on x86-64, the default CPU isn't changed (aka it's
> left as "x86-64").
> - For ppc32/ppc64, the default CPU should be power7 but
> tune for power8.
> - For ppc64le, the default CPU should be power8.
> - On ARM (hf), armv7-a should be the default cpu,
> vfpv3-d16 the default fpu, and it should default to thumb
> mode.
> etc...
>
> Note that those defaults are different on different
> releases of the distro.
>
> The way you do this with GCC is via options passed to the
> configure script: --with-arch-32= --with-arch-64=
> --with-fpu= --with-mode= etc. which turn into values used
> in the target-specific OPTION_DEFAULT_SPECS macro. Since
> GCC only builds for one target at a time (or two, if you
> count 32/64 separately), and you're expected to need to
> build a new gcc any time you want to cross-compile, that's
> sufficient.
>
> Clang is intrinsically a cross-compiler, so gcc's solution
> isn't good enough for clang (nor is clang's current
> behavior enough). So, what's the plan to actually solve this?
>
>
> Initial versions of this proposal defined two kinds of config
> files:
> - named, which should be explicitly specified by a user by
> option --config or be encoded into executable name as
> `armv7l-clang`.
> - default, which is loaded always much like `.bashrc` or any
> similar file.
> These two kinds of config file shared implementation but
> addressed different use cases, which made confusion during
> discussion. To facilitate review process the support of
> default config files was removed from the proposal. The issues
> you mention should mostly be solved by the default config files.
>
>
> OK -- focusing on cross-compilation for a minute. Clang and LLD
> have the potential to make cross-compilation *MUCH* simpler than
> it's traditionally been, and I'd like to be careful to think about
> how to take the best advantage of the fact that we're getting
> really close to having a single toolchain which is able to easily
> build programs for any target architecture/environment.
>
> Here's the scenario I'm using in my mind:
>
> Let's say I've got a super-fast RISC-V machine running OpenBSD,
> and want to cross-build from that to all of the various supported
> Debian architectures. I want to use my existing OpenBSD-supplied
> clang+lld toolchain, because that toolchain can natively
> cross-compile to everything, if given appropriate command-line
> arguments. So, I don't need to make a special build of the
> compiler or linker. Woohoo!
>
> I have copied over the root partition of the target Debian system
> into a directory on this machine, so I have headers and libraries
> to link against.
>
> (I also need the crtbegin/end and libgcc or compiler-rt. For this
> thread, let's assume I copied those over as well, even though
> they're typically "part of" the compiler toolchain suite. A
> separate discussion should be had about what to do about making
> THAT part of the cross-compile story easier -- it's almost the
> only tricky part remaining!)
>
> Now, given that scenario --
>
> How do I specify that I want to cross-build to that debian system?
>
> I need to know the proper configuration, and I need to know how
> I'm supposed to invoke clang to use the configuration.
>
> At the moment, you might run something like:
> "clang --target x86_64-linux-gnu
> --sysroot=/path/to/debian-installation
> --other-super-important-cpu/abi-flags"
>
> With your proposed patch, I would create a config file,
> "x86_64-debian.cfg", containing the above flags. I could put it in
> /etc/something or ~/.something, and run:
> $ clang -config x86_64-debian
>
> Alternatively, I could put it in ~/bin/x86_64-debian.cfg and
> $ ln -s /usr/bin/clang ~/bin/x86_64-debian-clang
> $ x86_64-debian-clang
>
>
> In general you are right I think.
>
> If the default config files were implemented in clang, driver
> would search binary directory for default configuration
> description. If compiler is named as `armv7l-clang`, driver
> first tries file `armv7l.cfg` then `clang.cfg`. If config file
> is found, options listed there are put into the set of driver
> arguments before any option specified in command line.
>
>
> With this feature a distribution or SDK can supply set of
> config files as a part of clang package and tunes compiler
> appropriately.
>
>
> 1. There needs to a way to be able to configure the
> defaults for all supported architectures of the platform
> (either in a single config, or in multiple that clang
> knows how to select).
>
>
> Each supported target can have separate config file.
>
>
> But, despite the target being x86_64, some software builds 32-bit
> x86 code, and will run "x86_64-debian-clang -m32" to do so. In
> that case, where does clang get the appropriate default x86 cpu
> from? (Note again that GCC compiles-in separate defaults for the
> 32 and 64-bit architecture variants to handle this).
>
>
> This is a real problem. In fact -m32 convert the specified target
> `x86_64-debian-clang` into `i686-debian-clang`. We cannot rely on the
> specified target as effective target can be different. This is the
> root of evil.
>
> Simple solution as config file cannot solve this problem. We could:
> - extend syntax of config file by allowing some sort of conditional
> directives, or
> - add additional logic (target specific) that reloads config file if
> effective target changes. For instance, invocation of
> `x86_64-debian-clang -m32` first loads `x86_64-debian.cfg` which
> contains -target=..., which sets defaults for 64-bit target, then
> command line is scanned to calculate effective target, the option
> `-m32` changes the effective target to `i686-debian-clang`. Driver
> then removes all options read from config file and puts defaults from
> `i686-debian.cfg`. Only then the command line options are processed by
> regular logic. It sounds complex, it fact it can be easily implemented.
I agree. This is an unfortunate special case. I also agree that proposed
solution (gather command line arguments including from config files,
check for -m32/64, discard as necessary and reprocess) is a relatively
clean solution. I'll note that our triple class has
get32BitArchVariant/get64BitArchVariant helpers which make adjusting the
triple relatively easy.
>
> More complex solution such as spec file does not suffer this problem,
> it can check presence of options like `m32` and conditionally choose
> appropriate defaults.
>
> If it's the case that each supported target has its own config
> file, clang will somehow need to know how to choose the
> "i386-debian.cfg" file when invoked as "x86_64-debian-clang
> -m32"....or something like that.
>
> 2. Those platform defaults should, somehow, avoid
> interfering with the use of clang to cross-compile TO a
> different platform, and be easy to use FROM a different
> host platform. (find default config via sysroot, maybe?)
>
>
> Default config in sysroot could be included by default clang
> config, however driver must know where the sysroot is. We
> could support a set of macros, for instance expand `$TARGET`
> in config file to target name as `armv7l-clang`. This topic
> is not elaborated yet.
>
>
> So -- I should've separated out the two points in my comment above.
>
> 2.1 (TO different): What happens if you do something like
> "x86_64-debian-clang -target armv7a-apple-darwin". Is that an
> error? Or, if not, what does it do? Does it still use the
> x86_64-debian-clang config file? Or, does it completely ignore the
> config file?
>
>
> I think multiple options `-target` must produce error. We now can call
> clang in such way: `clang -target x86_64-linux -target
> armv7-apple-ios6.0.0 abc.c` and clang successfully executes. Is there
> any use case for such target override?
I agree. I think we should handle this in a manner consistent with our
current processing (we can already take a triple from the executable
prefix and a target command-line argument).
> This problem isn't so major for explicitly-specified configs
> ("Don't do that!"), but becomes a larger issue if there's a
> default config file.
>
> This is why I was pondering if -target perhaps should not be the
> recommended way for end-users to select an architecture. The
> "-target" flag could be made to mean "ignore the platform
> configuration, just use this target with clang defaults". And, the
> "-arch" flag could mean: "within the current configuration, select
> this named architecture configuration".
>
>
> It would mean that clang must somehow calculate other things as ABI.
>
> 2.2 (FROM different): Default config files could be searched for
> inside the specified sysroot. That is, "clang
> --sysroot=/wherever/" would find the proper configs for the
> platform automatically, if they've been distributed in the
> sysroot. The canonical way to cross-build with clang would be to
> use your existing compiler, and specify two additional arguments:
> --sysroot and -target (or -arch?).
>
> 3. How do we recommend users select a target?
> Do we need to look for the proper defaults when the user
> specifies "--target $TARGET" to clang?
> Or maybe we favor of the "$TARGET-clang" symlink method?
> Or maybe "--target" is a low-level feature not recommended
> for end-users, and we should steer people to using
> something like "-arch", to select a named architecture
> within the current platform configuration, like apple does
> with darwin-based platforms now?
>
>
> To specify a target looks like a more flexible solution.
> "$TARGET-clang" symlink method was already implemented in
> early versions of https://reviews.llvm.org/D24933
> <https://reviews.llvm.org/D24933>. It is possible also to
> extend treatment of `--target` so that it acted similar to
> `--config`.
>
>
> I note that Apple has hardcoded behavior in clang for their
> targets, that lets it do something sensible when specifying the
> sysroot:
>
> That is, you can do:
> $ clang -isysroot /path/to/iPhoneOS9.3.sdk -arch arm64
> [Note, the "i" in the "isysroot" in that commandline there is
> correct, but a presumably-historical bogosity. It actually finds
> libraries in the specified "i"sysroot, too.]
>
> Given the above, it:
> 1. Sets the target platform to ios9.3 (*-apple-ios9.3.0), based on
> matching the pathname specified by isysroot.
> 2. Sets the target to aarch64 (printed arm64-apple-watchos2.2.0),
> based on mapping the -arch name back to a triple.
> 3. Also sets the default CPU to "cyclone". (-target-cpu cyclone),
> based on hardcoded behavior for "-arch arm64".
>
> That seems a pretty good example of what I'm talking about, except
> it's been implemented with driver hacks instead of a generic
> config file support.
>
James, what exactly are you proposing in this regard? Do you want us to
factor out the driver hack so that it can also be used by the logic
which looks for config files? Or you're trying to replace that hack
altogether with some kind of config-file-based setup?
-Hal
>
> That would be one more way to specify config file, in addition to
> option `--config` and symlink `x86_64-debian-clang`. Does such way
> have advantage over these ways?
> $ clang --config /path/to/iPhoneOS9.3.sdk.cfg
> or
> $ clang --config iPhoneOS9.3.sdk.cfg
>
> Sysroot directories could contain config files that set defaults
> appropriate for this SDK. For instance, directory
> /path/to/iPhoneOS9.3.sdk contains file platform.cfg. Config file
> loaded by clang could contain command
> `@/path/to/iPhoneOS9.3.sdk/platform.cfg`. That could be used to
> separate per-SDK defaults from other defaults.
>
> I don't object to such use case, we just need to find balance between
> solution complexity, user convenience and maintainability.
>
> Thanks,
> --Serge
--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20170320/a7608627/attachment.html>
More information about the cfe-dev
mailing list