[PATCH] D24933: Enable configuration files in clang

[Serge Pavlov via cfe-commits]

sepavloff added a comment.

Summary of proposal and discussion in mail list.

**Problems**

The two main problems this feature addresses are:

1. Ability to modify default compiler settings.

As an example, the warning `-Wundefined-var-template` can be helpful for people doing module-enabled builds   (https://llvm.org/bugs/show_bug.cgi?id=24425), but makes headache for others, who even come with ideas to turn this warning off by default (http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20160808/167354.html). To cope with this problem clang need a mechanism that effectively turns off a warning as if it were disabled in compiler sources.

2. Support of cross builds.

Doing cross builds require to specify lots of options that set target, backend options, include and library directories etc. There is interest in making this action more convenient.

**Solution**

Configuration file groups related options together and allow to specify them in more simple and less error prone way than just listing them somewhere in build scripts. Configuration may be thought as a "macro" names option set and is expanded when compiler is called. There are two use patterns of configuration files:

- Default configuration file that is applied without need to specify anything in compiler invocation. It can be read from predefined location or be specified by environmental variable.
- Named configuration file that is specified explicitly by a user, either by command line option `--config`, or by using special executable file names, such as `armv7l-clang`.

Only one config file is used, if it is specified explicitly, default config file is not searched. Config file may contain comments, references to other config files using syntax `@file`. Default config file is searched in the directory where clang executable resides.

**Existing experience**

Configuration file is not a new concept, it has been used in various form in many compilers including clang.

1. Intel compiler supports default configuration files (https://software.intel.com/en-us/node/522780), the file is searched for in the directory where executable resides or can be specified by setting environmental variable. The feature is pretty convenient in practice.
2. Clang allows to specify target by using special executable names, like `armv7l-clang`. Only target name can be specified by this way, which is not sufficient to tune compiler for new target. Obvious question arise - why not extend this mechanism for other target specific options too.
3. Clang allows modifying options using environmental variable `CCC_OVERRIDE_OPTIONS`. This is undocumented way and is looks more like a hack for internal use. Its existence however indicates that such functionality is useful.
4. Cross compilation environment ELLCC (http://ellcc.org), based on clang, supports advanced config files in YAML format. It has proven to be very handy to support a wide variety of targets.
5. GCC has support of spec files (https://gcc.gnu.org/onlinedocs/gcc/Spec-Files.html), which can be taken from default location or be specified by command line option `--spec`. Spec files are more powerful tool than just config files as they not only set options but also program driver logic. Using custom spec files allows GCC to be tuned to particular target without need to specify lots of options in command line.

**Concerns**

Somewhat random set of concerns extracted from discussions.

1. "what flags to pass to the compiler should be configured in a project's build system"

A project may include dozens of components each come with own build systems, which are tuned differently. Retargeting such build system can be time consuming and error prone. On the other hand, setting compiler and target specific options is not a business of a component, ideally it may be unaware of using in cross compilation environment. Also, not all component build systems are error free.

2. "If a project wants to disable some warnings by default, add it to the Makefile".

The problem is that some users do not want modify their build files, probably because it is not simple process as it appears to us. They would prefer customized compiler in which some options are on/off by default.

3. "The idea of having the compiler search for config files in random places on the filesystem seems scary"

Most of elaborated environments allows search for libraries, packages, classes etc in some predefined places. Running simple command `gcc file.c` make compiler search set of various places, and the set may change depending on version and OS variant. Complex problems may require complicated tools for solving.

4. "config files can make difficult to reproduce failure conditions"

Actual set of compiler options is provided in error dump files, in output of `clang -v` and `clang -###`, these invocations also report used config file.

5. "Right now when you invoke clang without any additional options you know that you are in a well-known state"

"If I don't pass any flags, I want Clang to work the same everywhere"
	In fact invocation of clang without additional argument does not provide a well known state. Actual state is represented by options passed from clang driver to clang compiler, but these options depend on used OS, CPU. In theory a system upgrade could cause compilation change. Hardly bare driver arguments can represent compilation options.

6. "we already have `@file` for bundling up compiler flags"

Config file make this feature a bit more convenient by providing a way to choose proper option set and making files containing options more user friendly by allowing comments and nested `@file`.

7. "There is already a lot of magic in the driver to figure out how to invoke the tools"

Orchestrating tools made by clang driver is already a complex task and IMO there is no hope that it can be solved by simple way. But it can be made more structured, config file may be a step in this direction making existing target selection in clang more powerful and useful.


https://reviews.llvm.org/D24933