<div dir="ltr"><div>The proposal has been evolved due to feedback. Below is a summary of updated version and feedback from various sources.</div><div><br></div><div>**Problems**</div><div><br></div><div>The two main problems this feature addresses are:</div><div><br></div><div>1. Ability to modify default compiler settings.</div><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>As an example, the warning `-Wundefined-var-template` can be helpful for people doing module-enabled builds (<a href="https://llvm.org/bugs/show_bug.cgi?id=24425">https://llvm.org/bugs/show_bug.cgi?id=24425</a>), but it makes headache for others, who even come with ideas to turn this warning off by default (<a href="http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20160808/167354.html">http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20160808/167354.html</a>).</div></blockquote><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>To cope with this and similar problem clang need a mechanism that effectively change default settings hardcoded in compiler sources. </div></blockquote><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>2. Support of cross builds.</div><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>Doing cross builds requires to specify lots of options that set target, backend options, include and library directories etc. These options could be placed into config files and then be specified with single option or even without such. This feature could make cross compilation with clang more convenient.</div></blockquote><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>**Solution**</div><div><br></div><div>Configuration file groups related options together and allow to specify them in more simple and less error prone way than just listing them somewhere in build scripts. According to the two problems to solve, there are two use patterns of configuration files:</div><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>- Default config file is applied without need to specify anything in compiler invocation. It can be read from predefined location or be specified by environmental variable. It allows to override compiler defaults.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>- Named config file is specified explicitly by a user, either by command line option `--config`, or by using special executable file names, such as `armv7l-clang`. Such configuration may be thought as a "macro" that designates an option set and is expanded when compiler is called.</div></blockquote><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>Only one config file is used, if it is specified explicitly, default config file is not searched. Config file may contain comments (# style), references to other config files using syntax `@file`.</div><div><br></div><div>Config file are searched in the directory where clang executable resides. User may specify any config file using option `--config` followed by full path of the file. Alternatively the path to config file may be specified in environmental variable CLANGCFG.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>**Existing experience**</div><div><br></div><div>Configuration file is not a new concept, it has been used in various form in many compilers including clang.</div><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>1. Intel compiler supports default configuration files (<a href="https://software.intel.com/en-us/node/522780">https://software.intel.com/en-us/node/522780</a>), the file is searched for in the directory where executable resides or can be specified by setting environmental variable. The feature is pretty convenient in practice.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>2. Clang allows to specify target by using special executable names, like `armv7l-clang`. Only target name can be specified by this way, which is not sufficient to tune compiler for a new target. Obvious question arise - why not extend this mechanism for other target specific options too.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>3. Clang allows modifying options using environmental variable `CCC_OVERRIDE_OPTIONS`. This is undocumented way and is looks more like a hack for internal use. Its existence however indicates that such functionality is useful.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>4. Cross compilation environment ELLCC (<a href="http://ellcc.org">http://ellcc.org</a>), based on clang, supports advanced config files in YAML format. It has proven to be very handy to support a wide variety of targets.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>5. GCC has support of spec files (<a href="https://gcc.gnu.org/onlinedocs/gcc/Spec-Files.html">https://gcc.gnu.org/onlinedocs/gcc/Spec-Files.html</a>), which can be taken from default location or be specified by command line option `--spec`. Spec files are more powerful tool than just config files as they not only set options but also program driver logic. Using custom spec files allows GCC to be tuned to particular target without need to specify lots of options in command line.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>6. Visual Studio supports custom project templates (<a href="https://msdn.microsoft.com/en-us/library/xkh1wxd8.aspx">https://msdn.microsoft.com/en-us/library/xkh1wxd8.aspx</a>), which may specify any compiler flags as default setting. Although it can be considered as feature of build system rather than compiler, it allows to override default compiler settings.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>7. Keil C compilers allow to specify compiler options in environmental variables, ARMCOMPILER6_CLANGOPT (<a href="http://www.keil.com/support/man/docs/armclang_intro/armclang_intro_chr1374139991387.htm">http://www.keil.com/support/man/docs/armclang_intro/armclang_intro_chr1374139991387.htm</a>) or ARMCC5_CCOPT (<a href="http://infocenter.arm.com/help/topic/com.arm.doc.dui0592e/pge1399635613963.html">http://infocenter.arm.com/help/topic/com.arm.doc.dui0592e/pge1399635613963.html</a>) depending on compiler version.</div></blockquote><div><span class="gmail-Apple-tab-span" style="white-space:pre"><br></span></div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>**Concerns**</div><div><br></div><div>Somewhat random set of concerns extracted from discussions with answers to them.</div><div><br></div><div>1. With default config files the options specified for clang in command line do not represent definite state anymore, because some options may come from the config file. A user that prepares bug report can forget to specify content of the used default config file, it can make difficult to reproduce failure conditions.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>Answer: Actual set of compiler options is provided in error dump files, in output of `clang -v` and `clang -###`, these invocations also report used config file. If a project wants to disable some warnings by default, add it to the Makefile.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>2. It is not a business of compiler to adjust options. What flags to pass to the compiler should be configured in a project's build system.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>Answer: A project may include dozens of components each come with own build systems, which are tuned differently. Retargeting such build system can be time consuming and error prone. On the other hand, setting compiler and target specific options is not a business of a component, ideally it may be unaware of using in cross compilation environment. Also, not all component build systems are flexible enough. As a result some users do not want modify their build files, they would prefer customized compiler.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>Config files brings more power and convenience to the compiler.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>3. Clang already has `@file` for bundling up compiler flags.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>Answer: Config file make this feature a bit more convenient by providing a way to choose proper option set and making files containing options more user friendly by allowing comments and nested `@file`. Also `@file` cannot solve the problem of overriding compiler defaults.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>4. The idea of having the compiler search for config files in random places on the filesystem seems scary.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>Answer: Default config files are searched only in the directory where compiler executable file resides. If compiler is installed from package, this file is placed in system directories and should not be changed by a simple user. It is assumed that such file is a part of SDK or some package prepared by experienced person, it is not changed frequently. The same applies to target-specific config files. A user may specify full path to config file, in this case the file may be anywhere but this explicit action.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>Most of elaborated environments allows search for libraries, packages, classes etc in some predefined places. Running simple command `gcc file.c` make compiler search set of various places, and the set may change depending on version and OS variant.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>5. There is already a lot of magic in the driver to figure out how to invoke the tools, this feature makes driver more complex.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>Answer: Orchestrating tools made by clang driver is already a complex task and hardly it can be made simple. But it can be made more structured, config file may be a step in this direction making existing target selection in clang more powerful and useful.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>6. This is too primitive solution:</div>- A good solution must be able to deduce target from arguments (-m32).<br>- We should also probably automatically silence unused options from config files. For example, if my cross-compile toolchain passes some extra linker arguments with -Wl,-foo, then I don’t want every single line in my build to complain that -Wl is unused in conjunction with -c.<br>- It should provide default mapping from targets to sysroots and associated paths; we currently hardcode a lot of this into the compiler itself.<br>- Any workable config-file scheme *must* interact with the various target specification command-line arguments, and allow for setting options dependent upon the actually-selected target.<br><div>- Maybe it'd be better to create a regular, .ini/.conf-style configuration file.</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div>Answer: This is a simple solution, that solves a good deal of problems. It does not require big changes in documentation and substantial maintenance efforts. It can be extended in future by allowing more complex constructs in config files.</div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature">Thanks,<br>--Serge<br></div></div>
<br><div class="gmail_quote">2016-10-05 0:29 GMT+07:00 Reid Kleckner <span dir="ltr"><<a href="mailto:rnk@google.com" target="_blank">rnk@google.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">My main concern with configuration files is that they should interact with regular command line flags in a predictable and maintainable way. Your approach of extending response files seems like it's going in the right direction.<div><br></div><div>I'm not excited about adding something like ELLC's YAML format to Clang (<a href="http://ellcc.org/viewvc/svn/ellcc/trunk/libecc/config/ppc64el-linux?view=markup" target="_blank">http://ellcc.org/viewvc/svn/<wbr>ellcc/trunk/libecc/config/<wbr>ppc64el-linux?view=markup</a>) because then we will have to document and maintain the semantics of things like multiple configuration files, in what order they override each other, and how they interact with flags. Our command line flag logic is already too complicated. We have one public interface, and it's flags, and I'm hesitant to add a new one.</div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="h5">On Mon, Sep 19, 2016 at 1:06 AM, Serge Pavlov via cfe-dev <span dir="ltr"><<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5"><div dir="ltr"><div>Hi all,</div><div><br></div><div>I would like to propose implementing configuration files in clang. The idea is to read some set of options prior to the options specified in a command line in every tool invocation. These options are taken from a file which is searched either in predefined place or its location is specified by environmental variable.</div><div><br></div><div>Few words about particular problem this facility can solve. Clang issues many warnings, some refer to potential errors, some merely attract attention to code that can cause problems in some circumstances but otherwise is pretty innocent. For example, warning -Wundefined-var-template is issued for template usages, which nor cannot be instantiated implicitly neither are mentioned in explicit instantiation declarations. Such usage is not an error or bad style, but the warning may be helpful for a user in a complex case, such as described in PR24425. For other users this warning may be annoying and they would prefer to turn it off by default (see discussion in <a href="http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20160808/167354.html" target="_blank">http://lists.llvm.org/pipermai<wbr>l/cfe-commits/Week-of-Mon-<wbr>20160808/167354.html</a>). Different categories of users like to have different set of warnings enabled by default.</div><div><br></div><div>If configuration files were supported by clang, a user can put -Wno-undefined-var-template to such file and this would have the same effect as if the unwanted warning were turned off by default. No changes to build scripts would be required.</div><div><br></div><div>Configuration files are supported by the Intel Compiler (<a href="https://software.intel.com/en-us/node/522780" target="_blank">https://software.intel.com/en<wbr>-us/node/522780</a>) and similar behavior is proposed to be implemented by clang.</div><div><br></div><div>By default the configuration file is searched in the same directory where tool executable resides. In the case of clang the file would be 'clang.cfg', for other tool named 'foo'- 'foo.cfg'. User can specify any location for this file if he sets up environmental variable 'CLANGCFG' (or 'FOOCFG'). The variable should contain path to configuration file. If the configuration file does not exist, it is silently ignored. Content of configuration file is treated as a part of command line of each tool invocation preceding any arguments specified in command line.</div><div><br></div><div>Does implementation of this facility makes sense?</div><div><br></div><div><div class="m_7522212463212906839m_-6253681486997706616gmail_signature">Thanks,<br>--Serge<br></div></div>
</div>
<br></div></div><span class="">______________________________<wbr>_________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/cfe-dev</a><br>
<br></span></blockquote></div><br></div>
</blockquote></div><br></div>