<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.7: http://docutils.sourceforge.net/" />
<title>Clang driver requirements</title>
<style type="text/css">
/*
:Author: David Goodger (goodger@python.org)
:Id: $Id: html4css1.css 6253 2010-03-02 00:24:53Z milde $
:Copyright: This stylesheet has been placed in the public domain.
Default cascading style sheet for the HTML output of Docutils.
See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to
customize this style sheet.
*/
/* used to remove borders from tables and images */
.borderless, table.borderless td, table.borderless th {
border: 0 }
table.borderless td, table.borderless th {
/* Override padding for "table.docutils td" with "! important".
The right padding separates the table cells. */
padding: 0 0.5em 0 0 ! important }
.first {
/* Override more specific margin styles with "! important". */
margin-top: 0 ! important }
.last, .with-subtitle {
margin-bottom: 0 ! important }
.hidden {
display: none }
a.toc-backref {
text-decoration: none ;
color: black }
blockquote.epigraph {
margin: 2em 5em ; }
dl.docutils dd {
margin-bottom: 0.5em }
/* Uncomment (and remove this text!) to get bold-faced definition list terms
dl.docutils dt {
font-weight: bold }
*/
div.abstract {
margin: 2em 5em }
div.abstract p.topic-title {
font-weight: bold ;
text-align: center }
div.admonition, div.attention, div.caution, div.danger, div.error,
div.hint, div.important, div.note, div.tip, div.warning {
margin: 2em ;
border: medium outset ;
padding: 1em }
div.admonition p.admonition-title, div.hint p.admonition-title,
div.important p.admonition-title, div.note p.admonition-title,
div.tip p.admonition-title {
font-weight: bold ;
font-family: sans-serif }
div.attention p.admonition-title, div.caution p.admonition-title,
div.danger p.admonition-title, div.error p.admonition-title,
div.warning p.admonition-title {
color: red ;
font-weight: bold ;
font-family: sans-serif }
/* Uncomment (and remove this text!) to get reduced vertical space in
compound paragraphs.
div.compound .compound-first, div.compound .compound-middle {
margin-bottom: 0.5em }
div.compound .compound-last, div.compound .compound-middle {
margin-top: 0.5em }
*/
div.dedication {
margin: 2em 5em ;
text-align: center ;
font-style: italic }
div.dedication p.topic-title {
font-weight: bold ;
font-style: normal }
div.figure {
margin-left: 2em ;
margin-right: 2em }
div.footer, div.header {
clear: both;
font-size: smaller }
div.line-block {
display: block ;
margin-top: 1em ;
margin-bottom: 1em }
div.line-block div.line-block {
margin-top: 0 ;
margin-bottom: 0 ;
margin-left: 1.5em }
div.sidebar {
margin: 0 0 0.5em 1em ;
border: medium outset ;
padding: 1em ;
background-color: #ffffee ;
width: 40% ;
float: right ;
clear: right }
div.sidebar p.rubric {
font-family: sans-serif ;
font-size: medium }
div.system-messages {
margin: 5em }
div.system-messages h1 {
color: red }
div.system-message {
border: medium outset ;
padding: 1em }
div.system-message p.system-message-title {
color: red ;
font-weight: bold }
div.topic {
margin: 2em }
h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
margin-top: 0.4em }
h1.title {
text-align: center }
h2.subtitle {
text-align: center }
hr.docutils {
width: 75% }
img.align-left, .figure.align-left, object.align-left {
clear: left ;
float: left ;
margin-right: 1em }
img.align-right, .figure.align-right, object.align-right {
clear: right ;
float: right ;
margin-left: 1em }
img.align-center, .figure.align-center, object.align-center {
display: block;
margin-left: auto;
margin-right: auto;
}
.align-left {
text-align: left }
.align-center {
clear: both ;
text-align: center }
.align-right {
text-align: right }
/* reset inner alignment in figures */
div.align-right {
text-align: left }
/* div.align-center * { */
/* text-align: left } */
ol.simple, ul.simple {
margin-bottom: 1em }
ol.arabic {
list-style: decimal }
ol.loweralpha {
list-style: lower-alpha }
ol.upperalpha {
list-style: upper-alpha }
ol.lowerroman {
list-style: lower-roman }
ol.upperroman {
list-style: upper-roman }
p.attribution {
text-align: right ;
margin-left: 50% }
p.caption {
font-style: italic }
p.credits {
font-style: italic ;
font-size: smaller }
p.label {
white-space: nowrap }
p.rubric {
font-weight: bold ;
font-size: larger ;
color: maroon ;
text-align: center }
p.sidebar-title {
font-family: sans-serif ;
font-weight: bold ;
font-size: larger }
p.sidebar-subtitle {
font-family: sans-serif ;
font-weight: bold }
p.topic-title {
font-weight: bold }
pre.address {
margin-bottom: 0 ;
margin-top: 0 ;
font: inherit }
pre.literal-block, pre.doctest-block {
margin-left: 2em ;
margin-right: 2em }
span.classifier {
font-family: sans-serif ;
font-style: oblique }
span.classifier-delimiter {
font-family: sans-serif ;
font-weight: bold }
span.interpreted {
font-family: sans-serif }
span.option {
white-space: nowrap }
span.pre {
white-space: pre }
span.problematic {
color: red }
span.section-subtitle {
/* font-size relative to parent (h1..h6 element) */
font-size: 80% }
table.citation {
border-left: solid 1px gray;
margin-left: 1px }
table.docinfo {
margin: 2em 4em }
table.docutils {
margin-top: 0.5em ;
margin-bottom: 0.5em }
table.footnote {
border-left: solid 1px black;
margin-left: 1px }
table.docutils td, table.docutils th,
table.docinfo td, table.docinfo th {
padding-left: 0.5em ;
padding-right: 0.5em ;
vertical-align: top }
table.docutils th.field-name, table.docinfo th.docinfo-name {
font-weight: bold ;
text-align: left ;
white-space: nowrap ;
padding-left: 0 }
h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
font-size: 100% }
ul.auto-toc {
list-style-type: none }
</style>
</head>
<body>
<div class="document" id="clang-driver-requirements">
<h1 class="title">Clang driver requirements</h1>
<div class="section" id="changelog">
<h1>Changelog</h1>
<p>2011-Nov-04: JamesM: Updated usecases, s/driver/plugin/g, sent to list.
2011-Oct-25: JamesM: Initial draft.</p>
</div>
<div class="section" id="introduction">
<h1>Introduction</h1>
<p>The current Clang driver is inadequate, as shown by many mailing list
threads, the latest being <a class="citation-reference" href="#ml1" id="id1">[ML1]</a>. Its failings seem to stem from the
lack of real extensibility without touching lots of the codebase, and
many interlinked special cases.</p>
<p>The current driver, while OK for hosted compilation is very difficult
to set up for cross-compilation.</p>
<p>This document aims initially to merely document the requirements for
<em>a compiler driver for an Clang/LLVM based compiler</em>, specifically aimed at but
not limited to the Clang codebase - that is, it should be possible to
reuse this driver component for a similar but different tool, much
like the rest of Clang's design.</p>
<p>This document aims at building/documenting a consensus on driver
design. There are many different use cases in the community so this
design is expected to be a living document which will have input from
the entire (interested) community.</p>
</div>
<div class="section" id="potential-use-cases">
<h1>Potential use cases</h1>
<p>Where these use cases spawn out requirements, these are indexed in
square brackets ([X]).</p>
<div class="section" id="usecase-1-unix-windows-distribution-maintainer-creating-a-distribution">
<h2>Usecase 1: UNIX/Windows distribution maintainer creating a distribution</h2>
<p>Wants to take Clang ToT or release tag, compile and package. The only
change he needs to make is to tell Clang where to pick up headers and
libraries. This differs between distributions, and with multilib
support it has started differing even more. This is due to differing
directory structures for multilib and the expectation of where to find
headers, libraries and crt*.o's. The ideal would be to not
have to recompile Clang or maintain diffs against the clang tree to
make this change [2].</p>
<p>The library and header locations may depend on the resolved target
triple, along with other possible parameters (command line options,
environment variables, etc.).</p>
</div>
<div class="section" id="usecase-2-developer-creating-a-clang-based-derivative-tool">
<h2>Usecase 2: Developer creating a Clang-based derivative tool</h2>
<p>Researchers or companies may wish to adapt Clang in more complex ways
than Usecase 1, such as adding new command line flags, adding
compatibility modes or changing the subtools invoked (for example,
invoking an alternate linker which takes different command line arguments).</p>
<p>Wants to adapt the command line parsing in a more complex way -
perhaps by ([3]):</p>
<blockquote>
<ul class="simple">
<li>Adding new command line flags.</li>
<li>Editing the functionality of current command line flags, possibly
in a complex (non-declarative) way.</li>
<li>Altering the way subtools are invoked.</li>
</ul>
</blockquote>
<p>These changes should be easy to maintain - diffs should be able to be
separate from the main Driver and not subject to clobbering by
enthusiastic Driver developers [4].</p>
</div>
<div class="section" id="usecase-3-clang-developer-developing">
<h2>Usecase 3: Clang developer, developing</h2>
<p>Wants no functionality to change - things keep working as normal [1]</p>
</div>
<div class="section" id="usecase-4-apple-darwin-developer-using-fat-binaries">
<h2>Usecase 4: Apple/Darwin developer, using fat binaries</h2>
<p>Requires fat-binary support. This entails multiple "-arch" arguments
being supported. [1]</p>
<div class="note">
<p class="first admonition-title">Note</p>
<p class="last">Describe this some more?</p>
</div>
</div>
</div>
<div class="section" id="functional-requirements">
<h1>Functional requirements</h1>
<p>The following requirements follow from the use cases above and attempt
to formalise those use cases more precisely.</p>
<dl class="docutils">
<dt>[1] No functional regressions</dt>
<dd>The driver <strong>must</strong> be able to be configured such that it can parse
command lines that the current Clang driver accepts. The driver
<strong>must</strong> invoke all subtools in the same manner as the current Clang
driver, with the possible exception of obtuse, undefined, legacy or
otherwise incorrect behaviour, permission for which must be obtained
from the mailing list and documented in a subsection of this
document for decision tracking.</dd>
<dt>[2] Adaptability</dt>
<dd><p class="first">The driver's parameters (search paths, header locations etc)
<strong>must</strong> be able to be changed with minimal intervention.</p>
<p class="last">These parameters <strong>should</strong> be able to be changed without a
recompile of Clang or any changes to the source base.</p>
</dd>
<dt>[3] Extensibility</dt>
<dd><p class="first">All parts of the driver that are to interact with outside
environment (such as interpreting command lines and launching
subtools) <strong>must</strong> be able to have their behaviour easily modified.</p>
<p class="last">While there is no requirement for this to be able to be done with no
source changes, there <strong>could</strong> be scope for allowing dynamically
loadable modules (in the spirit of <tt class="docutils literal">opt <span class="pre">-load</span></tt>) to change the
driver's behaviour at invoke-time.</p>
</dd>
<dt>[4] Maintainability of downstream changes</dt>
<dd><p class="first">There must be a highly defined and documented API that can be
followed by a developer attempting to modify the driver's
behaviour. This API should make it possible to, at a minimum:</p>
<blockquote class="last">
<ul class="simple">
<li>Add, remove and modify command line flags with possibly complex
imperative rules.</li>
<li>Hook into and mutate commands to be passed to subtools.</li>
<li>Pass "Clang-like" diagnostics to the user.</li>
<li>Set up default parameters such as include paths.</li>
<li>Separate their modification from the Clang sourcebase, at least
to the extent that existing Clang source files should not need
to be modified with anything other than a trivial patch.</li>
</ul>
</blockquote>
</dd>
</dl>
</div>
<div class="section" id="proposed-design-a">
<h1>Proposed design "A"</h1>
<p>The following design is proposed as a strawman to expose possible
flaws in the requirements and to generate more targetted
discussion. That said, I (JamesM) certainly think it's a decent
design, else I wouldn't propose it.</p>
<p>As the requirements change (and they will!) this design should change with
them.</p>
<p>The high level overview is of three parts: a driver "Kernel", one or
more "Driver Plugins" and a "Config file".</p>
<div class="note">
<p class="first admonition-title">Note</p>
<p class="last">Find a better word than "plugin" so it won't cause confusion
between this and normal Clang plugins?</p>
</div>
<p>This solves the requirements in the following ways:</p>
<ol class="arabic simple">
<li>As a pure rearchitecting exercise there should be no need for any
functional differences to take place.</li>
<li>The config files allow for easy tweaking of configuration
without a recompile.</li>
<li>The driver framework forms a stable API for adding to and defining
driver functionality, and also easily allows this to be imported at
runtime via shared object.</li>
<li>The driver framework means that ideally a developer can create
his/her own driver, plug it in and it not be affected at all by any
Clang change other than the driver API.</li>
</ol>
<div class="section" id="driver-kernel">
<h2>Driver Kernel</h2>
<p>The driver Kernel will be responsible for handling user input and
calling subprocesses. In particular it will parse command lines in a
generic, almost POSIX-compliant manner, launch subprocesses and
display their diagnostics, and emit diagnostics of its own.</p>
<p>It will maintain a list of plugins which are partially ordered (so that the
order they are linked in / loaded is not important), which will each
expose a list of command line options they handle, and how to handle
them (for example, setting a parameter or handing off to a handler
function).</p>
<p>The kernel should maintain a state as a sort of dictionary/hash which
the plugins can access and mutate, as well as add extra entries to.</p>
<p>Once all plugins have mutated the state due to command line options,
the state is handed to the adapter file to mutate further, after which
the kernel generates command lines to invoke subprocesses.</p>
<p>These command lines are then sent to the plugins for possible mutation
before being executed.</p>
</div>
<div class="section" id="driver-plugin">
<h2>Driver Plugin</h2>
<p>A driver plugin is a C++ module, either statically or dynamically
linked, which implements a specific API.</p>
<p>The API should consist of at least:</p>
<blockquote>
<ul class="simple">
<li>A function for the Kernel to obtain a list of command line options
the driver can handle and how to handle them. This is yet to be
defined, but should be nearly (may require a few changes)
compatible with the current output generated by the Clang argument
parser tablegen backend.</li>
<li>A function for the Kernel to obtain the "priority" of the
driver. Priority is a way for the developer to define which
drivers are queried first for command line argument resolution and
subprocess command mutation.</li>
<li>A function that the Kernel will call on every subprocess
invocation to allow the plugin to mutate that invocation.</li>
<li>A function to allow the plugin to emit diagnostics to the user via
the Kernel, or to abort compilation.</li>
</ul>
</blockquote>
<p>Statically linking plugins to the driver will result in the fastest
compilation speed, but because the API is so defined I suggest
offering the ability to dynamically load plugins at runtime - this may
possibly make development easier for some users (TODO: Does anyone
care about this? Would it help anyone?)</p>
</div>
<div class="section" id="driver-config-files">
<h2>Driver Config files</h2>
<p>These are designed to allow the <em>user</em> to tweak settings at invoke
time, without requiring a recompile. For speed, they allow a
restricted set of operations in comparison to driver plugins - they
are pure declarative with no imperative constructs and can modify or
add to the kernel state.</p>
<p>These files could be written in whatever lightweight markup language
we choose, which is not really important at this stage. The important
thing is that it is simple enough to parse speedily with no interpret
overhead and no extra dependencies.</p>
<p>Suggestions include JSON, YAML, XML or a INI style, similar to Daniel
Dunbar's recent build system changes.</p>
</div>
</div>
<div class="section" id="references">
<h1>References</h1>
<table class="docutils citation" frame="void" id="ml1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#id1">[ML1]</a></td><td><a class="reference external" href="http://lists.cs.uiuc.edu/pipermail/cfe-dev/2011-October/018059.html">http://lists.cs.uiuc.edu/pipermail/cfe-dev/2011-October/018059.html</a></td></tr>
</tbody>
</table>
</div>
</div>
</body>
</html>