<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Hi, Peter,</p>
<p><br>
</p>
<p>I think that this is the right approach. First, Clang's driver contains a lot of painstakingly-constructed code to find system libraries and link programs, including with sanitizers and other relevant things, on various targets, and aside from the fact that
Fortran has some additional runtime libraries, the Fortran driver will need all of that logic. Second, mixed C/Fortran and C++/Fortran applications are very common, and so the driver which links a Fortran application should also know how to link in anything
also relevant for C++ compilation. Third, Clang already has support for Fortran-specific options and knows how to compile Fortran code (by passing it to GCC's driver), and extending this support for Fortran code (in a general sense) to support LLVM's own Fortran
compiler makes a lot of sense to me.</p>
<p><br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 6/24/19 4:34 AM, Peter Waller via cfe-dev wrote:<br>
</div>
<blockquote type="cite" cite="mid:VI1PR08MB39847ED128EC541D88B1048F92E00@VI1PR08MB3984.eurprd08.prod.outlook.com">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style><span>Hi All,<br>
</span>
<div><br>
</div>
<div>I'm starting work on a driver and frontend for the new flang (the<br>
</div>
<div>recently renamed f18 - not to be confused with any previous project also<br>
</div>
<div>of the same name). This has implications for the clang subproject, which<br>
</div>
<div>I discuss below.<br>
</div>
<div><br>
</div>
<div># Preliminaries<br>
</div>
<div><br>
</div>
<div>The goal is to make the in-progress flang compiler usable as soon as<br>
</div>
<div>possible.<br>
</div>
<div><br>
</div>
<div>I am motivated to keep things simple and deliver soon, hopefully in<br>
</div>
<div>digestible chunks of work.<br>
</div>
<div><br>
</div>
<div>I have split this task into two smaller pieces in a top-down approach. First, a<br>
</div>
<div>driver needs to land which can invoke the flang compiler as part of a multi-file<br>
</div>
<div>compile+link when it is passed a Fortran input. Subsequently, the flang compiler<br>
</div>
<div>needs logic analogous to clang::CompilerInvocation, which takes a Fortran file<br>
</div>
<div>as input and produces output in the appropriate form for the command line<br>
</div>
<div>options specified.<br>
</div>
<div><br>
</div>
<div>This RFC is focused on the first part. There are implications for clang,<br>
</div>
<div>since there exists a significant body of driver code there, much of the<br>
</div>
<div>logic it would be beneficial to share and reuse. The first change will<br>
</div>
<div>be to clang. The flang changes will come in future RFCs, and I envisage<br>
</div>
<div>will be more an internal detail of the flang subproject. There will be<br>
</div>
<div>some leakage as we seek to share as much logic as makes sense to do so.<br>
</div>
<div><br>
</div>
<div># Approach<br>
</div>
<div><br>
</div>
<div>I assuming that flang intends to implement a gcc-like interface.<br>
</div>
<div><br>
</div>
<div>From <a class="moz-txt-link-freetext" href="https://clang.llvm.org/docs/DriverInternals.html">
https://clang.llvm.org/docs/DriverInternals.html</a>:<br>
</div>
<div><br>
</div>
<div>> [...] most of the driver functionality is kept in a library which can<br>
</div>
<div>> be used to build other tools which want to implement or accept a gcc<br>
</div>
<div>> like interface.<br>
</div>
<div><br>
</div>
<div>libclangDriver has the goal of being a reusable and extensible driver.<br>
</div>
<div>If it gained knowledge of the flang toolchain, implementing a flang<br>
</div>
<div>driver as powerful as the clang driver in terms of libclangDriver would<br>
</div>
<div>be a relatively small task.<br>
</div>
<div><br>
</div>
<div>I propose to put together a new "bin/flang" binary which would be<br>
</div>
<div>structured and behave in a similar manner as clang. The intent is to<br>
</div>
<div>mirror clang, for both the driver and CompilerInvocation, as much as<br>
</div>
<div>makes sense to do so. The aim is to avoid re-inventing the wheel and to<br>
</div>
<div>enable people who have worked with either the clang or flang<br>
</div>
<div>entrypoints, drivers, and frontends to easily understand the other.<br>
</div>
<div><br>
</div>
<div># The user-facing result will be:<br>
</div>
<div><br>
</div>
<div>When I do `bin/flang -o foobar foobar.f90`, the driver will invoke<br>
</div>
<div>`bin/flang -fc1 foobar.f90 -o /tmp/foobar_cafe1234.o` and subsequently<br>
</div>
<div>link it.<br>
</div>
<div><br>
</div>
<div># Preliminary prototype<br>
</div>
<div><span><br>
</span></div>
<div><span><a class="moz-txt-link-freetext" href="https://reviews.llvm.org/D63607?id=206172">https://reviews.llvm.org/D63607?id=206172</a> introduces the above</span></div>
<div><span></span></div>
<div>behaviour, to illustrate the approach. At this point I seek advice<br>
</div>
<div>primarily on the overall approach. If acceptable, I will tidy it up and<br>
</div>
<span>submit it for review. There will be more details to get right.</span>
<div><br>
</div>
<div>The patch is it stands has a small amount of logic which makes<br>
</div>
<div>`bin/flang` a symlink to `clang`, which causes the driver to go into<br>
</div>
<div>Fortran mode and invoke `bin/flang -fc1`; at the moment this is just a<br>
</div>
<div>stub which prints the arguments. In the real implementation, `bin/flang`<br>
</div>
<div>will be a standalone binary in the flang project. This hack enables<br>
</div>
<div>playing with my proposed driver approach now. At no point will flang<br>
</div>
<div>be required to build clang.<br>
</div>
<div><br>
</div>
<div># More details<br>
</div>
<div><br>
</div>
<div>`bin/flang` will naturally support mixed C/C++/Fortran inputs, invoke<br>
</div>
<div>the relevant compilers for each input and link everything together.<br>
</div>
<div><br>
</div>
<div>libclangDriver will gain a new "Fortran" mode (--driver-mode=fortran).<br>
</div>
<div>When in this mode, when given a Fortran input, the driver will invoke<br>
</div>
<div>the flang compiler, which would live at "flang -fc1" by analogy with<br>
</div>
<div>"clang -cc1". "clang", when not in `--driver-mode=fortran`, would retain<br>
</div>
<div>its existing default behaviour of falling back to gcc (and thus<br>
</div>
<div>gfortran), for now. This means that existing build processes relying on<br>
</div>
<div>the existing behaviour will not be touched, and that the new behaviour<br>
</div>
<div>is opt-in by either using `bin/flang` or `--driver-mode=fortran`.<br>
</div>
<div><br>
</div>
<div>`bin/flang` will use clang::Driver, in Fortran mode. This seems a source<br>
</div>
<div>of potential problems since it requires a clang::DiagnosticsEngine to<br>
</div>
<div>construct, and these things seem clang-specific. However, this<br>
</div>
<div>diagnostics engine is primarily used by the driver to print warnings<br>
</div>
<div>about user errors made in the options supplied, so maybe this is not a<br>
</div>
<div>big problem. I have not investigated the full implications of this yet.<br>
</div>
</blockquote>
<p><br>
</p>
<p>Given that Fortran code is often processed by the C preprocessor, and Clang's diagnostic infrastructure knows how to generate diagnostics including macro tracebacks, this is an area where potential reuse might make sense.<br>
</p>
<p><br>
</p>
<blockquote type="cite" cite="mid:VI1PR08MB39847ED128EC541D88B1048F92E00@VI1PR08MB3984.eurprd08.prod.outlook.com">
<div></div>
<div><br>
</div>
<div>The clang driver already has an option group for gfortran options.<br>
</div>
<div>libclangDriver will need to know about all of the options that the flang<br>
</div>
<div>frontend supports. The complete set is under development and will grow<br>
</div>
<div>over time. All options in this group would be passed through to the<br>
</div>
<div>flang compiler. It's unclear to me at the moment whether the<br>
</div>
<div>gfortran-specific options will remain in their own group, or whether<br>
</div>
<div>they can be merged into a 'fortran group'. This can be decided once the<br>
</div>
<div>full flang compiler options story is better understood.<br>
</div>
</blockquote>
<p><br>
</p>
<p>This will depend on the Flang options, but I assume that we'll want to accept a lot of gfortran options. In any case, I recommend treating this like we treat passing options to clang -cc1 (even if that means writing more explicit options to pass gfortran
options through to gcc in cases where the same options require more-complex logic for Flang).<br>
</p>
<p><br>
</p>
<blockquote type="cite" cite="mid:VI1PR08MB39847ED128EC541D88B1048F92E00@VI1PR08MB3984.eurprd08.prod.outlook.com">
<div></div>
<div><br>
</div>
<div>Where it makes sense to do so, the flang compiler will accept many of<br>
</div>
<div>the same options that clang does, particularly those relating to the<br>
</div>
<div>output form (e.g. -emit-llvm, -S, optimization options, etc).<br>
</div>
<div><br>
</div>
<div>I would act under the assumption that where possible, the<br>
</div>
<div>non-passthrough logic in the driver should be minimized, and the heavy<br>
</div>
<div>lifting should occur inside the flang compiler. The compiler will reuse LLVM<br>
</div>
<div>option parsing logic where it makes sense to do so.<br>
</div>
</blockquote>
<p><br>
</p>
<p>We should decide this in the same way as this is decided for Clang logic. We put options processing where it will yield the most robust, and highest-quality, user experience.<br>
</p>
<p><br>
</p>
<blockquote type="cite" cite="mid:VI1PR08MB39847ED128EC541D88B1048F92E00@VI1PR08MB3984.eurprd08.prod.outlook.com">
<div></div>
<div><br>
</div>
<div>cmake and project structure: flang at the moment is an independent<br>
</div>
<div>project. Taking the above proposed approach will "grow a dependency<br>
</div>
<div>edge" from flang onto the clang subproject. This may be undesirable in<br>
</div>
<div>the long run. It implies that building flang will require building (at<br>
</div>
<div>least part of) clang. </div>
</blockquote>
<p><br>
</p>
<p>I don't see this as a problem. We also need to build LLVM itself before building Clang, LLD, etc. At some point, we might refactor things, but the coupling is not actually unnatural.
<br>
</p>
<p><br>
</p>
<p>Thanks again,</p>
<p>Hal<br>
</p>
<p><br>
</p>
<blockquote type="cite" cite="mid:VI1PR08MB39847ED128EC541D88B1048F92E00@VI1PR08MB3984.eurprd08.prod.outlook.com">
<div>I think this should be addressed at some point so<br>
</div>
<div>that flang can be built independently, but perhaps not initially, as<br>
</div>
<div>moving the clang driver into another top-level subproject would be<br>
</div>
<div>another significant task. The approach and value for that task may be<br>
</div>
<div>clearer once a second dependency on libclangDriver exists under the<br>
</div>
<div>llvm-project tree.<br>
</div>
<div><br>
</div>
<div>Help text contamination: I'm operating under the belief that clang<br>
</div>
<div>should not mention the Fortran language options and likewise flang<br>
</div>
<div>should not mention the C-like-language options.<br>
</div>
<div><br>
</div>
<div>When in Fortran mode, the driver will need to pass additional options to<br>
</div>
<div>the linker, in particular to link the Fortran runtime and intrinsics.<br>
</div>
<div>I have not yet considered this in detail.<br>
</div>
<div><br>
</div>
<div># Implications<br>
</div>
<div><br>
</div>
<div>Building flang will require building libclangDriver, at least.<br>
</div>
<div><br>
</div>
<div>This approach gives a future path to having clang invoke flang for<br>
</div>
<div>Fortran inputs instead of falling back to gcc as it currently does,<br>
</div>
<div>giving better gcc drop-in compatibility without depending on having gcc<br>
</div>
<div>available.<br>
</div>
<div><br>
</div>
<div>Having this arrangement does mean that some changes - for example,<br>
</div>
<div>adding a new compiler option - will require patches which touch both the<br>
</div>
<div>clang and flang subprojects. In particular, Fortran language specific<br>
</div>
<div>options will continue to reside in clang/include/clang/Driver/Options.td.<br>
</div>
<div><br>
</div>
<div># Prior art<br>
</div>
<div><br>
</div>
<div>There is a lot of prior art in this domain. While considering the<br>
</div>
<div>problem, I have looked at the following projects:<br>
</div>
<div><br>
</div>
<div>* <a class="moz-txt-link-freetext" href="https://github.com/flang-compiler/flang-driver">
https://github.com/flang-compiler/flang-driver</a><br>
</div>
<div><br>
</div>
<div> This is a fork of clang, with the main goal of implementing the<br>
</div>
<div> toolchains/flang tool. The implementation that I propose in this RFC<br>
</div>
<div> is similar to that, but the tool would be simpler since the frontend<br>
</div>
<div> is starting from fresh.<br>
</div>
<div><br>
</div>
<div>* <a class="moz-txt-link-freetext" href="https://github.com/apple/swift">https://github.com/apple/swift</a><br>
</div>
<div><br>
</div>
<div> Swift's driver looks similar to that of clang, but is a separate<br>
</div>
<div> implementation.<br>
</div>
<div><br>
</div>
<div>* llgo and other languages<br>
</div>
<div><br>
</div>
<div> These are written in other languages, and do not appear to reuse the<br>
</div>
<div> driver logic.<br>
</div>
<div><br>
</div>
<div>I observe that most prior implementations do not try to share logic with<br>
</div>
<div>clang. I surmise that the reasons for this are partly due to those<br>
</div>
<div>projects being external, and also due to those projects having quite<br>
</div>
<div>different user interfaces. Fortran on the other hand shares many<br>
</div>
<div>similarities with the C-like user interfaces, and there is a large body<br>
</div>
<div>of existing code relying on gcc-like behaviour implemented by the clang<br>
</div>
<div>driver.<br>
</div>
<div><br>
</div>
<div># Alternative approaches<br>
</div>
<div><br>
</div>
<div>The other approach I have considered is to do what Swift have done and<br>
</div>
<div>reimplement significant amounts of the driver, copying the structure<br>
</div>
<div>from clang. I don't currently see a good reason to do this at this<br>
</div>
<div>stage, since flang is joining the LLVM project and therefore should be<br>
</div>
<div>able to share the clang implementation directly.<br>
</div>
<div><br>
</div>
<div>There may be other good reasons for doing this I've missed, in which<br>
</div>
<div>case I would be interested in hearing about it.<br>
</div>
<div><br>
</div>
<div># Future work and next steps<br>
</div>
<div><br>
</div>
<div>If there are no show-stoppers in my above proposal, I will submit the<br>
</div>
<div>attached patch for review, and commence further work on proposing,<br>
</div>
<div>prototyping and implementing `bin/flang` and `bin/flang -fc1` in the<br>
</div>
<div>flang project.<br>
</div>
<div><br>
</div>
<div>All input appreciated.<br>
</div>
<div><br>
</div>
<div>Thanks for reading,<br>
</div>
<div><br>
</div>
<div>Peter Waller<br>
</div>
<div><a class="moz-txt-link-abbreviated" href="mailto:peter.waller@arm.com">peter.waller@arm.com</a><br>
</div>
<div>Arm Limited<br>
</div>
<span></span><br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
cfe-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:cfe-dev@lists.llvm.org">cfe-dev@lists.llvm.org</a>
<a class="moz-txt-link-freetext" href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory</pre>
</body>
</html>