[cfe-dev] RFC: Adding a fortran mode to the clang driver for flang

Wed Jul 17 09:27:45 PDT 2019

Hi, Peter,

I think that this is the right approach. First, Clang's driver contains a lot of painstakingly-constructed code to find system libraries and link programs, including with sanitizers and other relevant things, on various targets, and aside from the fact that Fortran has some additional runtime libraries, the Fortran driver will need all of that logic. Second, mixed C/Fortran and C++/Fortran applications are very common, and so the driver which links a Fortran application should also know how to link in anything also relevant for C++ compilation. Third, Clang already has support for Fortran-specific options and knows how to compile Fortran code (by passing it to GCC's driver), and extending this support for Fortran code (in a general sense) to support LLVM's own Fortran compiler makes a lot of sense to me.

On 6/24/19 4:34 AM, Peter Waller via cfe-dev wrote:
Hi All,

I'm starting work on a driver and frontend for the new flang (the
recently renamed f18 - not to be confused with any previous project also
of the same name). This has implications for the clang subproject, which
I discuss below.

# Preliminaries

The goal is to make the in-progress flang compiler usable as soon as
possible.

I am motivated to keep things simple and deliver soon, hopefully in
digestible chunks of work.

I have split this task into two smaller pieces in a top-down approach. First, a
driver needs to land which can invoke the flang compiler as part of a multi-file
compile+link when it is passed a Fortran input. Subsequently, the flang compiler
needs logic analogous to clang::CompilerInvocation, which takes a Fortran file
as input and produces output in the appropriate form for the command line
options specified.

This RFC is focused on the first part. There are implications for clang,
since there exists a significant body of driver code there, much of the
logic it would be beneficial to share and reuse. The first change will
be to clang. The flang changes will come in future RFCs, and I envisage
will be more an internal detail of the flang subproject. There will be
some leakage as we seek to share as much logic as makes sense to do so.

# Approach

I assuming that flang intends to implement a gcc-like interface.

>From https://clang.llvm.org/docs/DriverInternals.html:

> [...] most of the driver functionality is kept in a library which can
> be used to build other tools which want to implement or accept a gcc
> like interface.

libclangDriver has the goal of being a reusable and extensible driver.
If it gained knowledge of the flang toolchain, implementing a flang
driver as powerful as the clang driver in terms of libclangDriver would
be a relatively small task.

I propose to put together a new "bin/flang" binary which would be
structured and behave in a similar manner as clang. The intent is to
mirror clang, for both the driver and CompilerInvocation, as much as
makes sense to do so. The aim is to avoid re-inventing the wheel and to
enable people who have worked with either the clang or flang
entrypoints, drivers, and frontends to easily understand the other.

# The user-facing result will be:

When I do `bin/flang -o foobar foobar.f90`, the driver will invoke
`bin/flang -fc1 foobar.f90 -o /tmp/foobar_cafe1234.o` and subsequently
link it.

# Preliminary prototype

https://reviews.llvm.org/D63607?id=206172 introduces the above
behaviour, to illustrate the approach. At this point I seek advice
primarily on the overall approach. If acceptable, I will tidy it up and
submit it for review. There will be more details to get right.

The patch is it stands has a small amount of logic which makes
`bin/flang` a symlink to `clang`, which causes the driver to go into
Fortran mode and invoke `bin/flang -fc1`; at the moment this is just a
stub which prints the arguments. In the real implementation, `bin/flang`
will be a standalone binary in the flang project. This hack enables
playing with my proposed driver approach now. At no point will flang
be required to build clang.

# More details

`bin/flang` will naturally support mixed C/C++/Fortran inputs, invoke
the relevant compilers for each input and link everything together.

libclangDriver will gain a new "Fortran" mode (--driver-mode=fortran).
When in this mode, when given a Fortran input, the driver will invoke
the flang compiler, which would live at "flang -fc1" by analogy with
"clang -cc1". "clang", when not in `--driver-mode=fortran`, would retain
its existing default behaviour of falling back to gcc (and thus
gfortran), for now. This means that existing build processes relying on
the existing behaviour will not be touched, and that the new behaviour
is opt-in by either using `bin/flang` or `--driver-mode=fortran`.

`bin/flang` will use clang::Driver, in Fortran mode. This seems a source
of potential problems since it requires a clang::DiagnosticsEngine to
construct, and these things seem clang-specific. However, this
diagnostics engine is primarily used by the driver to print warnings
about user errors made in the options supplied, so maybe this is not a
big problem. I have not investigated the full implications of this yet.

Given that Fortran code is often processed by the C preprocessor, and Clang's diagnostic infrastructure knows how to generate diagnostics including macro tracebacks, this is an area where potential reuse might make sense.

The clang driver already has an option group for gfortran options.
libclangDriver will need to know about all of the options that the flang
frontend supports. The complete set is under development and will grow
over time. All options in this group would be passed through to the
flang compiler. It's unclear to me at the moment whether the
gfortran-specific options will remain in their own group, or whether
they can be merged into a 'fortran group'. This can be decided once the
full flang compiler options story is better understood.

This will depend on the Flang options, but I assume that we'll want to accept a lot of gfortran options. In any case, I recommend treating this like we treat passing options to clang -cc1 (even if that means writing more explicit options to pass gfortran options through to gcc in cases where the same options require more-complex logic for Flang).

Where it makes sense to do so, the flang compiler will accept many of
the same options that clang does, particularly those relating to the
output form (e.g. -emit-llvm, -S, optimization options, etc).

I would act under the assumption that where possible, the
non-passthrough logic in the driver should be minimized, and the heavy
lifting should occur inside the flang compiler. The compiler will reuse LLVM
option parsing logic where it makes sense to do so.

We should decide this in the same way as this is decided for Clang logic. We put options processing where it will yield the most robust, and highest-quality, user experience.

cmake and project structure: flang at the moment is an independent
project. Taking the above proposed approach will "grow a dependency
edge" from flang onto the clang subproject. This may be undesirable in
the long run. It implies that building flang will require building (at
least part of) clang.

I don't see this as a problem. We also need to build LLVM itself before building Clang, LLD, etc. At some point, we might refactor things, but the coupling is not actually unnatural.

Thanks again,

Hal

I think this should be addressed at some point so
that flang can be built independently, but perhaps not initially, as
moving the clang driver into another top-level subproject would be
another significant task. The approach and value for that task may be
clearer once a second dependency on libclangDriver exists under the
llvm-project tree.

Help text contamination: I'm operating under the belief that clang
should not mention the Fortran language options and likewise flang
should not mention the C-like-language options.

When in Fortran mode, the driver will need to pass additional options to
the linker, in particular to link the Fortran runtime and intrinsics.
I have not yet considered this in detail.

# Implications

Building flang will require building libclangDriver, at least.

This approach gives a future path to having clang invoke flang for
Fortran inputs instead of falling back to gcc as it currently does,
giving better gcc drop-in compatibility without depending on having gcc
available.

Having this arrangement does mean that some changes - for example,
adding a new compiler option - will require patches which touch both the
clang and flang subprojects. In particular, Fortran language specific
options will continue to reside in clang/include/clang/Driver/Options.td.

# Prior art

There is a lot of prior art in this domain. While considering the
problem, I have looked at the following projects:

* https://github.com/flang-compiler/flang-driver

  This is a fork of clang, with the main goal of implementing the
  toolchains/flang tool. The implementation that I propose in this RFC
  is similar to that, but the tool would be simpler since the frontend
  is starting from fresh.

* https://github.com/apple/swift

  Swift's driver looks similar to that of clang, but is a separate
  implementation.

* llgo and other languages

  These are written in other languages, and do not appear to reuse the
  driver logic.

I observe that most prior implementations do not try to share logic with
clang. I surmise that the reasons for this are partly due to those
projects being external, and also due to those projects having quite
different user interfaces. Fortran on the other hand shares many
similarities with the C-like user interfaces, and there is a large body
of existing code relying on gcc-like behaviour implemented by the clang
driver.

# Alternative approaches

The other approach I have considered is to do what Swift have done and
reimplement significant amounts of the driver, copying the structure
from clang. I don't currently see a good reason to do this at this
stage, since flang is joining the LLVM project and therefore should be
able to share the clang implementation directly.

There may be other good reasons for doing this I've missed, in which
case I would be interested in hearing about it.

# Future work and next steps

If there are no show-stoppers in my above proposal, I will submit the
attached patch for review, and commence further work on proposing,
prototyping and implementing `bin/flang` and `bin/flang -fc1` in the
flang project.

All input appreciated.

Thanks for reading,

Peter Waller
peter.waller at arm.com<mailto:peter.waller at arm.com>
Arm Limited

_______________________________________________
cfe-dev mailing list
cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190717/59a1f137/attachment.html>