[flang-dev] Flang driver - next steps

Mon Oct 4 08:25:03 PDT 2021

Hi Pete,

Thank you for your feedback!

On 03/10/2021 13:53, Peter Steinfeld wrote:
> 
> Eventually, we'll need to be able to specify whether to generate debug information, and we'll need an option for that.
> 

Perhaps we could introduce `-g` (and other debug options) once we start 
discussing optimisation pipelines and `-O{0|1|2|3|s|z}` flags? That 
would come once basic code-generation is available.

> Also, I don't understand why we have two drivers, one for the compiler and another for the front end which is invoked by the "-fc1" option.  Are we planning to simplify this in the future
Tl;Dr The plan is to keep the two drivers separate.

In Clang's "toolchain driver" model, frontend driver is just yet another
specialised tool next to a linker, assembler or a sanitizer. Hence the
separation. We probably could try to implement it as one monolith (i.e.
together with the compiler driver), but that would be against the grain 
of `clangDriver`.

W decided to re-use Clang's painstakingly-constructed code for a 
compiler driver so that in LLVM Flang we don't have to re-write the 
logic to e.g. find system libraries or to link programs on various 
targets/platforms. This will also simplify the path towards a compiler 
that can work with mixed C/Fortran and C++/Fortran applications. 
Currently we are not benefiting too much from re-using `clangDriver` - 
first we need to teach the new driver how to generate code that could be 
linked. Once we are there, we should be able to see the benefits of this 
approach.

I think that we could achieve a bit cleaner design (both in LLVM Flang 
and Clang) by extracting `clangDriver` out of Clang and by making 
various drivers share less. Currently it can be tricky to identify bits 
that belong strictly to the compiler driver and bits that belong to the 
frontend driver. One could start by creating a dedicated Options.td file 
for `clang`, `clang -cc1`, `flang-new` and `flang-new -fc1`. That's just 
an idea. I'm not aware of anyone working on this.

Thank you,
-Andrzej

> 
> -----Original Message-----
> From: flang-dev <flang-dev-bounces at lists.llvm.org> On Behalf Of Andrzej Warzynski via flang-dev
> Sent: Wednesday, September 29, 2021 8:06 AM
> To: via flang-dev <flang-dev at lists.llvm.org>
> Subject: [flang-dev] Flang driver - next steps
> 
> External email: Use caution opening links or attachments
> 
> 
> Hello,
> 
> [Flang == "LLVM Flang"]
> 
> Until recently we were focusing on replacing `f18` (the old driver) with `flang-new`, the new driver. This was completed in [1]. The natural next step is to add support for code-generation and linking phases. Although we have been discussing most of this in our community calls, not everyone has been able to attend them. Here's a quick overview and some open questions.
> 
> # *Code generation*
> This will almost entirely be implemented inside the frontend driver (i.e. llvm-project/flang). It will support the following stages:
>     1. ParseTree --> MLIR
>     2. MLIR --> LLVM IR
>     3. LLVM IR --> assembly/object code
> This might be better expressed in terms of compiler options. Basically, we plan to add the following options to the frontend driver:
>     * -emit-mlir (ParseTree --> MLIR)
>     * -emit-llvm (ParseTree --> MLIR --> LLVM IR)
>     * -emit-obj (ParseTree --> MLIR LLVM IR --> object code)
>     * -S (ParseTree --> MLIR --> LLVM IR --> assembly)
>     * -save-temps/-save-temps=<value>
>     * --target=<value>
>     * -print-target-triple
>     * -print-targets
> Do you see any key code-generation related options missing here? We would like to keep this list short and focus on the bare minimum that will best inform the design. Note that adding these options will lead to some duplication in implementation _and_ functionality between `flang-new -fc1` and `bbc`/`tco`. We will need to make sure that these tools stay in sync and that the level of testing for all is similar.
> 
> This work has already started and a patch implementing `-emit-mlir` is currently under review [2]. Note that we are only able to work on the above in the `fir-dev` branch of f18-llvm-project [3].
> 
> # *Linking*
> This phase will be managed by `clangDriver`, so it will likely require some changes there. Compared to Clang, we do have one additional intermediate representation to support (MLIR). We may want to/need to make `clangDriver` aware of it. Otherwise, do you see anything that we might be missing here? Or any specific options that may require extra work within LLVM Flang?
> 
> We have not started working on this yet. Instead, we have been just using `clang` manually to drive the linking.
> 
> # *Implementation details*
> Quite a few of us have been using glue scripts to achieve end-to-end workflow (i.e. Fortran source --> executable). We want to focus on supporting this workflow in `flang-new` as soon as we can. We will try to achieve this without major design changes and that's why we want to focus on the bare minimum of options required here.
> 
> However, we probably want to introduce `ParseTreeConsumer` and `FIRConsumer` (or something similar) at some point. These abstraction layers would be similar to `ASTConsumer` in Clang and would allow a cleaner separation between the driver and various consumers of the intermediate representations in LLVM Flang. This could be a good time to introduce them, but it's an "implementation detail" that we can refine/revisit later.
> 
> # *Bash script*
> Once `flang-new` is capable of generating executables, shall we rename it as `flang` and then rename the `flang` [4] bash script as e.g.
> `flang-to-gfortran` (or something similar)?
> 
> Thank you for reading,
> Andrzej
> 
> 
> [1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freviews.llvm.org%2FD105811&data=04%7C01%7Cpsteinfeld%40nvidia.com%7C4dc3f386ec024dd6e73508d9835aa3aa%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637685247587498149%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=loFLSH6Tk0gX2ebuIY0Y4Cmu1O5vdl02mEaDZPviVzs%3D&reserved=0
> [2] https://github.com/flang-compiler/f18-llvm-project/pull/1008
> [3] https://github.com/flang-compiler/f18-llvm-project
> [4] https://github.com/llvm/llvm-project/blob/main/flang/tools/f18/flang
> _______________________________________________
> flang-dev mailing list
> flang-dev at lists.llvm.org
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.llvm.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fflang-dev&data=04%7C01%7Cpsteinfeld%40nvidia.com%7C4dc3f386ec024dd6e73508d9835aa3aa%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637685247587498149%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=fPk%2B968OUbk3bNLJ0gVRg4UFRziRYHlBLziBsALSzO0%3D&reserved=0
> 