[flang-dev] RFC: Is `flang-new` ready to replace `f18`?

Tue Apr 27 02:50:51 PDT 2021

Hello,

As the new Flang driver has recently reached feature parity with the 
"throwaway" driver, I wanted to ask whether we are ready to remove `f18`?

To better inform this discussion, below you will find an up-to-date 
status update. I wanted to avoid sending separate e-mails - I hope that 
this fine!

*SUPPORTED OPTIONS*
A summary of the supported options is available in [1]. I shared that 
spreadsheet in November last year [2]. I've been updating it while new 
features were being added. Most notable changes:
* `-fdebug-no-semantics` is replaced with 2 options: 
`-fdebug-unparse-no-sema` [3] and `-fdebug-dump-parse-tree-no-sema` [4]
* `-fconvert=swap/-byteswapio` is not supported in the "throwaway" 
driver [5] and hence has not been added in `flang-new`
* `-ffree-line-length=<arg>` is currently not supported by the frontend 
and hence has not been added in `flang-new`
* `-futf-8`/`-flatin` are replaced with `-finput-charset=<val>` (the 
latter is the GCC/Clang spelling that we didn't know about when 
compiling [1])
As discussed previously [2], `flang-new` uses GCC style spellings.

*TESTING*
Apart from adding tests specifically for the new driver, all existing 
Flang tests have been ported to use `flang-new` when 
`FLANG_BUILD_NEW_DRIVER` is set. In order to facilitate this, 2 new LIT 
variables have been added. Their meaning depend on whether 
`FLANG_BUILD_NEW_DRIVER` was set or not (unset by default):
* `%flang` expands to either `flang-new` or `f18` and represents 
compiler driver
* `%flang_fc1` expands to either `flang-new -fc1` or `f18` and 
represents frontend driver
This is similar to `%clang` and `%clang_cc1` in Clang and allows a clear 
separation between the drivers when testing. Please use `%flang_fc1` or 
`%flang` in your tests from now on. `%f18` will be removed shortly [6]. 
In terms of upstream testing, all but 1 of our Buildbot workers already 
set `FLANG_BUILD_NEW_DRIVER` to `On` (i.e. test with the new rather than 
the old driver).

*NEW VS OLD DRIVER - COMPARISON*
In terms of _driving the frontend_, `flang-new -fc1` and `f18` are fully 
compatible. Otherwise we wouldn't be able to run the frontend tests with 
the new driver.

Comparing `flang-new` and `f18` in terms of _driving the whole compiler_ 
(frontend/codegen/asm/linking) is tricky. The former does not support 
code-generation yet (it's not available in llvm-project/flang yet). The 
latter calls a separate Fortran compiler to do the code-generation, so 
that's more like an experimental feature. This won't be supported in 
`flang-new`, as discussed in [2].

Lastly, the new driver introduces a dependency on Clang. The steps to 
remove this dependency were outlined in [7], but we've not been able to 
make any progress here.

*ADDING NEW OPTIONS*
All options for the new Flang driver are defined in Options.td in Clang 
[8]. I will be documenting the steps required to add new options in the 
coming weeks. In the meantime, please feel free to ask me directly 
(Slack/e-mail) if you have any questions. You can also use one of the 
existing options for reference.

*NEXT STEPS*
Maintaining multiple drivers is not ideal and we would like to replace 
`f18` with `flang-new` sooner rather than later. We are keen do it now, 
despite the differences between `flang-new` and `f18` highlighted above. 
The new driver covers all use cases that are relevant for us, but we 
appreciate that this might not be the case for other community members.

Taking the above into account, do you think that `flang-new` is ready to 
replace `f18`? If not, what's the blocking issue for you? Is it:
* the ability to call an external compiler?
* the dependency on Clang?
* something else?
Please let me know if there's anything that's unclear!

Thank you for reading and for your feedback!

-Andrzej

[1] 
https://docs.google.com/spreadsheets/d/1JRe39lP_KhtkYxFEIvwrCFlE5v1Ofa_krOHI-XXXWPY/
[2] https://lists.llvm.org/pipermail/flang-dev/2020-November/000588.html
[3] 
https://github.com/llvm/llvm-project/blob/main/flang/test/Parser/omp-allocate-unparse.f90
[4] 
https://github.com/llvm/llvm-project/blob/main/flang/test/Driver/dump-parse-tree-no-sema.f90
[5] 
https://github.com/llvm/llvm-project/blob/b6db6f5530d2df8bc3a1acf6d68b2bfa2bd053b4/flang/tools/f18/f18.cpp#L671
[6] https://reviews.llvm.org/D101281
[7] https://lists.llvm.org/pipermail/cfe-dev/2020-July/066393.html
[8] 
https://github.com/llvm/llvm-project/blob/main/clang/include/clang/Driver/Options.td