[flang-dev] RFC: Is `flang-new` ready to replace `f18`?

Mon May 3 06:19:53 PDT 2021

Hi,

This is just a quick summary of the discussion that we had on this RFC 
on Slack and during our most recent community call.

* Peter Klausler and Steve Scalpone do use `flang` (the bash wrapper 
script for `f18`) to call external compilers. They suggest that we find 
a way to replicate this with `flang-new` (the new driver) before `f18` 
can be removed.
* Peter Klausler asked about the dependency on Clang and the ability to 
build out-of-tree. Just to clarify: the new driver can be built 
out-of-tree and we have a buildbot to make sure that this works fine: 
https://lab.llvm.org/buildbot/#/builders/149. The dependency on Clang 
will take a while to be removed. There's been no active development in 
this direction since [1]
* Johannes Doerfert suggested that the dependency on Clang shouldn't be 
a blocker. Flang will depend on Clang regardless of the new driver 
(through the OpenMP runtime). Also, given previous design decisions, it 
feels that compiler build times are not the highest priority for Flang.

Please correct me if I misunderstood anything or missed any comments.

Thank you for your feedback!
-Andrzej

[1] https://lists.llvm.org/pipermail/cfe-dev/2020-November/067263.html

On 27/04/2021 10:50, Andrzej Warzynski via flang-dev wrote:
> Hello,
> 
> As the new Flang driver has recently reached feature parity with the 
> "throwaway" driver, I wanted to ask whether we are ready to remove `f18`?
> 
> To better inform this discussion, below you will find an up-to-date 
> status update. I wanted to avoid sending separate e-mails - I hope that 
> this fine!
> 
> *SUPPORTED OPTIONS*
> A summary of the supported options is available in [1]. I shared that 
> spreadsheet in November last year [2]. I've been updating it while new 
> features were being added. Most notable changes:
> * `-fdebug-no-semantics` is replaced with 2 options: 
> `-fdebug-unparse-no-sema` [3] and `-fdebug-dump-parse-tree-no-sema` [4]
> * `-fconvert=swap/-byteswapio` is not supported in the "throwaway" 
> driver [5] and hence has not been added in `flang-new`
> * `-ffree-line-length=<arg>` is currently not supported by the frontend 
> and hence has not been added in `flang-new`
> * `-futf-8`/`-flatin` are replaced with `-finput-charset=<val>` (the 
> latter is the GCC/Clang spelling that we didn't know about when 
> compiling [1])
> As discussed previously [2], `flang-new` uses GCC style spellings.
> 
> *TESTING*
> Apart from adding tests specifically for the new driver, all existing 
> Flang tests have been ported to use `flang-new` when 
> `FLANG_BUILD_NEW_DRIVER` is set. In order to facilitate this, 2 new LIT 
> variables have been added. Their meaning depend on whether 
> `FLANG_BUILD_NEW_DRIVER` was set or not (unset by default):
> * `%flang` expands to either `flang-new` or `f18` and represents 
> compiler driver
> * `%flang_fc1` expands to either `flang-new -fc1` or `f18` and 
> represents frontend driver
> This is similar to `%clang` and `%clang_cc1` in Clang and allows a clear 
> separation between the drivers when testing. Please use `%flang_fc1` or 
> `%flang` in your tests from now on. `%f18` will be removed shortly [6]. 
> In terms of upstream testing, all but 1 of our Buildbot workers already 
> set `FLANG_BUILD_NEW_DRIVER` to `On` (i.e. test with the new rather than 
> the old driver).
> 
> *NEW VS OLD DRIVER - COMPARISON*
> In terms of _driving the frontend_, `flang-new -fc1` and `f18` are fully 
> compatible. Otherwise we wouldn't be able to run the frontend tests with 
> the new driver.
> 
> Comparing `flang-new` and `f18` in terms of _driving the whole compiler_ 
> (frontend/codegen/asm/linking) is tricky. The former does not support 
> code-generation yet (it's not available in llvm-project/flang yet). The 
> latter calls a separate Fortran compiler to do the code-generation, so 
> that's more like an experimental feature. This won't be supported in 
> `flang-new`, as discussed in [2].
> 
> Lastly, the new driver introduces a dependency on Clang. The steps to 
> remove this dependency were outlined in [7], but we've not been able to 
> make any progress here.
> 
> *ADDING NEW OPTIONS*
> All options for the new Flang driver are defined in Options.td in Clang 
> [8]. I will be documenting the steps required to add new options in the 
> coming weeks. In the meantime, please feel free to ask me directly 
> (Slack/e-mail) if you have any questions. You can also use one of the 
> existing options for reference.
> 
> *NEXT STEPS*
> Maintaining multiple drivers is not ideal and we would like to replace 
> `f18` with `flang-new` sooner rather than later. We are keen do it now, 
> despite the differences between `flang-new` and `f18` highlighted above. 
> The new driver covers all use cases that are relevant for us, but we 
> appreciate that this might not be the case for other community members.
> 
> Taking the above into account, do you think that `flang-new` is ready to 
> replace `f18`? If not, what's the blocking issue for you? Is it:
> * the ability to call an external compiler?
> * the dependency on Clang?
> * something else?
> Please let me know if there's anything that's unclear!
> 
> Thank you for reading and for your feedback!
> 
> -Andrzej
> 
> 
> [1] 
> https://docs.google.com/spreadsheets/d/1JRe39lP_KhtkYxFEIvwrCFlE5v1Ofa_krOHI-XXXWPY/ 
> 
> [2] https://lists.llvm.org/pipermail/flang-dev/2020-November/000588.html
> [3] 
> https://github.com/llvm/llvm-project/blob/main/flang/test/Parser/omp-allocate-unparse.f90 
> 
> [4] 
> https://github.com/llvm/llvm-project/blob/main/flang/test/Driver/dump-parse-tree-no-sema.f90 
> 
> [5] 
> https://github.com/llvm/llvm-project/blob/b6db6f5530d2df8bc3a1acf6d68b2bfa2bd053b4/flang/tools/f18/f18.cpp#L671 
> 
> [6] https://reviews.llvm.org/D101281
> [7] https://lists.llvm.org/pipermail/cfe-dev/2020-July/066393.html
> [8] 
> https://github.com/llvm/llvm-project/blob/main/clang/include/clang/Driver/Options.td 
> 
> _______________________________________________
> flang-dev mailing list
> flang-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/flang-dev