[llvm-dev] Plan for landing flang in monorepo

[Richard Barton via llvm-dev]

Hi llvm-dev

It's been a few weeks since I last gave an update on F18 and our progress on readying it for inclusion into the monorepo. Last time we discussed this the community challenged us to make the F18 source code look more like an LLVM project and to come up with a plan and schedule for completing this work (http://lists.llvm.org/pipermail/llvm-dev/2020-January/137989.html)

The full list of changes that could be made to make F18 more LLVM-like is very long. We're interested in identifying what the absolute dealbreakers are that block inclusion into the monorepo and which changes would be acceptable to make after inclusion to the monorepo. We've come up with strawman lists for each and would like to propose the following plan of action:
1.      We have captured our strawman proposal for all the changes that need to happen to F18 to make it ready for inclusion into the monorepo on a github project board: https://github.com/orgs/flang-compiler/projects/8 (also repeated at the end of this mail.)
2.      We are working through this list and we believe that we can complete this work in time for a new upstreaming date of 16th March.
3.      We have captured further work that we plan to complete on F18 after merging to the monorepo https://github.com/orgs/flang-compiler/projects/10 (also repeated below)
4.      We believe that we can complete this work before the LLVM11 branching date in June.
5.      After this date, we'll keep improving the code as we go along and not on any specific timescale.

We'd really appreciate feedback on the two lists of changes, specifically: are these lists complete? Is everyone satisfied that with all the items on https://github.com/orgs/flang-compiler/projects/8, we'd be happy to accept F18 into the monorepo? Are there any further changes that would need to be made to F18 for this to happen?

Thanks
Rich

More info on the lists:

Pre-merge list: https://github.com/orgs/flang-compiler/projects/8

The status today is that many of the items on the pre-merge list are well underway or complete.

1.      Integrate into the monorepo CMake
a.      This will be as an optional project, and default to not building.
b.      This also adds Doxygen infrastructure so we can start to improve interface documentation and continue post-merge.
2.      F18 changes to make it more LLVM-like in code style
a.      Rationalise headers to put public headers in /include and not /lib
b.      Examine F18's clang-format file and minimise deviations to the LLVM clang-format
c.      Rename all .cc files to .cpp
d.      Capitalize the module directory names in /lib and /include (e.g. /lib/Parser)
3.      Increase use of LLVM APIs and utilities in F18
a.      Switch F18 custom File handling to LLVM's File handling (helps with non-POSIX platform support)
b.      Change uses of C++ standard stream IO library to LLVM's equivalent library
c.      Audit use of std::list and consider migrating to a suitable alternative in LLVM's API
d.      Use llvm_unreachable with an error message for unreachable cases
4.      Convert the regression test suite to using lit rather than ctest
a.      Porting off the custom scripts to FileCheck will continue after this but we think it should not gate inclusion to the monorepo.
5.      Ensure that F18 builds with the same compilers as the rest of the monorepo
a.      One caveat is that we can only support C++17 compilers
b.      We propose to defer Windows support until after we merge
c.      We will specifically also check with the latest LLVM 10 rc

Post-merge list: https://github.com/orgs/flang-compiler/projects/10

This is the work that will happen right away after merging to the monorepo

1.      F18 changes to make it more LLVM-like in code style - We will perform a one-off exercise where we audit the code to find these instances and bring them in line. We'll look at:
a.      Braces on all single-line if statements
b.      Uses of else-after-return.
2.      Increase use of LLVM APIs and utilities in F18 - We'll audit the uses of these datatypes and try to move to a suitable LLVM alternative
a.      std::string/std::string_view
b.      std::vector
c.      std::set
d.      std::map
3.      Further work on porting the test suite to make it more LLVM-like
a.      Port lit tests to FileCheck
b.      Port unit tests to gtest
c.      Implement equivalent to clang -verify and port tests to that
4.      Support Windows
a.      Porting to LLVM file I/O is the main blocker - included in the post-merge worklist - but there will be more to do after this.
b.      Isuru Fernando is going to lead this effort
5.      Set up official builders
a.      Arm will handle bots for AArch64
b.      Nvidia will handle X86
c.      Tarique Islam at IBM will set up a builder for Power: http://lists.llvm.org/pipermail/flang-dev/2020-February/000232.html
d.      Any further help from community bot maintainers to cover all the platforms and compilers would be greatly appreciated.

One specific ask in the last round of feedback was on sharing lib/common/Although we see the benefit of doing this exercise, we feel it is a bit too early to start. One design principle we wish to stick to is for the Fortran runtime and compiler align on their implementations. For the specific example of https://github.com/flang-compiler/f18/blob/master/include/flang/common/bit-population-count.h we'd want the compiler and runtime POPCNT intrinsic to align on implementation. The F18 runtime is still a work in progress. We need to decide on how or if this could share code with LLVM libraries and then we can revisit the implementations in include/flang/common.

Future work
After all the above is done, we will continue to bring the code more in line with LLVM style and API usage by fixing things as we find them during development and enforce the new style through code review. A few specific areas that have been mentioned before that we will tackle in this way are:
1.      Add Doxygen style comments to interfaces
2.      Classes, files, names, etc. where a more LLVM-standard naming can be used.
3.      Refactor code to use early exits when suitable
4.      Audit functions in include/flang/common and port to LLVM equivalents (e.g. builtin_popcount)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200220/9b9373f7/attachment.html>