[llvm-dev] [flang-dev] [cfe-dev] [RFC] Refactor Clang: move frontend/driver/diagnostics code to LLVM
Andrzej Warzynski via llvm-dev
llvm-dev at lists.llvm.org
Thu Aug 6 07:45:37 PDT 2020
Just a small FYI, I've restarted this discussion separately on cfe-dev
(required refactoring within Clang) and flang-dev (changes in Flang):
* cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2020-July/066393.html
* flang-dev: http://lists.llvm.org/pipermail/flang-dev/2020-July/000470.html
-Andrzej
On 12/06/2020 17:53, Andrzej Warzynski via llvm-dev wrote:
> Hi Tim,
>
> The key idea behind this refactoring is to re-use the compiler driver
> code [1] that's currently part of Clang. We initially thought that we'd
> be re-using bits of Clang's frontend-driver too, but from the feedback
> so far it looks like writing those bits from scratch (e.g.
> CompilerInstance or CompilerInvocation) would be better.
>
> The main benefit of re-using Clang's design is the flexibility that
> Flang will inherit at a relatively low cost. It will make adding new
> compiler flags, driver diagnostics or fine-tuning compilation pipelines
> really easy and powerful. It will enable compilation of mixed-source
> projects (e.g. C++ + Fortran, perhaps Rust + Fortran one day?). Also,
> Flang will gain access to Clang’s Toolchains [5], so adoption on new
> platforms should be more straightforward. And, on top of that, we could
> start bringing up infrastructure for compiler plugins so that our users
> can extend Flang as they do Clang. None of this will happen overnight,
> but given how popular, powerful and widely adopted Clang is, it feels
> like a good first step.
>
> Currently we are focusing on implementing the Flang driver in terms of
> libclangDriver (which we want to refactor and move out of Clang first).
> libclangDriver requires some infrastructure code and Diagnostics [2] are
> a good example. The interface is defined in one header file in Clang [3]
> (see DiagnosticsEngine, DiagnosticBuilder, DiagnosticConsumer). For now
> we will only use them for the Flang driver. Wider adoption in Flang
> would IMHO be great, but that's outside the scope of this RFC and is not
> required for this to work. The actual diagnostics are defined in
> TableGen files - mechanism that we also want to adopt. Diagnostics for
> the driver are defined in [4] (some adoption will be needed there too).
>
> In other words - this will only introduce a new Flang driver implemented
> in terms of `frontend-support` (the new subproject) and hopefully
> replace the current Flang driver. I can't think of any small/big changes
> to the Flang codebase that would be required for this. I might be
> missing something though! As for SourceManager, SourceLocation,
> FileManager, VFS - to be perfectly honest I think that we need to make
> more progress with our proof-of-concept implementation before we are
> ready to discuss these in more detail. We will sent separate RFCs once
> we are ready.
>
> I appreciate that there's not that many technical details here, but we
> are still working these out.
>
> -Andrzej
>
>
> [1]
> https://clang.llvm.org/docs/DriverInternals.html#driver-design-internals
> [2] https://clang.llvm.org/diagnostics.html
> [3]
> https://github.com/llvm/llvm-project/blob/release/10.x/clang/include/clang/Basic/Diagnostic.h
>
> [4]
> https://github.com/llvm/llvm-project/blob/release/10.x/clang/include/clang/Basic/DiagnosticDriverKinds.td
>
> [5]
> https://github.com/llvm/llvm-project/bloc/release/10.x/clang/lib/Driver/ToolChains
>
>
> On 12/06/2020 15:25, Timothy Keith wrote:
>> For those of us not familiar with clang internals, it would be helpful
>> if you could describe the parts of clang that you're considering
>> sharing and explain what existing code they would replace in flang (if
>> any) and what benefits we gain by sharing them.
>>
>> In particular, these were mentioned previously: DiagnosticsEngine,
>> SourceManager, SourceLocation, FileManager, VFS
>>
>> Thanks,
>> Tim
>>
>> On 6/12/20, 3:25 AM, "flang-dev on behalf of Andrzej Warzynski via
>> flang-dev" <flang-dev-bounces at lists.llvm.org on behalf of
>> flang-dev at lists.llvm.org> wrote:
>>
>> External email: Use caution opening links or attachments
>>
>>
>> FYI: this RFC will be discussed in the next Flang Technical
>> Community
>> call:
>> http://lists.llvm.org/pipermail/flang-dev/2020-June/000379.html
>>
>> -Andrzej
>>
>> On 11/06/2020 23:06, Doerfert, Johannes wrote:
>> > On 6/11/20 4:04 PM, James Y Knight wrote:
>> >> I think the expectation is that LLVM remains at the bottom of the
>> >> dependency tree, with frontend-support depending on LLVM, and
>> Clang and
>> >> Flang depending on frontend-support (and LLVM).
>> >>
>> >> Not everything which makes sense to share between clang and
>> flang makes
>> >> sense to be part of llvm core. E.g., implementation of a
>> GCC-compatible
>> >> command-line compiler-driver seems like it would be very much
>> out-of-place
>> >> in llvm core.
>> >
>> > That clears things up, thanks a lot. So we could have a
>> frontend-support
>> >
>> > subproject for non-LLVM(-IR) related things and
>> `llvm/lib/Frontend` for
>> >
>> > LLVM(-IR) related stuff, e.g., IR generation parts.
>> >
>> >
>> > Thanks,
>> >
>> > Johannes
>> >
>> >
>> >> On Thu, Jun 11, 2020 at 10:28 AM Doerfert, Johannes via cfe-dev <
>> >> cfe-dev at lists.llvm.org> wrote:
>> >>
>> >>> On 6/11/20 3:32 AM, Andrzej Warzynski wrote:
>> >>>>
>> >>>> On 11/06/2020 00:49, Michael Kruse wrote:
>> >>>>> Am Mi., 10. Juni 2020 um 10:04 Uhr schrieb Doerfert,
>> Johannes via
>> >>>>> flang-dev <flang-dev at lists.llvm.org>:
>> >>>>>> I'm not against a subproject *but* if we also move the
>> existing
>> >>>>>> llvm/lib/Frontend stuff, that would introduce a dependence
>> from
>> >>>>>> llvm-core to this project, which I think is uncommon. We
>> could also
>> >>>>>> have
>> >>>>>> both. At the end of the day it depends on the benefit we
>> would get from
>> >>>>>> an independent subproject.
>> >>>>> If we have non-conditional dependencies
>> llvm-core->frontend-support
>> >>>>> and frontend-support->llvm-core, what advantage is there
>> left of
>> >>>>> making frontend-support a subproject? Both of them are
>> required all
>> >>>>> the time.
>> >>>>>
>> >>>>> Michael
>> >>>>>
>> >>>> Thanks for the feedback! It sounds like we should keep
>> >>>> llvm/lib/Frontend in LLVM and `frontend-support` would be a new
>> >>>> subproject for things unrelated to LLVM IR.
>> >>> Sure, if we expect usage of such a subproject without
>> llvm-core. (Maybe
>> >>> there are other reasons to separate it but I'm not sure I
>> understood them.)
>> >>>
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Johannes
>> >>>
>> >>>
>> >>>> -Andrzej
>> >>>
>> >>> _______________________________________________
>> >>> cfe-dev mailing list
>> >>> cfe-dev at lists.llvm.org
>> >>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>> >>>
>> >
>> _______________________________________________
>> flang-dev mailing list
>> flang-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/flang-dev
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
More information about the llvm-dev
mailing list