[cfe-dev] [llvm-dev] [flang-dev] [RFC] Refactor Clang: move frontend/driver/diagnostics code to LLVM

Andrzej Warzynski via cfe-dev cfe-dev at lists.llvm.org
Thu Aug 6 07:45:37 PDT 2020


Just a small FYI, I've restarted this discussion separately on cfe-dev 
(required refactoring within Clang) and flang-dev (changes in Flang):

* cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2020-July/066393.html
* flang-dev: http://lists.llvm.org/pipermail/flang-dev/2020-July/000470.html


-Andrzej


On 12/06/2020 17:53, Andrzej Warzynski via llvm-dev wrote:
> Hi Tim,
> 
> The key idea behind this refactoring is to re-use the compiler driver 
> code [1] that's currently part of Clang. We initially thought that we'd 
> be re-using bits of Clang's frontend-driver too, but from the feedback 
> so far it looks like writing those bits from scratch (e.g. 
> CompilerInstance or CompilerInvocation) would be better.
> 
> The main benefit of re-using Clang's design is the flexibility that 
> Flang will inherit at a relatively low cost. It will make adding new 
> compiler flags, driver diagnostics or fine-tuning compilation pipelines 
> really easy and powerful. It will enable compilation of mixed-source 
> projects (e.g. C++ + Fortran, perhaps Rust + Fortran one day?). Also, 
> Flang will gain access to Clang’s Toolchains [5], so adoption on new 
> platforms should be more straightforward.  And, on top of that, we could 
> start bringing up infrastructure for compiler plugins so that our users 
> can extend Flang as they do Clang. None of this will happen overnight, 
> but given how popular, powerful and widely adopted Clang is, it feels 
> like a good first step.
> 
> Currently we are focusing on implementing the Flang driver in terms of 
> libclangDriver (which we want to refactor and move out of Clang first). 
> libclangDriver requires some infrastructure code and Diagnostics [2] are 
> a good example. The interface is defined in one header file in Clang [3] 
> (see DiagnosticsEngine, DiagnosticBuilder, DiagnosticConsumer).  For now 
> we will only use them for the Flang driver. Wider adoption in Flang 
> would IMHO be great, but that's outside the scope of this RFC and is not 
> required for this to work. The actual diagnostics are defined in 
> TableGen files - mechanism that we also want to adopt. Diagnostics for 
> the driver are defined in [4] (some adoption will be needed there too).
> 
> In other words - this will only introduce a new Flang driver implemented 
> in terms of `frontend-support` (the new subproject) and hopefully 
> replace the current Flang driver. I can't think of any small/big changes 
> to the Flang codebase that would be required for this. I might be 
> missing something though! As for SourceManager, SourceLocation, 
> FileManager, VFS - to be perfectly honest I think that we need to make 
> more progress with our proof-of-concept implementation before we are 
> ready to discuss these in more detail. We will sent separate RFCs once 
> we are ready.
> 
> I appreciate that there's not that many technical details here, but we 
> are still working these out.
> 
> -Andrzej
> 
> 
> [1] 
> https://clang.llvm.org/docs/DriverInternals.html#driver-design-internals
> [2] https://clang.llvm.org/diagnostics.html
> [3] 
> https://github.com/llvm/llvm-project/blob/release/10.x/clang/include/clang/Basic/Diagnostic.h 
> 
> [4] 
> https://github.com/llvm/llvm-project/blob/release/10.x/clang/include/clang/Basic/DiagnosticDriverKinds.td 
> 
> [5] 
> https://github.com/llvm/llvm-project/bloc/release/10.x/clang/lib/Driver/ToolChains 
> 
> 
> On 12/06/2020 15:25, Timothy Keith wrote:
>> For those of us not familiar with clang internals, it would be helpful 
>> if you could describe the parts of clang that you're considering 
>> sharing and explain what existing code they would replace in flang (if 
>> any) and what benefits we gain by sharing them.
>>
>> In particular, these were mentioned previously: DiagnosticsEngine, 
>> SourceManager, SourceLocation, FileManager, VFS
>>
>> Thanks,
>> Tim
>>
>> On 6/12/20, 3:25 AM, "flang-dev on behalf of Andrzej Warzynski via 
>> flang-dev" <flang-dev-bounces at lists.llvm.org on behalf of 
>> flang-dev at lists.llvm.org> wrote:
>>
>>      External email: Use caution opening links or attachments
>>
>>
>>      FYI: this RFC will be discussed in the next Flang Technical 
>> Community
>>      call: 
>> http://lists.llvm.org/pipermail/flang-dev/2020-June/000379.html
>>
>>      -Andrzej
>>
>>      On 11/06/2020 23:06, Doerfert, Johannes wrote:
>>      > On 6/11/20 4:04 PM, James Y Knight wrote:
>>      >> I think the expectation is that LLVM remains at the bottom of the
>>      >> dependency tree, with frontend-support depending on LLVM, and 
>> Clang and
>>      >> Flang depending on frontend-support (and LLVM).
>>      >>
>>      >> Not everything which makes sense to share between clang and 
>> flang makes
>>      >> sense to be part of llvm core. E.g., implementation of a 
>> GCC-compatible
>>      >> command-line compiler-driver seems like it would be very much 
>> out-of-place
>>      >> in llvm core.
>>      >
>>      > That clears things up, thanks a lot. So we could have a 
>> frontend-support
>>      >
>>      > subproject for non-LLVM(-IR) related things and 
>> `llvm/lib/Frontend` for
>>      >
>>      > LLVM(-IR) related stuff, e.g., IR generation parts.
>>      >
>>      >
>>      > Thanks,
>>      >
>>      >     Johannes
>>      >
>>      >
>>      >> On Thu, Jun 11, 2020 at 10:28 AM Doerfert, Johannes via cfe-dev <
>>      >> cfe-dev at lists.llvm.org> wrote:
>>      >>
>>      >>> On 6/11/20 3:32 AM, Andrzej Warzynski wrote:
>>      >>>>
>>      >>>> On 11/06/2020 00:49, Michael Kruse wrote:
>>      >>>>> Am Mi., 10. Juni 2020 um 10:04 Uhr schrieb Doerfert, 
>> Johannes via
>>      >>>>> flang-dev <flang-dev at lists.llvm.org>:
>>      >>>>>> I'm not against a subproject *but* if we also move the 
>> existing
>>      >>>>>> llvm/lib/Frontend stuff, that would introduce a dependence 
>> from
>>      >>>>>> llvm-core to this project, which I think is uncommon. We 
>> could also
>>      >>>>>> have
>>      >>>>>> both. At the end of the day it depends on the benefit we 
>> would get from
>>      >>>>>> an independent subproject.
>>      >>>>> If we have non-conditional dependencies 
>> llvm-core->frontend-support
>>      >>>>> and frontend-support->llvm-core, what advantage is there 
>> left of
>>      >>>>> making frontend-support a subproject? Both of them are 
>> required all
>>      >>>>> the time.
>>      >>>>>
>>      >>>>> Michael
>>      >>>>>
>>      >>>> Thanks for the feedback! It sounds like we should keep
>>      >>>> llvm/lib/Frontend in LLVM and `frontend-support` would be a new
>>      >>>> subproject for things unrelated to LLVM IR.
>>      >>> Sure, if we expect usage of such a subproject without 
>> llvm-core. (Maybe
>>      >>> there are other reasons to separate it but I'm not sure I 
>> understood them.)
>>      >>>
>>      >>>
>>      >>> Thanks,
>>      >>>
>>      >>>      Johannes
>>      >>>
>>      >>>
>>      >>>> -Andrzej
>>      >>>
>>      >>> _______________________________________________
>>      >>> cfe-dev mailing list
>>      >>> cfe-dev at lists.llvm.org
>>      >>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>      >>>
>>      >
>>      _______________________________________________
>>      flang-dev mailing list
>>      flang-dev at lists.llvm.org
>>      https://lists.llvm.org/cgi-bin/mailman/listinfo/flang-dev
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


More information about the cfe-dev mailing list