[cfe-dev] [LLVMdev] llvm-abi: A library for generating ABI-compliant LLVM IR
Andrew Wilkins
axwalk at gmail.com
Tue Jun 30 18:24:15 PDT 2015
On Wed, 1 Jul 2015 at 08:19 Stephen Cross <scross at scross.co.uk> wrote:
> Hi everyone,
>
> I was wondering if anyone could answer the questions in my first
> email. These were:
>
> * Why does Clang generate 8 byte alignment for 16+ byte arrays on
> x86-64, even though the AMD64 ABI seems to require 16 byte alignment?
> * Clang has some logic in lib/Basic/Targets.cpp to determine CPU
> features, but LLVM also does the same task in its backend. Would it be
> possible to modify LLVM to expose this logic and modify Clang to use
> that?
> * (Slightly modified) Clang has some logic in lib/Driver/Tools.cpp
> ('getCPUName') for selecting a generic CPU. Would it be sensible to
> expose this in a header in include/clang/Driver/?
>
> > Refactor the logic in Clang's lib/CodeGen/TargetInfo.cpp so that it can
> be used by external clients (just as Clang's AST headers are available to
> external clients).
>
> Yes, this sounds like a good approach. While I'm not entirely
> convinced this is where we want to end up (it doesn't seem right to
> split the ABI implementation between the frontend and the backend), I
> definitely agree this is a sensible starting point. Unless anyone is
> against this I'll start working out the changes needed under the
> assumption this is how I should proceed.
>
> My approach would be to just refactor the code to separate classes
> within CodeGen and add the necessary headers to
> include/clang/CodeGen/. It might make sense to create a separate
> library directory (e.g. lib/ABI) but I expect there will be circular
> dependency issues to overcome. The very first thing is probably to
> split each target's code out of TargetInfo.cpp; I'd suggest that the
> structure of CodeGen could mirror <llvm>/lib/Target/ so we'd have
> directories for ARM, PowerPC, X86 etc. What do you think?
>
> Note that there's also functionality in CGCall.cpp (and a few other
> files) which would need to be separated out to be usable for other
> frontends.
>
> > Write your library as a wrapper around that logic so that simple cases
> can be handled without burdening other frontends with the subtleties of
> Clang's AST data structures.
>
> This will be how it ends up, but while the modifications to Clang are
> ongoing llvm-abi is and will continue to be an entirely separate
> ABI-implementing codebase. This is just because the Loci frontend
> needs this functionality right now. I'm also expecting there will be
> functionality going into llvm-abi that for good reason won't be found
> in Clang. There's some duplication of effort here but it's
> unavoidable.
>
> I like the idea of frontends using the library as a convenient API and
> the library's implementation using code from Clang. I know that some
> frontends (including the Loci frontend) are designed to be able to be
> built on systems that could have slightly older versions of LLVM/Clang
> (for Loci, it's LLVM 3.3+), so this factor will have to be considered
> when integrating the code from Clang.
>
> > At the moment we have support for x86-64 only:
> http://llvm.org/klaus/llgo/blob/master/irgen/cabi.go.
>
> I'll add other architectures to llvm-abi soon (it's largely a fairly
> trivial process of moving code from Clang and replacing its in-tree
> dependencies) so hopefully this will be useful. I can see from other
> non-Clang frontends that non-X86 support does tend to be at least
> quite incomplete.
>
> > It would be nice to have a simplified API to TargetInfo, ideally with a
> C API that other languages to can easily bind to.
>
> Absolutely, I agree. I appear to be in the rare situation of having a
> frontend written in C++ (i.e. not self-hosting) but I know most
> frontends rely on having a C API.
>
> It seems like the approach to this should be to have a C API in
> llvm-abi that wraps around its C++ API. Unfortunately my time is
> currently fairly thin-spread; would you be able to contribute to this
> effort?
>
Sounds good. My time is likewise thinly spread, but I'll be happy to
contribute to the effort when I can.
Cheers,
Andrew
> Thanks,
> Stephen
>
> On Tue, Jun 30, 2015 at 12:47 AM, Andrew Wilkins <axwalk at gmail.com> wrote:
> > On Tue, 30 Jun 2015 at 06:02 Hal Finkel <hfinkel at anl.gov> wrote:
> >>
> >> ----- Original Message -----
> >> > From: "Stephen Cross" <scross at scross.co.uk>
> >> > To: "Reid Kleckner" <rnk at google.com>
> >> > Cc: "Clang Developers List" <cfe-dev at cs.uiuc.edu>, "LLVM Developers
> >> > Mailing List" <llvmdev at cs.uiuc.edu>
> >> > Sent: Monday, June 29, 2015 3:03:40 PM
> >> > Subject: Re: [cfe-dev] llvm-abi: A library for generating
> ABI-compliant
> >> > LLVM IR
> >> >
> >> > Hi Reid,
> >> >
> >> > Thanks for your response.
> >> >
> >> > The issue is that every LLVM frontend needing ABI compliance has to
> >> > re-implement the same target-dependent logic, which is a significant
> >> > burden; the ABI compliance code inside Clang isn't really usable for
> >> > other frontends as-is. We haven't got many good options here :-). I
> >> > think a lot of people would've hoped that LLVM would provide the
> >> > means
> >> > for achieving ABI compliance (at least for C), though I'm well aware
> >> > of the complexity involved and understand the decisions taken thus
> >> > far.
> >> >
> >> > Within this context, llvm-abi is an immediately actionable way of
> >> > sharing code between frontends, that should lead to a higher quality
> >> > codebase for a significantly reduced effort on everyone's part. It
> >> > seems like this would further build on LLVM's success because I think
> >> > many people would like to generate code that can interact with C
> >> > APIs.
> >> >
> >> > Ultimately it would be great to see this functionality be provided in
> >> > an accessible form inside LLVM and hence for Clang to use that
> >> > functionality. This would move much of the target-dependent logic out
> >> > of Clang while at the same time making this functionality available
> >> > to
> >> > other LLVM-based tools.
> >> >
> >> > I am sure this is a substantial and complex long term project, but I
> >> > think it is a worthwhile aim. Do you (and the LLVM community as a
> >> > whole) agree? (Acknowledging that some of the details would need to
> >> > be
> >> > worked out.)
> >> >
> >> > I have a long term and substantial interest in this (as I expect do
> >> > other frontend developers), so I'm willing to contribute significant
> >> > effort to move this forward. Hopefully the llvm-abi library can
> >> > provide a better understanding of what needs to be represented and
> >> > help non-Clang frontends :-).
> >> >
> >> > Implementation-wise the changes to LLVM could involve encoding ABI
> >> > information into LLVM IR or simply providing C++ APIs for generating
> >> > ABI-compliant code inside LLVM (much like llvm-abi). Each solution
> >> > has
> >> > advantages and disadvantages and I'm planning to make a proposal
> >> > about
> >> > this on the mailing list later (hence I'd suggest leaving that
> >> > discussion until I make the proposal).
> >> >
> >> > In terms of the mentioned deficiencies of the Type structure, these
> >> > are all problems that I intend to address (it would be great if you
> >> > could provide details on any of the more difficult areas).
> >>
> >> First, let me say that I also think this is an incredibly important area
> >> in definite need of a solution. However, I also think that we really
> need to
> >> work from Clang's logic here: We don't want to have two different
> >> implementations of the ABI rules. They're complicated (and subtle), and
> it
> >> is hard enough to have one correct implementation in our C/C++ compiler.
> >> Given that we're talking about the C/C++ ABI, this is a perfectly
> >> appropriate place for the logic to live. The issue, as you well know, is
> >> that other languages need to interact with C/C++ functions, and so need
> to
> >> understand the C/C++ ABI. My recommended strategy is this:
> >>
> >> 1. Refactor the logic in Clang's lib/CodeGen/TargetInfo.cpp so that it
> >> can be used by external clients (just as Clang's AST headers are
> available
> >> to external clients).
> >>
> >> 2. Write your library as a wrapper around that logic so that simple
> cases
> >> can be handled without burdening other frontends with the subtleties of
> >> Clang's AST data structures.
> >
> >
> > This is what I was thinking of doing for llgo in the long term. At the
> > moment we have support for x86-64 only:
> > http://llvm.org/klaus/llgo/blob/master/irgen/cabi.go. It would be nice
> to
> > have a simplified API to TargetInfo, ideally with a C API that other
> > languages to can easily bind to. Stephen, if you do go down that route,
> I'd
> > be keen to hear about it.
> >
> > Cheers,
> > Andrew
> >
> >> -Hal
> >>
> >> >
> >> > Thanks again,
> >> > Stephen
> >> >
> >> > On Mon, Jun 29, 2015 at 7:32 PM, Reid Kleckner <rnk at google.com>
> >> > wrote:
> >> > > I agree it would be really nice to build a library for ABI
> >> > > lowering, but any
> >> > > solution that isn't clang or isn't ultimately picked up by clang
> >> > > will
> >> > > necessarily be incomplete. Perhaps that is OK for your frontend's
> >> > > uses, but
> >> > > I think it's the main reason that we haven't done something like
> >> > > this in
> >> > > LLVM already.
> >> > >
> >> > > Any solution that doesn't involve actual Clang ASTs is unlikely to
> >> > > be able
> >> > > to represent all C-with-extensions types (unions, bitfields,
> >> > > alignment
> >> > > attributes, transparent_union attribute). I took a look at Type.hpp
> >> > > in your
> >> > > project, and it seems to be missing some of these things. Keeping
> >> > > such a
> >> > > library up to date with new extensions is going to be a maintenance
> >> > > burden.
> >> > >
> >> > > That said, I wish you luck, and I hope the project eases some of
> >> > > the
> >> > > difficulties for new frontends. It is very possible that the corner
> >> > > cases
> >> > > that keep me up at night are not the problems that users actually
> >> > > face. :)
> >> > >
> >> > > On Sun, Jun 28, 2015 at 3:22 PM, Stephen Cross
> >> > > <scross at scross.co.uk> wrote:
> >> > >>
> >> > >> Hi everyone,
> >> > >>
> >> > >> (Also CC'ed cfe-dev since this seems relevant to Clang,
> >> > >> particularly
> >> > >> the questions at the end.)
> >> > >>
> >> > >> I've been working on a library to generate LLVM IR that complies
> >> > >> with
> >> > >> platform ABIs (the current focus is on C but I'm also interested
> >> > >> in
> >> > >> ABIs for other languages).
> >> > >>
> >> > >> You can find it here: https://github.com/scross99/llvm-abi
> >> > >>
> >> > >> To explain further (for those who are unfamiliar), LLVM frontends
> >> > >> have
> >> > >> to modify function argument types, attributes etc. in order to
> >> > >> ensure
> >> > >> the backend generates code that satisfies the ABI; this is needed
> >> > >> because LLVM's type system can't encode all the necessary
> >> > >> information.
> >> > >> This is a complex task and involves substantial amounts of
> >> > >> target-dependent logic. Clang performs this encoding and indeed
> >> > >> much
> >> > >> of the current functionality is derived from Clang's source.
> >> > >>
> >> > >> This project originated as a necessary piece of functionality for
> >> > >> the
> >> > >> Loci compiler frontend [1] and is now an external dependency; my
> >> > >> aim
> >> > >> is to make this usable for other LLVM frontends that also need to
> >> > >> generate ABI-compliant IR (I assume this is a fairly large subset
> >> > >> of
> >> > >> the frontends).
> >> > >>
> >> > >> I'd be very interested in any suggestions/queries/comments.
> >> > >>
> >> > >> I made a few interesting discoveries while working on this:
> >> > >>
> >> > >> * Clang generates 8 byte alignment for 16+ byte arrays on x86-64,
> >> > >> even
> >> > >> though the AMD64 ABI seems to require that arrays of 16+ bytes are
> >> > >> aligned to 16 bytes. Is this a bug or am I missing something
> >> > >> obvious?
> >> > >>
> >> > >> * Clang determines the features for a CPU (e.g. whether we have
> >> > >> AVX
> >> > >> support on x86-64 CPUs), even though this functionality is already
> >> > >> available in LLVM (but appears to be very difficult to query).
> >> > >> Would
> >> > >> it be possible to expose the information from LLVM and hence
> >> > >> eliminate
> >> > >> the duplication in Clang?
> >> > >>
> >> > >> * Clang determines a 'generic CPU' if the user doesn't specify a
> >> > >> CPU.
> >> > >> My understanding is that we don't usually generate code for the
> >> > >> native
> >> > >> CPU because it may have features unavailable on other similar
> >> > >> CPUs.
> >> > >> LLVM can provide full details of the native CPU but can't
> >> > >> determine a
> >> > >> generic CPU; could this functionality be added to LLVM?
> >> > >>
> >> > >> (Separately I've also been considering a proposal to add ABI
> >> > >> information directly inside LLVM IR in a language-independent way
> >> > >> and
> >> > >> I'll discuss this in a later email.)
> >> > >>
> >> > >> Thanks,
> >> > >> Stephen
> >> > >>
> >> > >> [1] https://github.com/scross99/locic
> >> > >> _______________________________________________
> >> > >> cfe-dev mailing list
> >> > >> cfe-dev at cs.uiuc.edu
> >> > >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
> >> > >
> >> > >
> >> > _______________________________________________
> >> > cfe-dev mailing list
> >> > cfe-dev at cs.uiuc.edu
> >> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
> >> >
> >>
> >> --
> >> Hal Finkel
> >> Assistant Computational Scientist
> >> Leadership Computing Facility
> >> Argonne National Laboratory
> >> _______________________________________________
> >> LLVM Developers mailing list
> >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20150701/2042db5f/attachment.html>
More information about the cfe-dev
mailing list