[llvm-dev] The future of LLVM's C APIs: Notes and BoF.

Wed Oct 21 18:52:33 PDT 2015

Yup that is quite similar to my use case, but in addition I use LLVM to JIT
as well from C using MCJIT.

I have a patch for a DIBuilder binding, but it does change the API a bit,
so depending on how this discussion goes, it can or cannot be added. It was
necessary as the current binding wrap all metadata in values, which is very
wasteful when dealing with a large amount of metadata. The breakage is
small (all my existing code still compile and work even using the old
binding API) but very real. I obviously think the change is good or I
wouldn't have made it, but that may be too much for others.

Back on point, the importants things for me to do through the C API are :
 - Write IR, including debug infos and other metadata (it is mostly good,
some IR specificities are not covered, like ordered loads/stores, but these
are edges cases that ca be added. I can prepare a patch if people want
these).
 - Read IR, so I can do transformations.
 - Codegen to object files
 - JIT (using MCJIT is fine, Orca would be better but making a good C api
for it may prove challenging).

I preferably would like that the ABI is not broken, as typecheking won't
work cross language boundary. That being said, I'm ok with this being a
"best practice" or whatever people want to call it rather than a strong,
enforced, requirement. However, if it is not a strong, enforced
requirement, we need a way to test what versions of LLVM is currently in
use as to choose the right version of the function to call. As long as ABI
breakage are not too frequent, this is workable as far as I'm concerned,
and put less constraint on the evolution of the whole thing.

Last but not least, I started working on a test that would read IR and
regenerate it. This would enforce that IR is readable and writable from the
C API, and that breakage are not done unknowingly like it happened in the
past several times. I really would like this to move forward whatever the
policy is chosen. Random breakage + detective work to figure out what
happened are the worst when we could know things in an automated manner.

As a final note, I'd like for us to focus on resolving this soon enough so
that some work can be done related to the C API before 3.8 is out. Nothing
is on fire yet, but let's not let this slip too much.

2015-10-20 13:16 GMT-07:00 Rodney M. Bates via llvm-dev <
llvm-dev at lists.llvm.org>:

>
>
> On 10/17/2015 01:20 PM, Justin Bogner via llvm-dev wrote:
>
>> (Moving this to llvm-dev)
>>
>>
>> On Friday, October 16, 2015, Justin Bogner <mail at justinbogner.com
>> <mailto:mail at justinbogner.com>> wrote:
>>
>>     Some users of llvm-c want stable API interfaces into various parts of
>>     the LLVM infrasture, others want further ABI guarantees about this
>>     usage, and still others simply want a way to bind to LLVM through
>> their
>>     language frontend’s existing FFI support for C.
>>
>>     If we want to improve the situation for any of these users, we need to
>>     properly understand how these APIs are being used (or abused)
>>     today. Juergen and I will be hosting a BoF at the dev meeting where we
>>     can discuss what the requirements of a sustainable C API are, and how
>> we
>>     can organize things in LLVM to support this.
>>
>>     There's been a fair amount of discussion about this lately, and it's
>>     pretty clear that the "hopefully stable bindings to whatever APIs
>>     somebody needed" approach isn't really good enough for anybody. There
>>     are a couple of points that I think are fairly non-controversial, but
>>     will more or less drive the discussion at the BoF:
>>
>>     1. It isn't practical to keep a bindings API stable, unless the
>>         underlying API is also stable.
>>
>>     2. Handrolling bindings as they're needed tends to leave conspicuous
>>         gaps where some API is inaccessible for no good reason.
>>
>>     So based on (1), we'll really want to create some purpose built APIs
>>     that we can keep stable for various tasks. What's needed here? People
>>     want to do things like building a pass manager, setting up a canned
>> JIT
>>     config, and to some degree even emit IR. We'll discuss what's
>> practical
>>     and what people want, and hopefully strike a good balance.
>>
>>     Similarly, (2) implies that if we really need a *full* bindings API
>>     we'll want to automate it. But what is a full bindings API? Who uses
>> it,
>>     and what do they want from it? If it's automated, should installing
>> LLVM
>>     install this API, or should we simply provide an easy way to generate
>>     it?
>>
>>     In the end, I hope to have a good idea of what people are actually
>> using
>>     these APIs for, and both to support stable API users less haphazardly
>>     and to make unstable API more thorough and/or easier to create.
>>
>>
> One actual use:
>
> A couple of us in the Modula3 support community are working on splicing a
> llvm back end onto our Modula3 compiler front end.  We have zero-thickness
> bindings written in Modula3 which match C bindings like core.h.  In our
> front
> end executable, we only build llvm IR in memory, then use
> LLVMPrintModuleToFile
> and/or LLVMWriteBitcodeToFile.  We only link in the needed parts of the
> llvm
> infrastructure to build and write the llvm IR.  Then we run stock llc on
> the
> IR code in the emitted file.
>
> We have had to write an additional binding to DIBuilder for this purpose,
> as well as to a few odd other C++ functions here and there.  The latest
> version of this is for llvm 3.6.1, after the separation of metadata from
> values.  It has had minimal testing, but is fairly complete.  Older
> bindings
> to an older DIBuilder exist, and are at least partly working, but only
> contain specifically needed functions.
>
> This scheme is passing a majority of preexisting compiler tests of code
> function, but there is not much testing of emitted debug info.  Speaking
> for myself, better debugger support is one of the primary motivations for
> using llvm.  Our older back ends use a really cobbled-up extension of
> stabs+.  It's a mess, and lots of debugger function is difficult to
> provide.
>
> The DIBuilder binding includes .h and .cpp files similar to core, that
> provide
> a C binding, with lots of [un]wrapping, etc.  Like core, it loses the type
> hierarchy.  Modula3 has a type hierarchy, but machine-level representation
> is undoubtedly not ABI compatible with any C++ compiler.  The current
> handling
> of this is not thoroughly thought out, just using lots of unsafe casting of
> pointers.  I would like to think of something better, but haven't had much
> time
> for that, so far.
>
>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
> --
> Rodney Bates
> rodney.m.bates at acm.org
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151021/359e9a7e/attachment.html>