[llvm-dev] [LLVMdev] [RFC] Developer Policy for LLVM C API

Mon Aug 17 13:47:22 PDT 2015

On Mon, Aug 17, 2015 at 12:31 PM, James Y Knight via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> As someone who used the LLVM C API for an experiment back in 2009ish
> (porting the SBCL lisp compiler to target LLVM as a backend -- never
> finished), I thought it was great that LLVM provided the C API. I was sad
> that it wasn't properly updated to include support for all the newly
> introduced IR features, though. (E.g. atomics). I tried to send patches for
> some of that stuff a while later, but didn't manage to get it in. (I wasn't
> active in llvm at that point, I was just trying to contribute a patch. I
> gave up.).
>
> It doesn't make sense to me to have the C API be less than a full wrapping
> of the IR building functions. Having less than that, you almost might as
> well not have it at all. (Which would be sad.)
>
> I'd like suggest that the most important thing for the C API is NOT a 100%
> promise of eternal compatibility, but to _strive_ for compatibility, as
> much as is realistically possible. If it's not possible, oh well. We're not
> talking about libc here, so the costs of changing it incompatibly -- with
> careful deliberation -- are not impossibly huge. I mean, even libstdc++
> changes its ABI occassionally! LLVM's C API certainly shouldn't be
> considered more important to keep stable than libstdc++!
>
> That said, incompatible changes ought not be made just for trivial
> reasons: they should be made when there's a true need, e.g. when an
> existing LLVM-C function cannot sensibly maintain its existing behavior
> anymore because the underlying functionality of LLVM was removed, or
> drastically changed.
>

FWIW we do this (drastically change LLVM) pretty often. See Debug Info
metadata, DataLayout, my typeless pointer work, Chandler's pass manager
work, etc.

>
> I'd propose that the only 100% strict rule should be that if the ABI/API
> changes, it is done in a way that *loudly* breaks old programs -- e.g. they
> fail to compile, link, or run (depending on how the other-lang wrappers are
> accessing the API functions) -- not that you get some random weird
> misbehavior because a function's argument types or return type has been
> changed.
>
> Ideally, if you're going to remove an API, you'd deprecate it first, if
> you can see the problem coming. Otherwise, oh well -- it's not reasonable
> to hold up LLVM development to adhere to deprecation policy on the C API.
>
> That is:
> 1) Adding new functions for new LLVM functionality: great.
> 2) Removing old functions: if it's required -- after careful deliberation.
> 3) Modifying the signature of existing functions: no way.
>
> A couple of concrete examples:
>
> - Let's say you run out of space in an existing enum type (ahem,
> LLVMAttribute...).
>
> You should introduce new function names, taking a better type (perhaps
> some sort of set type, instead of a fixed-width integer...), mark the old
> ones deprecated so that users will update to the new functions. But, keep
> the old ones around so that old code can keep working. There's no real need
> to remove the old deprecated functions -- they'll keep working as well as
> they ever did with the smaller-valued attributes.
>

But does it? What if the code is checking if two functions have the same
attributes (perhaps this is an invalid notion in the absence of information
about specific attributes) - if you compare the smaller-ranged attributes,
you might incorrectly conclude that these two functions have the same
attributes. Similarly for "no attributes specified" - you look at the enum,
see that none are set & conclude that it's a plain, uninteresting function
- so you create another plain, no-attributes function you think you can
make equivalent.

>
> - Let's say you remove types from pointers.
>
> This is a change that can be seen coming up. So, one should fairly early
> on add an LLVMBuildGEP2 function that takes the extra type argument. Leave
> the old function name around for now (deprecated), because with the current
> state of LLVM, it can continue to work just fine.
>
> When LLVM is changed to *actually* remove all pointer types in its core,
> then the LLVMBuildGEP function cannot reasonably continue to exist. It
> simply cannot work, if pointers don't have types, and the type isn't
> provided. So, remove it.
>

Yep, this is one we /might/ be able to manage - because we'll probably have
to keep typed pointer information around for old bitcode deserialization.
(but it does make the API awkward - when we finally do remove LLVMBuildGEP,
do we rename LLVMBuildGEP2? Do we leave it there? & then we need
LLVMBuildGEP3, etc, as things evolve?))

Also this works for writing, but not for reading - if code was expecting to
read IR and assumed that pointers had types and bitcasts were present, etc
- it's going to have a Bad Time reading new IR... the APIs wouldn't make
any sense (there would be no getPointeeType on PointerType anymore - there
would be no Type that could be returned in response to that query)

That's part of Eric's point, I think, once the API is low level enough, it
will churn with every change in LLVM or possible require some heroics to
avoid that churn which will slow down LLVM's API velocity (which is
somewhere around "ludicrous speed"), which is something we rather like. The
only way to make the C API more stable is to make it higher level (see
libLTO for an example) so that we can own the migration & shelter users
from it through that abstraction. If it's low level, it's going to churn or
slow us down (and probably both).

>
> Preferably, there would be a release of LLVM in between with both
> functions available to let people have some time to adjust, but if not, oh
> well.
>
>
> On Mon, Aug 17, 2015 at 1:28 PM, deadal nix via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>>
>>
>> 2015-08-17 8:34 GMT-07:00 Reid Kleckner <rnk at google.com>:
>>
>>> On Sun, Aug 16, 2015 at 10:34 PM, deadal nix via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>>
>>>> 2015-08-16 21:51 GMT-07:00 Eric Christopher <echristo at gmail.com>:
>>>>>
>>>>> The promise of stability. We don't promise that the C++ API will stay
>>>>> stable.
>>>>>
>>>>
>>>>
>>>> Why was that promise be made in the first place ? Has it been made in
>>>> the first place ?
>>>>
>>>
>>> It sounds like you're in favor of dropping C API stability then, if it's
>>> holding us back? That feedback is actually really helpful. :)
>>>
>>> There are really three goals here: flexibility to change LLVM IR,
>>> completeness of the C API, and stability of the C API. Pick two.
>>>
>>>
>> Yes but these aren't black and white IMO. These are need in tension,
>> sure, but all API have these kind of problem not specifically the C API.
>>
>>
>>
>>> The goals are mutually incompatible and we have to trade off one for the
>>> other. Most of the LLVM core developers value goal #1, the ability to
>>> change the IR. Look at the pointee type changes that David Blaikie is
>>> working on, and the new EH representation. If we promise both stability and
>>> completeness, these things are impossible.
>>>
>>>
>> Yes I was actually thinking about that. This is the kind of change that
>> justify a change in the C API IMO. We should try our best to keep the C API
>> stable, but if somethign fundamental changes in LLVM, and that there is no
>> nice way to map it to the old API, then
>>
>>
>>
>>> One way forward is to guarantee stability, but limit completeness. This
>>> would mean limiting the C API to constructs that will always and forever be
>>> easily represented in LLVM.
>>>
>>>
>> That would be a problem for me. It seems like other in this thread
>> mentioned the incompleteness of the C API as a problem for them and
>> mentioned the use of various hacks to work around it. On my side I tried to
>> propose diff to increase completeness of the C API in the past, but the
>> review process was frankly too painful and I mostly give up. I'd be up for
>> being a maintainer of the C API if that is needed.
>>
>>
>>
>>> The other choice is to forget stability and wrap the C++ API completely,
>>> potentially with something auto-generated. We could make a more limited
>>> stability promise along the lines of "these APIs will be updated to reflect
>>> changes to the IR, and are otherwise stable." I think everyone would be
>>> fine with that.
>>>
>>
>> I do not think this is a good idea. The C API is used a lot to use LLVM
>> from other languages than C++ . When used directly from C, you get function
>> signature, but when used from other languages, changing method randomly
>> cause undefined behavior. For instance, the landing pad instruction was
>> changed recently and the C API to create a landing pad was updated. As a
>> result, 2 thing happened :
>>  - It was not possible to set the personality function from C, which is a
>> problem for many frontends.
>>  - In other languages, as mangling remained the same, the thing compiles
>> and fail randomly at runtime.
>>
>> This need to be avoided as much as possible. I'd propose the following
>> policy :
>> 1/ Do change the IR as per need of LLVM development. The C API must not
>> prevent LLVM to move forward.
>> 2/ If the change can be mapped nicely without changing the C API, then do
>> that.
>> 3/ If there is no way to map it nicely, then update the C API.
>>
>> For instance, the pointer update work would probably be a valid reason to
>> go with 3/ .
>>
>> Right now, as far as I'm concerned, the main limitation of the C API are
>> the capability to set attributes and the capability to use atomic
>> loads/stores.
>>
>> In any ways, I do think the patch adding tests for existing API should be
>> merged. The need for test for the C API was mentioned various times in this
>> thread and I do think that whatever the road taken, having test is a good
>> thing.
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org         http://llvm.cs.uiuc.edu
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org         http://llvm.cs.uiuc.edu
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150817/f0a6a294/attachment-0001.html>