[llvm-dev] [LLVMdev] [RFC] Developer Policy for LLVM C API

Mon Aug 17 14:19:25 PDT 2015

On Mon, Aug 17, 2015 at 1:47 PM David Blaikie via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> On Mon, Aug 17, 2015 at 12:31 PM, James Y Knight via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> As someone who used the LLVM C API for an experiment back in 2009ish
>> (porting the SBCL lisp compiler to target LLVM as a backend -- never
>> finished), I thought it was great that LLVM provided the C API. I was sad
>> that it wasn't properly updated to include support for all the newly
>> introduced IR features, though. (E.g. atomics). I tried to send patches for
>> some of that stuff a while later, but didn't manage to get it in. (I wasn't
>> active in llvm at that point, I was just trying to contribute a patch. I
>> gave up.).
>>
>> It doesn't make sense to me to have the C API be less than a full
>> wrapping of the IR building functions. Having less than that, you almost
>> might as well not have it at all. (Which would be sad.)
>>
>> I'd like suggest that the most important thing for the C API is NOT a
>> 100% promise of eternal compatibility, but to _strive_ for compatibility,
>> as much as is realistically possible. If it's not possible, oh well. We're
>> not talking about libc here, so the costs of changing it incompatibly --
>> with careful deliberation -- are not impossibly huge. I mean, even
>> libstdc++ changes its ABI occassionally! LLVM's C API certainly shouldn't
>> be considered more important to keep stable than libstdc++!
>>
>> That said, incompatible changes ought not be made just for trivial
>> reasons: they should be made when there's a true need, e.g. when an
>> existing LLVM-C function cannot sensibly maintain its existing behavior
>> anymore because the underlying functionality of LLVM was removed, or
>> drastically changed.
>>
>
> FWIW we do this (drastically change LLVM) pretty often. See Debug Info
> metadata, DataLayout, my typeless pointer work, Chandler's pass manager
> work, etc.
>
>
>>
>> I'd propose that the only 100% strict rule should be that if the ABI/API
>> changes, it is done in a way that *loudly* breaks old programs -- e.g. they
>> fail to compile, link, or run (depending on how the other-lang wrappers are
>> accessing the API functions) -- not that you get some random weird
>> misbehavior because a function's argument types or return type has been
>> changed.
>>
>> Ideally, if you're going to remove an API, you'd deprecate it first, if
>> you can see the problem coming. Otherwise, oh well -- it's not reasonable
>> to hold up LLVM development to adhere to deprecation policy on the C API.
>>
>> That is:
>> 1) Adding new functions for new LLVM functionality: great.
>> 2) Removing old functions: if it's required -- after careful deliberation.
>> 3) Modifying the signature of existing functions: no way.
>>
>> A couple of concrete examples:
>>
>> - Let's say you run out of space in an existing enum type (ahem,
>> LLVMAttribute...).
>>
>> You should introduce new function names, taking a better type (perhaps
>> some sort of set type, instead of a fixed-width integer...), mark the old
>> ones deprecated so that users will update to the new functions. But, keep
>> the old ones around so that old code can keep working. There's no real need
>> to remove the old deprecated functions -- they'll keep working as well as
>> they ever did with the smaller-valued attributes.
>>
>
> But does it? What if the code is checking if two functions have the same
> attributes (perhaps this is an invalid notion in the absence of information
> about specific attributes) - if you compare the smaller-ranged attributes,
> you might incorrectly conclude that these two functions have the same
> attributes. Similarly for "no attributes specified" - you look at the enum,
> see that none are set & conclude that it's a plain, uninteresting function
> - so you create another plain, no-attributes function you think you can
> make equivalent.
>
>
>>
>> - Let's say you remove types from pointers.
>>
>> This is a change that can be seen coming up. So, one should fairly early
>> on add an LLVMBuildGEP2 function that takes the extra type argument. Leave
>> the old function name around for now (deprecated), because with the current
>> state of LLVM, it can continue to work just fine.
>>
>> When LLVM is changed to *actually* remove all pointer types in its core,
>> then the LLVMBuildGEP function cannot reasonably continue to exist. It
>> simply cannot work, if pointers don't have types, and the type isn't
>> provided. So, remove it.
>>
>
> Yep, this is one we /might/ be able to manage - because we'll probably
> have to keep typed pointer information around for old bitcode
> deserialization. (but it does make the API awkward - when we finally do
> remove LLVMBuildGEP, do we rename LLVMBuildGEP2? Do we leave it there? &
> then we need LLVMBuildGEP3, etc, as things evolve?))
>
> Also this works for writing, but not for reading - if code was expecting
> to read IR and assumed that pointers had types and bitcasts were present,
> etc - it's going to have a Bad Time reading new IR... the APIs wouldn't
> make any sense (there would be no getPointeeType on PointerType anymore -
> there would be no Type that could be returned in response to that query)
>
> That's part of Eric's point, I think, once the API is low level enough, it
> will churn with every change in LLVM or possible require some heroics to
> avoid that churn which will slow down LLVM's API velocity (which is
> somewhere around "ludicrous speed"), which is something we rather like. The
> only way to make the C API more stable is to make it higher level (see
> libLTO for an example) so that we can own the migration & shelter users
> from it through that abstraction. If it's low level, it's going to churn or
> slow us down (and probably both).
>
>

Precisely :)

-eric

>
>> Preferably, there would be a release of LLVM in between with both
>> functions available to let people have some time to adjust, but if not, oh
>> well.
>>
>>
>> On Mon, Aug 17, 2015 at 1:28 PM, deadal nix via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>>
>>>
>>> 2015-08-17 8:34 GMT-07:00 Reid Kleckner <rnk at google.com>:
>>>
>>>> On Sun, Aug 16, 2015 at 10:34 PM, deadal nix via llvm-dev <
>>>> llvm-dev at lists.llvm.org> wrote:
>>>>
>>>>>
>>>>> 2015-08-16 21:51 GMT-07:00 Eric Christopher <echristo at gmail.com>:
>>>>>>
>>>>>> The promise of stability. We don't promise that the C++ API will stay
>>>>>> stable.
>>>>>>
>>>>>
>>>>>
>>>>> Why was that promise be made in the first place ? Has it been made in
>>>>> the first place ?
>>>>>
>>>>
>>>> It sounds like you're in favor of dropping C API stability then, if
>>>> it's holding us back? That feedback is actually really helpful. :)
>>>>
>>>> There are really three goals here: flexibility to change LLVM IR,
>>>> completeness of the C API, and stability of the C API. Pick two.
>>>>
>>>>
>>> Yes but these aren't black and white IMO. These are need in tension,
>>> sure, but all API have these kind of problem not specifically the C API.
>>>
>>>
>>>
>>>> The goals are mutually incompatible and we have to trade off one for
>>>> the other. Most of the LLVM core developers value goal #1, the ability to
>>>> change the IR. Look at the pointee type changes that David Blaikie is
>>>> working on, and the new EH representation. If we promise both stability and
>>>> completeness, these things are impossible.
>>>>
>>>>
>>> Yes I was actually thinking about that. This is the kind of change that
>>> justify a change in the C API IMO. We should try our best to keep the C API
>>> stable, but if somethign fundamental changes in LLVM, and that there is no
>>> nice way to map it to the old API, then
>>>
>>>
>>>
>>>> One way forward is to guarantee stability, but limit completeness. This
>>>> would mean limiting the C API to constructs that will always and forever be
>>>> easily represented in LLVM.
>>>>
>>>>
>>> That would be a problem for me. It seems like other in this thread
>>> mentioned the incompleteness of the C API as a problem for them and
>>> mentioned the use of various hacks to work around it. On my side I tried to
>>> propose diff to increase completeness of the C API in the past, but the
>>> review process was frankly too painful and I mostly give up. I'd be up for
>>> being a maintainer of the C API if that is needed.
>>>
>>>
>>>
>>>> The other choice is to forget stability and wrap the C++ API
>>>> completely, potentially with something auto-generated. We could make a more
>>>> limited stability promise along the lines of "these APIs will be updated to
>>>> reflect changes to the IR, and are otherwise stable." I think everyone
>>>> would be fine with that.
>>>>
>>>
>>> I do not think this is a good idea. The C API is used a lot to use LLVM
>>> from other languages than C++ . When used directly from C, you get function
>>> signature, but when used from other languages, changing method randomly
>>> cause undefined behavior. For instance, the landing pad instruction was
>>> changed recently and the C API to create a landing pad was updated. As a
>>> result, 2 thing happened :
>>>  - It was not possible to set the personality function from C, which is
>>> a problem for many frontends.
>>>  - In other languages, as mangling remained the same, the thing compiles
>>> and fail randomly at runtime.
>>>
>>> This need to be avoided as much as possible. I'd propose the following
>>> policy :
>>> 1/ Do change the IR as per need of LLVM development. The C API must not
>>> prevent LLVM to move forward.
>>> 2/ If the change can be mapped nicely without changing the C API, then
>>> do that.
>>> 3/ If there is no way to map it nicely, then update the C API.
>>>
>>> For instance, the pointer update work would probably be a valid reason
>>> to go with 3/ .
>>>
>>> Right now, as far as I'm concerned, the main limitation of the C API are
>>> the capability to set attributes and the capability to use atomic
>>> loads/stores.
>>>
>>> In any ways, I do think the patch adding tests for existing API should
>>> be merged. The need for test for the C API was mentioned various times in
>>> this thread and I do think that whatever the road taken, having test is a
>>> good thing.
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org         http://llvm.cs.uiuc.edu
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org         http://llvm.cs.uiuc.edu
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org         http://llvm.cs.uiuc.edu
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150817/052a1c3c/attachment.html>