[cfe-dev] Proposal: Managing ABI changes in libc++

[Evgenii Stepanov]

It seems like there is a general agreement on the general direction
this should go. In the hope of pushing this forward, I've uploaded a
very basic implementation to http://reviews.llvm.org/D11740. It does
not include any build system changes, because it's not clear if we
want, for example, to bump soname each time ABI version is increased.

On Mon, Dec 29, 2014 at 10:51 PM, Ted Kremenek <kremenek at apple.com> wrote:
> My apologies as well for the delay.  Chandler's response was quite
> excellent, so I'm going to piggy back on his comments.
>
> On Dec 19, 2014, at 6:28 PM, Chandler Carruth <chandlerc at google.com> wrote:
>
> First, sorry for delays. I do owe you feedback here though, and then I'll go
> look at the patch. =]
>
> On Mon, Dec 8, 2014 at 7:42 AM, Marshall Clow <mclow.lists at gmail.com> wrote:
>>
>> In general, we try to avoid making changes to the ABI for libc++.
>> ABI changes can lead to subtle, hard to find bugs, when part of a piece of
>> software (a dylib or static library, say) is build to the old ABI, and the
>> rest to the new ABI. People have been burned in the past by inadvertent
>> changes to the libc++  ABI. (not to be confused with the libc++abi project)
>>
>> Eric Fiselier has been working on a tool to detect ABI changes, so that
>> (hopefully) all future changes will be intentional.
>>
>> ABI-breaking changes can include things like:
>>         * Changes to structures (sizes, layout)
>>         * Addition/removal of virtual functions (vtable layouts)
>>         * Changes to template parameters (addition, removal)
>>
>> Also, there are times that a change to the standard will mandate an ABI
>> change. I tend to argue against those in the committee meetings, but I don’t
>> always get my way.
>>
>> In the LLVM community, there are two differing opinions about changes in
>> lib++ that are ABI-breaking. Broadly speaking:
>>
>> a) There are the people who ship libc++ in production systems, who say:
>> Whoa! Don’t do that! Ever! (or at least “let us decide when”).
>>
>> b) There are the people who use libc++ internally, who say: Is it faster?
>> Does it work better? Do it!
>
>
> (FWIW, there are also people that support users in both camps (a) and (b).
> I'm one of those.)
>
>>
>>
>> === Proposal ===
>>
>> Goals:
>> 1) Make the default be “ABI is stable” (modulo changes in the C++
>> standard)
>> 2) Make it possible for people to propose (and use) ABI-breaking changes
>> for libc++, and have them live in tree.
>> Note: This would make it possible, not trivial. We still want to avoid
>> gratuitously changing the ABI.
>>
>> Concrete steps:
>> 1) Give each ABI-breaking change its own "enabling macro”, that starts
>> with “_LIBCPP_ABI_”
>>
>> We have an example of this today. There are two different
>> std::basic_string layouts defined in <string>, and the
>> second (ABI changing) one is controlled by the macro
>> _LIBCPP_ALTERNATE_STRING_LAYOUT
>>
>> Under my proposal, I would change this to
>> _LIBCPP_ABI_ALTERNATE_STRING_LAYOUT, and keep the old name as a synonym.
>>
>> 2) Create a global macro “_LIBCPP_ABI_UNSTABLE” which, when defined, turns
>> on ALL of the _LIBCPP_ABI_* changes.
>>
>> Adding a new, ABI-incompatible change to the library would consist of:
>> * Choosing an enabling macro name of the form _LIBCPP_ABI_XXXXXXX
>> * Wrapping the code in #ifdef _LIBCPP_ABI_XXXXXXX
>> * Enabling the macro if _LIBCPP_ABI_UNSTABLE is defined.
>>
>> I think that this convention will make it possible both camps ((a) and (b)
>> above) to coexist in the same code base.
>>
>> Comments?
>
>
> As far as this goes, I'm 100% in favor.
>
>
> I'm supportive of this initiative; my main concern is the social aspect that
> we have a manageable way to discern different ABI configurations and that
> they are all testable.
>
>
>
> I think there are two more ABI concerns that we should really figure out a
> plan for now in order to ensure they fit cohesively with the whole.
>
> 1) I think we need a way to more quickly roll standard-mandated or bug-fix
> ABI breaks into something much more stable than "unstable".
> 2) I think we need to figure out how to maintain at least two stable ABIs at
> the same time.
>
> I'll expand a bit below.
>
> A terminology point, when I say a "minor" or "major" ABI break, I am not
> classifying the *nature* of the break, but the *scope*. Changing the layout
> of std::string has a radically different scope in its impact than fixing the
> return type of a infrequently used function for example.
>
> For (1), let's consider two cases.
> 1.1) We introduce an ABI-significant bug and need to fix it. What do we do?
> This is exacerbated when the bug has shipped to customers. Some users of
> libc++ update *very* rapidly, and so even with a very narrow window of
> fallout in-tree, it would be advantageous in my opinion to have a non-silent
> way to fix these issues. Note that this only really applies to minor ABI
> breaks. A massive breaking change would be sufficiently disruptive to
> warrant more extreme measures and I certainly hope i
> 1.2) The standard changes in some *minor* way that necessitates an ABI
> break. Note that I'm not talking about "C++1z requires a whole new ABI" kind
> of break, I'm talking about the standard equivalent to 1.1 -- we ship a bug,
> we fix it, but it requires some small ABI break.
>
> In both of these cases, I think the right thing for the default
> configuration of libc++ is to make the change and take the ABI break. I
> think we should be standard conforming and correct above all else out of the
> box. But I think it is important to provide some mechanism to opt *out* of
> such changes to the ABI, at least in order to control when they arrive in
> systems that are very susceptible to ABI fallout.
>
>
> This seems reasonable to me, and provides an option for vendors to curate
> the ABI based on their particular needs.
>
>
> For (2), my motivation is to chart out a path forward, likely measured in
> years if not decades. This would include the ability to follow any massive
> upheaval in the standard's ABI, as well as the ability to pick up
> improvements which go in under the unstable bucket after a resonable
> interval and in a way that platform vendors are comfortable with. I picked
> "two" specifically for a reason. I think we should be able to create a new
> stable major ABI without changing the default while customers test and
> evaluate it, etc. Then we should be able to switch the default at some point
> without ever touching the old ABI. Finally, after some lengthy period
> (likely also measured in years if not decades) we should be able to remove
> the old ABI and start the process again. When Howard first discussed the
> time scale at which this kind of breaking change could possibly be
> acceptable to customers, he used decades. I'm echoing that, as it matches my
> experience with customers that have hard ABI requirements.
>
>
> I like this goal as well.  I'm concerned about the practicalities of being
> able to manage such divergences in the libc++ project in a coherent way, but
> I think it really comes down to the scale of the changes we are talking
> about as well as the time scale.
>
>
> So, here is my initial proposal for how to handle the above two issues.
>
> First, extend the ABI definitions to include what they already (somewhat)
> do: versions. Specifically, both minor and major versions to handle the
> above two use cases respectively. We already sort-of have this for the
> stable ABI, I just think we should formalize it, document it, and
> incorporate it into the macro naming convention.
>
> The resulting pattern would be that when a bug-fix or minor
> standards-motivated change is introduced which breaks the ABI, it too is
> guarded behind an ABI macro, but that macro is by-default enabled. A new
> high-level macro (i'm trying to avoid picking names here, I suspect Marshall
> will pick better ones than I will) would be introduced to restrict libc++ to
> the prior minor ABI version, and that macro would disable the bug-fix. If at
> some point a contributor wants to build a new major version stable ABI out
> of a sub set of the unstable changes in the tree (and everyone agrees that
> is reasonable to do), then the expected new high-level macros for those
> versions would be introduced, and the per-feature abi-breaking macros would
> be flipped based on them.
>
> Does this make sense to folks?
>
>
> This makes sense to me.
>
>
>
> One goal I have throughout this is that we *have* per-feature ABI-break
> macros, but that users essentially never need to use them. I would like to
> have something closer to version numbers that they interact with in order to
> request specific sets of features that can be documented together.
>
>
> I very much like this goal as well.
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>