[LLVMdev] Clarification on the backward compatibility promises

[Eric Christopher]

On Thu, Jul 10, 2014 at 11:40 AM, Eric Christopher <echristo at gmail.com> wrote:
> On Thu, Jul 10, 2014 at 11:29 AM, Evan Cheng <evan.cheng at apple.com> wrote:
>>
>>> On Jul 9, 2014, at 9:52 PM, Eric Christopher <echristo at gmail.com> wrote:
>>>
>>> On Wed, Jul 9, 2014 at 9:33 PM, Owen Anderson <resistor at mac.com> wrote:
>>>>
>>>> On Jul 9, 2014, at 3:51 PM, Eric Christopher <echristo at gmail.com> wrote:
>>>>
>>>> On Wed, Jul 9, 2014 at 3:12 PM, Evan Cheng <evan.cheng at apple.com> wrote:
>>>>
>>>>
>>>> On Jun 17, 2014, at 2:10 PM, Eric Christopher <echristo at gmail.com> wrote:
>>>>
>>>> 2. Metadata compatibility. We already had precedence of introducing
>>>> incompatible changes into metadata format in the past within release.
>>>> Should we use relaxes rules for metadata compatibility?
>>>>
>>>>
>>>> I think we have a special case for debug metadata (and should document
>>>> that), but otherwise I think we should hold metadata to the same
>>>> standard as the rest of the IR.
>>>>
>>>>
>>>> The idea with metadata is that it can be removed and everything still
>>>> works. I'm definitely not ready to lock down the debug metadata format
>>>> and I really don't think we should for any of the other uses since
>>>> stripping already works. (Note, I don't consider function attributes
>>>> etc as metadata)
>>>>
>>>>
>>>> We may need to rethink this. If metadata is used only as optimization /
>>>> codegen hints, then yes I agree they can be dropped. But I suspect there is
>>>> a need for metadata that’s *required* for correctness. As LLVM continues to
>>>> gain clients beyond “just” compilers, we will need to be sensitive to their
>>>> needs. I anticipate use of LLVM bitcode files as persistent object format.
>>>>
>>>>
>>>> I think that metadata that's required for correctness should be baked
>>>> into the IR and not be metadata - so if there's something we need for
>>>> correctness we need to come up with an IR extension. See the recent
>>>> comdat work as an example.
>>>>
>>>>
>>>> That’s not really a practical suggestion for clients that aren’t essentially
>>>> clang.  The bar to changing the IR is (correctly) very high, essentially
>>>> unreachable if the client is out-of-tree.
>>>>
>>>
>>> Sure, but they likely have their own metadata format with their own
>>> needs and can keep their own local patches for their out of tree
>>> extensions right? As far as I know we don't have any metadata
>>> extensions in tree that are required for any correctness. If so,
>>> they've explicitly gone against the rules we set for metadata a long
>>> time back:
>>>
>>> http://blog.llvm.org/2010/04/extensible-metadata-in-llvm-ir.html
>>>
>>> Unless I'm missing your point completely of course :)
>>
>> I don’t disagree with this. I am only cautioning against taking an absolutely hardline attitude towards metadata compatibility. As long as we take a pragmatic approach by providing ways for clients to maintain backward compatibility, I don’t think anyone will have problems with the stated policy. We will need to be very careful if / when we propose fundamental changes.
>>
>
> I'm not inclined to change any metadata consumer just for kicks, and
> if we want to change how metadata itself works then that should be a
> more reasoned and careful change.
>
> I.e. I see a user of metadata as something that we don't guarantee
> compatibility for, but the metadata system itself I can see as a more
> formal IR construct.
>
> Make sense?
>

An addendum, versioning metadata once you've got a stable format would
probably be a good idea if you want to worry about future
compatibility. As an example, I can see the loop metadata getting a
version that can be easily read by a consumer and "I can't cope with
this" can just mean it can be ignored etc.

However, that's a maintainability/backward compatibility thing for
each metadata producer to deal with - I removed it from debug info
because we were changing so fast that it was a significant overhead.
That said, if someone wants to change a metadata format to be more
efficient or expressive then it should have a significantly lower bar
than an IR level change.

-eric