[PATCH] D12618: Reserve a vendor reserved block ID for bitcode

[Chandler Carruth via llvm-commits]

Let's sort out how the epoch thing might reasonably work first. Until we
understand that, it will be hard to compare options and understand them.

I've not spent a lot of time thinking about this, and so I'm not 100%
confident, but here is my initial thought:

We essentially want to be able to *detect* (but of course, not correct) if
an incoming bitcode file is plausible for our reader to consume and
upgrade. Essentially, if the incoming bitcode file is from a supported
epoch, the only way to see an error is with incompatible builds or a newer
bitcode going to an older llvm[1]. However, if the incoming bitcode file is
from an earlier epoch than supported, it might be read without error but
produce garbage, and so we could definitively diagnose it.

Based on this, the epoch should be incremented when the horizon of support
changes. The policy about backwards compatibility of bitcode clearly spells
out that this occurs on major version increments in upstream. Regardless of
when your releases are, there is a single point which forms the new
baseline of backwards compatibility support. So at some point, when we make
a 4.0 release, we would bump the epoch. Every 4.x release would support
reading the new epoch. The 4.0 release would support reading both the old
*and* new epoch. Make sense?

One particular advantage of having this is that one might keep a
"llvm-upgrade-N" binary (or library) around which is built based on the N.0
release and which can read the *old* epoch but writes the *new* epoch. By
archiving enough of these, you can extend the backwards compatibility as
far as is necessary for your use case. Without the epoch number in the
actual bitcode, you don't know *which* of these to start with in order to
incrementally upgrade to your current bitcode format.


[1] We might want to be more aggressive. I could conceive of us actually
having a proper version. This would change *every time the bitcode format
changes*. Literally every time. Then we could both detect too-new *and*
too-old bitcode: if the epoch isn't supported it is too-new or too-old; if
the epoch *is* supported and the version is past our version, it is
too-new. We could reset the version number to 0 with each bump of the
epoch. While the epochs would clearly correspond with major releases
(although they would be incremented with a normal commit, perhaps the one
that bumps the version number, but really any commit), the version would
not correspond to anything. It would just be incremented every time any
change is made to any part of the bitcode or IR format.


Thoughts?

On Wed, Sep 9, 2015 at 5:04 PM Mehdi Amini <mehdi.amini at apple.com> wrote:

>
> > On Sep 9, 2015, at 4:59 PM, Rafael Espíndola <rafael.espindola at gmail.com>
> wrote:
> >
> > On 9 September 2015 at 18:36, Chandler Carruth <chandlerc at google.com>
> wrote:
> >> Two points.
> >>
> >> One, we have the idea of backwards compat until a major version bump. If
> >> we're going to do this, we should write a record with the epoch of
> >> compatibility as a monotonically increasing integer IMO. It seems crazy
> to
> >> not have a way to check that the bitcode file is at least *plausible* to
> >> auto-upgrade. That seems reasonable for the open source project in
> general.
> >>
> >> Second, unless there is a pressing need to handle this in an opaque
> >> vendor-specific way, I'd love to find an approach that could even be
> useful
> >> to open source developers. Let me propose an alternative, and maybe you
> can
> >> tell me where it falls over: we could add both a hard compatibility
> epoch
> >> counter, and a version string. The version string would literally
> contain a
> >> munged string just like 'clang --version' does. We can stash SVN
> revisions,
> >> release numbers, and whatever else your heart desires in there. Then,
> when
> >> the bitcode reader fails for some reason, as part of a more helpful
> error
> >> message, we use these strings in the two bitcode files to try to form a
> >> helpful error message:
> >>
> >> "ld: Error parsing bitcode file in libFooBar.a (generated by LLVM rNNNN,
> >> read by LLVM rMMMM)" or some such.
> >>
> >> Thoughts?
> >
> > I like the idea of an unstructured version string that is printed once
> > an error is found by some other way.
> >
> > Note that we don't have to add the epoch right now, since its absence
> > can be read as epoch 0.
> >
> > One issue with the epoch idea is that we would then be dropping big
> > version ranges at a time. Once we get to 4.1 we can choose to drop
> > compatibility with 3.X. If we want to drop with all of 3.X the epoch
> > works fine, but what if only revisions < 3.5 are problematic enough
> > that we want to drop it? Should the epoch be updated more often than
> > that?
>
>
> Our internal prototype of the vendor block contains a “magic” at the
> beginning of the block, followed by the version number, followed by a
> string to be user friendly.
> It is not very different from what the epoch would provide, but I’m not
> sure I understand the proposal about when to increment it?
>
> Also our release does not match the open-source LLVM release, it is not
> clear how the epoch is supposed to interact on this aspect?
>
>
> Thanks,
>
> —
> Mehdi
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20150910/fa61db66/attachment.html>