[cfe-dev] [LLVMdev] Proposal to add Bitcode version field to bitcode file wrapper

[Renato Golin]

On 28 September 2014 23:10, Robinson, Paul
<Paul_Robinson at playstation.sony.com> wrote:
> | Bitcode backward compatibility, at least for the current major version, is
> supposed to make this unnecessary.

It didn't use to work that way, and I'm not sure we want it at all.


> I think the "at least for the current major version" part is one thing that
> concerns us.  LLVM 4.0 will promise to read LLVM 3.4 bitcode, but LLVM 4.1
> will not, according to my understanding of the current promise.

I've never heard such promises, but even if you're right (that there
is a promise), we cannot enforce it, since we have no tests to make
sure we do.

Right now, the only guarantee I know exists (and it's a new one, from
3.4 onwards) is that minor releases won't break ABI or API
compatibility, which includes IR logic. So 3.4.2 is guaranteed to
parse 3.4 IR but not 3.3 or 3.5.x.


> Smoothly identifying that point and being able to provide an intelligent diagnostic
> seems like goodness. Hard to distinguish "old" bitcode from "broken" bitcode
> without recording version info of some kind, and the sooner we start
> recording the version number the more completely we're able to diagnose the
> situation properly when the time comes.

There are two problems with this:

1. Due to the nature of our development strategy, IR compatibility can
be broken between two releases, which means any two commits within the
same revision can fail to parse (or parse incorrectly) IR from each
other. Do we care about between-release compatibility?

Some people get specific commits, rather than releases, for timing
reasons, for their products, and in doing so, you could get a commit
that is actually IR incompatible with the next major release. If you
care about compatibility, you should increment the IR version every
time something radical changes, which can be multiple times between
the same two releases, or spawn across multiple releases.

IR versioning should be completely independent of major / minor
release cycles. The hard part is to truly detect, and validate, IR
compatibility changes.

2. IR incompatibility is different from metadata incompatibility. If
the IR is incompatible (say we drop or add a new type, or we change
how exceptions are propagated), the new parser will not understand the
old and vice-versa. But if metadata changes, it can still be parsed,
and as David said, if we can't understand it, we just drop it.

If you want your parser to break the least, you'll have to have at
least two version: IR and Debug. Other metadata versioning can be done
individually (since they change at different rates). You may want to
warn on stale metadata status (since it's not an error), but you
should stop on stale IR.


Finally, both problems end up in the same place: how do you validate
this? We'd have to add a new class of tests, and for every new change
in IR/metadata, we'd increase the version number and create a test
that checks old parser+new syntax and old syntax+new parser and makes
sure they fail/warn.

You'd also need to have a table of major releases vs. IR versions, so
that in the error/warning message you tell: please use LLVM M.N
instead.

cheers,
--renato