[LLVMdev] Using the unused "version" field in the bitcode wrapper (redux)

[Robinson, Paul]

It should report that the
particular new feature is not support. In this case, something like
"unknown 'g' flag in datalayout".

Currently that would hit an llvm_unreachable().  How should we re-engineer this to provide sensible error recovery in an LTO context, while continuing to provide the die-horribly fallback behavior that it currently (quite reasonably) provides in all other contexts?

And that's just one example of a place that would have to be re-engineered this way, to provide the necessary degree of future-proofing across some indeterminate number of modules within LLVM that never expected to see this particular problem.

Again, *any* solution involving the bitcode wrapper is not interesting
to the open source project since it is not required.

I fail to see why all that re-engineering effort IS "required" to support the use-cases we've provided, and why future-proofing a large body of software that quite honestly was never designed for it and has genuinely very few reasons why it should be.

If we're going to support bitcode as a long-term storage format, everything that pulls information out of bitcode MUST be future-proofed.  Doing it piece by piece is a LOT of engineering effort to make LLVM user-friendly in this way; it was never intended to have that degree of self-defense.  The wrapper provides that as a simple low-cost high-reliability mechanism, and allows the LLVM community to continue not having to worry about future-proofing.  It adds the necessary layer that provides the necessary protection, to those very few vendors who actually need this kind of feature.  And it costs *nothing* for anyone else, because nobody else will actually have the wrapper.

If you're asserting that "bitcode as a long-term storage format" has been completely rejected by the open source project, well, that's a different statement, and not one that I have been hearing.

Sony is definitely moving in the direction of supporting bitcode as a long-term format, therefore we DO require future-proofing, and the wrapper is the cheapest engineering solution to get that.
--paulr

From: Sean Silva [mailto:chisophugis at gmail.com]
Sent: Monday, November 24, 2014 4:19 PM
To: Rafael Espíndola
Cc: Robinson, Paul; Bruce Hoult; Yung, Douglas; llvmdev at cs.uiuc.edu
Subject: Re: [LLVMdev] Using the unused "version" field in the bitcode wrapper (redux)



On Mon, Nov 24, 2014 at 7:53 AM, Rafael Espíndola <rafael.espindola at gmail.com<mailto:rafael.espindola at gmail.com>> wrote:
> Right, that version number is used to resolve *ambiguities* in how to
> interpret some chunk of bitcode.  It is not a generic bitcode version
> scheme, because most bitcode format changes involve things like adding
> new operands or opcodes, which are easily identified without needing
> an explicit version number.

That is what it is used for at the moment. It is is just a number, we
can increment it as often as we want.

> The scenario I am most concerned about is this:
>
> - We as a vendor publish toolchain #12 based on SVN r250000.
> - During subsequent LLVM development, changes happen (!).
>   For example, a new key letter 'g' in the Data Layout.  This is
>   not a bitcode ambiguity so MODULE_CODE_VERSION is unchanged.
> - We as a vendor publish toolchain #13 based on SVN r300000.
> - Some middleware provider publishes libIncrediblyUseful.bc built
>   using spiffy new toolchain #13.
> - Some hapless game developer tries to use libIncrediblyUseful.bc
>   but is still on toolchain #12. This causes an error during some
>   LTO build phase, of course; the question is, what kind of error
>   and how does Hapless Game Developer know what to do?

In summary. An old tool reading new bitcode. It should report that the
particular new feature is not support. In this case, something like
"unknown 'g' flag in datalayout".

> This does *nothing* for Hapless Game Developer.  HGD wants to see
> "this bitcode file was generated by a newer version, I don't understand
> how to interpret it" because _that's_ the actual problem.

We can probably add a note saying "newer or corrupt BC".

This makes sense to me. I think it is pretty safe to assume that any "invalid bitcode" that an end-user would get their hands on is just because the bitcode is from a newer version.

-- Sean Silva


> Proposed solution:
>
> Whether to emit a bitcode wrapper becomes a target-dependent predicate.

Again, *any* solution involving the bitcode wrapper is not interesting
to the open source project since it is not required.

Cheers,
Rafael
_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu>         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20141125/3b83bcd4/attachment.html>