[LLVMdev] Using the unused "version" field in the bitcode wrapper (redux)

Fri Oct 31 23:44:19 PDT 2014

Jumping late at these threads, but my opinion with my open source hat on is:

* We want to support backwards compatibility in the bitcode file. In
the past, we have not done a good job at it, but if someone wants to
do the extra testing, any issues found would be considered real bugs
and if reported are very likely to be fixed.

* The backwards compatibility for debug info is far less strict, but
we already have a version info for that. We currently drop it on
mismatch, but it would be trivial to have an option to error instead.

* When we get to 4.1 we might want to drop compatibility with some
older bitcodes from the 3.X era. If we then also want to be able to
reuse enum values (linkage for example), adding a marker/version when
we get to 4.0 might make sense, but it would include only the major
version.

* For the 4.1 transition we would have to detect any bitcode files, so
using the wrapper would not be sufficient, we would need to have the
version/marker somewhere in the main bitcode.

Tanks to Sean for getting me thinking about the 4.0 to 4.1 transition.

On 31 October 2014 21:41, Yung, Douglas
<douglas_yung at playstation.sony.com> wrote:
> Hi Sean,
>
>
>
>> Rafael gave me some of the backstory on this. Basically it is to work
>> around some buggy behavior in the Darwin ar. Adding that on the front of the
>> bitcode file just to get a version doesn't seem
>
>> like a very clean thing to do.
>
>>
>
>> Doug, what other alternatives did you guys consider before settling on
>> this?
>
>>
>
>> As for #2 above, the non-universality of the wrapper makes using the
>> wrapper as a version indicator sort of a non-starter.
>
>
>
> Using the bitcode wrapper was our main idea as we discovered it fairly early
> one when looking for ways to store the version information. Originally we
> wanted to propose adding extra fields at the end of the bitcode wrapper, but
> we found that we only needed the version to be embedded by the compiler, and
> the wrapper already included an unused version field, so we were hoping to
> use that. We also liked that the existing bitcode wrapper was documented,
> implemented and already worked with most if not all of the existing LLVM
> tools.
>
>
>
> One alternative that was suggested was we could create a version section in
> the bitcode that the compiler would then embed the version into. But we did
> not want to take that approach because it would have required the linker to
> parse the bitcode to just to extract the version. That functionality is not
> available (to my knowledge) in our proprietary linker. By using the bitcode
> wrapper, our linker is easily able to extract the version and find the
> bitcode without needing to understand bitcode at all.
>
>
>
> The non-universality of the wrapper could be an issue, but we were going to
> work around that by forcing our compiler to always produce the bitcode
> wrapper for our target, so in our case that would not be an issue. If
> versioning of bitcode files is something that might be useful to other
> teams, it might make sense to just apply it on all bitcode files produced by
> the compiler. In essence, make it required instead of optional as it
> currently stands. The bitcode wrapper would only add 20 bytes to the size of
> the file and most LLVM tools should be able to handle them without any
> changes.
>
>
>
> Douglas Yung
>
>
>
> From: Sean Silva [mailto:chisophugis at gmail.com]
> Sent: Friday, October 31, 2014 21:13
> To: llvmdev at cs.uiuc.edu
> Cc: Yung, Douglas
> Subject: Re: Using the unused "version" field in the bitcode wrapper (redux)
>
>
>
>
>
>
>
> On Fri, Oct 31, 2014 at 6:21 PM, Sean Silva <chisophugis at gmail.com> wrote:
>
> Hi all,
>
>
>
> Doug Yung started a discussion earlier
> (http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-September/077227.html)
> about using the unused "version" field in the bitcode wrapper, and I think
> there was some misunderstanding. I'd like to clarify the motivation.
>
>
>
> The reason we want to add the version field is to easily identify "old"
> bitcode. It is only LLVM version granularity, but that is good enough for
> us. The obvious thing is that we offer compatibility, so why do we need to
> know the bitcode version? There are really two reasons:
>
>
>
> 1. In the short term, LLVM does theoretically provide compatibility.
> However, we at Sony are under a much stronger commitment to our customers
> than the open source project here, so until the test infrastructure is
> beefed up quite a bit to improve this confidence in the backwards
> compatibility promise, the version field is a quick unobtrusive way for us
> to behave correctly in the short term. Beefing up the compatibility testing
> is a separate discussion that everybody realizes is a much larger long-term
> endeavor; we at Sony are glad to help with that. As we prepare for our first
> official SDK release with LTO, where our customers are officially sanctioned
> to feed bitcode to our tools, a solution is needed though. I think that
> existing toolchains with LTO will also benefit from this in the near-term.
>
>
>
> 2. In the long term, it *will* break across major releases. Currently I
> don't think we have a way to identify this. This is a bit longer-term
> perspective, but it will eventually come up and this fixes it.
>
>
>
>
>
> Also, is there a reason why the bitcode wrapper is Darwin-specific?
>
>
>
> Rafael gave me some of the backstory on this. Basically it is to work around
> some buggy behavior in the Darwin ar. Adding that on the front of the
> bitcode file just to get a version doesn't seem like a very clean thing to
> do.
>
>
>
> Doug, what other alternatives did you guys consider before settling on this?
>
>
>
> As for #2 above, the non-universality of the wrapper makes using the wrapper
> as a version indicator sort of a non-starter.
>
>
>
> Looks like I've taken my second U-turn on this proposal :/
>
>
>
>
>
> Can we just always use it and simplify the code path?
>
>
>
> -- Sean Silva
>
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>