[PATCH] Bitcode: Add bitcode format compatibility test

[Sean Silva]

On Wed, Jul 22, 2015 at 3:36 PM, Reid Kleckner <rnk at google.com> wrote:

> I don't think everyone has been as vigilant. Most people aren't checking
> in old bitcode when they switch the writer to produce a new record.
>
> I thought the plan of record for doing this compatibility testing was to
> have one compatibility.ll file which exercises all IR features, and then at
> every release point we assemble that IR file to compatibility-N.M.ll and
> add a RUN line for it. If your IR is exhaustive, could it serve as the
> basis for this going forwards?
>

I don't recall us ever settling on anything. There was a thread `[LLVMdev]
[RFC] Exhaustive bitcode compatibility tests for IR features` where Steven
Wu proposed something similar, but that discussion tailed off. Key domain
experts in bitcode compatibility like the RenderScript folks didn't
participate strongly.

Overall I think that the approach Steven was suggesting (and which it seems
like Vedant is following) is an improvement over the current state of
affairs, though is hardly "exhaustive" (which is part of the confusion in
the prevoius discussion; realistically a different approach is needed for
being "exhaustive"; but "exhaustive" is not actually a goal for anyone I
don't think).

It would be interesting to collect some empirical data about how much
effort this release-to-release work is, and which, if any, existing
compatibility problems are caught. Vedant, could you maybe retroactively do
the expected release-to-release work for the last couple LLVM revisions and
see what you find, and how much effort it is?

-- Sean Silva


>
> On Wed, Jul 22, 2015 at 3:10 PM, David Blaikie <dblaikie at gmail.com> wrote:
>
>> Not sure about everyone else, but I tried to add regression tests
>> whenever I've changed the bitcode format, so I'm not sure how much it's
>> necessary to have a generic test like this. I could imagine coverage isn't
>> great & this might help - if it's easy to quantify, that might be
>> interesting to know.
>>
>> On Wed, Jul 22, 2015 at 2:02 PM, Vedant Kumar <vsk at apple.com> wrote:
>>
>>> Successive versions of LLVM should retain the ability to parse bitcode
>>> generated by old releases of the compiler. To that end, I've attached a
>>> patch which contains a bitcode format compatibility test. It's set up as
>>> a
>>> regression test. It achieves good coverage of the 3.7 LangRef (excepting
>>> a
>>> bunch of intrinsics).
>>>
>>> The test consists of two files:
>>>
>>>     o test/Bitcode/compatibility-3.7.ll
>>>     o test/Bitcode/compatibility-3.7.ll.bc
>>>
>>> We should always be able to disassemble the bitcode file and exactly
>>> recover
>>> the original IR.
>>>
>>> I've tried to stress as much of the LangRef as possible. I'm still trying
>>> to find a good way of testing format compatibility for intrinsics.
>>>
>>> I'd have preferred to split this patch up into smaller ones, but since it
>>> contains a binary file I thought it'd be best to get it done in one
>>> shot. I
>>> don't have commit rights, so I'd appreciate someone taking a look at the
>>> patch and getting it into trunk :).
>>>
>>> thanks!
>>> vedant
>>>
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>>>
>>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>>
>>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20150723/160b37a4/attachment.html>