[PATCH] Bitcode: Add bitcode format compatibility test

[Vedant Kumar]

> Overall I think that the approach Steven was suggesting (and which it seems like Vedant is following) is an improvement over the current state of affairs, though is hardly "exhaustive" (which is part of the confusion in the prevoius discussion; realistically a different approach is needed for being "exhaustive"; but "exhaustive" is not actually a goal for anyone I don't think).

I agree with you that this approach can’t be perfectly exhaustive. However, I think it does buy us a *pretty* good safeguard for a very reasonable amount of overhead.


> It would be interesting to collect some empirical data about how much effort this release-to-release work is, and which, if any, existing compatibility problems are caught. Vedant, could you maybe retroactively do the expected release-to-release work for the last couple LLVM revisions and see what you find, and how much effort it is?

Here’s some data :).

I think a reasonable way of updating compatibility-N.M.ll is by diff’ing LangRef.rst between the relevant commits and adding coverage for all affected areas. For the 3.6 => 3.7 update, we’d have to add support for;

    o ‘-‘ signs in identifier names
    o comdat specifiers for global variables
    o function prologues and personalities
    o dereferenceable_or_null(<n>)
    o the convergent, argmemonly, safestack, and thunk attributes
    o the ‘distinct’ metadata keyword
    o the revamped landingpad instruction  

These aren’t the only changes in the LangRef between the two releases, but they strike me as the only salient ones from a bitcode compatibility point-of-view.

It takes maybe a couple minutes to look over the diff, and maybe an hour to extend the .ll file with the necessary changes. I haven’t caught any compatibility problems (excepting the uselistorder serialization issue I mentioned earlier) — however, this gives us a nice way of avoiding those problems in the first place.

I only went back one revision. I can pull data from previous revisions if you need more convincing :).


> I think that the best way forward for bitcode compatibility is for an interested party to set up a watch in Phabricator over all the bitcode-related parts of the tree, then monitor it like a hawk and ensure that all changes needing tests are tested.


I don’t know about this. This seems pretty involved, and the people in charge of it might get bored. It’s unclear if the extra coverage this might achieve is cost-effective.

vedant