[LLVMdev] [PATCH] -emit-bitcode-version

[Joe Abbey]

Sent from my iPhone

On Nov 9, 2012, at 4:55 PM, "Chris Lattner" <clattner at apple.com> wrote:

> On Nov 8, 2012, at 7:09 AM, Joe Abbey <jabbey at arxan.com> wrote:
>>>> The problem we face is that bitcode changes, and when it does… future versions
>>>> can read it, but past versions are left in the lurch.  For instance LLVM 3.2svn
>>>> can BitcodeReader from LLVM 3.1, but LLVM 3.1 can't BitcodeReader LLVM 3.2
>>>> (after r165739.)   There was an element of this patch which would have helped
>>>> enable bitcode compatibility (use-abs-operands), but alas it was not committed.
>>> 
>>> can't you use a combination of llvm-dis (from LLVM 3.2) and llvm-as (from LLVM
>>> 3.1) to convert the bitcode?
>>> 
>>> Ciao, Duncan.
>>> 
>> 
>> I could for my immediate headache.  My concern is that there will be future updates to the bitcode representation that may not work in this flow.  That's impossible to predict, and llvm-dis -> .ll -> llvm-as is one path which certainly seems attractive.  By providing an internal switch to emit bitcode, I think we save a step and have a method for future proofing.
>> 
>> I'll be talking about this in a lightning talk, perhaps others have thoughts on this.
> 
> 
> Hi Joe,
> 
> As you mentioned in your talk, I'm sympathetic to your desires, but highly skeptical of this approach - for a number of reasons.  Off the top of my head:
> 
> 1. This will (over time) accrete a ton of old gunk in the bitcode writer, and also slow down progress.
> 
> 3. The open source project as a whole benefits from "forcing" users of LLVM to "stay up" on mainline… which this feature acts in opposition to.
> 
> -Chris
> 

Hi Chris!

Thanks for the reply.  The talk has  been quite beneficial in connecting me with Nico Weber and Stephen Hines on this problem space.  Thanks again for enabling the community with the developers' conference, and trying out lightning talks. 

> 1. This will (over time) accrete a ton of old gunk in the bitcode writer, and also slow down progress.

Well there's actually three layers of compatibility, possibly more. 

1. The LLVM API

The LLVM API is simply a cost I have to eat every now and then when moving from version n to n+1.  And after our leap from 2.8 -> 3.0 the API layer has been generally stable.  A couple refactors here and there in recent days, like Attributes, but nothing major.  I'm trying to assess our internal usage and ponder an API layer which abstracts LLVM from our tool.  Still weighing pros and cons on that.  

2. IR

Since 3.0 it's been very stable, there was a blip in 3.1 to 3.2 where switch records changed but overall not the problem I faced with 2.8 to 3.0.  I don't see any way to provide IR compatibility.  If it happens to change we'll just have to roll with those punches. Pretty sure even an llvm-dis to llvm-as fails us. 

3. Bitcode encoding

This one is a bit of a head scratcher.  The encoding offered 0 new features, and was a relatively arbitrary change. Yeah it saves 15% size, but it's completely optional.  And for a couple users with unique use-cases it breaks reading 3.2 written code for not a whole lot of gain. 

Turning this on and off doesn't add gunk or an unmaintainable path.  Maybe in a few months we try something more comprehensive like Espindola's SLEB suggestion.  What about the original use-abs-operands?  Do you think that what more reasonably scope this?

And so properly scoping my proposal I see that I don't want IR compatibility as much as I want encoding options.  So with that in mind maybe I need to go back to the drawing board and propose a BitcodeEncoder?BitcodeCompressor?

> 2. The use case for it is also very narrow (in contrast to having the *reader* handle old files, which many scenarios benefit from).

Yep can't argue that. And I really like that reader is backwards compatible. 

> 3. The open source project as a whole benefits from "forcing" users of LLVM to "stay up" on mainline… which this feature acts in opposition to.

Ah, I hadn't considered that.

Thanks for taking the time to add your input to this.  I'll go back to the drawing board with these points in mind. 

Cheers

Joe