<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Oct 5, 2014 at 8:10 PM, Yung, Douglas <span dir="ltr"><<a href="mailto:douglas_yung@playstation.sony.com" target="_blank">douglas_yung@playstation.sony.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi –<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I realize the thread has drifted a little, but I wanted to get back to my original proposal. I would like to make a change to the bitcode file wrapper to include
the version of llvm that produced the bitcode. I would like to write this version into the unused version field that currently exists. Would there be any objections to this change?</span></p></div></div></blockquote><div><br></div><div>If the version field is currently unused, I don't see the harm in filling it in. It probably would be fine to just use it to store the LLVM major version, so that we can detect incompatibilities. I still would be opposed to using a granularity finer than the "intended" breakage cycle (major versions), since that would give the impression that it is "ok" to break compatibility since it can be detected through the version field.</div><div><br></div><div>-- Sean Silva</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Since the original wrapper is only emitted for Darwin targets, I ran an experiment where I took bitcode files produced by the official Apple tools and then
modified the version field to be non-zero. From my simple tests, there seemed to be no problems with existing tools when the version field is non-zero.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Douglas Yung<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> David Blaikie [mailto:<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>]
<br>
<b>Sent:</b> Sunday, September 28, 2014 1:02<br>
<b>To:</b> Alex Rosenberg<br>
<b>Cc:</b> Greg Bedwell; Yung, Douglas; <a href="mailto:cfe-dev@cs.uiuc.edu" target="_blank">cfe-dev@cs.uiuc.edu</a>; <a href="mailto:llvmdev@cs.uiuc.edu" target="_blank">llvmdev@cs.uiuc.edu</a><br>
<b>Subject:</b> Re: [cfe-dev] [LLVMdev] Proposal to add Bitcode version field to bitcode file wrapper<u></u><u></u></span></p>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Sat, Sep 27, 2014 at 11:35 PM, Alex Rosenberg <<a href="mailto:alexr@leftfield.org" target="_blank">alexr@leftfield.org</a>> wrote:<u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">How is this use case different from the LTO-supported toolchains shipped by other vendors such as Apple? Do they have this theoretical problem too?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">If the issue is solely constrained to debug info metadata, then why not use metadata to describe the format/version of the debug info?<u></u><u></u></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">FWIW (I haven't followed the rest of this thread) - that's what we/Apple have done. There's a module flag metadata that specifies the debug info metadata schema version, then the verifier (or some other pass, I forget how it's phrased)
can check if the version matches the current LLVM's debug info metadata version, and if it doesn't match, it strips out all the debug info metadata.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<div>
<p class="MsoNormal"><span style="color:#888888"><br>
Alex<u></u><u></u></span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
On Sep 27, 2014, at 3:19 AM, Greg Bedwell <<a href="mailto:gregbedwell@gmail.com" target="_blank">gregbedwell@gmail.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal">As I understand it, the bitcode compatibility promise doesn't extend as far as debug info metadata (happy to be wrong here!). I think we have a usecase where need to guarantee that debug information from any two arbitrary bitcode files
going into an LTO link will result in the expected/correct debug information going into the resulting ELF file; unless we can be sure that this will always work between bitcode files generated by different versions we'd need some way of flagging up an incompatibility
and providing useful information on the reason to the user.<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">--Greg Bedwell<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">SN Systems - Sony Computer Entertainment Group<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On 27 September 2014 02:24, Sean Silva <<a href="mailto:chisophugis@gmail.com" target="_blank">chisophugis@gmail.com</a>> wrote:<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Fri, Sep 26, 2014 at 5:18 PM, Yung, Douglas <<a href="mailto:douglas_yung@playstation.sony.com" target="_blank">douglas_yung@playstation.sony.com</a>> wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Sorry if I was unclear. There are currently no “known incompatibilities” that I am aware of, although
I fully admit to not being an expert on the topic. The idea is that we add versioning information to the bitcode so that if an issue were discovered, it could be easily detected and dealt with.</span><u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">It sounds like time would be better invested in improving the testing of our bitcode compatibility promise.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#888888"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#888888">-- Sean Silva<u></u><u></u></span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Douglas Yung</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Bob Wilson [mailto:<a href="mailto:bob.wilson@apple.com" target="_blank">bob.wilson@apple.com</a>]
<br>
<b>Sent:</b> Friday, September 26, 2014 16:39<br>
<b>To:</b> Yung, Douglas<br>
<b>Cc:</b> <a href="mailto:llvmdev@cs.uiuc.edu" target="_blank">llvmdev@cs.uiuc.edu</a>;
<a href="mailto:cfe-dev@cs.uiuc.edu" target="_blank">cfe-dev@cs.uiuc.edu</a><br>
<b>Subject:</b> Re: [LLVMdev] Proposal to add Bitcode version field to bitcode file wrapper</span><u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Bitcode backward compatibility, at least for the current major version, is supposed to make this unnecessary. Can you provide more information about what “known incompatibilities”
you’re seeing?<u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Sep 26, 2014, at 2:40 PM, Yung, Douglas <<a href="mailto:douglas_yung@playstation.sony.com" target="_blank">douglas_yung@playstation.sony.com</a>> wrote:<u></u><u></u></p>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Hi,</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">We would like to add a version number to the bitcode wrapper. This feature would allow easier identification of
what compiler produced the bitcode so that known incompatibilities with LTO compilation could be detected. Roughly speaking, this version number would consist of the major, minor and optionally the patch version of the compiler used to produce the bitcode.
The version information would be encoded in 4 bytes, with the first byte representing the major version number, the second byte the minor version number, and the third and fourth bytes optionally encoding the patch version or other information. As to where
to place this information, we are considering two different possibilities for updating the bitcode wrapper specification.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">The first is to simply add a single 32bit wide field at the end of the existing bitcode wrapper format field. This
would result in the new structure looking like this: <br>
<br>
[Magic_{32}, Version_{32}, Offset_{32}, Size_{32}, CPUType_{32}, BitcodeVersion_{32}] <br>
<br>
All of the existing fields would keep their current meanings, and the new field BitcodeVersion is simply appended with the format described in the first paragraph. <br>
<br>
A second idea was to use the existing Version field in the bitcode wrapper format to store the bitcode version information. According to the documentation (<a href="http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format" target="_blank"><span style="color:purple">http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format</span></a>)
this field is currently always set to 0. This would allow us to make use of what is (presumably) an unused field.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">As this is a feature that we feel would be beneficial to the community, we wanted to get feedback on the design
for our upcoming patches. Any thoughts or opinions on this would be greatly appreciated.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Thanks!</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Douglas Yung</span><u></u><u></u></p>
</div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica","sans-serif"">_______________________________________________<br>
LLVM Developers mailing list<br>
</span><a href="mailto:LLVMdev@cs.uiuc.edu" target="_blank"><span style="font-size:9.0pt;font-family:"Helvetica","sans-serif";color:purple">LLVMdev@cs.uiuc.edu</span></a><span style="font-size:9.0pt;font-family:"Helvetica","sans-serif""> </span><a href="http://llvm.cs.uiuc.edu/" target="_blank"><span style="font-size:9.0pt;font-family:"Helvetica","sans-serif";color:purple">http://llvm.cs.uiuc.edu</span></a><span style="font-size:9.0pt;font-family:"Helvetica","sans-serif""><br>
</span><a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank"><span style="font-size:9.0pt;font-family:"Helvetica","sans-serif";color:purple">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</span></a><u></u><u></u></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:LLVMdev@cs.uiuc.edu" target="_blank">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><u></u><u></u></p>
</blockquote>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@cs.uiuc.edu" target="_blank">cfe-dev@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev</a><u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:LLVMdev@cs.uiuc.edu" target="_blank">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><u></u><u></u></p>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@cs.uiuc.edu" target="_blank">cfe-dev@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div></div></div>
</div>
<br>_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@cs.uiuc.edu">cfe-dev@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev</a><br>
<br></blockquote></div><br></div></div>