<div dir="ltr"><div>I'm now thinking that there's a bug in either obj2yaml or yaml2obj, because if I run just those two tools on my codebase it generates yaml files that can't be decoded, will try now to not add any section to the obj file in llvm-objcopy to see if I can link with obj files that I rewrite (but without adding symbols or sections).</div><div><br></div><div>One of the bugs that do annoy me is that the timedatestamp is not carried when obj2yaml writes a file, and that the layout function on yaml2coff does generate different indexes to the sections, none that look wrong, but it seems that it leaves some padding, but I didn't have time to look to closely at why.<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jan 26, 2018 at 6:52 PM, Zachary Turner <span dir="ltr"><<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hmm, ok. In that case let me try again without my local changes. Maybe they are getting in the way :-/<div><div class="h5"><br><br><div class="gmail_quote"><div dir="ltr">On Fri, Jan 26, 2018 at 9:51 AM Leonardo Santagada <<a href="mailto:santagada@gmail.com" target="_blank">santagada@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">it is identical to me... wierd.<br></div><div class="gmail_extra"></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jan 26, 2018 at 6:49 PM, Zachary Turner <span dir="ltr"><<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">(Ignore the fact that my hashes are 8 byte in the "good" file, this is due to some local changes I've been experimenting with)</div><div class="m_962751798852437708m_-4300639605599606656HOEnZb"><div class="m_962751798852437708m_-4300639605599606656h5"><br><div class="gmail_quote"><div dir="ltr">On Fri, Jan 26, 2018 at 9:48 AM Zachary Turner <<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I did this:<div><br></div><div>// a.cpp</div><div><div>static int x = 0;</div><div>void b(int);<br></div><div>void a(int) {<br></div><div> if (x)</div><div> b(x);</div><div>}</div><div>int main(int argc, char **argv) {<br></div><div> a(argc);</div><div> return x;</div><div>}</div></div><div><br></div><div><br></div><div>clang-cl /Z7 /c a.cpp /Foa.noghash.obj</div><div>clang-cl /Z7 /c a.cpp -mllvm -emit-codeview-ghash-section /Foa.ghash.good.obj</div><div>llvm-objcopy a.noghash.obj a.ghash.bad.obj</div><div>obj2yaml a.ghash.good.obj > a.ghash.good.yaml</div><div>obj2yaml a.ghash.bad.obj > a.ghash.bad.yaml</div><div><br></div><div>Then open these 2 yaml files up in a diff viewer. It looks like the hashes aren't getting emitted at all. For example, in the good yaml file I see this:</div><div><br></div><div><div> - Name: '.debug$H'</div><div> Characteristics: [ IMAGE_SCN_CNT_INITIALIZED_<wbr>DATA, IMAGE_SCN_MEM_DISCARDABLE, IMAGE_SCN_MEM_READ ]</div><div> Alignment: 4</div><div> SectionData: <wbr>C5C93301000001005549419E78044E<wbr>3896D45CD7009428758BE4A1E2B3E0<wbr>22BA267DEE221F5C42B17BCA182AF8<wbr>4584814A8B5E7E3FB17B397A9E3DEA<wbr>75CD5627</div><div> GlobalHashes: </div><div> Version: 0</div><div> HashAlgorithm: 1</div><div> HashValues: </div><div> - 5549419E78044E38</div><div> - 96D45CD700942875</div><div> - 8BE4A1E2B3E022BA</div><div> - 267DEE221F5C42B1</div><div> - 7BCA182AF8458481</div><div> - 4A8B5E7E3FB17B39</div><div> - 7A9E3DEA75CD5627</div></div><div> - Name: .pdata <br></div><div><br></div><div>And in the bad yaml file I see this:</div><div><div> - Name: '.debug$H'</div><div> Characteristics: [ IMAGE_SCN_CNT_INITIALIZED_<wbr>DATA, IMAGE_SCN_MEM_DISCARDABLE, IMAGE_SCN_MEM_READ ]</div><div> Alignment: 4</div><div> SectionData: C5C9330100000000</div><div> GlobalHashes: </div><div> Version: 0</div><div> HashAlgorithm: 0</div><div> - Name: .pdata</div></div><div><br></div><div>Don't focus too much on trying to figure out weird linker errors. Just get the output of obj2yaml to be identical when run under a diff utility, then everything should work fine.</div></div><br><div class="gmail_quote"><div dir="ltr">On Fri, Jan 26, 2018 at 7:27 AM Leonardo Santagada <<a href="mailto:santagada@gmail.com" target="_blank">santagada@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>I'm so close I can almost smell it :)</div><div><br></div><div>I know how bad the code looks, I don't intend to submit this, but if you want to try it out its at: <a href="https://gist.github.com/santagada/544136b1ee143bf31653b1158ac6829e" target="_blank">https://gist.github.com/<wbr>santagada/<wbr>544136b1ee143bf31653b1158ac682<wbr>9e</a><br></div><div><br></div><div>I'm seeing: lld-link.exe: error: duplicate symbol: "<redacted_unmangled>" (<redacted>) in <internal> and in <redacted_filename>.obj, looking at the .yaml dump the symbols are all similar to this:<br></div><div><br></div><div><div><div><span style="color:rgb(0,0,0)"> - </span><span style="color:rgb(128,0,0)">Name</span><span style="color:rgb(0,0,0)">: </span><redacted></div><div><span style="color:rgb(0,0,0)"> </span><span style="color:rgb(128,0,0)">Value</span><span style="color:rgb(0,0,0)">: </span><span style="color:rgb(9,136,90)">0</span></div><div><span style="color:rgb(0,0,0)"> </span><span style="color:rgb(128,0,0)">SectionNumber</span><span style="color:rgb(0,0,0)">: </span><span style="color:rgb(9,136,90)">0</span></div><div><span style="color:rgb(0,0,0)"> </span><span style="color:rgb(128,0,0)">SimpleType</span><span style="color:rgb(0,0,0)">: </span><span style="color:rgb(0,0,255)">IMAGE_SYM_TYPE_NULL</span></div><div><span style="color:rgb(0,0,0)"> </span><span style="color:rgb(128,0,0)">ComplexType</span><span style="color:rgb(0,0,0)">: </span><span style="color:rgb(0,0,255)">IMAGE_SYM_DTYPE_FUNCTION</span></div><div><span style="color:rgb(0,0,0)"> </span><span style="color:rgb(128,0,0)">StorageClass</span><span style="color:rgb(0,0,0)">: </span><span style="color:rgb(0,0,255)">IMAGE_SYM_CLASS_WEAK_EXTERNAL</span></div><div><span style="color:rgb(0,0,0)"> </span><span style="color:rgb(128,0,0)">WeakExternal</span><span style="color:rgb(0,0,0)">: </span></div><div><span style="color:rgb(0,0,0)"> </span><span style="color:rgb(128,0,0)">TagIndex</span><span style="color:rgb(0,0,0)">: </span><span style="color:rgb(9,136,90)">134</span></div><div><span style="color:rgb(0,0,0)"> </span><span style="color:rgb(128,0,0)">Characteristics</span><span style="color:rgb(0,0,0)">: </span><span style="color:rgb(0,0,255)">IMAGE_WEAK_EXTERN_SEARCH_<wbr>LIBRARY</span></div></div></div></div><div class="gmail_extra"></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 25, 2018 at 8:01 PM, Zachary Turner <span dir="ltr"><<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I haven't really dabbled in this part of the COFF format personally, so hopefully I'm not leading you astray :)<div><br></div><div>But I checked the code for coff2yaml, and I see this:</div><div><br></div><div><div> } else if (Symbol.isSectionDefinition()) {</div><div> // This symbol represents a section definition.</div><div> assert(Symbol.<wbr>getNumberOfAuxSymbols() == 1 &&</div><div> "Expected a single aux symbol to describe this section!");</div></div><div><div> const object::coff_aux_section_<wbr>definition *ObjSD =<br></div><div> reinterpret_cast<const object::coff_aux_section_<wbr>definition *>(</div><div> AuxData.data());</div></div><div><br></div><div>So it looks like you need exactly 1 aux symbol for each section symbol.</div><div><br></div><div>I then scrolled up in this function to figure out where AuxData comes from, and it comes from COFFObjectFile::<wbr>getSymbolAuxData. I think that function holds the clue to what you need to do. It looks like you need to set coff::symbol::<wbr>NumberOfAuxSymbols to 1, and then there is a comment in getSymbolAuxData which says:</div><div><br></div><div><div> // AUX data comes immediately after the symbol in COFF</div><div> Aux = reinterpret_cast<const uint8_t *>(Symbol.getRawPtr()) + SymbolSize;</div></div><div><br></div><div>So I think you just need to write the bytes immediately after the coff::symbol. The thing you need to write looks like a coff::coff_aux_section_<wbr>definition structure.</div><div><br></div><div>For the CheckSum, look at WinCOFFObjectWriter::<wbr>writeSection. It looks like its a CRC32 of the actual section contents, which you can generate with a couple of lines of code:</div><div><br></div><div><div> JamCRC JC(/*Init=*/0);</div><div> JC.update(DebugHContents);</div><div> AuxSymbol.CheckSum = JC.getCRC();</div></div><div><br></div><div>Hope this helps</div></div></blockquote></div></div></blockquote></div></blockquote></div></div></div></blockquote></div></div><div class="gmail_extra">
</div></blockquote></div></div></div></div>
</blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><br>Leonardo Santagada</div>
</div>