<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On May 8, 2015, at 4:08 PM, Adrian Prantl <<a href="mailto:aprantl@apple.com" class="">aprantl@apple.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On May 7, 2015, at 11:25 AM, Adrian Prantl <<a href="mailto:aprantl@apple.com" class="">aprantl@apple.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On May 6, 2015, at 3:27 PM, David Blaikie <<a href="mailto:dblaikie@gmail.com" class="">dblaikie@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Wed, May 6, 2015 at 3:20 PM, Adrian Prantl <span dir="ltr" class=""><<a href="mailto:aprantl@apple.com" target="_blank" class="">aprantl@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><div class=""><div class="h5"><blockquote type="cite" class=""><div class="">On May 6, 2015, at 3:05 PM, David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank" class="">dblaikie@gmail.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Wed, May 6, 2015 at 3:02 PM, Adrian Prantl <span dir="ltr" class=""><<a href="mailto:aprantl@apple.com" target="_blank" class="">aprantl@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On May 6, 2015, at 2:55 PM, David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank" class="">dblaikie@gmail.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Wed, May 6, 2015 at 2:50 PM, Adrian Prantl <span dir="ltr" class=""><<a href="mailto:aprantl@apple.com" target="_blank" class="">aprantl@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On May 6, 2015, at 2:44 PM, David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank" class="">dblaikie@gmail.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Wed, May 6, 2015 at 2:41 PM, Adrian Prantl <span dir="ltr" class=""><<a href="mailto:aprantl@apple.com" target="_blank" class="">aprantl@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On May 6, 2015, at 2:38 PM, David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank" class="">dblaikie@gmail.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Wed, May 6, 2015 at 2:25 PM, Adrian Prantl <span dir="ltr" class=""><<a href="mailto:aprantl@apple.com" target="_blank" class="">aprantl@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On May 5, 2015, at 1:23 PM, David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank" class="">dblaikie@gmail.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Tue, May 5, 2015 at 11:52 AM, Adrian Prantl <span dir="ltr" class=""><<a href="mailto:aprantl@apple.com" target="_blank" class="">aprantl@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On May 5, 2015, at 9:42 AM, David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank" class="">dblaikie@gmail.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Tue, May 5, 2015 at 8:35 AM, Adrian Prantl <span dir="ltr" class=""><<a href="mailto:aprantl@apple.com" target="_blank" class="">aprantl@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">David, I’m not entirely sure if you are ok with the naming of the field as is.. is this good to commit from your side?<br class=""></blockquote><div class=""><br class="">It's a bit generic/misleading given that for the common case of plain Fission the DWO ID is generated based on the DWARF in the backend, so it doesn't make sense to carry it as part of the metadata because it cannot be known then.<br class=""><br class="">What's the particular part of the module debug info this will be used for? Your comment says it's specifically for the skeleton side (so we'll take the module ID in the frontend and emit a skeleton CU into the metadata)? What about the concrete side for the module's debug info? (it'll need the same dwo ID)<br class=""></div></div></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">I casually mentioned that in one of the emails in the other thread; the idea is that when emitting a clang module the backend computes the regular DWARF DWOId for it. When emitting an external type reference, the frontend is using libDebugInfo to extract the DWOId from the module and emits a skeleton CU with pcm filename and DWOId.</div></div></div></blockquote><div class=""><br class="">That seems a bit heavy... I thought we discussed with Richard Smith that the module has an actual identifier we could use here?</div></div></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">I realize now that I haven't communicated this problem back to the list after discovering it, and I apologize for that: It is true that each type has a unique TypeId in the module,</div></div></div></blockquote><div class=""><br class="">I don't think I was suggesting using a module based type id - I figured we'd use the type IDs that type units already use (a hash of the fully qualified name - same as the type uniquing in debug info metadata that was implemented years ago (well the type uniquing uses the mangled name itself - I just hashed that to create the type identifier for the comdat section and type signature for type units))<br class=""><br class="">In any case, I wasn't asking about the type identifier, I was asking about the module identifier (the DWO ID).<br class=""></div></div></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">Oh. The module does have a module id, but that value is the result of running the random number generator in a loop until we hit a non-zero value, so it’s not exactly resilient against rebuilding the module.</div></div></div></blockquote><div class=""><br class="">OK, that sounds like the module ID that Chandler came across - apparently it was added to work around instability in the module output. Chandler's since fixed the module instability and removed this under a flag (I guess the flag is still on by default for implicit modules builds).<br class=""><br class="">Though I'm confused - wouldn't rebuilding the module /want/ to change the identifier so these don't match anymore? The module might have different things compared to what the original code was built against... <br class=""></div></div></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">Remember that on Darwin the module cache is this (sort of) global shared space that in theory could have been cleared by the time llvm-dsymutil wants to link the debug info.</div><div class="">In that case I want to be able to rebuild the module and have the exact same DWO ID or error.</div></div></div></blockquote><div class=""><br class="">Using the DWO ID seems insufficient if you want the ASTs to all match up - DWARF doesn't have all the information in the AST, it's lossy. Which means you could have a change to the ASTs that makes the modules incompatible that wouldn't be reflected in a hash of the DWARF.<br class=""></div></div></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">That’s ok as far as dsymutil is concerned. Dsymutil itself only cares about DWARF and is happy as long as it can resolve all external type references and replace them with DW_FORM_ref_addrs.</div><div class="">[The dSYM bundle created by dsymutil would still contain the TAG_module, so if a debugger later decides to import the module it could still run into problems, but as far as dsymutil is concerned the DWARF is all that matters.]</div></div></div></blockquote><div class=""><br class="">OK - but for LLDB you're going to want to use some kind of identifier that's resilient to benign rebuilds with AST accuracy, perhaps? So we could use that number for the DWO ids?<br class=""><br class="">I'd like to seriously consider options that would avoid parsing the DWARF in a module during compilation to retrieve the DWO ID to reference from the .o debug info... that just seems a bit heavy weight, as I said.<br class=""></div></div></div></div></div></blockquote><div class=""><br class=""></div></div></div><div class="">That’s a valid concern. It would be an easier sell if the DWO ID were part of a header. Keep in mind that we will have to open the module with libObject anyway to get to the AST section </div></div></div></blockquote><div class=""><br class="">I assume we're just going to merge the module object file and the debug info object file? Or is there something else in mind? (I'd like to avoid having to read from/write to either after they've been written out - (& if we could avoid writing them to separate files then merging, that'd be a bonus, certainly))<br class=""></div></div></div></div></div></blockquote><div class=""><br class=""></div><div class="">Can you clarify what you mean with module object file vs debug info object file, to ensure we’re talking about the same thing?</div><div class="">What I meant was that in the .pcm file there is a .clang_ast section that contains the serialized AST and also (among others) a .debug_info section that contains the debug info for the module. The LLVMModuleProvider (there’s a patch waiting on Richard’s approval floating around in cfe-commits) that is responsible for loading a clang module from disk opens the .pcm file hands a MemoryBuffer with the .clang_ast section to clang’s module deserializer while also extracting the DWOid from the .debug_info section. There is no other parsing of debug info besides getting to the DWOId.</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">We could eliminate the need for linking against DebugInfo by emitting a .debug_cu_index section into the .pcm with only a single entry and have the frontend parse that first hash table entry to</div></div></div></div></blockquote><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><div class="">extract the dwo id.</div></div></div></div></blockquote><div><br class=""></div><div>I’ve been thinking some more about this and I’m no longer opposed to just straight using the clang module id as the dwo_id. It has the big advantage of being very cheap to extract from the module, plus we don’t need to compute DWARF hash then. Realizing that it is quite improbable that the clang module cache is purged while building a project, we might as well go ahead and use the existing random module hash and then fix clang’s module hash to become deterministic if that should turn out to be problem in practice.</div><div><br class=""></div><div>-- adrian</div><div><br class=""></div><blockquote type="cite" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><div class=""><br class=""></div><div class="">-- adrian</div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><div class="">and the DWARF parsing would be restricted to the first abbreviation entry and the first compile unit DIE and no relocations necessary.</div><div class=""><div class="h5"><div class=""><br class=""></div><div class="">-- adrian</div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><br class="">- David<br class=""> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><span class=""><font color="#888888" class=""><div class="">-- adrian</div></font></span><div class=""><div class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><div class="">In other situations the requirements are less strict. If I’m just debugging without a dSYM I could imagine being happy with a best effort after rebuilding a module that is “close enough”.</div><span class=""><font color="#888888" class=""><div class=""><br class=""></div><div class="">-- adrian</div></font></span><div class=""><div class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><span class=""><font color="#888888" class=""><div class=""><br class=""></div><div class="">-- adrian</div></font></span><div class=""><div class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><br class=""> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><div class=""> but this is just a integer counter that is not necessarily stable across rebuilds. With modules becoming more deterministic, an id should survive a rebuild of the module, but if the header file the module is built with is altered in any way, the ids could be shuffled. It is more resilient (but still fast enough) to look up a type by decl context + name.</div><span class=""><font color="#888888" class=""><div class=""><br class=""></div><div class="">-- adrian</div></font></span><div class=""><div class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><span class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class="">While I'm all for incremental development, a little context of the path you've got in mind may be helpful.<br class=""></div></div></div></div></div></blockquote><div class=""><br class=""></div></span>Right, the examples in cfe-dev were always DWARF and never showed the IR:</div><div class="">A skeleton CU emitted by the frontend would be represented as a</div><div class=""> !0 = DICompileUnit(file: MDFile(“/path/to”, “module.pcm”), dwoId: ABCD1234)</div><div class="">and is emitted as a compile_unit with an AT_dwo_name and an AT_dwo_id by the backend.</div></div></blockquote><div class=""><br class="">OK.<br class=""> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><span class=""><font color="#888888" class=""><div class=""><br class=""></div></font></span><div class=""><span class=""><font color="#888888" class="">-- adrian</font></span><div class=""><div class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><br class="">- David<br class=""> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span class=""><font color="#888888" class=""><br class="">
-- adrian<br class="">
</font></span><div class=""><div class=""><br class="">
> On May 4, 2015, at 9:04 PM, Duncan P. N. Exon Smith <<a href="mailto:dexonsmith@apple.com" target="_blank" class="">dexonsmith@apple.com</a>> wrote:<br class="">
><br class="">
><br class="">
>> On 2015 May 4, at 20:17, Adrian Prantl <<a href="mailto:aprantl@apple.com" target="_blank" class="">aprantl@apple.com</a>> wrote:<br class="">
>><br class="">
>> Now with autoupgrade testcase.<br class="">
>><br class="">
>><br class="">
>> <a href="http://reviews.llvm.org/D9488" target="_blank" class="">http://reviews.llvm.org/D9488</a><br class="">
>><br class="">
>> Files:<br class="">
>> include/llvm/IR/DIBuilder.h<br class="">
>> include/llvm/IR/DebugInfoMetadata.h<br class="">
>> lib/AsmParser/LLParser.cpp<br class="">
>> lib/Bitcode/Reader/BitcodeReader.cpp<br class="">
>> lib/Bitcode/Writer/BitcodeWriter.cpp<br class="">
>> lib/IR/AsmWriter.cpp<br class="">
>> lib/IR/DIBuilder.cpp<br class="">
>> lib/IR/DebugInfoMetadata.cpp<br class="">
>> lib/IR/LLVMContextImpl.h<br class="">
>> test/Assembler/mdcompileunit.ll<br class="">
>> test/Bitcode/DICompileUnit-upgrade.test<br class="">
>> test/Bitcode/Inputs/DICompileUnit-no-DWOId.bc<br class="">
>> unittests/IR/MetadataTest.cpp<br class="">
>><br class="">
>> EMAIL PREFERENCES<br class="">
>> <a href="http://reviews.llvm.org/settings/panel/emailpreferences/" target="_blank" class="">http://reviews.llvm.org/settings/panel/emailpreferences/</a><br class="">
>> <D9488.24927.patch><br class="">
><br class="">
> LGTM with some changes to the autoupgrade test.<br class="">
><br class="">
>> Index: test/Bitcode/DICompileUnit-upgrade.test<br class="">
>> ===================================================================<br class="">
>> --- /dev/null<br class="">
>> +++ test/Bitcode/DICompileUnit-upgrade.test<br class="">
><br class="">
> Can you be more descriptive? Perhaps dicompileunit-no-dwoid.ll.<br class="">
> (Feel free to use the camel-case filename; I personally avoid<br class="">
> capitals but clearly there's no harm.)<br class="">
><br class="">
>> @@ -0,0 +1,6 @@<br class="">
>> +RUN: llvm-dis %p/Inputs/DICompileUnit-no-DWOId.bc -o - | FileCheck %s<br class="">
><br class="">
> %p was undocumented last I checked. I recommend the documented %S<br class="">
> when this sort of functionality is necessary.<br class="">
><br class="">
> But I wouldn't even use the `Inputs` folder here. The usual bitcode<br class="">
> upgrade tests say:<br class="">
><br class="">
> ; RUN: llvm-dis < %s.bc | FileCheck %s<br class="">
> ; RUN: verify-uselistorder < %s.bc<br class="">
><br class="">
> and just drop a .ll.bc file next to a .ll test in `test/Bitcode`.<br class="">
><br class="">
> Moreover, the test file should be the `.ll` file that was used to<br class="">
> generate the bitcode. Please add a comment that says what revision of<br class="">
> LLVM was used to generate the bitcode, something like:<br class="">
><br class="">
> ; Bitcode generated from llvm-as @ r123456.<br class="">
><br class="">
>> +The input uses the older form without a dwoId field.<br class="">
>> +This should default to 0,<br class="">
>> +which is not displayed at all in the textual representation.<br class="">
>> +CHECK: !DICompileUnit<br class="">
>> +CHECK-NOT: dwoId:<br class="">
><br class="">
<br class="">
</div></div></blockquote></div><br class=""></div></div>
</div></blockquote></div></div></div><br class=""></div></blockquote></div><br class=""></div></div>
</div></blockquote></div></div></div><br class=""></div></blockquote></div><br class=""></div></div>
</div></blockquote></div></div></div><br class=""></div></blockquote></div><br class=""></div></div>
</div></blockquote></div></div></div><br class=""></div></blockquote></div><br class=""></div></div>
</div></blockquote></div></div></div><br class=""></div></blockquote></div><br class=""></div></div>
</div></blockquote></div></div></div><br class=""></div></blockquote></div><br class=""></div></div>
</div></blockquote></div><br class=""></div>_______________________________________________<br class="">llvm-commits mailing list<br class=""><a href="mailto:llvm-commits@cs.uiuc.edu" class="">llvm-commits@cs.uiuc.edu</a><br class=""><a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" class="">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</a><br class=""></div></blockquote></div><br class=""></div>_______________________________________________<br class="">llvm-commits mailing list<br class=""><a href="mailto:llvm-commits@cs.uiuc.edu" class="">llvm-commits@cs.uiuc.edu</a><br class="">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits<br class=""></blockquote></div><br class=""></body></html>