[llvm-dev] DWARF Fission + ThinLTO
David Blaikie via llvm-dev
llvm-dev at lists.llvm.org
Wed May 3 19:51:52 PDT 2017
On Wed, May 3, 2017 at 7:48 PM Adrian Prantl <aprantl at apple.com> wrote:
> On May 3, 2017, at 7:43 PM, Adrian Prantl via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>
> On May 3, 2017, at 2:59 PM, David Blaikie <dblaikie at gmail.com> wrote:
>
>
>
> On Wed, May 3, 2017 at 2:09 PM Adrian Prantl <aprantl at apple.com> wrote:
>
>>
>> > On May 3, 2017, at 2:00 PM, David Blaikie <dblaikie at gmail.com> wrote:
>> >
>> > So Dehao and I have been dealing with some of the nitty gritty details
>> of debug info with ThinLTO, specifically with Fission(Split DWARF).
>> >
>> > This applies to LTO as well, so I won't single out ThinLTO here.
>> >
>> > 1) Multiple CUs in a .dwo file
>> > Clang/LLVM produces a CU for each original source file - these CUs are
>> kept through IR linking (thin or full) and produced as distinct CUs in the
>> resulting DWARF.
>> > This preserves semantics as much as possible - eg: file-local functions
>> and types (those in anonymous namespaces or declared file-static) con
>> co-exist even if their names collide.
>> > GDB does good things with this, except with Fission.
>>
>> Can you lay out the terminology you are using here? Is Fission as used
>> here different/more than creating .dwo files?
>>
>
> Fission as used in that sentence means .dwo files and anything beyond
> (.dwp files - https://gcc.gnu.org/wiki/DebugFissionDWP ). GDB
> warns/errors/complains if two CUs are in a single .dwo, and ignores all but
> the first.
>
>
> It sounds like having more than one CU per .dwo is outside of the
> intention of the DWARF specification (though not explicitly forbidden),
> since there is an implied 1-1 relationship between skeleton CU and .dwo.
>
> What do you think is the *right* solution here:
> - Allowing more than one CU in gdb?
> - Emitting many little .dwo files for the cross-inlined functions' CUs?
>
>
> Hmm.. I guess that would make the cross-CU references impossible. (Unless
> we can refer to everything via signatures).
>
Precisely!
It'd be pretty tricky to do by signature - /maybe/ possible. But these are
internal symbols - so they're not very uniqueable - in terms of mangled
name, etc to hash.
and I hadn't mentioned one other wrinkle:
The CU fragments that result from selectively importing functions in
ThinLTO are going to be a problem - if two ThinLTO shards both import the
same chunks from a third module - they'll potentially produce two CUs with
the same DWO ID hash. So I was thinking there might be some need to
cross-polinate the hashes of the various CUs in ThinLTO, to create 'more
unique' identifiers... but I don't know exactly how it'd all look there.
>
>
>
>> > Binutils DWP produces usable DWPs from DWO files with multiple CUs
>> >
>> > 2) Cross-CU references
>> > This is where it gets trickier.
>> > LLVM produces cross-CU references (DW_FORM_ref_addr) to refer to types
>> or functions defined in one CU used from another. This only happens with
>> (thin or plain) LTO. LLVM's had this for a while.
>> > This helps fully express interesting cases outlined in (1) - it's
>> possible that a file-local function is inlined into another CU - by using
>> ref_addr the semantics described in (1) can be preserved, while also
>> explaining the inlining that has occurred. (similar cases can arise with
>> intra-CU type references too)
>> > GDB handles this, except with Fission (I mean, it doesn't get this far
>> - but even with patches to handle (1)+Fission, it's still not enough -
>> needs more work)
>> > Binutils DWP and the DWP format in general... maybe can't cope with
>> this.
>> >
>> > Talking about (2)+DWP:
>>
>> You mean (2)+DWO+DWP?
>>
>
> DWP is a package containing the contents of multiple DWOs - so, yes and
> no? Not sure how to answer.
>
> (2)+DWO could be made to work without changing much in the
> contents/format/etc, I think - consumers could be made to understand the
> 'obvious' form (no change to producers I think would be needed).
>
> But once you go to a DWP file, then there are repreesntational problems
> that make it currently impossible to use cross-CU references. The semantics
> (& possibly the syntax) of DWP files would have to change to enable this
> functionality.
>
>
> Why can a .dwo cope with cross-CU-references but not a .dwp?
>
>
>
>>
>> > It looks like this may require adjusting at least the semantics, if not
>> the syntax of the indexes in the DWP file. I've started a thread on
>> dwarf-discuss to start trying to hash that out.
>> >
>> >
>> >
>> > So, to cut a long story short: Fission+LTO is currently unusable and
>> may take a while to make the DWARF side of this usable.
>> >
>> > In the interim, I'd like to propose adding a flag to LLVM to support
>> something not very nice: When merging modules (or importing things into a
>> module in ThinLTO), do not maintain separate CUs but redirect the cu
>> pointer in any DISubprogram (& probably DIGlobalVariable too - I forget if
>> ThinLTO can import them) metadata to refer to the original CU in the
>> destination module. (in the full LTO case, it'd also be necessary to import
>> any retained types)
>> >
>> > This flag could be passed to the task doing the merging/importing, or
>> could be passed to the frontend compile and stashed in per-CU metadata
>> (where it would be respected when importing/merging from that CU).
>> >
>> > The intent would be to support this flag until the DWARF functionality
>> can be decided on and implemented - and kept for some time after that for
>> backwards compatibility for those who want to use Fission but haven't got
>> the latest toolchains with the fixes/improvements in them.
>> >
>>
>
> If the other tools cannot be fixed quickly, then this would be a pragmatic
> interim solution.
>
> >
>> >
>> > How's this sound to everyone? Reasonable? Unreasonable? Need more
>> details about the impact of these changes, etc? (I have examples with DWARF
>> output, GDB behavior, etc)
>>
>> If you can post an example, that would help.
>>
>
> OK, so, here's an example of how GDB, without fission, benefits from the
> two distinct CUs (rather than bundling all the types & functions into a
> single CU):
>
> given:
> a.cpp:
> namespace {
> struct foo {
> int i;
> };
> }
> static void f1() {
> }
> void f2() {
> f1();
> }
> b.cpp:
> namespace {
> struct foo {
> float f;
> };
> }
> static void f1() {
> }
> void f2();
> int main() {
> f1();
> f2();
> }
> $ clang++ {a,b}.cpp -g -c -emit-llvm -S
> $ llvm-link {a,b}.ll -S -o ab.ll
> $ clang++-tot ab.ll
> $ gdb a.out
> (gdb) start
> ..., main () at b.cpp:11
> 11 f1();
> (gdb) ptype foo
> type = struct (anonymous namespace)::foo {
> float f;
> }
> (gdb) p &f1
> $1 = (void (*)(void)) 0x400530 <f1()>
> (gdb) n
> 12 f2();
> (gdb) s
> f2 () at a.cpp:10
> 10 f1();
> (gdb) ptype foo
> type = struct (anonymous namespace)::foo {
> int i;
> }
> (gdb) p &f1
> $2 = (void (*)(void)) 0x400500 <f1()>
>
> So in that case, the foo type and f1 function are properly identified
> depending on the context in which the expression is evaluated.
>
> If the types and subprograms are tied into the same CU (with a bit of
> manual IR editing), two distinct types and functions are emitted into the
> DWARF, but GDB only finds the first one, every time, in both contexts.
>
>
> To look into the larger motivation, cross-CU inlining and how it interacts
> with Fission:
>
> a.cpp:
> __attribute__((optnone)) void f1() {
> }
> __attribute__((always_inline)) void f2() {
> f1();
> };
> b.cpp:
> void f2();
> void f3() {
> f2();
> }
> c.cpp:
> void f3();
> int main() {
> f3();
> }
> $ clang++ {a,b}.cpp -g -c -emit-llvm -S
> $ llvm-link {a,b}.ll -S -o ab.ll
> $ clang++-tot c.cpp ab.ll -gsplit-dwarf
> $ dwp {c,ab}.dwo -o cab.dwp
>
> This produces two dwo files - one per object file (c.dwo, ab.dwo). If we
> look at the contents, they make a fair bit of sense:
>
> 0x0b: DW_TAG_compile_unit [1] *
> 0x19: DW_TAG_subprogram [2]
> 0x25: DW_TAG_subprogram [3]
> 0x31: DW_TAG_subprogram [4]
> 0x43: DW_TAG_compile_unit [1] *
> 0x51: DW_TAG_subprogram [5] *
> 0x5d: DW_TAG_inlined_subroutine [6]
> DW_AT_abstract_origin [DW_FORM_ref_addr] (0x31 "_Z2f2v")
>
> The ref_addr can be resolved relative to this whole .dwo file & the
> abstract subprogram is found. Yay.
>
> So GDB can't currently handle this:
>
> Could not find DWO CU ab.dwo(0x32dd6d7121dd1d9a) referenced by CU at
> offset 0x66 [in module a.out]
>
> And that's after some local fixes I have so it doesn't error out on the
> two-CUs-in-a-dwo situation (that fix at least allows the first example
> (local foo/f1, etc) to work with GDB - but fixing ref_addr to resolve
> correctly would require deeper changes I haven't figured out/prototyped
> yet).
>
> So, up until this point I was pretty satisfied there would be a way
> forward...
>
> Then I hit DWP files.
>
> DWP files represent multiple DWO files stacked together - mostly to
> deduplicate strings and type units as a sort of archival (like dsym) format.
>
> The way DWPs work, roughly (full/more details here:
> https://gcc.gnu.org/wiki/DebugFissionDWP ) is that all the sections are
> concatenated together, except the debug_str and debug_str_offsets sections
> which have special handling to do what you'd expect - deduplicate strings,
> and adjust the str_offsets to point to the right offsets in the
> deduplicated string section.
>
> The other thing that DWP has, is an index, or two. cu_index and tu_index.
> We'll just look at cu_index.
>
> A cu_index for the cab.dwp above, is:
>
> Index Signature INFO ABBR LINE STR_OFF
> ----- ------------------ -------- -------- -------- --------
> 2 0x7bd765349b7e7631 [2d, 65) [38, ae) [11, 22) [14, 3c)
> 8 0x66f4e160661d2687 [00, 2d) [00, 38) [00, 11) [00, 14)
> 11 0x32dd6d7121dd1d9a [65, 98) [38, ae) [11, 22) [14, 3c)
>
> What this is for is to tell the consumer, which portions of each section
> relate to which CUs - since the DIEs in the CU don't contain any
> relocations, for example the abbrev offset in the CU header is probably
> zero. It must be resolved relative to the ABBR section in the above table
> to find the chunk of the debug_abbrev that came from that CU.
>
> So, this is all good and well, except that the INFO range isn't the whole
> range of the debug info from the DWO file - it's /just/ the range of this
> CU. Which means there's no way to resolve the ref_addr relative to the
> whole range, you don't know where it starts.
>
>
> Ah thanks, that explains my question above!
>
>
>
> So this leads us to a few options - the 'simplest', that changes the
> semantics but not the syntax of the table - would be to widen the INFO
> range to cover the whole portion that came from the original DWO file.
>
> Consumers would have to be adjusted to cope with the fact that the INFO
> contribution for a given DWO ID would be a range that contains the CU
> somewhere, along with other CUs - and they'd have to search (in full LTO,
> this would mean searching through /all/ the CUs).
>
>
> Would generating a .dwo per CU help with this?
>
> This doesn't cover all the use cases I'd have in mind, but it would at
> least be possible to implement & support ref_addr. The full depth of design
> choices I'll likely leave to the dwarf-discuss thread, hopefully.
>
>
> Yes, given all the non-LLVM tools that will need to support this, that is
> the right forum to discuss this.
>
> -- adrian
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170504/97f3ceb2/attachment.html>
More information about the llvm-dev
mailing list