[llvm-dev] DWARF Fission + ThinLTO
Eric Christopher via llvm-dev
llvm-dev at lists.llvm.org
Thu May 4 11:50:33 PDT 2017
On Thu, May 4, 2017 at 11:36 AM David Blaikie via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Thu, May 4, 2017 at 11:23 AM Robinson, Paul <paul.robinson at sony.com>
> wrote:
>
>> Sorry, trying to catch up a bit late…
>>
>>
>>
>> It sounds like having more than one CU per .dwo is outside of the
>> intention of the DWARF specification (though not explicitly forbidden),
>> since there is an implied 1-1 relationship between skeleton CU and .dwo.
>>
>>
>>
>> There is an explicit 1-1 relationship between skeleton CU and split-full
>> CU (not .dwo). This suggests to me that if you want a .dwo to have
>> multiple CUs, then the .o should have multiple corresponding skeleton CUs.
>>
>
> Right - that's closest to/as things are today, for sure.
>
>
This is similar to my proposal as well when Dave and I were chatting the
other day. However..
> Each skeleton/split-full pair would have the same dwo_name but different
>> dwo_IDs. You're making it sound like LTO is actually producing only one
>> skeleton CU? That would be a bug.
>>
>
> It isn't at the moment - at the moment it produces a skeleton per CU, a
> single DWO file with multiple CUs, and all this (with a relatively small
> patch to GDB) works in GDB today so long as there aren't any cross-cu
> references and so long as it's DWO files and not a DWP file.
>
>
>> I am not finding anywhere that explicitly says a .dwo file can have
>> only one unit? Although it does seem that the definition of the cu_index
>> assumes each unit should be treated as being in an independent .debug_info
>> section, which is a problem for making cross-CU references in a given .dwo
>> file. Thanks for bringing that up on dwarf-discuss already.
>>
>
> Right - I'm not sure if binutils DWP was implemented with this in mind, or
> if it fell out naturally from the debug_types support (walk the info/types
> section for each unit, keep it if it's not already present in the output -
> so it would naturally want to trim the info/types contribution to that
> specific unit since other units may not be included in the output). My
> llvm-dwp failed this - and brings in the first CU, but I think it takes the
> whole unit CU contribution, maybe... - so that's a bit broken in any case
> here.
>
>
... this is one of the problems. It's mostly bugs in other tools to have to
fix. :)
>
>>
>> So this leads us to a few options - the 'simplest', that changes the
>> semantics but not the syntax of the table - would be to widen the INFO
>> range to cover the whole portion that came from the original DWO file.
>>
>> Consumers would have to be adjusted to cope with the fact that the INFO
>> contribution for a given DWO ID would be a range that contains the CU
>> somewhere, along with other CUs - and they'd have to search (in full LTO,
>> this would mean searching through /all/ the CUs).
>>
>>
>>
>> But each skeleton/split-full pair needs a distinct DWO ID (per the v5
>> spec).
>>
>
> Each skeleton/split-full pair would still have a distinct DWO ID (each CU
> still has a separate hash) from what I was describing above/in the section
> you quoted. But the debug_info contribution in the cu_index would be
> 'vague' - it would be widened to describe the whole info contribution from
> that DWO file.
>
>
>> I guess we could address this by replicating the index entry for each
>> DWO ID in the .dwo file, so lookup-by-ID would still work, but the INFO
>> ranges would always describe the full .debug_info section from the .dwo
>> file.
>>
>
> Right, that ^ so, yeah "replicated" in the sense that for each DWO ID in
> the same DWO file, they'd have a separate entry in the cu_index with each
> unique DWO ID - but the contributions would be the same for every one of
> those DWO IDs from the same DWO file.
>
>
>> You'd still need to search the .dwo file's part of the INFO range to find
>> the actual CU, unless we change the syntax of the table to provide those
>> offsets. But then ref_addr would work across CUs from the same .dwo file.
>>
>
> Precisely.
>
> In the worst case, with full LTO - that'd be potentially a lot of
> searching across because it'd be a single DWO file with all the CUs.
>
> hopefully we can talk about this more on the dwarf-discuss list (I haven't
> seen any replies there, maybe I should just subscribe to the dwarf
> committee list & discuss it there? I dunno - open to suggestions)
>
>
... and everything Dave has said here.
That said, let's take this to dwarf-discuss and talk about it there.
-eric
> - Dave
>
>
>> --paulr
>>
>>
>>
>>
>>
>> *From:* llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] *On Behalf Of *David
>> Blaikie via llvm-dev
>> *Sent:* Wednesday, May 03, 2017 7:52 PM
>> *To:* Adrian Prantl
>> *Cc:* llvm-dev
>> *Subject:* Re: [llvm-dev] DWARF Fission + ThinLTO
>>
>>
>>
>>
>>
>> On Wed, May 3, 2017 at 7:48 PM Adrian Prantl <aprantl at apple.com> wrote:
>>
>> On May 3, 2017, at 7:43 PM, Adrian Prantl via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>
>>
>>
>> On May 3, 2017, at 2:59 PM, David Blaikie <dblaikie at gmail.com> wrote:
>>
>>
>>
>>
>>
>> On Wed, May 3, 2017 at 2:09 PM Adrian Prantl <aprantl at apple.com> wrote:
>>
>>
>> > On May 3, 2017, at 2:00 PM, David Blaikie <dblaikie at gmail.com> wrote:
>> >
>> > So Dehao and I have been dealing with some of the nitty gritty details
>> of debug info with ThinLTO, specifically with Fission(Split DWARF).
>> >
>> > This applies to LTO as well, so I won't single out ThinLTO here.
>> >
>> > 1) Multiple CUs in a .dwo file
>> > Clang/LLVM produces a CU for each original source file - these CUs are
>> kept through IR linking (thin or full) and produced as distinct CUs in the
>> resulting DWARF.
>> > This preserves semantics as much as possible - eg: file-local functions
>> and types (those in anonymous namespaces or declared file-static) con
>> co-exist even if their names collide.
>> > GDB does good things with this, except with Fission.
>>
>> Can you lay out the terminology you are using here? Is Fission as used
>> here different/more than creating .dwo files?
>>
>>
>> Fission as used in that sentence means .dwo files and anything beyond
>> (.dwp files - https://gcc.gnu.org/wiki/DebugFissionDWP ). GDB
>> warns/errors/complains if two CUs are in a single .dwo, and ignores all but
>> the first.
>>
>>
>>
>> It sounds like having more than one CU per .dwo is outside of the
>> intention of the DWARF specification (though not explicitly forbidden),
>> since there is an implied 1-1 relationship between skeleton CU and .dwo.
>>
>>
>>
>> What do you think is the *right* solution here:
>>
>> - Allowing more than one CU in gdb?
>>
>> - Emitting many little .dwo files for the cross-inlined functions' CUs?
>>
>>
>>
>> Hmm.. I guess that would make the cross-CU references impossible. (Unless
>> we can refer to everything via signatures).
>>
>>
>> Precisely!
>>
>> It'd be pretty tricky to do by signature - /maybe/ possible. But these
>> are internal symbols - so they're not very uniqueable - in terms of mangled
>> name, etc to hash.
>>
>> and I hadn't mentioned one other wrinkle:
>>
>> The CU fragments that result from selectively importing functions in
>> ThinLTO are going to be a problem - if two ThinLTO shards both import the
>> same chunks from a third module - they'll potentially produce two CUs with
>> the same DWO ID hash. So I was thinking there might be some need to
>> cross-polinate the hashes of the various CUs in ThinLTO, to create 'more
>> unique' identifiers... but I don't know exactly how it'd all look there.
>>
>>
>>
>>
>>
>>
>>
>> > Binutils DWP produces usable DWPs from DWO files with multiple CUs
>> >
>> > 2) Cross-CU references
>> > This is where it gets trickier.
>> > LLVM produces cross-CU references (DW_FORM_ref_addr) to refer to types
>> or functions defined in one CU used from another. This only happens with
>> (thin or plain) LTO. LLVM's had this for a while.
>> > This helps fully express interesting cases outlined in (1) - it's
>> possible that a file-local function is inlined into another CU - by using
>> ref_addr the semantics described in (1) can be preserved, while also
>> explaining the inlining that has occurred. (similar cases can arise with
>> intra-CU type references too)
>> > GDB handles this, except with Fission (I mean, it doesn't get this far
>> - but even with patches to handle (1)+Fission, it's still not enough -
>> needs more work)
>> > Binutils DWP and the DWP format in general... maybe can't cope with
>> this.
>> >
>> > Talking about (2)+DWP:
>>
>> You mean (2)+DWO+DWP?
>>
>>
>> DWP is a package containing the contents of multiple DWOs - so, yes and
>> no? Not sure how to answer.
>>
>> (2)+DWO could be made to work without changing much in the
>> contents/format/etc, I think - consumers could be made to understand the
>> 'obvious' form (no change to producers I think would be needed).
>>
>> But once you go to a DWP file, then there are repreesntational problems
>> that make it currently impossible to use cross-CU references. The semantics
>> (& possibly the syntax) of DWP files would have to change to enable this
>> functionality.
>>
>>
>>
>> Why can a .dwo cope with cross-CU-references but not a .dwp?
>>
>>
>>
>>
>> > It looks like this may require adjusting at least the semantics, if not
>> the syntax of the indexes in the DWP file. I've started a thread on
>> dwarf-discuss to start trying to hash that out.
>> >
>> >
>> >
>> > So, to cut a long story short: Fission+LTO is currently unusable and
>> may take a while to make the DWARF side of this usable.
>> >
>> > In the interim, I'd like to propose adding a flag to LLVM to support
>> something not very nice: When merging modules (or importing things into a
>> module in ThinLTO), do not maintain separate CUs but redirect the cu
>> pointer in any DISubprogram (& probably DIGlobalVariable too - I forget if
>> ThinLTO can import them) metadata to refer to the original CU in the
>> destination module. (in the full LTO case, it'd also be necessary to import
>> any retained types)
>> >
>> > This flag could be passed to the task doing the merging/importing, or
>> could be passed to the frontend compile and stashed in per-CU metadata
>> (where it would be respected when importing/merging from that CU).
>> >
>> > The intent would be to support this flag until the DWARF functionality
>> can be decided on and implemented - and kept for some time after that for
>> backwards compatibility for those who want to use Fission but haven't got
>> the latest toolchains with the fixes/improvements in them.
>> >
>>
>>
>>
>> If the other tools cannot be fixed quickly, then this would be a
>> pragmatic interim solution.
>>
>>
>>
>> >
>> >
>> > How's this sound to everyone? Reasonable? Unreasonable? Need more
>> details about the impact of these changes, etc? (I have examples with DWARF
>> output, GDB behavior, etc)
>>
>> If you can post an example, that would help.
>>
>>
>> OK, so, here's an example of how GDB, without fission, benefits from the
>> two distinct CUs (rather than bundling all the types & functions into a
>> single CU):
>>
>> given:
>> a.cpp:
>>
>> namespace {
>>
>> struct foo {
>>
>> int i;
>>
>> };
>>
>> }
>>
>> static void f1() {
>>
>> }
>>
>> void f2() {
>>
>> f1();
>>
>> }
>> b.cpp:
>>
>> namespace {
>>
>> struct foo {
>>
>> float f;
>>
>> };
>>
>> }
>>
>> static void f1() {
>>
>> }
>>
>> void f2();
>>
>> int main() {
>>
>> f1();
>>
>> f2();
>>
>> }
>> $ clang++ {a,b}.cpp -g -c -emit-llvm -S
>> $ llvm-link {a,b}.ll -S -o ab.ll
>>
>> $ clang++-tot ab.ll
>> $ gdb a.out
>> (gdb) start
>>
>> ..., main () at b.cpp:11
>>
>> 11 f1();
>>
>> (gdb) ptype foo
>>
>> type = struct (anonymous namespace)::foo {
>>
>> float f;
>>
>> }
>>
>> (gdb) p &f1
>>
>> $1 = (void (*)(void)) 0x400530 <f1()>
>>
>> (gdb) n
>>
>> 12 f2();
>>
>> (gdb) s
>>
>> f2 () at a.cpp:10
>>
>> 10 f1();
>>
>> (gdb) ptype foo
>>
>> type = struct (anonymous namespace)::foo {
>>
>> int i;
>>
>> }
>>
>> (gdb) p &f1
>>
>> $2 = (void (*)(void)) 0x400500 <f1()>
>>
>> So in that case, the foo type and f1 function are properly identified
>> depending on the context in which the expression is evaluated.
>>
>> If the types and subprograms are tied into the same CU (with a bit of
>> manual IR editing), two distinct types and functions are emitted into the
>> DWARF, but GDB only finds the first one, every time, in both contexts.
>>
>>
>> To look into the larger motivation, cross-CU inlining and how it
>> interacts with Fission:
>>
>> a.cpp:
>>
>> __attribute__((optnone)) void f1() {
>>
>> }
>>
>> __attribute__((always_inline)) void f2() {
>>
>> f1();
>>
>> };
>>
>> b.cpp:
>>
>> void f2();
>>
>> void f3() {
>>
>> f2();
>>
>> }
>> c.cpp:
>>
>> void f3();
>>
>> int main() {
>>
>> f3();
>>
>> }
>>
>> $ clang++ {a,b}.cpp -g -c -emit-llvm -S
>> $ llvm-link {a,b}.ll -S -o ab.ll
>> $ clang++-tot c.cpp ab.ll -gsplit-dwarf
>> $ dwp {c,ab}.dwo -o cab.dwp
>>
>> This produces two dwo files - one per object file (c.dwo, ab.dwo). If we
>> look at the contents, they make a fair bit of sense:
>>
>> 0x0b: DW_TAG_compile_unit [1] *
>>
>> 0x19: DW_TAG_subprogram [2]
>>
>> 0x25: DW_TAG_subprogram [3]
>>
>> 0x31: DW_TAG_subprogram [4]
>>
>> 0x43: DW_TAG_compile_unit [1] *
>>
>> 0x51: DW_TAG_subprogram [5] *
>>
>> 0x5d: DW_TAG_inlined_subroutine [6]
>>
>> DW_AT_abstract_origin [DW_FORM_ref_addr] (0x31
>> "_Z2f2v")
>>
>> The ref_addr can be resolved relative to this whole .dwo file & the
>> abstract subprogram is found. Yay.
>>
>> So GDB can't currently handle this:
>>
>> Could not find DWO CU ab.dwo(0x32dd6d7121dd1d9a) referenced by CU at
>> offset 0x66 [in module a.out]
>>
>> And that's after some local fixes I have so it doesn't error out on the
>> two-CUs-in-a-dwo situation (that fix at least allows the first example
>> (local foo/f1, etc) to work with GDB - but fixing ref_addr to resolve
>> correctly would require deeper changes I haven't figured out/prototyped
>> yet).
>>
>> So, up until this point I was pretty satisfied there would be a way
>> forward...
>>
>> Then I hit DWP files.
>>
>> DWP files represent multiple DWO files stacked together - mostly to
>> deduplicate strings and type units as a sort of archival (like dsym) format.
>>
>> The way DWPs work, roughly (full/more details here:
>> https://gcc.gnu.org/wiki/DebugFissionDWP ) is that all the sections are
>> concatenated together, except the debug_str and debug_str_offsets sections
>> which have special handling to do what you'd expect - deduplicate strings,
>> and adjust the str_offsets to point to the right offsets in the
>> deduplicated string section.
>>
>> The other thing that DWP has, is an index, or two. cu_index and tu_index.
>> We'll just look at cu_index.
>>
>> A cu_index for the cab.dwp above, is:
>>
>> Index Signature INFO ABBR LINE STR_OFF
>>
>> ----- ------------------ -------- -------- -------- --------
>>
>> 2 0x7bd765349b7e7631 [2d, 65) [38, ae) [11, 22) [14, 3c)
>>
>> 8 0x66f4e160661d2687 [00, 2d) [00, 38) [00, 11) [00, 14)
>>
>> 11 0x32dd6d7121dd1d9a [65, 98) [38, ae) [11, 22) [14, 3c)
>>
>>
>> What this is for is to tell the consumer, which portions of each section
>> relate to which CUs - since the DIEs in the CU don't contain any
>> relocations, for example the abbrev offset in the CU header is probably
>> zero. It must be resolved relative to the ABBR section in the above table
>> to find the chunk of the debug_abbrev that came from that CU.
>>
>> So, this is all good and well, except that the INFO range isn't the whole
>> range of the debug info from the DWO file - it's /just/ the range of this
>> CU. Which means there's no way to resolve the ref_addr relative to the
>> whole range, you don't know where it starts.
>>
>>
>>
>> Ah thanks, that explains my question above!
>>
>>
>>
>> So this leads us to a few options - the 'simplest', that changes the
>> semantics but not the syntax of the table - would be to widen the INFO
>> range to cover the whole portion that came from the original DWO file.
>>
>> Consumers would have to be adjusted to cope with the fact that the INFO
>> contribution for a given DWO ID would be a range that contains the CU
>> somewhere, along with other CUs - and they'd have to search (in full LTO,
>> this would mean searching through /all/ the CUs).
>>
>>
>>
>> Would generating a .dwo per CU help with this?
>>
>>
>>
>> This doesn't cover all the use cases I'd have in mind, but it would at
>> least be possible to implement & support ref_addr. The full depth of design
>> choices I'll likely leave to the dwarf-discuss thread, hopefully.
>>
>>
>>
>> Yes, given all the non-LLVM tools that will need to support this, that is
>> the right forum to discuss this.
>>
>>
>>
>> -- adrian
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>>
>> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170504/0d8e53f5/attachment.html>
More information about the llvm-dev
mailing list