[llvm] r191792 - Debug Info: remove duplication of DIEs when a DIE is part of the type system

[Manman Ren]

On Thu, Oct 3, 2013 at 2:06 PM, David Blaikie <dblaikie at gmail.com> wrote:

>
> 3GB memory reduction for xalan built with lto -g (without removing DIE
>>> duplication, the memory usage is 7GB)
>>>
>>
>> OK, so you're looking at memory usage during LTO, not the size of the
>> resulting debug info? (that's fine, just trying to understand exactly what
>> metrics you're targeting)
>>
>
> One thought I had (and just discussed with Eric - he seemed to think it
> was plausible, at least):
>
> If you're interested in reducing peak memory usage of LLVM during LTO,
> have you considered modifying debug info generation not to cache all the
> compile_units before emitting them? If we emitted one CU at a time as they
> were created I imagine the memory footprint would be much lower.
>

Yes, that is on my todo list after type uniquing is done. Right now, we
don't release any memory and we have layers of memory usage related to
debug info: MDNodes, DIEs and some data in MC layer.
 All these layers exist for the whole program with LTO.


> (and we could still, potentially, add the type cross-referencing by just
> caching the section offsets or labels, etc, of the types emitted in prior
> units - though that wouldn't be perfect for types that contain some members
> (implicit special members) in one CU and a different set of members in
> another CU (and it'll be a bit trickier for type units - we'd need to cache
> any subgraph of potentially self-referencing type units until all the
> signatures are resolved, then they could all be emitted))
>
> This isn't necessarily a place you should start with - I can appreciate
> that your current approach is probably higher value to you for lower
> engineering cost (and thus gets you closer sooner) but if it's still not
> going to be sufficient for your needs, especially if you're going to have
> to implement something like emit-CU-at-a-time as I describe above anyway,
> it might be more valuable to lay that foundation first.
>

Even if we emit one CU at a time, it is still good to share the types
across-CU so we don't waist time constructing the type DIEs for each CU,
and this approach (remove DIE duplication) also reduces the raw DWARF size.


>
> If not, I'd still probably like to discuss the design of the cross-CU
> referencing work you have here to decide on the right design in a normal
> patch review process rather than trying to think about how the current code
> can be morphed into a better design.
>

What other designs do you have in mind?

Manman

>
>
> - David
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20131003/8c338de3/attachment.html>