[LLVMdev] Proposal: release MDNodes for source modules (LTO+debug info)

[Manman Ren]

On Thu, Nov 14, 2013 at 10:11 AM, Eric Christopher <echristo at gmail.com>wrote:

> On Thu, Nov 14, 2013 at 5:19 AM, Rafael Espíndola
> <rafael.espindola at gmail.com> wrote:
> >> Letting the module own MDNodes may not be a win for gold since it is
> going
> >> to create multiple copies of MDNodes that could be shared with Context
> >> owning MDNodes.
> >>
> >> For example, with debug info type uniquing, the type nodes can be shared
> >> across modules, but with module owning MDNodes, each module will create
> its
> >> own copy of the type nodes. The advantage is that the MDNodes can be
> deleted
> >> easily by deleting the module. It is not clearly a win to me.
> >
> > But gold has at most 2 objects loaded at any time.
> >
>
> Agreed. Dave and I were chatting about this some and from my
> perspective the only disadvantage to this is the time to copy metadata
> from one place to the other. Peak memory usage will still go down
> since you could then free up each module as you added. The "uniquing"
> works basically the same way as adding the same node 400 times to the
> folding set - it just returns the same one each time.
>

Let's clear up a few things first, I believe Chandler was proposing moving
the folding set from LLVMContext to Module.
The "uniquing" happens only within the module. It is not "just returning
the same one each time", I would like to rephrase it as creating the same
thing for each module.

With that proposal, we will have:
1> more MDNodes will be created.
     This can be a big issue because it means we are going to recreate the
types for each module.
2> it is easier to free up the MDNodes, it happens when we delete the
source modules.

My proposal earlier was to implement an interface to "garbage-collect" the
MDNodes (and maybe other things  LLVMContext, as pointed out by Chris).


>
> I guess I'm confused at where this 40% overhead of MDNodes is coming
> from though? Do we know what they are?
>

The overhead is the MDNodes that are only used by the source modules. I
gave a few examples earlier in another thread:

There are a few cases where we generate MDNodes when linking modules:
1> when a MDNode points to a value that is different from the source module
such as Function*.
2> when we have a cycle in the MDNode graph, all nodes in the cycle will be
created for the destination module.

When the linked module generates different MDNodes, the MDNodes in the
source module become garbage once the source module is deleted.

Cheers,
Manman


>
> -eric
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131114/ea1e2c22/attachment.html>