[LLVMdev] Proposal: release MDNodes for source modules (LTO+debug info)

[Chandler Carruth]

On Tue, Nov 12, 2013 at 6:02 PM, Chris Lattner <clattner at apple.com> wrote:

> On Nov 12, 2013, at 4:59 PM, Chandler Carruth <chandlerc at google.com>
> wrote:
>
> Other options are:
>>>> 1> Using a different LLVMContext for the destination module, but it
>>>> didn’t work out since Linker was not designed to work with different
>>>> LLVMContexts for source vs destination.
>>>> 2> removeUnusedMDNodes checks if a MDNode is used in a different way
>>>>  (i.e use_empty() && !hasValueHandler()), but it does not remove MDNodes
>>>> that form cycles.
>>>>
>>>
>>> 3) Make the MDNode be owned by the module that uses it?
>>>
>>
>> MDNode is shared among modules so multiple modules can use it, if we
>> specify an owner for a MDNode, that will prevent sharing.
>>
>
> From your stats (40% stuck in the old module) it doesn't sound like this
> is buying us anything...
>
>
>
> If the old module is deleted, then these MDNodes can be reclaimed.
>
> I think this proposal amounts to a “garbage collector” that clears out
> now-dead IR objects that are uniqued in the LLVM Context.  While MDNodes
> are your focus, the same thing would apply equally well to ConstantInt and
> other things that may become unreachable.
>

I think the big difference is that there appears to be very little
(relatively speaking) overlap between modules for MDNodes. This is
different from types and many other things I suspect.

My question was: if we're not seeing a higher degree of sharing between
modules by using context-uniqued MDNodes, why not have them be owned by the
module rather than the context?


>
> Details matter on this (for it to be efficient), but I think that it would
> be very useful for LLVMContext to have a method that goes through and
> releases IR objects that aren’t used.  This could be used by the LTO
> driver, and if it actually reduces memory by 40%, that would be huge.
>

My only worry is doing a relatively expensive walk over all types and
constants when most are actually shared and don't get deleted. It sounds
like the metadata nodes are weird in that they are mostly disjoint between
modules.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131112/b4ef29e5/attachment.html>