[LLVMdev] Proposal: release MDNodes for source modules (LTO+debug info)

[Chandler Carruth]

On Tue, Nov 12, 2013 at 4:29 PM, Manman Ren <manman.ren at gmail.com> wrote:

> Hi All,
>
> In LTO, we load in the source modules and link the source modules into a
> destination module.
> Lots of MDNodes are only used by the source modules, for example Xalan
> used 649MB for MDNodes after loading and linking, but the actual
> destination module only has 393MB of MDNodes. There are 649-393MB (40% of
> 649MB) not used.
>
> MDNodes belong to the Context, deleting modules will not release the
> MDNodes.
>
> One possible solution is:
>
> In LLVMContext, add “removeUnusedMDNodes" function
>   It goes through OwnedModules and check if a MDNode is used by any of the
> modules, if not remove it.
>   One implementation is to mark a visited MDNode used when traversing the
> module. After done traversing all modules, we can delete MDNodes in
> MDNodeSet that are not marked.
>
> In LTOCodeGenerator, add a vector of source modules that are added (these
> source modules will be linked with DestroySource mode).
> In LTOCodeGenerator:: compile_to_file, delete all source modules that are
> linked in, then call LLVMContext::removeUnusedMDNodes
> —> I can’t find a better place to call the function. When we
> call compile_to_file, we should have done linking in all source modules.
> Another possibility is to add a lto API so the linker can delete the
> source modules and call the API to release MDNodes.
>
> Other options are:
> 1> Using a different LLVMContext for the destination module, but it didn’t
> work out since Linker was not designed to work with different LLVMContexts
> for source vs destination.
> 2> removeUnusedMDNodes checks if a MDNode is used in a different way  (i.e
> use_empty() && !hasValueHandler()), but it does not remove MDNodes that
> form cycles.
>

3) Make the MDNode be owned by the module that uses it?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131112/22442859/attachment.html>