[LLVMdev] Proposal: release MDNodes for source modules (LTO+debug info)

[Manman Ren]

On Tue, Nov 12, 2013 at 4:38 PM, Chandler Carruth <chandlerc at google.com>wrote:

>
> On Tue, Nov 12, 2013 at 4:29 PM, Manman Ren <manman.ren at gmail.com> wrote:
>
>> Hi All,
>>
>>  In LTO, we load in the source modules and link the source modules into a
>> destination module.
>> Lots of MDNodes are only used by the source modules, for example Xalan
>> used 649MB for MDNodes after loading and linking, but the actual
>> destination module only has 393MB of MDNodes. There are 649-393MB (40% of
>> 649MB) not used.
>>
>> MDNodes belong to the Context, deleting modules will not release the
>> MDNodes.
>>
>> One possible solution is:
>>
>> In LLVMContext, add “removeUnusedMDNodes" function
>>   It goes through OwnedModules and check if a MDNode is used by any of
>> the modules, if not remove it.
>>   One implementation is to mark a visited MDNode used when traversing the
>> module. After done traversing all modules, we can delete MDNodes in
>> MDNodeSet that are not marked.
>>
>> In LTOCodeGenerator, add a vector of source modules that are added (these
>> source modules will be linked with DestroySource mode).
>> In LTOCodeGenerator:: compile_to_file, delete all source modules that are
>> linked in, then call LLVMContext::removeUnusedMDNodes
>> —> I can’t find a better place to call the function. When we
>> call compile_to_file, we should have done linking in all source modules.
>> Another possibility is to add a lto API so the linker can delete the
>> source modules and call the API to release MDNodes.
>>
>> Other options are:
>> 1> Using a different LLVMContext for the destination module, but it
>> didn’t work out since Linker was not designed to work with different
>> LLVMContexts for source vs destination.
>> 2> removeUnusedMDNodes checks if a MDNode is used in a different way
>>  (i.e use_empty() && !hasValueHandler()), but it does not remove MDNodes
>> that form cycles.
>>
>
> 3) Make the MDNode be owned by the module that uses it?
>

MDNode is shared among modules so multiple modules can use it, if we
specify an owner for a MDNode, that will prevent sharing.

Manman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131112/01bce254/attachment.html>