Free memory used by initializers of global variables, created during linking

[Manman Ren]

In r226592.

Thanks for reviewing!

Manman

> On Jan 19, 2015, at 12:34 PM, Duncan P. N. Exon Smith <dexonsmith at apple.com> wrote:
> 
>> 
>> On 2015-Jan-19, at 11:25, Manman Ren <mren at apple.com> wrote:
>> 
>> Thanks Rafael and Duncan.
>> 
>> Updated patch is attached. It adds
>> +  /// Destroy ConstantArrays in LLVMContext if they are not used.
>> +  /// ConstantArrays constructed during linking can cause quadratic memory
>> +  /// explosion. Releasing all unused constants can cause a 20% LTO compile-time
>> +  /// slowdown for a large application.
>> +  /// NOTE: Constants are currently owned by LLVMContext. This can then only
>> +  /// be called where all uses of the LLVMContext are understood.
>> +  void dropTriviallyDeadConstantArrays();
>> 
>> 
>>> On Jan 16, 2015, at 6:17 PM, Rafael Espíndola <rafael.espindola at gmail.com> wrote:
>>> 
>>>> A 20% slowdown seems excessive.  Do I understand correctly that if you
>>>> don't delete unused `ConstantExpr`s there's no real slowdown?
>> 
>> Without deleting “ConstantExprs”, the run time is 10 minutes, and it reduces the memory footprint from 22GB to 6GB.
>> If we delete unused “ConstantExprs”, the run time is 12 minutes, the memory footprint is about the same.
>> 
>> Thanks,
>> Manman
> 
> LGTM, once you fix a couple of comment nitpicks:
> 
>> Index: include/llvm/IR/Module.h
>> ===================================================================
>> --- include/llvm/IR/Module.h	(revision 226024)
>> +++ include/llvm/IR/Module.h	(working copy)
>> @@ -630,6 +630,11 @@
>>                                                          named_metadata_end());
>>   }
>> 
>> +  /// Destroy ConstantArrays in LLVMContext if they are not used.
>> +  /// ConstantArrays constructed during linking can cause quadratic memory
>> +  /// explosion. Releasing all unused constants can cause a 20% LTO compile-time
>> +  /// slowdown for a large application.
> 
> Can you add an empty (but commented) line here?  This makes the NOTE
> jump out a bit more clearly.
> 
>    ///
> 
>> +  /// NOTE: Constants are currently owned by LLVMContext. This can then only
>> +  /// be called where all uses of the LLVMContext are understood.
>> +  void dropTriviallyDeadConstantArrays();
>> +
>> /// @}
>> /// @name Utility functions for printing and dumping Module objects
>> /// @{
>> Index: lib/IR/LLVMContextImpl.cpp
>> ===================================================================
>> --- lib/IR/LLVMContextImpl.cpp	(revision 226024)
>> +++ lib/IR/LLVMContextImpl.cpp	(working copy)
>> @@ -154,6 +154,95 @@
>>   MDStringCache.clear();
>> }
>> 
>> +void LLVMContextImpl::dropTriviallyDeadConstantArrays() {
>> +  bool Changed;
>> +  do {
>> +    Changed = false;
>> +
>> +    for (auto I = ArrayConstants.map_begin(), E = ArrayConstants.map_end();
>> +         I != E; ) {
>> +      auto *C = I->first;
>> +      I++;
>> +      if (C->use_empty()) {
>> +        Changed = true;
>> +        C->destroyConstant();
>> +      }
>> +    }
>> +
>> +  } while (Changed);
>> +}
>> +
>> +void Module::dropTriviallyDeadConstantArrays() {
>> +  Context.pImpl->dropTriviallyDeadConstantArrays();
>> +}
>> +
>> // ConstantsContext anchors
>> void UnaryConstantExpr::anchor() { }
>> 
>> Index: lib/IR/LLVMContextImpl.h
>> ===================================================================
>> --- lib/IR/LLVMContextImpl.h	(revision 226024)
>> +++ lib/IR/LLVMContextImpl.h	(working copy)
>> @@ -392,6 +392,10 @@
>> 
>>   LLVMContextImpl(LLVMContext &C);
>>   ~LLVMContextImpl();
>> +
>> +  /// Destroy the ConstantArrays if they are not
> 
> This comment is incomplete.
> 
>> +  void dropTriviallyDeadConstantArrays();
>> };
>> 
>> }
>> Index: lib/Linker/LinkModules.cpp
>> ===================================================================
>> --- lib/Linker/LinkModules.cpp	(revision 226024)
>> +++ lib/Linker/LinkModules.cpp	(working copy)
>> @@ -1721,7 +1721,9 @@
>> bool Linker::linkInModule(Module *Src) {
>>   ModuleLinker TheLinker(Composite, IdentifiedStructTypes, Src,
>>                          DiagnosticHandler);
>> -  return TheLinker.run();
>> +  bool RetCode = TheLinker.run();
>> +  Composite->dropTriviallyDeadConstantArrays();
>> +  return RetCode;
>> }
>> 
>> //===----------------------------------------------------------------------===//

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20150204/59922176/attachment.html>