Free memory used by initializers of global variables, created during linking

[Duncan P. N. Exon Smith]

> On 2015-Jan-19, at 11:25, Manman Ren <mren at apple.com> wrote:
> 
> Thanks Rafael and Duncan.
> 
> Updated patch is attached. It adds
> +  /// Destroy ConstantArrays in LLVMContext if they are not used.
> +  /// ConstantArrays constructed during linking can cause quadratic memory
> +  /// explosion. Releasing all unused constants can cause a 20% LTO compile-time
> +  /// slowdown for a large application.
> +  /// NOTE: Constants are currently owned by LLVMContext. This can then only
> +  /// be called where all uses of the LLVMContext are understood.
> +  void dropTriviallyDeadConstantArrays();
> 
> 
>> On Jan 16, 2015, at 6:17 PM, Rafael Espíndola <rafael.espindola at gmail.com> wrote:
>> 
>>> A 20% slowdown seems excessive.  Do I understand correctly that if you
>>> don't delete unused `ConstantExpr`s there's no real slowdown?
> 
> Without deleting “ConstantExprs”, the run time is 10 minutes, and it reduces the memory footprint from 22GB to 6GB.
> If we delete unused “ConstantExprs”, the run time is 12 minutes, the memory footprint is about the same.
> 
> Thanks,
> Manman

LGTM, once you fix a couple of comment nitpicks:

> Index: include/llvm/IR/Module.h
> ===================================================================
> --- include/llvm/IR/Module.h	(revision 226024)
> +++ include/llvm/IR/Module.h	(working copy)
> @@ -630,6 +630,11 @@
>                                                           named_metadata_end());
>    }
>  
> +  /// Destroy ConstantArrays in LLVMContext if they are not used.
> +  /// ConstantArrays constructed during linking can cause quadratic memory
> +  /// explosion. Releasing all unused constants can cause a 20% LTO compile-time
> +  /// slowdown for a large application.

Can you add an empty (but commented) line here?  This makes the NOTE
jump out a bit more clearly.

    ///

> +  /// NOTE: Constants are currently owned by LLVMContext. This can then only
> +  /// be called where all uses of the LLVMContext are understood.
> +  void dropTriviallyDeadConstantArrays();
> +
>  /// @}
>  /// @name Utility functions for printing and dumping Module objects
>  /// @{
> Index: lib/IR/LLVMContextImpl.cpp
> ===================================================================
> --- lib/IR/LLVMContextImpl.cpp	(revision 226024)
> +++ lib/IR/LLVMContextImpl.cpp	(working copy)
> @@ -154,6 +154,95 @@
>    MDStringCache.clear();
>  }
>  
> +void LLVMContextImpl::dropTriviallyDeadConstantArrays() {
> +  bool Changed;
> +  do {
> +    Changed = false;
> +
> +    for (auto I = ArrayConstants.map_begin(), E = ArrayConstants.map_end();
> +         I != E; ) {
> +      auto *C = I->first;
> +      I++;
> +      if (C->use_empty()) {
> +        Changed = true;
> +        C->destroyConstant();
> +      }
> +    }
> +
> +  } while (Changed);
> +}
> +
> +void Module::dropTriviallyDeadConstantArrays() {
> +  Context.pImpl->dropTriviallyDeadConstantArrays();
> +}
> +
>  // ConstantsContext anchors
>  void UnaryConstantExpr::anchor() { }
>  
> Index: lib/IR/LLVMContextImpl.h
> ===================================================================
> --- lib/IR/LLVMContextImpl.h	(revision 226024)
> +++ lib/IR/LLVMContextImpl.h	(working copy)
> @@ -392,6 +392,10 @@
>    
>    LLVMContextImpl(LLVMContext &C);
>    ~LLVMContextImpl();
> +
> +  /// Destroy the ConstantArrays if they are not

This comment is incomplete.

> +  void dropTriviallyDeadConstantArrays();
>  };
>  
>  }
> Index: lib/Linker/LinkModules.cpp
> ===================================================================
> --- lib/Linker/LinkModules.cpp	(revision 226024)
> +++ lib/Linker/LinkModules.cpp	(working copy)
> @@ -1721,7 +1721,9 @@
>  bool Linker::linkInModule(Module *Src) {
>    ModuleLinker TheLinker(Composite, IdentifiedStructTypes, Src,
>                           DiagnosticHandler);
> -  return TheLinker.run();
> +  bool RetCode = TheLinker.run();
> +  Composite->dropTriviallyDeadConstantArrays();
> +  return RetCode;
>  }
>  
>  //===----------------------------------------------------------------------===//
>