Free memory used by initializers of global variables, created during linking

[Rafael Espíndola]

+  /// HACK: constants are currently owned by LLVMContext. It is conceptually
+  /// awkard for Module to delete them.


N.B. might be more canonical than hack :-)

Add something on the lines of: This can then only be called where all
uses of the context are understood.

+  // Go through ArrayConstants and destroy constants without usage.

The repeated comment in each loop doesn't add a lot of value. Just put
a larger one at the start of the function saying that it goes over all
constant stores.

Can at least some of these loops use range loops?

What about ExprConstants?


On 8 January 2015 at 13:02, Manman Ren <mren at apple.com> wrote:
> Hi All,
>
> ——————— The problem
> Here is a simple example where we can have memory issue during LTO:
> Suppose we have 3 .ll files (a.ll b.ll c.ll), each with "@llvm.compiler.used = appending global [3 x i8*]”.
>
> We currently create a new GlobalVariable with a new ConstantArray as its initializer (see linkAppendingVarProto and linkAppendingVarInit), when linking in each .ll file.
> With our example, when linking b.ll, we will create a GlobalVariable (V_2) and a ConstantArray (CA_2) with 3+3 operands;
> when linking c.ll, we will create another GlobalVariable (V_3) and a ConstantArray (CA_3) with 6+3 operands, we then delete the GlobalVariable V_2, but the memory occupied by CA_2 is not released.
>
> This will cause memory explosion when we are looking at 500 .ll files, each with "@llvm.compiler.used = appending global [100 x i8*]”.
> A ConstantArray will be created when linking in each .ll file and its number of operands will be a sum of the operands from the previous .ll files.
> We will create 499 ConstantArrays, with number of operands from 200, 300, …, 100*500. The total amount of memory allocated will be O(n^2) where n is # of .ll files linked.
>
> The application I was looking at used 18GB memory for these temporary ConstantArrays.
>
> ——————— The proposed solution
> After off-line discussions with Rafael and Duncan, we are adding Module::dropTriviallyDeadConstants that gets the LLVMContextImpl, walks the various constant stores and removes the ones with no uses.
>
> Currently the owner of the Constants is LLVMContext, so we added comments for Module::dropTriviallyDeadConstants.
> +  /// Destroy constants in LLVMContext if they are not used.
> +  /// HACK: constants are currently owned by LLVMContext. It is conceptually
> +  /// awkard for Module to delete them.
>
> The plan is to move constants to “Module” after we are done with moving Metadata to “Module”.
>
> The patch is attached, please review!
>
> Thanks,
> Manman