[llvm-dev] Memory Requirements For Compiling PassBuilder.cpp

[Stefan Pintilie via llvm-dev]

Thank you both for the advice.
I'm going to try each of these ideas (including splitting the functions 
that include PassRegistry.def, looking at the templates in PassManager.h 
and looking at the large header files) and see what I get.
If any of those give me gains I'll post a patch.

Thank you for the help!
Stefan



From:   Chandler Carruth <chandlerc at gmail.com>
To:     David Blaikie <dblaikie at gmail.com>
Cc:     Stefan Pintilie <stefanp at ca.ibm.com>, "llvm-dev at lists.llvm.org" 
<llvm-dev at lists.llvm.org>
Date:   2019/09/04 07:51 PM
Subject:        [EXTERNAL] Re: [llvm-dev] Memory Requirements For 
Compiling PassBuilder.cpp



On Wed, Sep 4, 2019 at 2:17 PM David Blaikie via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
Probably the simple thing that might help would be to split up the file a 
bit - especially the functions that include PassRegistry.def as part of 
their implementation. I'm guessing the parse.*Pass functions are the worst 
parts and having each of those in a separate file is probably helpful.

This may not help as much as you'd like. I suspect they're all going to 
instantiate a similar set of templates, so you may end up with N expensive 
files instead of 1.

=/
 

Other than that - actually changing the constructs to use less memory 
overall, I'm not sure. 

Things that would likely help a lot here:

1) Looking at the templates in PassManager.h that get instantiated 
per-pass and have nested types or other constructs and refactoring them so 
that they don't.
2) Looking at some of the especially large headers and refactoring them so 
that we don't need to include them. AliasAnalysis.h and ScalarEvolution.h 
would be my first targets, but that's just based on intuition, not 
profiling.

I suspect there is a decent amount of improvement available for both #1 
and #2.

The other thing that might help would be splitting apart the parsing code 
from the fixed pipeline building code -- those actually use somewhat 
different bits of pass management infrastructure and so we might get a win 
from separating their compilation.
 

On Tue, Sep 3, 2019 at 7:12 PM Stefan Pintilie via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
Hi Everyone,

Recently I have realised that compiling the file PassBuilder.cpp takes a 
significant amount of memory. I have seen anywhere between 700 MB to over 
1GB depending on which build compiler is used. This memory consumption 
recently caused a couple of build bots that run on a smaller machine to 
run out of memory and fail to build.

I have no intention of changing any of the functionality of this file but 
I was wondering if anyone could suggest a way to reduce its complexity 
(and hopefully memory consumption).
Any ideas of how I could try to go about this? Would it be possible to 
refactor parts into another file perhaps?

Thank you in advance,
Stefan

_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190906/d04bcb56/attachment.html>