[llvm-dev] Memory Requirements For Compiling PassBuilder.cpp

[Chandler Carruth via llvm-dev]

On Wed, Sep 4, 2019 at 2:17 PM David Blaikie via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Probably the simple thing that might help would be to split up the file a
> bit - especially the functions that include PassRegistry.def as part of
> their implementation. I'm guessing the parse.*Pass functions are the worst
> parts and having each of those in a separate file is probably helpful.
>

This may not help as much as you'd like. I suspect they're all going to
instantiate a similar set of templates, so you may end up with N expensive
files instead of 1.

=/


>
> Other than that - actually changing the constructs to use less memory
> overall, I'm not sure.
>

Things that would likely help a lot here:

1) Looking at the templates in PassManager.h that get instantiated per-pass
and have nested types or other constructs and refactoring them so that they
don't.
2) Looking at some of the especially large headers and refactoring them so
that we don't need to include them. AliasAnalysis.h and ScalarEvolution.h
would be my first targets, but that's just based on intuition, not
profiling.

I suspect there is a decent amount of improvement available for both #1 and
#2.

The other thing that might help would be splitting apart the parsing code
from the fixed pipeline building code -- those actually use somewhat
different bits of pass management infrastructure and so we might get a win
from separating their compilation.


>
> On Tue, Sep 3, 2019 at 7:12 PM Stefan Pintilie via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi Everyone,
>>
>> Recently I have realised that compiling the file PassBuilder.cpp takes a
>> significant amount of memory. I have seen anywhere between 700 MB to over
>> 1GB depending on which build compiler is used. This memory consumption
>> recently caused a couple of build bots that run on a smaller machine to run
>> out of memory and fail to build.
>>
>> I have no intention of changing any of the functionality of this file but
>> I was wondering if anyone could suggest a way to reduce its complexity (and
>> hopefully memory consumption).
>> Any ideas of how I could try to go about this? Would it be possible to
>> refactor parts into another file perhaps?
>>
>> Thank you in advance,
>> Stefan
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190904/6eaf913d/attachment.html>