[cfe-dev] [RFC] Clearing Clang AST before running backend optimizations/codegen to save memory

[Arthur Eubanks via cfe-dev]

We keep around the Clang AST when we do backend optimizations on the IR.
This causes the peak memory usage to be more than necessary since (I
believe) generally we don't need the Clang AST when running optimizations.
This gives us more room to work with things like caching analyses, at least
for frontend compilations.

Measuring the effects of this when building LLVM's PassBuilder.cpp (longest
LLVM file to compile), I measured a drop of peak memory usage
(/usr/bin/time's max rss) from ~1.3-1.4G to ~1.0G.

There are still a couple issues I haven't dug too deeply into yet, mostly
to do with cleaning things up when freeing memory, so right now it's only
enabled with -disable-free which works around those issues. Most clang
tests pass with this patch; there are a couple things that crash (e.g.
-Rpass, clang interpreter) where we can investigate further or just disable
this feature.

Prototype: https://reviews.llvm.org/D109781
llvm-compile-time-tracker
<https://llvm-compile-time-tracker.com/compare.php?from=167ff5280d7fcad731810d5d2bf10561ed2adacc&to=b08fcae3a02d5ebe58afd8f8658d798b62ff8eb7&stat=max-rss>
memory
metrics

Any concerns with this?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20210917/8180ef8d/attachment.html>