[llvm] 057e784 - [llvm-profgen] Clean up unnecessary memory reservations between phases.

[Hongtao Yu via llvm-commits]

Author: Hongtao Yu
Date: 2022-02-01T12:48:08-08:00
New Revision: 057e784b0962a7c5a17e858932bb6f03c7676c47

URL: https://github.com/llvm/llvm-project/commit/057e784b0962a7c5a17e858932bb6f03c7676c47
DIFF: https://github.com/llvm/llvm-project/commit/057e784b0962a7c5a17e858932bb6f03c7676c47.diff

LOG: [llvm-profgen] Clean up unnecessary memory reservations between phases.

Cleaning up data structures that are not used after a certain point. This further brings down peak memory usage by 15% for a large benchmark.

Before:
   note: Before parsePerfTraces
   note: VM: 40.73 GB   RSS: 39.18 GB
   note: Before parseAndAggregateTrace
   note: VM: 40.73 GB   RSS: 39.18 GB
   note: After parseAndAggregateTrace
   note: VM: 88.93 GB   RSS: 87.97 GB
   note: Before generateUnsymbolizedProfile
   note: VM: 88.95 GB   RSS: 87.99 GB
   note: After generateUnsymbolizedProfile
   note: VM: 93.50 GB   RSS: 92.53 GB
   note: After computeSizeForProfiledFunctions
   note: VM: 101.13 GB   RSS: 99.36 GB
   note: After generateProbeBasedProfile
   note: VM: 215.61 GB   RSS: 210.88 GB
   note: After postProcessProfiles
   note: VM: 237.48 GB   RSS: 212.50 GB

After:
   note: Before parsePerfTraces
   note: VM: 40.73 GB   RSS: 39.18 GB
   note: Before parseAndAggregateTrace
   note: VM: 40.73 GB   RSS: 39.18 GB
   note: After parseAndAggregateTrace
   note: VM: 88.93 GB   RSS: 87.96 GB
   note: Before generateUnsymbolizedProfile
   note: VM: 88.95 GB   RSS: 87.97 GB
   note: After generateUnsymbolizedProfile
   note: VM: 93.50 GB   RSS: 92.51 GB
   note: After computeSizeForProfiledFunctions
   note: VM: 93.50 GB   RSS: 92.53 GB
   note: After generateProbeBasedProfile
   note: VM: 164.87 GB   RSS: 163.55 GB
   note: After postProcessProfiles
   note: VM: 182.28 GB   RSS: 179.43 GB

Reviewed By: wenlei, wlei

Differential Revision: https://reviews.llvm.org/D118677

Added: 
    

Modified: 
    llvm/tools/llvm-profgen/PerfReader.cpp
    llvm/tools/llvm-profgen/llvm-profgen.cpp

Removed: 
    


################################################################################
diff  --git a/llvm/tools/llvm-profgen/PerfReader.cpp b/llvm/tools/llvm-profgen/PerfReader.cpp
index 46422ce470898..98b4c7cdf169a 100644
--- a/llvm/tools/llvm-profgen/PerfReader.cpp
+++ b/llvm/tools/llvm-profgen/PerfReader.cpp
@@ -1212,6 +1212,7 @@ void PerfScriptReader::parsePerfTraces() {
   warnTruncatedStack();
   warnInvalidRange();
   generateUnsymbolizedProfile();
+  AggregatedSamples.clear();
 
   if (SkipSymbolization)
     writeUnsymbolizedProfile(OutputFilename);

diff  --git a/llvm/tools/llvm-profgen/llvm-profgen.cpp b/llvm/tools/llvm-profgen/llvm-profgen.cpp
index f092df04d52b3..b2728ae41872a 100644
--- a/llvm/tools/llvm-profgen/llvm-profgen.cpp
+++ b/llvm/tools/llvm-profgen/llvm-profgen.cpp
@@ -158,6 +158,9 @@ int main(int argc, const char *argv[]) {
       ProfileGeneratorBase::create(Binary.get(), Reader->getSampleCounters(),
                                    Reader->profileIsCSFlat());
   Generator->generateProfile();
+  // The Reader object, espcially its SampleCounters field, is not needed at
+  // this point, so releasing it to reduce peak memory usage.
+  Reader.release();
   Generator->write();
 
   return EXIT_SUCCESS;