[PATCH] D79831: [PGO] Improve the working set size heuristics under the partial sample PGO.

[Hiroshi Yamauchi via Phabricator via llvm-commits]

yamauchi created this revision.
yamauchi added a reviewer: davidxl.
Herald added subscribers: dang, arphaman, dexonsmith, steven_wu, hiraditya, eraman, mgorny.
Herald added a project: LLVM.
yamauchi updated this revision to Diff 263584.
yamauchi added a comment.

clang-format.


yamauchi added a comment.

I'm going to split this patch into parts:

- Use Module::setModuleFlag rather than addModuleFlag for the ProfileSummary metadata.
- Add the PartialProfileRatio field in ProfileSummary metadata.
- Compute the block count through thin LTO.
- The ProfileSummaryInfo change.


The working set size heuristics (ProfileSummaryInfo::hasHugeWorkingSetSize)
under the partial sample PGO may not be accurate because the profile is partial
and the number of hot profile counters in the ProfileSummary may not reflect the
actual working set size of the program being compiled.

To improve this, the (approximated) ratio of the the number of profile counters
of the program being compiled to the number of profile counters in the partial
sample profile is computed (which is called the partial profile ratio) and the
working set size of the profile is scaled by this ratio to reflect the working
set size of the program being compiled and used for the working set size
heuristics.

The partial profile ratio is approximated based on the number of the basic
blocks in the program and the NumCounts field in the ProfileSummary and computed
through the thin LTO indexing. This means that there is the limitation that the
scaled working set size is available to the thin LTO post link passes only.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D79831

Files:
  llvm/include/llvm/Bitcode/LLVMBitCodes.h
  llvm/include/llvm/IR/Module.h
  llvm/include/llvm/IR/ModuleSummaryIndex.h
  llvm/include/llvm/IR/ProfileSummary.h
  llvm/lib/Analysis/ModuleSummaryAnalysis.cpp
  llvm/lib/Analysis/ProfileSummaryInfo.cpp
  llvm/lib/AsmParser/LLLexer.cpp
  llvm/lib/AsmParser/LLParser.cpp
  llvm/lib/AsmParser/LLParser.h
  llvm/lib/AsmParser/LLToken.h
  llvm/lib/Bitcode/Reader/BitcodeAnalyzer.cpp
  llvm/lib/Bitcode/Reader/BitcodeReader.cpp
  llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
  llvm/lib/IR/AsmWriter.cpp
  llvm/lib/IR/Module.cpp
  llvm/lib/IR/ModuleSummaryIndex.cpp
  llvm/lib/IR/ProfileSummary.cpp
  llvm/lib/LTO/LTOBackend.cpp
  llvm/lib/Transforms/IPO/FunctionImport.cpp
  llvm/test/Assembler/thinlto-vtable-summary.ll
  llvm/test/Bitcode/summary_version.ll
  llvm/test/Bitcode/thinlto-alias.ll
  llvm/test/Bitcode/thinlto-alias2.ll
  llvm/test/Bitcode/thinlto-function-summary-callgraph-cast.ll
  llvm/test/Bitcode/thinlto-function-summary-callgraph-pgo.ll
  llvm/test/Bitcode/thinlto-function-summary-callgraph-profile-summary.ll
  llvm/test/Bitcode/thinlto-function-summary-callgraph-relbf.ll
  llvm/test/Bitcode/thinlto-function-summary-callgraph-sample-profile-summary.ll
  llvm/test/Bitcode/thinlto-function-summary-callgraph.ll
  llvm/test/Bitcode/thinlto-function-summary-originalnames.ll
  llvm/test/Bitcode/thinlto-function-summary.ll
  llvm/test/ThinLTO/X86/distributed_indexes.ll
  llvm/test/Transforms/PGOProfile/unreachable_bb.ll
  llvm/test/tools/llvm-lto/thinlto.ll
  llvm/unittests/Analysis/ProfileSummaryInfoTest.cpp
  llvm/unittests/IR/CMakeLists.txt
  llvm/unittests/IR/ModuleSummaryIndexTest.cpp
  llvm/unittests/IR/ModuleTest.cpp
  llvm/unittests/ProfileData/SampleProfTest.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D79831.263584.patch
Type: text/x-patch
Size: 46351 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20200513/4c965999/attachment-0001.bin>