[PATCH] D93264: [CSSPGO] Introducing distribution factor for pseudo probe

[Hongtao Yu via Phabricator via cfe-commits]

hoy created this revision.
Herald added subscribers: dexonsmith, wenlei, hiraditya.
hoy requested review of this revision.
Herald added subscribers: llvm-commits, cfe-commits, jdoerfert.
Herald added projects: clang, LLVM.

Sample re-annotation is required in LTO time to achieve a reasonable post-inline profile quality. However, we have seen that such LTO-time re-annotation degrades profile quality. This is mainly caused by preLTO code duplication that is done by passes such as loop unrolling, jump threading, indirect call promotion etc, where samples corresponding to a source location are aggregated multiple times due to the duplicates. In this change we are introducing a concept of distribution factor for pseudo probes so that samples can be distributed for duplicated probes scaled by a factor, with the hope that optimizations duplicating code well maintains the branch frequency information (BFI) based on which probe distribution factors are calculated. Distribution factors are updated at the end of preLTO pipeline to reflect an estimated portion of the real execution count.

This change also introduces a pseudo probe verifier that can be run after each IR passes to detect duplicated pseudo probes.

A saturated distribution factor stands for 1.0. A duplicated probe will carry a factor with the value ranged from 0.0 to 1.0. A 64-bit integral distribution factor field that represents [0.0, 1.0] is associated to each block probe. Unfortunately this cannot be done for callsite probes due to the size limitation of a 32-bit Dwarf discriminator. A 7-bit distribution factor is used instead.

Changes are also needed to the sample profile inliner to deal with prorated callsite counts. The changes have a dependency on the CS inliner and thus not included here.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D93264

Files:
  clang/test/CodeGen/pseudo-probe-emit.c
  llvm/include/llvm/IR/IntrinsicInst.h
  llvm/include/llvm/IR/Intrinsics.td
  llvm/include/llvm/IR/PseudoProbe.h
  llvm/include/llvm/Passes/StandardInstrumentations.h
  llvm/include/llvm/Transforms/IPO/SampleProfileProbe.h
  llvm/lib/IR/PseudoProbe.cpp
  llvm/lib/Passes/PassBuilder.cpp
  llvm/lib/Passes/PassRegistry.def
  llvm/lib/Passes/StandardInstrumentations.cpp
  llvm/lib/Transforms/IPO/SampleProfile.cpp
  llvm/lib/Transforms/IPO/SampleProfileProbe.cpp
  llvm/test/Transforms/SampleProfile/Inputs/pseudo-probe-update.prof
  llvm/test/Transforms/SampleProfile/pseudo-probe-emit-inline.ll
  llvm/test/Transforms/SampleProfile/pseudo-probe-emit.ll
  llvm/test/Transforms/SampleProfile/pseudo-probe-profile.ll
  llvm/test/Transforms/SampleProfile/pseudo-probe-update.ll
  llvm/test/Transforms/SampleProfile/pseudo-probe-verify.ll

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D93264.311763.patch
Type: text/x-patch
Size: 39878 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20201215/c1ac3a2b/attachment-0001.bin>