[all-commits] [llvm/llvm-project] 47b075: [SampleFDO] Persist profile staleness metrics into...

[ictwanglei via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 47b0758049eab549c51e77c8f8b5560498af5b80
      https://github.com/llvm/llvm-project/commit/47b0758049eab549c51e77c8f8b5560498af5b80
  Author: wlei <wlei at fb.com>
  Date:   2022-11-09 (Wed, 09 Nov 2022)

  Changed paths:
    M llvm/include/llvm/IR/MDBuilder.h
    M llvm/include/llvm/MC/MCObjectFileInfo.h
    M llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp
    M llvm/lib/IR/MDBuilder.cpp
    M llvm/lib/MC/MCObjectFileInfo.cpp
    M llvm/lib/Transforms/IPO/SampleProfile.cpp
    M llvm/test/Transforms/SampleProfile/profile-mismatch.ll
    M llvm/test/Transforms/SampleProfile/pseudo-probe-profile-mismatch.ll

  Log Message:
  -----------
  [SampleFDO] Persist profile staleness metrics into binary

With https://reviews.llvm.org/D136627, now we have the metrics for profile staleness based on profile statistics, monitoring the profile staleness in real-time can help user quickly identify performance issues. For a production scenario, the build is usually incremental and if we want the real-time metrics, we should store/cache all the old object's metrics somewhere and pull them in a post-build time. To make it more convenient, this patch add an option to persist them into the object binary, the metrics can be reported right away by decoding the binary rather than polling the previous stdout/stderrs from a cache system.

For implementation, it writes the statistics first into a new metadata section(llvm.stats) then encode into a special ELF `.llvm_stats` section. The section data is formatted as a list of key/value pair so that future statistics can be easily extended. This is also under a new switch(`-persist-profile-staleness`)

In terms of size overhead, the metrics are computed at module level, so the size overhead should be small, measured on one of our internal service, it costs less than < 1MB for a 10GB+ binary.

Reviewed By: wenlei

Differential Revision: https://reviews.llvm.org/D136698