[all-commits] [llvm/llvm-project] b9d40a: [llvm-profgen] Improve sample profile density (#92...

[Lei Wang via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: b9d40a7ae4b3e5b9829eca8a497637c9fab6dd3e
      https://github.com/llvm/llvm-project/commit/b9d40a7ae4b3e5b9829eca8a497637c9fab6dd3e
  Author: Lei Wang <wlei at fb.com>
  Date:   2024-05-24 (Fri, 24 May 2024)

  Changed paths:
    M llvm/test/tools/llvm-profgen/profile-density.test
    M llvm/tools/llvm-profgen/ProfileGenerator.cpp
    M llvm/tools/llvm-profgen/ProfileGenerator.h

  Log Message:
  -----------
  [llvm-profgen] Improve sample profile density (#92144)

The profile density feature(the amount of samples in the profile
relative to the program size) is used to identify insufficient sample
issue and provide hints for user to increase sample count. A low-density
profile can be inaccurate due to statistical noise, which can hurt FDO
performance.

This change introduces two improvements to the current density work. 
1. The density calculation/definition is changed. Previously, the
density of a profile was calculated as the minimum density for all warm
functions (a function was considered warm if its total samples were
within the top N percent of the profile). However, there is a problem
that a high total sample profile can have a very low density, which
makes the density value unstable.
- Instead, we want to find a density number such that if a function's
density is below this value, it is considered low-density function. We
consider the whole profile is bad if a group of low-density functions
have the sum of samples that exceeds N percent cut-off of the total
samples.

- In implementation, we sort the function profiles by density, iterate
them in descending order and keep accumulating the body samples until
the sum exceeds the (100% - N) percentage of the total_samples, the
profile-density is the last(minimum) function-density of processed
functions. We introduce the a flag(`--profile-density-threshold`) for
this percentage threshold.

2. The density is now calculated based on final(compiler used) profiles
instead of merged context-less profiles.



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications