[all-commits] [llvm/llvm-project] 76c782: [PGO] Collect dense AMDGPU block wave counts

[Yaxun (Sam) Liu via All-commits]

  Branch: refs/heads/users/yxsamliu/pgo-device-wave-counters
  Home:   https://github.com/llvm/llvm-project
  Commit: 76c7824f3da2727c233bf4736784b501215140bc
      https://github.com/llvm/llvm-project/commit/76c7824f3da2727c233bf4736784b501215140bc
  Author: Yaxun (Sam) Liu <yaxun.liu at amd.com>
  Date:   2026-09-28 (Mon, 28 Sep 2026)

  Changed paths:
    M llvm/docs/InstrProfileFormat.md
    M llvm/include/llvm/ProfileData/InstrProf.h
    M llvm/include/llvm/ProfileData/InstrProfData.inc
    M llvm/include/llvm/ProfileData/InstrProfReader.h
    M llvm/include/llvm/ProfileData/InstrProfWriter.h
    M llvm/lib/ProfileData/InstrProfReader.cpp
    M llvm/lib/ProfileData/InstrProfWriter.cpp
    M llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp
    A llvm/test/Transforms/PGOProfile/dense-wave-cfg.ll
    A llvm/test/Transforms/PGOProfile/dense-wave-instrumentation.ll
    A llvm/test/Transforms/PGOProfile/dense-wave-profile-use.ll
    A llvm/test/tools/llvm-profdata/dense-wave-layout.test
    M llvm/tools/llvm-profdata/llvm-profdata.cpp
    M llvm/unittests/ProfileData/InstrProfTest.cpp

  Log Message:
  -----------
  [PGO] Collect dense AMDGPU block wave counts

Wave visits are not additive across divergent control flow. Sparse scalar
counter sites can leave repeated loop blocks unmeasured, so their wave
frequencies cannot be reconstructed from entry and edge counts.

Append zero-step instrumentation for eligible unmeasured AMDGPU blocks,
keeping the existing lane and select counter indices unchanged. Exclude the
appended slots from lane-flow reconstruction and uniformity annotation.

Identify the layout with a profile variant bit, preserve it through raw and
indexed readers/writers, and select it automatically during profile use.
Keep sparse profiles readable and reject incompatible merges, including
concatenated raw profiles. Ignore empty merge-worker contexts.

Enable dense collection for ordinary AMDGPU IR-PGO by default, with the
hidden -pgo-instrument-dense-wave-counts option for debugging. Leave
context-sensitive, coverage, and temporal instrumentation unchanged.

Test loop counter placement, critical-edge splitting, EH insertion points,
scalar/select reconstruction, uniformity coverage and merge layout.



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications