[all-commits] [llvm/llvm-project] a1b347: [CodeGen] Use validated wave counts for AMDGPU spi...

Yaxun (Sam) Liu via All-commits all-commits at lists.llvm.org
Mon Sep 28 20:24:21 PDT 2026


  Branch: refs/heads/users/yxsamliu/pgo-wave-profiled-spill
  Home:   https://github.com/llvm/llvm-project
  Commit: a1b347cdd7a3adbcd47b04d42cf1e105f95bc1b1
      https://github.com/llvm/llvm-project/commit/a1b347cdd7a3adbcd47b04d42cf1e105f95bc1b1
  Author: Yaxun (Sam) Liu <yaxun.liu at amd.com>
  Date:   2026-09-28 (Mon, 28 Sep 2026)

  Changed paths:
    M llvm/lib/CodeGen/SpillPlacement.cpp
    A llvm/test/CodeGen/AMDGPU/wave-profile-spill.mir

  Log Message:
  -----------
  [CodeGen] Use validated wave counts for AMDGPU spill costs

Lane-based block frequencies can understate the cost of a spill in
divergent GPU code: a wave still executes a block with only some lanes
active. Use measured block-wave counts to weight SpillPlacement's costs
relative to the original entry-wave count.

Opt in on AMDGPU only. Accept a measured count only when its IR block
maps uniquely to a machine block with matching predecessors and
successors. Keep the existing MBFI cost for unmeasured or rejected
blocks, and leave branch probabilities and general BFI unchanged.


  Commit: 623ddd98a1f8048ef1cec12a992a762c40281585
      https://github.com/llvm/llvm-project/commit/623ddd98a1f8048ef1cec12a992a762c40281585
  Author: Yaxun (Sam) Liu <yaxun.liu at amd.com>
  Date:   2026-09-28 (Mon, 28 Sep 2026)

  Changed paths:
    M llvm/lib/CodeGen/SpillPlacement.cpp
    A llvm/test/CodeGen/AMDGPU/wave-profile-spill-zero-cost.mir

  Log Message:
  -----------
  [CodeGen] Keep wave-profiled spill frequencies positive

SpillPlacement expects positive block weights, but a valid wave profile can
record zero executions for a CFG-reachable block. Giving such a block zero
spill cost can make the allocator choose a very different placement.

Clamp every accepted wave-derived frequency to at least one, as we already
do for nonzero counts that round down to zero. Unmeasured or rejected blocks
still use their existing MBFI frequency. Add a focused MIR test for a valid
zero-wave record.

This pattern arose in a profiled Composable Kernel convolution case. With
the separate spill correctness fixes and partial spilling enabled, the
zero-cost policy failed two CPU-reference checks; the positive floor passed
both. The test checks the cost directly; the application result was checked
separately on gfx950.


  Commit: 25e66e4a639a1008dfc7caeb28ec3770b0e7bc91
      https://github.com/llvm/llvm-project/commit/25e66e4a639a1008dfc7caeb28ec3770b0e7bc91
  Author: Yaxun (Sam) Liu <yaxun.liu at amd.com>
  Date:   2026-09-28 (Mon, 28 Sep 2026)

  Changed paths:
    M llvm/lib/CodeGen/SpillPlacement.cpp
    M llvm/test/CodeGen/AMDGPU/wave-profile-spill-zero-cost.mir
    M llvm/test/CodeGen/AMDGPU/wave-profile-spill.mir

  Log Message:
  -----------
  [CodeGen] Enable validated AMDGPU wave spill costs by default

Use available validated wave counts for AMDGPU spill placement without an
explicit opt-in. Keep the existing target, mapping and normalization checks
and the ordinary block-frequency fallback for unavailable or rejected data.

Retain -enable-wave-profiled-spill=false for debugging and matched performance
comparisons. Test default behavior, explicit disabling, equivalence to explicit
enabling, fallback cases and the positive frequency floor.


Compare: https://github.com/llvm/llvm-project/compare/2fc9681b49d6...25e66e4a639a

To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications


More information about the All-commits mailing list