[all-commits] [llvm/llvm-project] a1b347: [CodeGen] Use validated wave counts for AMDGPU spi...
Yaxun (Sam) Liu via All-commits
all-commits at lists.llvm.org
Mon Sep 28 20:24:21 PDT 2026
Branch: refs/heads/users/yxsamliu/pgo-wave-profiled-spill
Home: https://github.com/llvm/llvm-project
Commit: a1b347cdd7a3adbcd47b04d42cf1e105f95bc1b1
https://github.com/llvm/llvm-project/commit/a1b347cdd7a3adbcd47b04d42cf1e105f95bc1b1
Author: Yaxun (Sam) Liu <yaxun.liu at amd.com>
Date: 2026-09-28 (Mon, 28 Sep 2026)
Changed paths:
M llvm/lib/CodeGen/SpillPlacement.cpp
A llvm/test/CodeGen/AMDGPU/wave-profile-spill.mir
Log Message:
-----------
[CodeGen] Use validated wave counts for AMDGPU spill costs
Lane-based block frequencies can understate the cost of a spill in
divergent GPU code: a wave still executes a block with only some lanes
active. Use measured block-wave counts to weight SpillPlacement's costs
relative to the original entry-wave count.
Opt in on AMDGPU only. Accept a measured count only when its IR block
maps uniquely to a machine block with matching predecessors and
successors. Keep the existing MBFI cost for unmeasured or rejected
blocks, and leave branch probabilities and general BFI unchanged.
Commit: 623ddd98a1f8048ef1cec12a992a762c40281585
https://github.com/llvm/llvm-project/commit/623ddd98a1f8048ef1cec12a992a762c40281585
Author: Yaxun (Sam) Liu <yaxun.liu at amd.com>
Date: 2026-09-28 (Mon, 28 Sep 2026)
Changed paths:
M llvm/lib/CodeGen/SpillPlacement.cpp
A llvm/test/CodeGen/AMDGPU/wave-profile-spill-zero-cost.mir
Log Message:
-----------
[CodeGen] Keep wave-profiled spill frequencies positive
SpillPlacement expects positive block weights, but a valid wave profile can
record zero executions for a CFG-reachable block. Giving such a block zero
spill cost can make the allocator choose a very different placement.
Clamp every accepted wave-derived frequency to at least one, as we already
do for nonzero counts that round down to zero. Unmeasured or rejected blocks
still use their existing MBFI frequency. Add a focused MIR test for a valid
zero-wave record.
This pattern arose in a profiled Composable Kernel convolution case. With
the separate spill correctness fixes and partial spilling enabled, the
zero-cost policy failed two CPU-reference checks; the positive floor passed
both. The test checks the cost directly; the application result was checked
separately on gfx950.
Commit: 25e66e4a639a1008dfc7caeb28ec3770b0e7bc91
https://github.com/llvm/llvm-project/commit/25e66e4a639a1008dfc7caeb28ec3770b0e7bc91
Author: Yaxun (Sam) Liu <yaxun.liu at amd.com>
Date: 2026-09-28 (Mon, 28 Sep 2026)
Changed paths:
M llvm/lib/CodeGen/SpillPlacement.cpp
M llvm/test/CodeGen/AMDGPU/wave-profile-spill-zero-cost.mir
M llvm/test/CodeGen/AMDGPU/wave-profile-spill.mir
Log Message:
-----------
[CodeGen] Enable validated AMDGPU wave spill costs by default
Use available validated wave counts for AMDGPU spill placement without an
explicit opt-in. Keep the existing target, mapping and normalization checks
and the ordinary block-frequency fallback for unavailable or rejected data.
Retain -enable-wave-profiled-spill=false for debugging and matched performance
comparisons. Test default behavior, explicit disabling, equivalence to explicit
enabling, fallback cases and the positive frequency floor.
Compare: https://github.com/llvm/llvm-project/compare/2fc9681b49d6...25e66e4a639a
To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications
More information about the All-commits
mailing list