[llvm] [CodeGen] Cache `shouldOptimizeForSize()` when computing spill weights (PR #214905)
Peter Rong via llvm-commits
llvm-commits at lists.llvm.org
Fri Aug 7 18:25:09 PDT 2026
https://github.com/DataCorrupted created https://github.com/llvm/llvm-project/pull/214905
`VirtRegAuxInfo::weightCalcHelper()` calls LiveIntervals::getSpillWeight() once per use/def operand of every virtual register, and `getSpillWeight()` evaluates `(PSI && llvm::shouldOptimizeForSize(MF, PSI, MBFI))` repeatedly. `shouldOptimizeForSize` is expensive: `O(#operands * #blocks)`.
We had a function with 20k blocks, and this inefficient repeated query caused severe delay, the function took 30+ minutes to build.
Since the result is the same for every query in this MF, and `VirtRegAuxInfo` already holds `MF`, `PSI` and `MBFI`, all fixed for the lifetime of the object, so the value is provably invariant across every `getSpillWeight()` call it makes. We should cache it.
>From 97d00334a6c3ff58dac17c50cff44512e80c0d1a Mon Sep 17 00:00:00 2001
From: Peter Rong <PeterRong at meta.com>
Date: Thu, 6 Aug 2026 21:24:50 -0700
Subject: [PATCH] [CodeGen] Memoize shouldOptimizeForSize() when computing
spill weights
VirtRegAuxInfo::weightCalcHelper() calls LiveIntervals::getSpillWeight() once
per use/def operand of every virtual register, and getSpillWeight() evaluates
if (PSI && llvm::shouldOptimizeForSize(MF, PSI, MBFI))
on every call. That result is invariant for the whole MachineFunction, but it is
not cheap to compute. shouldOptimizeForSize(MF, ...) short-circuits only when the
function carries a size attribute; otherwise it reaches
ProfileSummaryInfo::isFunctionHotInCallGraphNthPercentile(), which walks every
block of the function calling MachineBlockFrequencyInfo::getBlockProfileCount()
and returns early only if it finds a hot block. Each of those block queries
constructs three heap-allocating 128-bit APInts and performs a 128-bit multiply
and udiv. ProfileSummaryInfo memoizes percentile thresholds but not per-function
hotness, so for a function with no hot block this is a full walk on every call.
The cost is therefore O(#operands * #blocks) 128-bit divisions to produce a
single loop-invariant bool.
This is not hard to reach. -O2/-O3 attach no size attributes and -pgso defaults
to true, so with an instrumentation or sample profile any function that is not
hot at -pgso-cutoff-instr-prof pays the walk. Note that the PGSO cutoff (default
950000) is a higher count threshold than the ordinary hot cutoff, so a function
can be "entry hot" by the usual measure and still take the slow path. Only
RABasic and PBQP are affected: RegAllocGreedy constructs its VirtRegAuxInfo
without a ProfileSummaryInfo and never makes this query.
VirtRegAuxInfo already holds MF, PSI and MBFI, all fixed for the lifetime of the
object, so the value is provably invariant across every getSpillWeight() call it
makes. Memoize it, and add getSpillWeight() overloads that take the precomputed
bool. The existing ProfileSummaryInfo * overloads are retained and now delegate
to them, so StackSlotColoring and WebAssemblyRegColoring, which pass no
ProfileSummaryInfo at all, are unaffected.
This is a compile-time change only; no functional change intended.
Measured with llc -O2 -regalloc=basic on a generated cold function whose entry
count falls below the PGSO threshold, where N is both the number of switch cases
and the number of blocks. The assembly output is identical at every size:
N before after speedup
1000 1.12s 0.23s 4.9x
2000 3.98s 0.62s 6.4x
4000 16.48s 2.09s 7.9x
8000 64.58s 8.07s 8.0x
check-llvm: 48221 passed, 72 expectedly failed, 0 unexpected failures.
---
llvm/include/llvm/CodeGen/CalcSpillWeights.h | 7 +++++++
llvm/include/llvm/CodeGen/LiveIntervals.h | 11 +++++++++++
llvm/lib/CodeGen/CalcSpillWeights.cpp | 10 +++++++++-
llvm/lib/CodeGen/LiveIntervals.cpp | 18 ++++++++++++++++--
4 files changed, 43 insertions(+), 3 deletions(-)
diff --git a/llvm/include/llvm/CodeGen/CalcSpillWeights.h b/llvm/include/llvm/CodeGen/CalcSpillWeights.h
index 3809356f16d6c..3f9d98001adcb 100644
--- a/llvm/include/llvm/CodeGen/CalcSpillWeights.h
+++ b/llvm/include/llvm/CodeGen/CalcSpillWeights.h
@@ -10,6 +10,7 @@
#define LLVM_CODEGEN_CALCSPILLWEIGHTS_H
#include "llvm/CodeGen/SlotIndexes.h"
+#include <optional>
namespace llvm {
@@ -51,6 +52,12 @@ class VirtRegMap;
ProfileSummaryInfo *PSI;
const MachineBlockFrequencyInfo &MBFI;
+ /// Memoized llvm::shouldOptimizeForSize(&MF, PSI, &MBFI).
+ std::optional<bool> OptForSize;
+
+ /// Lazily computes and caches the above.
+ bool shouldOptimizeForSize();
+
/// Returns true if Reg of live interval LI is used in instruction with many
/// operands like STATEPOINT.
bool isLiveAtStatepointVarArg(LiveInterval &LI);
diff --git a/llvm/include/llvm/CodeGen/LiveIntervals.h b/llvm/include/llvm/CodeGen/LiveIntervals.h
index 99a0122265d3b..9b99056e6ba2a 100644
--- a/llvm/include/llvm/CodeGen/LiveIntervals.h
+++ b/llvm/include/llvm/CodeGen/LiveIntervals.h
@@ -130,6 +130,17 @@ class LiveIntervals {
const MachineBasicBlock *MBB,
ProfileSummaryInfo *PSI = nullptr);
+ /// Variants taking a precomputed \p OptForSize rather than deriving it from a
+ /// ProfileSummaryInfo.
+ LLVM_ABI static float getSpillWeight(bool isDef, bool isUse,
+ const MachineBlockFrequencyInfo *MBFI,
+ const MachineInstr &MI, bool OptForSize);
+
+ LLVM_ABI static float getSpillWeight(bool isDef, bool isUse,
+ const MachineBlockFrequencyInfo *MBFI,
+ const MachineBasicBlock *MBB,
+ bool OptForSize);
+
LiveInterval &getInterval(Register Reg) {
if (hasInterval(Reg))
return *VirtRegIntervals[Reg.id()];
diff --git a/llvm/lib/CodeGen/CalcSpillWeights.cpp b/llvm/lib/CodeGen/CalcSpillWeights.cpp
index 51cce4eb78c94..f07ea2bff50d8 100644
--- a/llvm/lib/CodeGen/CalcSpillWeights.cpp
+++ b/llvm/lib/CodeGen/CalcSpillWeights.cpp
@@ -15,6 +15,7 @@
#include "llvm/CodeGen/MachineLoopInfo.h"
#include "llvm/CodeGen/MachineOperand.h"
#include "llvm/CodeGen/MachineRegisterInfo.h"
+#include "llvm/CodeGen/MachineSizeOpts.h"
#include "llvm/CodeGen/StackMaps.h"
#include "llvm/CodeGen/TargetInstrInfo.h"
#include "llvm/CodeGen/TargetRegisterInfo.h"
@@ -30,6 +31,12 @@ using namespace llvm;
#define DEBUG_TYPE "calcspillweights"
+bool VirtRegAuxInfo::shouldOptimizeForSize() {
+ if (!OptForSize)
+ OptForSize = PSI && llvm::shouldOptimizeForSize(&MF, PSI, &MBFI);
+ return *OptForSize;
+}
+
void VirtRegAuxInfo::calculateSpillWeightsAndHints() {
LLVM_DEBUG(dbgs() << "********** Compute Spill Weights **********\n"
<< "********** Function: " << MF.getName() << '\n');
@@ -318,7 +325,8 @@ float VirtRegAuxInfo::weightCalcHelper(LiveInterval &LI) {
// Calculate instr weight.
bool Reads, Writes;
std::tie(Reads, Writes) = MI->readsWritesVirtualRegister(LI.reg());
- Weight = LiveIntervals::getSpillWeight(Writes, Reads, &MBFI, *MI, PSI);
+ Weight = LiveIntervals::getSpillWeight(Writes, Reads, &MBFI, *MI,
+ shouldOptimizeForSize());
// Give extra weight to what looks like a loop induction variable update.
if (Writes && IsExiting && LIS.isLiveOutOfMBB(LI, MBB))
diff --git a/llvm/lib/CodeGen/LiveIntervals.cpp b/llvm/lib/CodeGen/LiveIntervals.cpp
index 3da53566ae0c2..1a305eba9db58 100644
--- a/llvm/lib/CodeGen/LiveIntervals.cpp
+++ b/llvm/lib/CodeGen/LiveIntervals.cpp
@@ -912,11 +912,25 @@ float LiveIntervals::getSpillWeight(bool isDef, bool isUse,
const MachineBlockFrequencyInfo *MBFI,
const MachineBasicBlock *MBB,
ProfileSummaryInfo *PSI) {
- float Weight = isDef + isUse;
const auto *MF = MBB->getParent();
+ return getSpillWeight(isDef, isUse, MBFI, MBB,
+ PSI && llvm::shouldOptimizeForSize(MF, PSI, MBFI));
+}
+
+float LiveIntervals::getSpillWeight(bool isDef, bool isUse,
+ const MachineBlockFrequencyInfo *MBFI,
+ const MachineInstr &MI, bool OptForSize) {
+ return getSpillWeight(isDef, isUse, MBFI, MI.getParent(), OptForSize);
+}
+
+float LiveIntervals::getSpillWeight(bool isDef, bool isUse,
+ const MachineBlockFrequencyInfo *MBFI,
+ const MachineBasicBlock *MBB,
+ bool OptForSize) {
+ float Weight = isDef + isUse;
// When optimizing for size we only consider the codesize impact of spilling
// the register, not the runtime impact.
- if (PSI && llvm::shouldOptimizeForSize(MF, PSI, MBFI))
+ if (OptForSize)
return Weight;
return Weight * MBFI->getBlockFreqRelativeToEntryBlock(MBB);
}
More information about the llvm-commits
mailing list