[llvm] [AArch64][RegAlloc] Enable callee-saved register optimization for AArch64 (PR #214773)
via llvm-commits
llvm-commits at lists.llvm.org
Fri Aug 7 08:21:52 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-backend-aarch64
Author: Shreeyash Pandey (bojle)
<details>
<summary>Changes</summary>
This patch enables the AArch64 backend to use the new CSR (callee-saved register) cost model introduced in https://github.com/llvm/llvm-project/pull/177226, which changed the computation of CSRCost.
This is a sister patch to https://github.com/llvm/llvm-project/pull/188609.
Fixes https://github.com/llvm/llvm-project/issues/214748.
I swept across different scale values to find the optimal one for this arch. The table below shows the % improvement in SPECrate vs. the trunk baseline (positive = faster). Values are median of 3 iterations.
```
bench 30 40 50 60 70 80 90* 100* 110 120
----------- ----- ----- ----- ----- ----- ----- ----- ----- ----- -----
perlbench_r +0.5 +1.2 +0.5 +1.7 +0.7 +1.0 +0.9 +2.1 +1.7 +2.1
gcc_r +0.4 +0.8 +1.0 +1.2 +1.3 +1.2 +0.9 +1.3 +1.5 +1.3
omnetpp_r +0.2 -0.7 0.0 -0.3 -1.2 -0.8 +0.2 -0.2 0.0 -0.8
mcf_r -0.7 -0.7 -0.7 -0.3 -0.8 -0.8 -0.7 -0.1 -0.3 -0.1
x264_r +0.6 +1.2 +1.2 +1.2 +1.2 +1.2 +1.2 +1.2 +1.2 +1.2
deepsjeng_r -0.5 -0.7 -0.5 -0.5 -0.5 -0.5 -0.7 -0.9 -1.3 -1.3
xz_r -1.6 -0.2 -0.9 -1.2 -0.7 -1.2 -0.5 -1.6 -1.4 -1.6
```
At scale 100, the wins are concentrated in the call-heavy benchmarks: perlbench_r (+2.1%), gcc_r (+1.3%), and x264_r (+1.2%), while omnetpp_r and mcf_r are essentially flat. The regressions at this scale are deepsjeng_r (−0.9%) and xz_r (−1.6%); both quite small, closer to run-to-run noise. CSR Value 100 looks like the most favourable scale value for this AArch64.
This change updates 14 AArch64 CodeGen tests, whose diffs are register-allocation churn from the new cost model.
At the moment, this change in the csr-cost-scale is applied globally to all targets (via the cl::init default), so CodeGen tests on other targets may also see similar churn. It was suggested in #<!-- -->188609 that the existing cost calculation be refactored so that each target can be enabled incrementally. Once that refactor lands, I'll rebase this to make the change AArch64-specific.
---
Patch is 85.96 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/214773.diff
16 Files Affected:
- (modified) llvm/lib/CodeGen/RegAllocGreedy.cpp (+1-1)
- (modified) llvm/lib/Target/AArch64/AArch64RegisterInfo.h (-6)
- (modified) llvm/test/CodeGen/AArch64/cgp-usubo.ll (+11-10)
- (modified) llvm/test/CodeGen/AArch64/cmpxchg-idioms.ll (+39-38)
- (modified) llvm/test/CodeGen/AArch64/combine-comparisons-by-cse.ll (+17-20)
- (modified) llvm/test/CodeGen/AArch64/csr-split.ll (+32-28)
- (modified) llvm/test/CodeGen/AArch64/div-i256.ll (+61-61)
- (modified) llvm/test/CodeGen/AArch64/pr166870.ll (+3-2)
- (modified) llvm/test/CodeGen/AArch64/pr51516.mir (+195-5)
- (modified) llvm/test/CodeGen/AArch64/sme-callee-save-restore-pairs.ll (+12-10)
- (modified) llvm/test/CodeGen/AArch64/sme-peephole-opts.ll (+21-14)
- (modified) llvm/test/CodeGen/AArch64/sme-streaming-checkvl.ll (+17-15)
- (modified) llvm/test/CodeGen/AArch64/sme-streaming-compatible-interface.ll (+19-15)
- (modified) llvm/test/CodeGen/AArch64/spill-reload-remarks.ll (+1-1)
- (modified) llvm/test/CodeGen/AArch64/statepoint-call-lowering.ll (+12-11)
- (modified) llvm/test/CodeGen/AArch64/sve-breakdown-scalable-vectortype.ll (+252-315)
``````````diff
diff --git a/llvm/lib/CodeGen/RegAllocGreedy.cpp b/llvm/lib/CodeGen/RegAllocGreedy.cpp
index ef23468b1e752..41ff45bfe1e87 100644
--- a/llvm/lib/CodeGen/RegAllocGreedy.cpp
+++ b/llvm/lib/CodeGen/RegAllocGreedy.cpp
@@ -118,7 +118,7 @@ CSRFirstTimeCost("regalloc-csr-first-time-cost",
static cl::opt<unsigned> CSRCostScale(
"regalloc-csr-cost-scale",
cl::desc("Scale for the callee-saved register cost, in percentage."),
- cl::init(80), cl::Hidden);
+ cl::init(100), cl::Hidden);
static cl::opt<unsigned long> GrowRegionComplexityBudget(
"grow-region-complexity-budget",
diff --git a/llvm/lib/Target/AArch64/AArch64RegisterInfo.h b/llvm/lib/Target/AArch64/AArch64RegisterInfo.h
index b31104aac6551..60af8f6da5fbf 100644
--- a/llvm/lib/Target/AArch64/AArch64RegisterInfo.h
+++ b/llvm/lib/Target/AArch64/AArch64RegisterInfo.h
@@ -54,12 +54,6 @@ class AArch64RegisterInfo final : public AArch64GenRegisterInfo {
const uint32_t *getDarwinCallPreservedMask(const MachineFunction &MF,
CallingConv::ID) const;
- unsigned getCSRCost() const override {
- // The cost will be compared against BlockFrequency where entry has the
- // value of 1 << 14. A value of 5 will choose to spill or split really
- // cold path instead of using a callee-saved register.
- return 5;
- }
unsigned getCSRFirstUseCost() const override {
// The cost of 2 means push and pop for each CSR.
return 2;
diff --git a/llvm/test/CodeGen/AArch64/cgp-usubo.ll b/llvm/test/CodeGen/AArch64/cgp-usubo.ll
index 71a873014a61f..8273013a5db2c 100644
--- a/llvm/test/CodeGen/AArch64/cgp-usubo.ll
+++ b/llvm/test/CodeGen/AArch64/cgp-usubo.ll
@@ -279,29 +279,30 @@ end:
define i1 @usubo_ult_cmp_dominates_i64(i64 %x, i64 %y, ptr %p, i1 %cond) nounwind {
; CHECK-LABEL: usubo_ult_cmp_dominates_i64:
; CHECK: // %bb.0: // %entry
-; CHECK-NEXT: stp x30, x23, [sp, #-48]! // 16-byte Folded Spill
+; CHECK-NEXT: str x30, [sp, #-48]! // 8-byte Folded Spill
; CHECK-NEXT: stp x20, x19, [sp, #32] // 16-byte Folded Spill
; CHECK-NEXT: mov w19, w3
; CHECK-NEXT: stp x22, x21, [sp, #16] // 16-byte Folded Spill
; CHECK-NEXT: tbz w3, #0, .LBB15_3
; CHECK-NEXT: // %bb.1: // %t
; CHECK-NEXT: cmp x0, x1
-; CHECK-NEXT: mov x22, x0
-; CHECK-NEXT: mov x20, x2
-; CHECK-NEXT: cset w21, lo
-; CHECK-NEXT: mov x23, x1
-; CHECK-NEXT: mov w0, w21
+; CHECK-NEXT: mov x21, x0
+; CHECK-NEXT: str x2, [sp, #8] // 8-byte Spill
+; CHECK-NEXT: cset w20, lo
+; CHECK-NEXT: mov x22, x1
+; CHECK-NEXT: mov w0, w20
; CHECK-NEXT: bl call
-; CHECK-NEXT: subs x8, x22, x23
+; CHECK-NEXT: subs x8, x21, x22
; CHECK-NEXT: b.hs .LBB15_3
; CHECK-NEXT: // %bb.2: // %end
-; CHECK-NEXT: mov w19, w21
-; CHECK-NEXT: str x8, [x20]
+; CHECK-NEXT: ldr x9, [sp, #8] // 8-byte Reload
+; CHECK-NEXT: mov w19, w20
+; CHECK-NEXT: str x8, [x9]
; CHECK-NEXT: .LBB15_3: // %common.ret
; CHECK-NEXT: and w0, w19, #0x1
; CHECK-NEXT: ldp x20, x19, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: ldp x22, x21, [sp, #16] // 16-byte Folded Reload
-; CHECK-NEXT: ldp x30, x23, [sp], #48 // 16-byte Folded Reload
+; CHECK-NEXT: ldr x30, [sp], #48 // 8-byte Folded Reload
; CHECK-NEXT: ret
entry:
br i1 %cond, label %t, label %f
diff --git a/llvm/test/CodeGen/AArch64/cmpxchg-idioms.ll b/llvm/test/CodeGen/AArch64/cmpxchg-idioms.ll
index 3f4dd116d91f8..51d00704ff1d2 100644
--- a/llvm/test/CodeGen/AArch64/cmpxchg-idioms.ll
+++ b/llvm/test/CodeGen/AArch64/cmpxchg-idioms.ll
@@ -164,53 +164,50 @@ declare void @baz()
define i1 @test_conditional2(i32 %a, i32 %b, ptr %c) {
; CHECK-LABEL: test_conditional2:
; CHECK: ; %bb.0: ; %entry
-; CHECK-NEXT: stp x22, x21, [sp, #-48]! ; 16-byte Folded Spill
-; CHECK-NEXT: stp x20, x19, [sp, #16] ; 16-byte Folded Spill
+; CHECK-NEXT: sub sp, sp, #48
; CHECK-NEXT: stp x29, x30, [sp, #32] ; 16-byte Folded Spill
; CHECK-NEXT: .cfi_def_cfa_offset 48
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
-; CHECK-NEXT: .cfi_offset w19, -24
-; CHECK-NEXT: .cfi_offset w20, -32
-; CHECK-NEXT: .cfi_offset w21, -40
-; CHECK-NEXT: .cfi_offset w22, -48
-; CHECK-NEXT: mov x19, x2
-; CHECK-NEXT: mov w20, w1
-; CHECK-NEXT: mov w21, w0
; CHECK-NEXT: LBB3_1: ; %cmpxchg.start
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
-; CHECK-NEXT: ldaxr w8, [x19]
-; CHECK-NEXT: cmp w8, w21
+; CHECK-NEXT: ldaxr w8, [x2]
+; CHECK-NEXT: cmp w8, w0
; CHECK-NEXT: b.ne LBB3_9
; CHECK-NEXT: ; %bb.2: ; %cmpxchg.trystore
; CHECK-NEXT: ; in Loop: Header=BB3_1 Depth=1
-; CHECK-NEXT: stlxr w8, w20, [x19]
+; CHECK-NEXT: stlxr w8, w1, [x2]
; CHECK-NEXT: cbnz w8, LBB3_1
; CHECK-NEXT: ; %bb.3:
; CHECK-NEXT: mov w8, #1 ; =0x1
; CHECK-NEXT: LBB3_4: ; %for.cond.preheader
-; CHECK-NEXT: mov w22, #2 ; =0x2
+; CHECK-NEXT: mov w9, #2 ; =0x2
; CHECK-NEXT: LBB3_5: ; %for.cond
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
-; CHECK-NEXT: cbz w22, LBB3_8
+; CHECK-NEXT: cbz w9, LBB3_8
; CHECK-NEXT: ; %bb.6: ; %for.body
; CHECK-NEXT: ; in Loop: Header=BB3_5 Depth=1
-; CHECK-NEXT: sub w22, w22, #1
-; CHECK-NEXT: orr w9, w21, w20
-; CHECK-NEXT: ldr w10, [x19, w22, sxtw #2]
-; CHECK-NEXT: cmp w9, w10
+; CHECK-NEXT: sub w9, w9, #1
+; CHECK-NEXT: orr w10, w0, w1
+; CHECK-NEXT: ldr w11, [x2, w9, sxtw #2]
+; CHECK-NEXT: cmp w10, w11
; CHECK-NEXT: b.eq LBB3_5
; CHECK-NEXT: ; %bb.7: ; %if.then
; CHECK-NEXT: ; in Loop: Header=BB3_5 Depth=1
-; CHECK-NEXT: str w9, [x19, w22, sxtw #2]
+; CHECK-NEXT: str w10, [x2, w9, sxtw #2]
+; CHECK-NEXT: str x2, [sp, #24] ; 8-byte Spill
+; CHECK-NEXT: stp w1, w0, [sp, #16] ; 8-byte Folded Spill
+; CHECK-NEXT: str w9, [sp, #12] ; 4-byte Spill
; CHECK-NEXT: bl _foo
+; CHECK-NEXT: ldp w9, w1, [sp, #12] ; 8-byte Folded Reload
+; CHECK-NEXT: ldr w0, [sp, #20] ; 4-byte Reload
+; CHECK-NEXT: ldr x2, [sp, #24] ; 8-byte Reload
; CHECK-NEXT: mov w8, wzr
; CHECK-NEXT: b LBB3_5
; CHECK-NEXT: LBB3_8: ; %for.cond.cleanup
; CHECK-NEXT: ldp x29, x30, [sp, #32] ; 16-byte Folded Reload
; CHECK-NEXT: and w0, w8, #0x1
-; CHECK-NEXT: ldp x20, x19, [sp, #16] ; 16-byte Folded Reload
-; CHECK-NEXT: ldp x22, x21, [sp], #48 ; 16-byte Folded Reload
+; CHECK-NEXT: add sp, sp, #48
; CHECK-NEXT: ret
; CHECK-NEXT: LBB3_9: ; %cmpxchg.nostore
; CHECK-NEXT: mov w8, wzr
@@ -219,10 +216,11 @@ define i1 @test_conditional2(i32 %a, i32 %b, ptr %c) {
;
; OUTLINE-ATOMICS-LABEL: test_conditional2:
; OUTLINE-ATOMICS: ; %bb.0: ; %entry
-; OUTLINE-ATOMICS-NEXT: stp x22, x21, [sp, #-48]! ; 16-byte Folded Spill
-; OUTLINE-ATOMICS-NEXT: stp x20, x19, [sp, #16] ; 16-byte Folded Spill
-; OUTLINE-ATOMICS-NEXT: stp x29, x30, [sp, #32] ; 16-byte Folded Spill
-; OUTLINE-ATOMICS-NEXT: .cfi_def_cfa_offset 48
+; OUTLINE-ATOMICS-NEXT: sub sp, sp, #64
+; OUTLINE-ATOMICS-NEXT: stp x22, x21, [sp, #16] ; 16-byte Folded Spill
+; OUTLINE-ATOMICS-NEXT: stp x20, x19, [sp, #32] ; 16-byte Folded Spill
+; OUTLINE-ATOMICS-NEXT: stp x29, x30, [sp, #48] ; 16-byte Folded Spill
+; OUTLINE-ATOMICS-NEXT: .cfi_def_cfa_offset 64
; OUTLINE-ATOMICS-NEXT: .cfi_offset w30, -8
; OUTLINE-ATOMICS-NEXT: .cfi_offset w29, -16
; OUTLINE-ATOMICS-NEXT: .cfi_offset w19, -24
@@ -234,29 +232,32 @@ define i1 @test_conditional2(i32 %a, i32 %b, ptr %c) {
; OUTLINE-ATOMICS-NEXT: mov w21, w0
; OUTLINE-ATOMICS-NEXT: bl ___aarch64_cas4_acq_rel
; OUTLINE-ATOMICS-NEXT: cmp w0, w21
-; OUTLINE-ATOMICS-NEXT: mov w22, #2 ; =0x2
-; OUTLINE-ATOMICS-NEXT: cset w8, eq
+; OUTLINE-ATOMICS-NEXT: mov w8, #2 ; =0x2
+; OUTLINE-ATOMICS-NEXT: cset w9, eq
; OUTLINE-ATOMICS-NEXT: LBB3_1: ; %for.cond
; OUTLINE-ATOMICS-NEXT: ; =>This Inner Loop Header: Depth=1
-; OUTLINE-ATOMICS-NEXT: cbz w22, LBB3_4
+; OUTLINE-ATOMICS-NEXT: cbz w8, LBB3_4
; OUTLINE-ATOMICS-NEXT: ; %bb.2: ; %for.body
; OUTLINE-ATOMICS-NEXT: ; in Loop: Header=BB3_1 Depth=1
-; OUTLINE-ATOMICS-NEXT: sub w22, w22, #1
-; OUTLINE-ATOMICS-NEXT: orr w9, w21, w20
-; OUTLINE-ATOMICS-NEXT: ldr w10, [x19, w22, sxtw #2]
-; OUTLINE-ATOMICS-NEXT: cmp w9, w10
+; OUTLINE-ATOMICS-NEXT: sub w8, w8, #1
+; OUTLINE-ATOMICS-NEXT: orr w10, w21, w20
+; OUTLINE-ATOMICS-NEXT: ldr w11, [x19, w8, sxtw #2]
+; OUTLINE-ATOMICS-NEXT: cmp w10, w11
; OUTLINE-ATOMICS-NEXT: b.eq LBB3_1
; OUTLINE-ATOMICS-NEXT: ; %bb.3: ; %if.then
; OUTLINE-ATOMICS-NEXT: ; in Loop: Header=BB3_1 Depth=1
-; OUTLINE-ATOMICS-NEXT: str w9, [x19, w22, sxtw #2]
+; OUTLINE-ATOMICS-NEXT: str w10, [x19, w8, sxtw #2]
+; OUTLINE-ATOMICS-NEXT: str w8, [sp, #12] ; 4-byte Spill
; OUTLINE-ATOMICS-NEXT: bl _foo
-; OUTLINE-ATOMICS-NEXT: mov w8, wzr
+; OUTLINE-ATOMICS-NEXT: ldr w8, [sp, #12] ; 4-byte Reload
+; OUTLINE-ATOMICS-NEXT: mov w9, wzr
; OUTLINE-ATOMICS-NEXT: b LBB3_1
; OUTLINE-ATOMICS-NEXT: LBB3_4: ; %for.cond.cleanup
-; OUTLINE-ATOMICS-NEXT: ldp x29, x30, [sp, #32] ; 16-byte Folded Reload
-; OUTLINE-ATOMICS-NEXT: and w0, w8, #0x1
-; OUTLINE-ATOMICS-NEXT: ldp x20, x19, [sp, #16] ; 16-byte Folded Reload
-; OUTLINE-ATOMICS-NEXT: ldp x22, x21, [sp], #48 ; 16-byte Folded Reload
+; OUTLINE-ATOMICS-NEXT: ldp x29, x30, [sp, #48] ; 16-byte Folded Reload
+; OUTLINE-ATOMICS-NEXT: and w0, w9, #0x1
+; OUTLINE-ATOMICS-NEXT: ldp x20, x19, [sp, #32] ; 16-byte Folded Reload
+; OUTLINE-ATOMICS-NEXT: ldp x22, x21, [sp, #16] ; 16-byte Folded Reload
+; OUTLINE-ATOMICS-NEXT: add sp, sp, #64
; OUTLINE-ATOMICS-NEXT: ret
entry:
%pair = cmpxchg ptr %c, i32 %a, i32 %b seq_cst seq_cst
diff --git a/llvm/test/CodeGen/AArch64/combine-comparisons-by-cse.ll b/llvm/test/CodeGen/AArch64/combine-comparisons-by-cse.ll
index 09e80ee936738..3bb09723381fa 100644
--- a/llvm/test/CodeGen/AArch64/combine-comparisons-by-cse.ll
+++ b/llvm/test/CodeGen/AArch64/combine-comparisons-by-cse.ll
@@ -582,26 +582,30 @@ declare void @do_something() #1
define i32 @do_nothing_if_resultant_opcodes_would_differ() #0 {
; CHECK-LABEL: do_nothing_if_resultant_opcodes_would_differ:
; CHECK: // %bb.0: // %entry
-; CHECK-NEXT: str x30, [sp, #-32]! // 8-byte Folded Spill
-; CHECK-NEXT: .cfi_def_cfa_offset 32
-; CHECK-NEXT: stp x20, x19, [sp, #16] // 16-byte Folded Spill
-; CHECK-NEXT: .cfi_offset w19, -8
-; CHECK-NEXT: .cfi_offset w20, -16
-; CHECK-NEXT: .cfi_offset w30, -32
-; CHECK-NEXT: adrp x19, :got:a
-; CHECK-NEXT: ldr x19, [x19, :got_lo12:a]
-; CHECK-NEXT: ldr w8, [x19]
+; CHECK-NEXT: adrp x8, :got:a
+; CHECK-NEXT: ldr x8, [x8, :got_lo12:a]
+; CHECK-NEXT: ldr w8, [x8]
; CHECK-NEXT: cmn w8, #2
; CHECK-NEXT: b.gt .LBB10_4
; CHECK-NEXT: // %bb.1: // %while.body.preheader
-; CHECK-NEXT: sub w20, w8, #1
+; CHECK-NEXT: stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
+; CHECK-NEXT: .cfi_def_cfa_offset 16
+; CHECK-NEXT: .cfi_offset w19, -8
+; CHECK-NEXT: .cfi_offset w30, -16
+; CHECK-NEXT: sub w19, w8, #1
; CHECK-NEXT: .LBB10_2: // %while.body
; CHECK-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-NEXT: bl do_something
-; CHECK-NEXT: adds w20, w20, #1
+; CHECK-NEXT: adds w19, w19, #1
; CHECK-NEXT: b.mi .LBB10_2
; CHECK-NEXT: // %bb.3: // %while.cond.while.end_crit_edge
-; CHECK-NEXT: ldr w8, [x19]
+; CHECK-NEXT: adrp x8, :got:a
+; CHECK-NEXT: ldr x8, [x8, :got_lo12:a]
+; CHECK-NEXT: ldr w8, [x8]
+; CHECK-NEXT: ldp x30, x19, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT: .cfi_def_cfa_offset 0
+; CHECK-NEXT: .cfi_restore w19
+; CHECK-NEXT: .cfi_restore w30
; CHECK-NEXT: .LBB10_4: // %while.end
; CHECK-NEXT: cmp w8, #1
; CHECK-NEXT: b.gt .LBB10_7
@@ -616,16 +620,9 @@ define i32 @do_nothing_if_resultant_opcodes_would_differ() #0 {
; CHECK-NEXT: b.ne .LBB10_7
; CHECK-NEXT: // %bb.6:
; CHECK-NEXT: mov w0, #123 // =0x7b
-; CHECK-NEXT: b .LBB10_8
+; CHECK-NEXT: ret
; CHECK-NEXT: .LBB10_7: // %if.end
; CHECK-NEXT: mov w0, wzr
-; CHECK-NEXT: .LBB10_8: // %return
-; CHECK-NEXT: ldp x20, x19, [sp, #16] // 16-byte Folded Reload
-; CHECK-NEXT: ldr x30, [sp], #32 // 8-byte Folded Reload
-; CHECK-NEXT: .cfi_def_cfa_offset 0
-; CHECK-NEXT: .cfi_restore w19
-; CHECK-NEXT: .cfi_restore w20
-; CHECK-NEXT: .cfi_restore w30
; CHECK-NEXT: ret
entry:
%0 = load i32, ptr @a, align 4
diff --git a/llvm/test/CodeGen/AArch64/csr-split.ll b/llvm/test/CodeGen/AArch64/csr-split.ll
index 7b092b00b9655..e99d4f30beb80 100644
--- a/llvm/test/CodeGen/AArch64/csr-split.ll
+++ b/llvm/test/CodeGen/AArch64/csr-split.ll
@@ -164,54 +164,58 @@ return: ; preds = %if.end, %entry, %if
define dso_local ptr @test3(ptr nocapture %p1, i8 zeroext %p2) local_unnamed_addr uwtable {
; CHECK-LABEL: test3:
; CHECK: // %bb.0: // %entry
-; CHECK-NEXT: str x30, [sp, #-32]! // 8-byte Folded Spill
+; CHECK-NEXT: mov x8, x0
+; CHECK-NEXT: ldr x0, [x0]
+; CHECK-NEXT: cbz x0, .LBB2_2
+; CHECK-NEXT: // %bb.1: // %land.rhs
+; CHECK-NEXT: sub sp, sp, #32
; CHECK-NEXT: .cfi_def_cfa_offset 32
-; CHECK-NEXT: stp x20, x19, [sp, #16] // 16-byte Folded Spill
+; CHECK-NEXT: stp x30, x19, [sp, #16] // 16-byte Folded Spill
; CHECK-NEXT: .cfi_offset w19, -8
-; CHECK-NEXT: .cfi_offset w20, -16
-; CHECK-NEXT: .cfi_offset w30, -32
-; CHECK-NEXT: ldr x19, [x0]
-; CHECK-NEXT: cbz x19, .LBB2_2
-; CHECK-NEXT: // %bb.1: // %land.rhs
-; CHECK-NEXT: mov x20, x0
-; CHECK-NEXT: mov x0, x19
+; CHECK-NEXT: .cfi_offset w30, -16
+; CHECK-NEXT: str x0, [sp, #8] // 8-byte Spill
+; CHECK-NEXT: mov x19, x8
; CHECK-NEXT: bl bar
-; CHECK-NEXT: str x0, [x20]
-; CHECK-NEXT: .LBB2_2: // %land.end
-; CHECK-NEXT: mov x0, x19
-; CHECK-NEXT: ldp x20, x19, [sp, #16] // 16-byte Folded Reload
-; CHECK-NEXT: ldr x30, [sp], #32 // 8-byte Folded Reload
+; CHECK-NEXT: mov x8, x0
+; CHECK-NEXT: ldr x0, [sp, #8] // 8-byte Reload
+; CHECK-NEXT: str x8, [x19]
+; CHECK-NEXT: ldp x30, x19, [sp, #16] // 16-byte Folded Reload
+; CHECK-NEXT: add sp, sp, #32
; CHECK-NEXT: .cfi_def_cfa_offset 0
; CHECK-NEXT: .cfi_restore w19
-; CHECK-NEXT: .cfi_restore w20
; CHECK-NEXT: .cfi_restore w30
+; CHECK-NEXT: .LBB2_2: // %land.end
; CHECK-NEXT: ret
;
; CHECK-APPLE-LABEL: test3:
; CHECK-APPLE: ; %bb.0: ; %entry
-; CHECK-APPLE-NEXT: stp x20, x19, [sp, #-32]! ; 16-byte Folded Spill
-; CHECK-APPLE-NEXT: .cfi_def_cfa_offset 32
-; CHECK-APPLE-NEXT: stp x29, x30, [sp, #16] ; 16-byte Folded Spill
+; CHECK-APPLE-NEXT: mov x8, x0
+; CHECK-APPLE-NEXT: ldr x0, [x0]
+; CHECK-APPLE-NEXT: cbz x0, LBB2_2
+; CHECK-APPLE-NEXT: ; %bb.1: ; %land.rhs
+; CHECK-APPLE-NEXT: sub sp, sp, #48
+; CHECK-APPLE-NEXT: .cfi_def_cfa_offset 48
+; CHECK-APPLE-NEXT: stp x20, x19, [sp, #16] ; 16-byte Folded Spill
+; CHECK-APPLE-NEXT: stp x29, x30, [sp, #32] ; 16-byte Folded Spill
; CHECK-APPLE-NEXT: .cfi_offset w30, -8
; CHECK-APPLE-NEXT: .cfi_offset w29, -16
; CHECK-APPLE-NEXT: .cfi_offset w19, -24
; CHECK-APPLE-NEXT: .cfi_offset w20, -32
-; CHECK-APPLE-NEXT: ldr x19, [x0]
-; CHECK-APPLE-NEXT: cbz x19, LBB2_2
-; CHECK-APPLE-NEXT: ; %bb.1: ; %land.rhs
-; CHECK-APPLE-NEXT: mov x20, x0
-; CHECK-APPLE-NEXT: mov x0, x19
+; CHECK-APPLE-NEXT: str x0, [sp, #8] ; 8-byte Spill
+; CHECK-APPLE-NEXT: mov x19, x8
; CHECK-APPLE-NEXT: bl _bar
-; CHECK-APPLE-NEXT: str x0, [x20]
-; CHECK-APPLE-NEXT: LBB2_2: ; %land.end
-; CHECK-APPLE-NEXT: ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
-; CHECK-APPLE-NEXT: mov x0, x19
-; CHECK-APPLE-NEXT: ldp x20, x19, [sp], #32 ; 16-byte Folded Reload
+; CHECK-APPLE-NEXT: mov x8, x0
+; CHECK-APPLE-NEXT: ldp x29, x30, [sp, #32] ; 16-byte Folded Reload
+; CHECK-APPLE-NEXT: str x8, [x19]
+; CHECK-APPLE-NEXT: ldp x20, x19, [sp, #16] ; 16-byte Folded Reload
+; CHECK-APPLE-NEXT: ldr x0, [sp, #8] ; 8-byte Reload
+; CHECK-APPLE-NEXT: add sp, sp, #48
; CHECK-APPLE-NEXT: .cfi_def_cfa_offset 0
; CHECK-APPLE-NEXT: .cfi_restore w30
; CHECK-APPLE-NEXT: .cfi_restore w29
; CHECK-APPLE-NEXT: .cfi_restore w19
; CHECK-APPLE-NEXT: .cfi_restore w20
+; CHECK-APPLE-NEXT: LBB2_2: ; %land.end
; CHECK-APPLE-NEXT: ret
entry:
%0 = load ptr, ptr %p1, align 8, !tbaa !6
diff --git a/llvm/test/CodeGen/AArch64/div-i256.ll b/llvm/test/CodeGen/AArch64/div-i256.ll
index 5f8d362c85bb2..02a65ec9f3207 100644
--- a/llvm/test/CodeGen/AArch64/div-i256.ll
+++ b/llvm/test/CodeGen/AArch64/div-i256.ll
@@ -110,36 +110,36 @@ define i256 @udiv256(i256 %a, i256 %b) nounwind {
; CHECK-NEXT: // %bb.3: // %udiv-preheader
; CHECK-NEXT: lsr x20, x8, #3
; CHECK-NEXT: stp x0, x1, [sp]
-; CHECK-NEXT: mov x1, sp
+; CHECK-NEXT: mov x19, sp
; CHECK-NEXT: stp q0, q0, [sp, #32]
; CHECK-NEXT: mov x18, xzr
-; CHECK-NEXT: mov x19, xzr
-; CHECK-NEXT: and x0, x20, #0x18
+; CHECK-NEXT: mov x17, xzr
+; CHECK-NEXT: and x1, x20, #0x18
; CHECK-NEXT: stp x2, x3, [sp, #16]
-; CHECK-NEXT: and x3, x8, #0x3f
-; CHECK-NEXT: add x0, x1, x0
; CHECK-NEXT: mvn w20, w8
-; CHECK-NEXT: eor x3, x3, #0x3f
-; CHECK-NEXT: ldp x1, x2, [x0, #16]
-; CHECK-NEXT: mov x17, xzr
-; CHECK-NEXT: ldp x23, x21, [x0]
-; CHECK-NEXT: lsl x0, x1, #1
-; CHECK-NEXT: lsl x22, x2, #1
-; CHECK-NEXT: lsr x24, x1, x8
-; CHECK-NEXT: lsl x1, x21, #1
+; CHECK-NEXT: add x1, x19, x1
+; CHECK-NEXT: and x19, x8, #0x3f
+; CHECK-NEXT: mov x0, xzr
+; CHECK-NEXT: ldp x2, x3, [x1, #16]
+; CHECK-NEXT: eor x19, x19, #0x3f
+; CHECK-NEXT: ldp x23, x21, [x1]
+; CHECK-NEXT: lsl x1, x2, #1
+; CHECK-NEXT: lsl x22, x3, #1
+; CHECK-NEXT: lsr x24, x2, x8
+; CHECK-NEXT: lsl x2, x21, #1
; CHECK-NEXT: lsr x26, x21, x8
; CHECK-NEXT: lsr x27, x23, x8
-; CHECK-NEXT: lsl x25, x0, x20
-; CHECK-NEXT: subs x0, x4, #1
-; CHECK-NEXT: lsl x22, x22, x3
-; CHECK-NEXT: lsl x3, x1, x3
-; CHECK-NEXT: sbcs x1, x5, xzr
-; CHECK-NEXT: lsr x21, x2, x8
-; CHECK-NEXT: sbcs x2, x6, xzr
+; CHECK-NEXT: lsl x25, x1, x20
+; CHECK-NEXT: subs x1, x4, #1
+; CHECK-NEXT: lsl x22, x22, x19
+; CHECK-NEXT: lsl x19, x2, x19
+; CHECK-NEXT: sbcs x2, x5, xzr
+; CHECK-NEXT: lsr x21, x3, x8
+; CHECK-NEXT: sbcs x3, x6, xzr
; CHECK-NEXT: orr x20, x22, x24
; CHECK-NEXT: orr x23, x26, x25
-; CHECK-NEXT: orr x22, x3, x27
-; CHECK-NEXT: sbc x3, x7, xzr
+; CHECK-NEXT: orr x22, x19, x27
+; CHECK-NEXT: sbc x19, x7, xzr
; CHECK-NEXT: .LBB0_4: // %udiv-do-while
; CHECK-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-NEXT: extr x24, x22, x15, #63
@@ -148,13 +148,13 @@ define i256 @udiv256(i256 %a, i256 %b) nounwind {
; CHECK-NEXT: extr x21, x21, x20, #63
; CHECK-NEXT: extr x15, x15, x13, #63
; CHECK-NEXT: extr x13, x13, x12, #63
-; CHECK-NEXT: cmp x0, x24
-; CHECK-NEXT: sbcs xzr, x1, x25
-; CHECK-NEXT: orr x13, x19, x13
+; CHECK-NEXT: cmp x1, x24
+; CHECK-NEXT: sbcs xzr, x2, x25
+; CHECK-NEXT: orr x13, x0, x13
; CHECK-NEXT: orr x15, x17, x15
-; CHECK-NEXT: sbcs xzr, x2, x26
+; CHECK-NEXT: sbcs xzr, x3, x26
; CHECK-NEXT: mov x17, xzr
-; CHECK-NEXT: sbc x20, x3, x21
+; CHECK-NEXT: sbc x20, x19, x21
; CHECK-NEXT: asr x27, x20, #63
; CHECK-NEXT: and x20, x27, x4
; CHECK-NEXT: subs x22, x24, x20
@@ -172,11 +172,11 @@ define i256 @udiv256(i256 %a, i256 %b) nounwind {
; CHECK-NEXT: sbcs x10, x10, xzr
; CHECK-NEXT: orr x12, x18, x24
; CHECK-NEXT: sbc x14, x14, xzr
-; CHECK-NEXT: orr x19, x8, x10
+; CHECK-NEXT: orr x0, x8, x10
; CHECK-NEXT: orr x18, x9, x14
-; CHECK-NEXT: orr x24, x19, x18
+; CHECK-NEXT: orr x24, x0, x18
; CHECK-NEXT: mov x18, xzr
-; CHECK-NEXT: mov x19, xzr
+; CHECK-NEXT: mov x0, xzr
; CHECK-NEXT: cbnz x24, .LBB0_4
; CHECK-NEXT: .LBB0_5: // %udiv-loop-exit
; CHECK-NEXT: ldp x20, x19, [sp, #192] // 16-byte Folded Reload
@@ -327,36 +327,36 @@ define i256 @sdiv256(i256 %a, i256 %b) nounwind {
; CHECK-NEXT: // %bb.3: // %udiv-preheader
; CHECK-NEXT: lsr x21, x13, #3
; CHECK-NEXT: stp x14, x15, [sp]
-; CHECK-NEXT: mov x15, sp
+; CHECK-NEXT: ...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/214773
More information about the llvm-commits
mailing list