[llvm] [AMDGPU] Check register class when folding a redundant AND (PR #217900)
via llvm-commits
llvm-commits at lists.llvm.org
Fri Aug 21 05:29:33 PDT 2026
https://github.com/michaelselehov created https://github.com/llvm/llvm-project/pull/217900
tryFoldRedundantAND accepts an SALU AND as the parent instruction. The child AND can be a VALU instruction, so the fold replaced a VGPR with an SGPR in every use of the child result. A use that requires a VGPR, for example data0 of DS_BPERMUTE_B32, then fails the machine verifier. This aborted a rocPRIM build for gfx1010.
Fold only when the register class of the replacement register is a subclass of the register class of the folded register. Every use of the folded register already accepts its class, so a subclass is legal in those uses too, and no walk over the uses is needed.
Also return early when the source of the AND is a physical register, because getVRegDef and getRegClass both expect a virtual register.
Add a negative test for the case above and a positive test where the classes differ but the replacement is a strict subclass. Run the machine verifier in the RUN lines, since the verifier detects exactly this kind of error.
Assisted-by: Claude Opus
>From 4878619296e9339b633a40ca89ec5b9180b8ef96 Mon Sep 17 00:00:00 2001
From: Michael Selehov <michael.selehov at amd.com>
Date: Fri, 21 Aug 2026 07:18:24 -0500
Subject: [PATCH] [AMDGPU] Check register class when folding a redundant AND
tryFoldRedundantAND accepts an SALU AND as the parent instruction. The child
AND can be a VALU instruction, so the fold replaced a VGPR with an SGPR in
every use of the child result. A use that requires a VGPR, for example data0
of DS_BPERMUTE_B32, then fails the machine verifier. This aborted a rocPRIM
build for gfx1010.
Fold only when the register class of the replacement register is a subclass
of the register class of the folded register. Every use of the folded
register already accepts its class, so a subclass is legal in those uses
too, and no walk over the uses is needed.
Also return early when the source of the AND is a physical register, because
getVRegDef and getRegClass both expect a virtual register.
Add a negative test for the case above and a positive test where the classes
differ but the replacement is a strict subclass. Run the machine verifier in
the RUN lines, since the verifier detects exactly this kind of error.
Assisted-by: Claude Opus
---
llvm/lib/Target/AMDGPU/SIFoldOperands.cpp | 17 +++-
llvm/test/CodeGen/AMDGPU/redundant-and.mir | 101 +++++++++++++++++++--
2 files changed, 106 insertions(+), 12 deletions(-)
diff --git a/llvm/lib/Target/AMDGPU/SIFoldOperands.cpp b/llvm/lib/Target/AMDGPU/SIFoldOperands.cpp
index a224d0c48e8bb..7b60f6a022e87 100644
--- a/llvm/lib/Target/AMDGPU/SIFoldOperands.cpp
+++ b/llvm/lib/Target/AMDGPU/SIFoldOperands.cpp
@@ -1904,6 +1904,9 @@ bool SIFoldOperandsImpl::tryFoldRedundantAND(MachineInstr &ChildMI) const {
if (!ChildResult)
return false;
+ if (!ChildResult->Reg.isVirtual())
+ return false;
+
MachineInstr *ParentMI = MRI->getVRegDef(ChildResult->Reg);
if (!ParentMI)
return false;
@@ -1925,11 +1928,21 @@ bool SIFoldOperandsImpl::tryFoldRedundantAND(MachineInstr &ChildMI) const {
return false;
Register Dst = ChildMI.getOperand(0).getReg();
- MRI->replaceRegWith(Dst, ChildResult->Reg);
+ Register Src = ChildResult->Reg;
+
+ // Replace Dst only if Src is at least as constrained. Every use of Dst
+ // accepts the register class of Dst, so it also accepts a subclass of it.
+ // An S_AND_B32 parent with a V_AND_B32 child defines Src in the scalar bank,
+ // and putting it into a use that requires a VGPR is illegal.
+ if (!Dst.isVirtual() ||
+ !MRI->getRegClass(Dst)->hasSubClassEq(MRI->getRegClass(Src)))
+ return false;
+
+ MRI->replaceRegWith(Dst, Src);
// Clear kill flags if the register operand is not marked as kill.
if (!ChildMI.getOperand(ChildResult->RegIdx).isKill())
- MRI->clearKillFlags(ChildResult->Reg);
+ MRI->clearKillFlags(Src);
ChildMI.eraseFromParent();
return true;
diff --git a/llvm/test/CodeGen/AMDGPU/redundant-and.mir b/llvm/test/CodeGen/AMDGPU/redundant-and.mir
index 7b220b7d7cdd6..d002c6d0e8114 100644
--- a/llvm/test/CodeGen/AMDGPU/redundant-and.mir
+++ b/llvm/test/CodeGen/AMDGPU/redundant-and.mir
@@ -1,7 +1,7 @@
# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
-# RUN: llc -mtriple=amdgpu8.03-amd-amdhsa -run-pass=si-fold-operands -o - %s | FileCheck -check-prefix=GFX8 %s
-# RUN: llc -mtriple=amdgpu9.06-amd-amdhsa -run-pass=si-fold-operands -o - %s | FileCheck -check-prefix=GFX9 %s
-# RUN: llc -mtriple=amdgpu10.30-amd-amdhsa -run-pass=si-fold-operands -o - %s | FileCheck -check-prefix=GFX10 %s
+# RUN: llc -mtriple=amdgpu8.03-amd-amdhsa -run-pass=si-fold-operands -verify-machineinstrs -o - %s | FileCheck -check-prefix=GFX8 %s
+# RUN: llc -mtriple=amdgpu9.06-amd-amdhsa -run-pass=si-fold-operands -verify-machineinstrs -o - %s | FileCheck -check-prefix=GFX9 %s
+# RUN: llc -mtriple=amdgpu10.30-amd-amdhsa -run-pass=si-fold-operands -verify-machineinstrs -o - %s | FileCheck -check-prefix=GFX10 %s
# Tests for tryFoldRedundantAnd optimization.
# This optimization eliminates a child AND instruction when its mask is a superset
@@ -2450,33 +2450,68 @@ body: |
...
---
-name: fold_redundant_and_scalar_parent_vgpr_imm_child
+name: fold_redundant_and_src_subclass
tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0
- ; GFX8-LABEL: name: fold_redundant_and_scalar_parent_vgpr_imm_child
+ ; GFX8-LABEL: name: fold_redundant_and_src_subclass
+ ; GFX8: liveins: $sgpr0
+ ; GFX8-NEXT: {{ $}}
+ ; GFX8-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $sgpr0
+ ; GFX8-NEXT: [[S_AND_B32_:%[0-9]+]]:sgpr_32 = S_AND_B32 [[COPY]], 255, implicit-def dead $scc
+ ; GFX8-NEXT: S_NOP 0, implicit [[S_AND_B32_]]
+ ;
+ ; GFX9-LABEL: name: fold_redundant_and_src_subclass
+ ; GFX9: liveins: $sgpr0
+ ; GFX9-NEXT: {{ $}}
+ ; GFX9-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $sgpr0
+ ; GFX9-NEXT: [[S_AND_B32_:%[0-9]+]]:sgpr_32 = S_AND_B32 [[COPY]], 255, implicit-def dead $scc
+ ; GFX9-NEXT: S_NOP 0, implicit [[S_AND_B32_]]
+ ;
+ ; GFX10-LABEL: name: fold_redundant_and_src_subclass
+ ; GFX10: liveins: $sgpr0
+ ; GFX10-NEXT: {{ $}}
+ ; GFX10-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $sgpr0
+ ; GFX10-NEXT: [[S_AND_B32_:%[0-9]+]]:sgpr_32 = S_AND_B32 [[COPY]], 255, implicit-def dead $scc
+ ; GFX10-NEXT: S_NOP 0, implicit [[S_AND_B32_]]
+ %0:sreg_32 = COPY $sgpr0
+ %1:sgpr_32 = S_AND_B32 %0:sreg_32, 255, implicit-def dead $scc
+ %2:sreg_32 = S_AND_B32 %1:sgpr_32, 65535, implicit-def dead $scc
+ S_NOP 0, implicit %2:sreg_32
+...
+
+---
+name: no_fold_redundant_and_scalar_parent_vgpr_imm_child
+tracksRegLiveness: true
+body: |
+ bb.0:
+ liveins: $sgpr0
+ ; GFX8-LABEL: name: no_fold_redundant_and_scalar_parent_vgpr_imm_child
; GFX8: liveins: $sgpr0
; GFX8-NEXT: {{ $}}
; GFX8-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $sgpr0
; GFX8-NEXT: [[S_AND_B32_:%[0-9]+]]:sreg_32 = S_AND_B32 [[COPY]], 255, implicit-def dead $scc
; GFX8-NEXT: [[V_MOV_B32_e32_:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 511, implicit $exec
- ; GFX8-NEXT: S_NOP 0, implicit [[S_AND_B32_]]
+ ; GFX8-NEXT: [[V_AND_B32_e64_:%[0-9]+]]:vgpr_32 = V_AND_B32_e64 [[V_MOV_B32_e32_]], [[S_AND_B32_]], implicit $exec
+ ; GFX8-NEXT: S_NOP 0, implicit [[V_AND_B32_e64_]]
;
- ; GFX9-LABEL: name: fold_redundant_and_scalar_parent_vgpr_imm_child
+ ; GFX9-LABEL: name: no_fold_redundant_and_scalar_parent_vgpr_imm_child
; GFX9: liveins: $sgpr0
; GFX9-NEXT: {{ $}}
; GFX9-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $sgpr0
; GFX9-NEXT: [[S_AND_B32_:%[0-9]+]]:sreg_32 = S_AND_B32 [[COPY]], 255, implicit-def dead $scc
; GFX9-NEXT: [[V_MOV_B32_e32_:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 511, implicit $exec
- ; GFX9-NEXT: S_NOP 0, implicit [[S_AND_B32_]]
+ ; GFX9-NEXT: [[V_AND_B32_e64_:%[0-9]+]]:vgpr_32 = V_AND_B32_e64 [[V_MOV_B32_e32_]], [[S_AND_B32_]], implicit $exec
+ ; GFX9-NEXT: S_NOP 0, implicit [[V_AND_B32_e64_]]
;
- ; GFX10-LABEL: name: fold_redundant_and_scalar_parent_vgpr_imm_child
+ ; GFX10-LABEL: name: no_fold_redundant_and_scalar_parent_vgpr_imm_child
; GFX10: liveins: $sgpr0
; GFX10-NEXT: {{ $}}
; GFX10-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $sgpr0
; GFX10-NEXT: [[S_AND_B32_:%[0-9]+]]:sreg_32 = S_AND_B32 [[COPY]], 255, implicit-def dead $scc
- ; GFX10-NEXT: S_NOP 0, implicit [[S_AND_B32_]]
+ ; GFX10-NEXT: [[V_AND_B32_e64_:%[0-9]+]]:vgpr_32 = V_AND_B32_e64 511, [[S_AND_B32_]], implicit $exec
+ ; GFX10-NEXT: S_NOP 0, implicit [[V_AND_B32_e64_]]
%0:sreg_32 = COPY $sgpr0
%1:sreg_32 = S_AND_B32 %0:sreg_32, 255, implicit-def dead $scc
%2:vgpr_32 = V_MOV_B32_e32 511, implicit $exec
@@ -2484,6 +2519,52 @@ body: |
S_NOP 0, implicit %3:vgpr_32
...
+---
+name: no_fold_redundant_and_sgpr_into_vgpr_use
+tracksRegLiveness: true
+body: |
+ bb.0:
+ liveins: $sgpr0, $vgpr0
+ ; GFX8-LABEL: name: no_fold_redundant_and_sgpr_into_vgpr_use
+ ; GFX8: liveins: $sgpr0, $vgpr0
+ ; GFX8-NEXT: {{ $}}
+ ; GFX8-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $sgpr0
+ ; GFX8-NEXT: [[COPY1:%[0-9]+]]:vgpr_32 = COPY $vgpr0
+ ; GFX8-NEXT: [[S_AND_B32_:%[0-9]+]]:sreg_32 = S_AND_B32 [[COPY]], 255, implicit-def dead $scc
+ ; GFX8-NEXT: [[V_MOV_B32_e32_:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 511, implicit $exec
+ ; GFX8-NEXT: [[V_AND_B32_e64_:%[0-9]+]]:vgpr_32 = V_AND_B32_e64 [[V_MOV_B32_e32_]], [[S_AND_B32_]], implicit $exec
+ ; GFX8-NEXT: [[DS_BPERMUTE_B32_:%[0-9]+]]:vgpr_32 = DS_BPERMUTE_B32 [[COPY1]], [[V_AND_B32_e64_]], 0, implicit $exec
+ ; GFX8-NEXT: S_NOP 0, implicit [[DS_BPERMUTE_B32_]]
+ ;
+ ; GFX9-LABEL: name: no_fold_redundant_and_sgpr_into_vgpr_use
+ ; GFX9: liveins: $sgpr0, $vgpr0
+ ; GFX9-NEXT: {{ $}}
+ ; GFX9-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $sgpr0
+ ; GFX9-NEXT: [[COPY1:%[0-9]+]]:vgpr_32 = COPY $vgpr0
+ ; GFX9-NEXT: [[S_AND_B32_:%[0-9]+]]:sreg_32 = S_AND_B32 [[COPY]], 255, implicit-def dead $scc
+ ; GFX9-NEXT: [[V_MOV_B32_e32_:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 511, implicit $exec
+ ; GFX9-NEXT: [[V_AND_B32_e64_:%[0-9]+]]:vgpr_32 = V_AND_B32_e64 [[V_MOV_B32_e32_]], [[S_AND_B32_]], implicit $exec
+ ; GFX9-NEXT: [[DS_BPERMUTE_B32_:%[0-9]+]]:vgpr_32 = DS_BPERMUTE_B32 [[COPY1]], [[V_AND_B32_e64_]], 0, implicit $exec
+ ; GFX9-NEXT: S_NOP 0, implicit [[DS_BPERMUTE_B32_]]
+ ;
+ ; GFX10-LABEL: name: no_fold_redundant_and_sgpr_into_vgpr_use
+ ; GFX10: liveins: $sgpr0, $vgpr0
+ ; GFX10-NEXT: {{ $}}
+ ; GFX10-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $sgpr0
+ ; GFX10-NEXT: [[COPY1:%[0-9]+]]:vgpr_32 = COPY $vgpr0
+ ; GFX10-NEXT: [[S_AND_B32_:%[0-9]+]]:sreg_32 = S_AND_B32 [[COPY]], 255, implicit-def dead $scc
+ ; GFX10-NEXT: [[V_AND_B32_e64_:%[0-9]+]]:vgpr_32 = V_AND_B32_e64 511, [[S_AND_B32_]], implicit $exec
+ ; GFX10-NEXT: [[DS_BPERMUTE_B32_:%[0-9]+]]:vgpr_32 = DS_BPERMUTE_B32 [[COPY1]], [[V_AND_B32_e64_]], 0, implicit $exec
+ ; GFX10-NEXT: S_NOP 0, implicit [[DS_BPERMUTE_B32_]]
+ %0:sreg_32 = COPY $sgpr0
+ %1:vgpr_32 = COPY $vgpr0
+ %2:sreg_32 = S_AND_B32 %0:sreg_32, 255, implicit-def dead $scc
+ %3:vgpr_32 = V_MOV_B32_e32 511, implicit $exec
+ %4:vgpr_32 = V_AND_B32_e64 %3:vgpr_32, %2:sreg_32, implicit $exec
+ %5:vgpr_32 = DS_BPERMUTE_B32 %1:vgpr_32, %4:vgpr_32, 0, implicit $exec
+ S_NOP 0, implicit %5:vgpr_32
+...
+
---
name: no_fold_redundant_and_scc_live
tracksRegLiveness: true
More information about the llvm-commits
mailing list