[llvm-branch-commits] [llvm] [AMDGPU] Verify VGPR tuple alignment from the operand register class (PR #219230)
Valery Pykhtin via llvm-branch-commits
llvm-branch-commits at lists.llvm.org
Tue Sep 1 08:48:52 PDT 2026
https://github.com/vpykhtin updated https://github.com/llvm/llvm-project/pull/219230
>From 73851c2fe015ac8f9afb47ad7648fb7894d4a190 Mon Sep 17 00:00:00 2001
From: Valery Pykhtin <valery.pykhtin at amd.com>
Date: Wed, 26 Aug 2026 17:57:52 +0000
Subject: [PATCH 1/2] [AMDGPU] Verify VGPR tuple alignment from the operand
register class
The machine verifier decided VGPR tuple alignment with isProperlyAlignedRC(),
which inspects only the register's own class. Alignment is not really a property
of the register in isolation: whether a 64-bit tuple must be even-aligned depends
on the operand it feeds, and on mixed-alignment targets the same register class
can be required to be aligned in one operand and exempt in another. Inspecting
only the register also conflates alignment with unrelated problems - a register
that is simply the wrong bank or size for the operand came out as "requires even
aligned vector registers" as well.
Make the operand's register class the source of truth instead: a register is
misaligned only when it does not satisfy the operand's class but its even-aligned
same-bank/width equivalent (SIRegisterInfo::getAlignedEquivalentRC) would. A
register that fits neither is a genuine class or bank mismatch and is left to the
illegal-register and sub-register checks. So an AGPR in a VGPR|SGPR (VS_64)
operand is now reported as an illegal register, and a wrong-size register (e.g. a
64-bit VGPR in a 128-bit MFMA source) or an invalid sub-register index is
reported by those checks alone, no longer doubled up as an "even aligned" error.
This drops the redundant diagnostics in tests.
Deriving the requirement from the operand class also lets several special cases
go away. The RegClass == -1 early-out is hoisted so the operand class is always
valid, and the V_MOV_B64_PSEUDO / AV_MOV_B64_IMM_PSEUDO / spill exemptions are
dropped: those operands use unaligned register classes (VReg_64, AV_64, and the
spill classes), which every register already satisfies, so the comparison never
flags them. Inline-asm operands (RegClass == -1) are no longer alignment-checked,
matching the prior FIXME that they were never meaningfully verified.
The same reasoning removes the DS_GWS-specific alignment check: on subtargets
that require aligned VGPRs the DS_GWS data0 operand has the AV_64_Align2 register
class, so the operand-class check above already diagnoses its alignment and the
separate check is redundant. The image vaddr operand is a plain VGPR_32 whose
class cannot encode even-alignment, so its dedicated position check is kept.
Co-Authored-By: Claude <noreply at anthropic.com>
---
llvm/lib/Target/AMDGPU/SIInstrInfo.cpp | 70 +++++++++----------
llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp | 15 ++++
llvm/lib/Target/AMDGPU/SIRegisterInfo.h | 10 +++
.../CodeGen/AMDGPU/verify-ds-gws-align.mir | 10 ---
.../AMDGPU/verify-gfx90a-aligned-vgprs.mir | 10 ---
...ported-subreg-index-aligned-vgpr-check.mir | 4 --
...rted-unaligned-vgpr-check-vsrc-operand.mir | 2 -
7 files changed, 57 insertions(+), 64 deletions(-)
diff --git a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
index 3094c1390e9cf..99458944b6685 100644
--- a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
@@ -5456,44 +5456,48 @@ bool SIInstrInfo::verifyInstruction(const MachineInstr &MI,
break;
}
+ // Operands without a fixed register class (RegClass == -1), such as inline
+ // asm operands, are not verified here.
+ if (RegClass == -1)
+ continue;
+
if (!MO.isReg())
continue;
Register Reg = MO.getReg();
if (!Reg)
continue;
- // FIXME: Ideally we would have separate instruction definitions with the
- // aligned register constraint.
- // FIXME: We do not verify inline asm operands, but custom inline asm
- // verification is broken anyway
- if (ST.needsAlignedVGPRs() && Opcode != AMDGPU::AV_MOV_B64_IMM_PSEUDO &&
- Opcode != AMDGPU::V_MOV_B64_PSEUDO && !isSpill(MI)) {
- const TargetRegisterClass *RC = RI.getRegClassForReg(MRI, Reg);
- if (RI.hasVectorRegisters(RC) && MO.getSubReg()) {
- if (const TargetRegisterClass *SubRC =
- RI.getSubRegisterClass(RC, MO.getSubReg())) {
- RC = RI.getCompatibleSubRegClass(RC, SubRC, MO.getSubReg());
- if (RC)
- RC = SubRC;
+ const TargetRegisterClass *OpRC = RI.getRegClass(RegClass);
+
+ if (ST.needsAlignedVGPRs()) {
+ const TargetRegisterClass *RegRC = RI.getRegClassForReg(MRI, Reg);
+ if (RI.hasVectorRegisters(RegRC)) {
+ if (MO.getSubReg()) {
+ // Narrow to the sub-register's class. getSubRegisterClass already
+ // accounts for the sub-register index's alignment within the tuple
+ // (an odd-aligned slice yields an unaligned class, caught below); a
+ // null result means an invalid sub-register index and is left to the
+ // sub-register check.
+ RegRC = RI.getSubRegisterClass(RegRC, MO.getSubReg());
+ }
+ // Flag an alignment-only mismatch: the register does not satisfy the
+ // operand's class, but its even-aligned same-bank/width equivalent
+ // would. A bank or size mismatch fails even when aligned, so it is left
+ // to the illegal-register / sub-register checks.
+ if (RegRC && !OpRC->hasSubClassEq(RegRC)) {
+ const TargetRegisterClass *AlignedRegRC =
+ RI.getAlignedEquivalentRC(RegRC);
+ if (AlignedRegRC && OpRC->hasSubClassEq(AlignedRegRC)) {
+ ErrInfo = "Subtarget requires even aligned vector registers";
+ return false;
+ }
}
- }
-
- // Check that this is the aligned version of the class.
- if (!RC || !RI.isProperlyAlignedRC(*RC)) {
- ErrInfo = "Subtarget requires even aligned vector registers";
- return false;
}
}
- if (RegClass != -1) {
- if (Reg.isVirtual())
- continue;
-
- const TargetRegisterClass *RC = RI.getRegClass(RegClass);
- if (!RC->contains(Reg)) {
- ErrInfo = "Operand has incorrect register class.";
- return false;
- }
+ if (Reg.isPhysical() && !OpRC->contains(Reg)) {
+ ErrInfo = "Operand has incorrect register class.";
+ return false;
}
}
@@ -6040,16 +6044,6 @@ bool SIInstrInfo::verifyInstruction(const MachineInstr &MI,
!(RI.getChannelFromSubReg(Op->getSubReg()) & 1);
};
- if (Opcode == AMDGPU::DS_GWS_INIT || Opcode == AMDGPU::DS_GWS_SEMA_BR ||
- Opcode == AMDGPU::DS_GWS_BARRIER) {
-
- if (!isAlignedReg(AMDGPU::OpName::data0)) {
- ErrInfo = "Subtarget requires even aligned vector registers "
- "for DS_GWS instructions";
- return false;
- }
- }
-
if (isMIMG(MI)) {
if (!isAlignedReg(AMDGPU::OpName::vaddr)) {
ErrInfo = "Subtarget requires even aligned vector registers "
diff --git a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
index 69dc3a66ab09a..601e29fa4c2e0 100644
--- a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
@@ -3899,6 +3899,21 @@ SIRegisterInfo::getDefaultVectorSuperClassForBitWidth(unsigned BitWidth) const {
: getVGPRClassForBitWidth(BitWidth);
}
+const TargetRegisterClass *
+SIRegisterInfo::getAlignedEquivalentRC(const TargetRegisterClass *RC) const {
+ assert(RC->isAllocatable() &&
+ "expected an allocatable register class; non-allocatable VS_* operand "
+ "classes cannot be a register's class");
+ unsigned Width = getRegSizeInBits(*RC);
+ if (isVGPRClass(RC))
+ return getVGPRClassForBitWidth(Width);
+ if (isAGPRClass(RC))
+ return getAGPRClassForBitWidth(Width);
+ if (isVectorSuperClass(RC))
+ return getVectorSuperClassForBitWidth(Width);
+ return nullptr;
+}
+
const TargetRegisterClass *
SIRegisterInfo::getSGPRClassForBitWidth(unsigned BitWidth) {
if (BitWidth == 16 || BitWidth == 32)
diff --git a/llvm/lib/Target/AMDGPU/SIRegisterInfo.h b/llvm/lib/Target/AMDGPU/SIRegisterInfo.h
index 296389ed88915..482e1d1003a3a 100644
--- a/llvm/lib/Target/AMDGPU/SIRegisterInfo.h
+++ b/llvm/lib/Target/AMDGPU/SIRegisterInfo.h
@@ -226,6 +226,16 @@ class SIRegisterInfo final : public AMDGPUGenRegisterInfo {
const TargetRegisterClass *
getDefaultVectorSuperClassForBitWidth(unsigned BitWidth) const;
+ /// \returns the even-aligned register class with the same register bank and
+ /// width as \p RC (its aligned "canonical" class), or null if \p RC has no
+ /// vector registers. Only meaningful on subtargets that require aligned
+ /// VGPRs, where the per-bitwidth getters return the aligned variant. \p RC
+ /// must be an allocatable register's class; the non-allocatable VS_* operand
+ /// classes are intentionally not handled.
+ LLVM_READONLY
+ const TargetRegisterClass *
+ getAlignedEquivalentRC(const TargetRegisterClass *RC) const;
+
LLVM_READONLY
static const TargetRegisterClass *getSGPRClassForBitWidth(unsigned BitWidth);
diff --git a/llvm/test/CodeGen/AMDGPU/verify-ds-gws-align.mir b/llvm/test/CodeGen/AMDGPU/verify-ds-gws-align.mir
index 8a7fc2a3af057..feffe3d27e451 100644
--- a/llvm/test/CodeGen/AMDGPU/verify-ds-gws-align.mir
+++ b/llvm/test/CodeGen/AMDGPU/verify-ds-gws-align.mir
@@ -10,29 +10,21 @@ body: |
bb.0:
; A 32-bit sub-register does not fit the 64-bit AV_64_Align2 data0 operand.
%0:areg_128_align2 = IMPLICIT_DEF
- ; GFX90A-ERR: *** Bad machine code: Subtarget requires even aligned vector registers for DS_GWS instructions ***
- ; GFX90A-ERR: - instruction: DS_GWS_INIT killed %0.sub1:areg_128_align2
; GFX90A-ERR: *** Bad machine code: Illegal virtual register for instruction ***
; GFX90A-ERR: - instruction: DS_GWS_INIT killed %0.sub1:areg_128_align2
; GFX90A-ERR: AReg_128_Align2.sub1 cannot be used for AV_64_Align2 operands.
DS_GWS_INIT killed %0.sub1, 0, implicit $m0, implicit $exec :: (store (s32) into custom "GWSResource")
%0:areg_128_align2 = IMPLICIT_DEF
- ; GFX90A-ERR: *** Bad machine code: Subtarget requires even aligned vector registers for DS_GWS instructions ***
- ; GFX90A-ERR: - instruction: DS_GWS_INIT killed %0.sub3:areg_128_align2
; GFX90A-ERR: *** Bad machine code: Illegal virtual register for instruction ***
; GFX90A-ERR: - instruction: DS_GWS_INIT killed %0.sub3:areg_128_align2
; GFX90A-ERR: AReg_128_Align2.sub3 cannot be used for AV_64_Align2 operands.
DS_GWS_INIT killed %0.sub3, 0, implicit $m0, implicit $exec :: (store (s32) into custom "GWSResource")
%1:vreg_64_align2 = IMPLICIT_DEF
- ; GFX90A-ERR: *** Bad machine code: Subtarget requires even aligned vector registers for DS_GWS instructions ***
- ; GFX90A-ERR: - instruction: DS_GWS_SEMA_BR killed %1.sub1:vreg_64_align2
; GFX90A-ERR: *** Bad machine code: Illegal virtual register for instruction ***
; GFX90A-ERR: - instruction: DS_GWS_SEMA_BR killed %1.sub1:vreg_64_align2
; GFX90A-ERR: VReg_64_Align2.sub1 cannot be used for AV_64_Align2 operands.
DS_GWS_SEMA_BR killed %1.sub1, 0, implicit $m0, implicit $exec :: (store (s32) into custom "GWSResource")
%2:vreg_64 = IMPLICIT_DEF
- ; GFX90A-ERR: *** Bad machine code: Subtarget requires even aligned vector registers for DS_GWS instructions ***
- ; GFX90A-ERR: - instruction: DS_GWS_BARRIER killed %2.sub0:vreg_64
; GFX90A-ERR: *** Bad machine code: Illegal virtual register for instruction ***
; GFX90A-ERR: - instruction: DS_GWS_BARRIER killed %2.sub0:vreg_64
; GFX90A-ERR: VReg_64.sub0 cannot be used for AV_64_Align2 operands.
@@ -40,8 +32,6 @@ body: |
; A whole 32-bit register is the wrong size for the 64-bit operand.
%3:vgpr_32 = IMPLICIT_DEF
- ; GFX90A-ERR: *** Bad machine code: Subtarget requires even aligned vector registers for DS_GWS instructions ***
- ; GFX90A-ERR: - instruction: DS_GWS_INIT killed %3:vgpr_32
; GFX90A-ERR: *** Bad machine code: Illegal virtual register for instruction ***
; GFX90A-ERR: - instruction: DS_GWS_INIT killed %3:vgpr_32
; GFX90A-ERR: Expected a AV_64_Align2 register, but got a VGPR_32 register
diff --git a/llvm/test/CodeGen/AMDGPU/verify-gfx90a-aligned-vgprs.mir b/llvm/test/CodeGen/AMDGPU/verify-gfx90a-aligned-vgprs.mir
index f8f8197cbc005..8cec088873a94 100644
--- a/llvm/test/CodeGen/AMDGPU/verify-gfx90a-aligned-vgprs.mir
+++ b/llvm/test/CodeGen/AMDGPU/verify-gfx90a-aligned-vgprs.mir
@@ -193,16 +193,12 @@ body: |
$vgpr0_vgpr1 = V_PK_ADD_F32 0, %13, 11, %12, 0, 0, 0, 0, 0, implicit $mode, implicit $exec
; src %14 is an AGPR - a bank mismatch, not an alignment problem
- ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers ***
- ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_ADD_F32 0, %13:vreg_64_align2, 11, %14.sub1_sub2:areg_96_align2
; CHECK: *** Bad machine code: Illegal virtual register for instruction ***
; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_ADD_F32 0, %13:vreg_64_align2, 11, %14.sub1_sub2:areg_96_align2
; CHECK: AReg_96_Align2.sub1_sub2 cannot be used for VS_64_Align2 operands.
$vgpr0_vgpr1 = V_PK_ADD_F32 0, %13, 11, %14.sub1_sub2, 0, 0, 0, 0, 0, implicit $mode, implicit $exec
; src %14 is an AGPR - a bank mismatch, not an alignment problem
- ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers ***
- ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_ADD_F32 0, %14.sub1_sub2:areg_96_align2, 11, %13:vreg_64_align2
; CHECK: *** Bad machine code: Illegal virtual register for instruction ***
; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_ADD_F32 0, %14.sub1_sub2:areg_96_align2, 11, %13:vreg_64_align2
; CHECK: AReg_96_Align2.sub1_sub2 cannot be used for VS_64_Align2 operands.
@@ -219,16 +215,12 @@ body: |
$vgpr0_vgpr1 = V_PK_MUL_F32 0, %13, 11, %12, 0, 0, 0, 0, 0, implicit $mode, implicit $exec
; src %14 is an AGPR - a bank mismatch, not an alignment problem
- ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers ***
- ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_MUL_F32 0, %13:vreg_64_align2, 11, %14.sub1_sub2:areg_96_align2
; CHECK: *** Bad machine code: Illegal virtual register for instruction ***
; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_MUL_F32 0, %13:vreg_64_align2, 11, %14.sub1_sub2:areg_96_align2
; CHECK: AReg_96_Align2.sub1_sub2 cannot be used for VS_64_Align2 operands.
$vgpr0_vgpr1 = V_PK_MUL_F32 0, %13, 11, %14.sub1_sub2, 0, 0, 0, 0, 0, implicit $mode, implicit $exec
; src %14 is an AGPR - a bank mismatch, not an alignment problem
- ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers ***
- ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_MUL_F32 0, %14.sub1_sub2:areg_96_align2, 11, %13:vreg_64_align2
; CHECK: *** Bad machine code: Illegal virtual register for instruction ***
; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_MUL_F32 0, %14.sub1_sub2:areg_96_align2, 11, %13:vreg_64_align2
; CHECK: AReg_96_Align2.sub1_sub2 cannot be used for VS_64_Align2 operands.
@@ -249,8 +241,6 @@ body: |
$vgpr0_vgpr1 = nofpexcept V_PK_FMA_F32 8, %13, 8, %12, 11, %14.sub0_sub1, 0, 0, 0, 0, 0, implicit $mode, implicit $exec
; src %14 is an AGPR - a bank mismatch (both %13 sources are aligned)
- ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers ***
- ; CHECK: - instruction: $vgpr0_vgpr1 = nofpexcept V_PK_FMA_F32 8, %13:vreg_64_align2, 8, %13:vreg_64_align2, 11, %14.sub1_sub2:areg_96_align2
; CHECK: *** Bad machine code: Illegal virtual register for instruction ***
; CHECK: - instruction: $vgpr0_vgpr1 = nofpexcept V_PK_FMA_F32 8, %13:vreg_64_align2, 8, %13:vreg_64_align2, 11, %14.sub1_sub2:areg_96_align2
; CHECK: AReg_96_Align2.sub1_sub2 cannot be used for VS_64_Align2 operands.
diff --git a/llvm/test/MachineVerifier/AMDGPU/unsupported-subreg-index-aligned-vgpr-check.mir b/llvm/test/MachineVerifier/AMDGPU/unsupported-subreg-index-aligned-vgpr-check.mir
index 5b1aaa6337261..6ddf0f62a6f1c 100644
--- a/llvm/test/MachineVerifier/AMDGPU/unsupported-subreg-index-aligned-vgpr-check.mir
+++ b/llvm/test/MachineVerifier/AMDGPU/unsupported-subreg-index-aligned-vgpr-check.mir
@@ -21,10 +21,6 @@ body: |
; CHECK-NEXT: - operand 1: %1.sub16_sub17_sub18_sub19:vreg_512_align2
; CHECK-NEXT: Register class VReg_512_Align2 does not support subreg index sub16_sub17_sub18_sub19
- ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers ***
- ; CHECK-NEXT: - function: uses_invalid_subregister_for_regclass
- ; CHECK-NEXT: - basic block: %bb.0
- ; CHECK-NEXT: - instruction: GLOBAL_STORE_DWORDX4_SADDR %0:vgpr_32, %2.sub16_sub17_sub18_sub19:vreg_512, undef $sgpr8_sgpr9, 80, 0, implicit $exec :: (store (s128), addrspace 1)
GLOBAL_STORE_DWORDX4_SADDR %0, %1.sub16_sub17_sub18_sub19, undef $sgpr8_sgpr9, 80, 0, implicit $exec :: (store (s128), addrspace 1)
; Test with unaligned class
diff --git a/llvm/test/MachineVerifier/AMDGPU/unsupported-unaligned-vgpr-check-vsrc-operand.mir b/llvm/test/MachineVerifier/AMDGPU/unsupported-unaligned-vgpr-check-vsrc-operand.mir
index c982b1a35d44a..3df4ce13fc15d 100644
--- a/llvm/test/MachineVerifier/AMDGPU/unsupported-unaligned-vgpr-check-vsrc-operand.mir
+++ b/llvm/test/MachineVerifier/AMDGPU/unsupported-unaligned-vgpr-check-vsrc-operand.mir
@@ -23,12 +23,10 @@ body: |
%1:vgpr_32 = V_MOV_B32_e32 1, implicit $exec
%2:vreg_64 = IMPLICIT_DEF
- ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers ***
; CHECK: *** Bad machine code: Illegal virtual register for instruction ***
%3:areg_128_align2 = V_MFMA_F32_4X4X1F32_e64 %0, %1, %2, 0, 0, 0, implicit $mode, implicit $exec
%4:vreg_64 = IMPLICIT_DEF
- ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers ***
; CHECK: *** Bad machine code: Illegal virtual register for instruction ***
%5:vreg_128_align2 = V_MFMA_F32_4X4X1F32_vgprcd_e64 %0, %1, %4, 0, 0, 0, implicit $mode, implicit $exec
...
>From d8b59c024c09e8318f3a264c233b929e4dde10d4 Mon Sep 17 00:00:00 2001
From: Valery Pykhtin <valery.pykhtin at amd.com>
Date: Mon, 31 Aug 2026 13:06:03 +0000
Subject: [PATCH 2/2] [AMDGPU] Match the AsmParser's operand-side check and
share its helper
In the upcoming PR AsmParser performs the same alignment check but differently:
it tests whether a concrete physreg, which carries no regclass, is in the
relaxed operand's regclass.
Change the verifier to work the same way, reusing
AMDGPUBaseInfo::getUnalignedEquivalentRC.
Co-Authored-By: Claude <noreply at anthropic.com>
---
llvm/lib/Target/AMDGPU/SIInstrInfo.cpp | 13 +--
llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp | 15 ----
llvm/lib/Target/AMDGPU/SIRegisterInfo.h | 10 ---
.../Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp | 87 +++++++++++++++++++
llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h | 5 ++
5 files changed, 99 insertions(+), 31 deletions(-)
diff --git a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
index 99458944b6685..3b3bf682a5d29 100644
--- a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
@@ -5481,13 +5481,14 @@ bool SIInstrInfo::verifyInstruction(const MachineInstr &MI,
RegRC = RI.getSubRegisterClass(RegRC, MO.getSubReg());
}
// Flag an alignment-only mismatch: the register does not satisfy the
- // operand's class, but its even-aligned same-bank/width equivalent
- // would. A bank or size mismatch fails even when aligned, so it is left
- // to the illegal-register / sub-register checks.
+ // operand's class, but does satisfy it with the alignment requirement
+ // relaxed (the operand class's unaligned equivalent). A bank or size
+ // mismatch fails even when relaxed, so it is left to the
+ // illegal-register / sub-register checks.
if (RegRC && !OpRC->hasSubClassEq(RegRC)) {
- const TargetRegisterClass *AlignedRegRC =
- RI.getAlignedEquivalentRC(RegRC);
- if (AlignedRegRC && OpRC->hasSubClassEq(AlignedRegRC)) {
+ int UnalignedRC = AMDGPU::getUnalignedEquivalentRC(RegClass);
+ if (UnalignedRC >= 0 &&
+ RI.getRegClass(UnalignedRC)->hasSubClassEq(RegRC)) {
ErrInfo = "Subtarget requires even aligned vector registers";
return false;
}
diff --git a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
index 601e29fa4c2e0..69dc3a66ab09a 100644
--- a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
@@ -3899,21 +3899,6 @@ SIRegisterInfo::getDefaultVectorSuperClassForBitWidth(unsigned BitWidth) const {
: getVGPRClassForBitWidth(BitWidth);
}
-const TargetRegisterClass *
-SIRegisterInfo::getAlignedEquivalentRC(const TargetRegisterClass *RC) const {
- assert(RC->isAllocatable() &&
- "expected an allocatable register class; non-allocatable VS_* operand "
- "classes cannot be a register's class");
- unsigned Width = getRegSizeInBits(*RC);
- if (isVGPRClass(RC))
- return getVGPRClassForBitWidth(Width);
- if (isAGPRClass(RC))
- return getAGPRClassForBitWidth(Width);
- if (isVectorSuperClass(RC))
- return getVectorSuperClassForBitWidth(Width);
- return nullptr;
-}
-
const TargetRegisterClass *
SIRegisterInfo::getSGPRClassForBitWidth(unsigned BitWidth) {
if (BitWidth == 16 || BitWidth == 32)
diff --git a/llvm/lib/Target/AMDGPU/SIRegisterInfo.h b/llvm/lib/Target/AMDGPU/SIRegisterInfo.h
index 482e1d1003a3a..296389ed88915 100644
--- a/llvm/lib/Target/AMDGPU/SIRegisterInfo.h
+++ b/llvm/lib/Target/AMDGPU/SIRegisterInfo.h
@@ -226,16 +226,6 @@ class SIRegisterInfo final : public AMDGPUGenRegisterInfo {
const TargetRegisterClass *
getDefaultVectorSuperClassForBitWidth(unsigned BitWidth) const;
- /// \returns the even-aligned register class with the same register bank and
- /// width as \p RC (its aligned "canonical" class), or null if \p RC has no
- /// vector registers. Only meaningful on subtargets that require aligned
- /// VGPRs, where the per-bitwidth getters return the aligned variant. \p RC
- /// must be an allocatable register's class; the non-allocatable VS_* operand
- /// classes are intentionally not handled.
- LLVM_READONLY
- const TargetRegisterClass *
- getAlignedEquivalentRC(const TargetRegisterClass *RC) const;
-
LLVM_READONLY
static const TargetRegisterClass *getSGPRClassForBitWidth(unsigned BitWidth);
diff --git a/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp b/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp
index a0648da67c06d..52803c2a9f494 100644
--- a/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp
@@ -3016,6 +3016,93 @@ unsigned getRegBitWidth(const MCRegisterClass &RC) {
return getRegBitWidth(RC.getID());
}
+int getUnalignedEquivalentRC(unsigned RCID) {
+ switch (RCID) {
+ case AMDGPU::VReg_64_Align2RegClassID:
+ return AMDGPU::VReg_64RegClassID;
+ case AMDGPU::VReg_96_Align2RegClassID:
+ return AMDGPU::VReg_96RegClassID;
+ case AMDGPU::VReg_128_Align2RegClassID:
+ return AMDGPU::VReg_128RegClassID;
+ case AMDGPU::VReg_160_Align2RegClassID:
+ return AMDGPU::VReg_160RegClassID;
+ case AMDGPU::VReg_192_Align2RegClassID:
+ return AMDGPU::VReg_192RegClassID;
+ case AMDGPU::VReg_224_Align2RegClassID:
+ return AMDGPU::VReg_224RegClassID;
+ case AMDGPU::VReg_256_Align2RegClassID:
+ return AMDGPU::VReg_256RegClassID;
+ case AMDGPU::VReg_288_Align2RegClassID:
+ return AMDGPU::VReg_288RegClassID;
+ case AMDGPU::VReg_320_Align2RegClassID:
+ return AMDGPU::VReg_320RegClassID;
+ case AMDGPU::VReg_352_Align2RegClassID:
+ return AMDGPU::VReg_352RegClassID;
+ case AMDGPU::VReg_384_Align2RegClassID:
+ return AMDGPU::VReg_384RegClassID;
+ case AMDGPU::VReg_512_Align2RegClassID:
+ return AMDGPU::VReg_512RegClassID;
+ case AMDGPU::VReg_1024_Align2RegClassID:
+ return AMDGPU::VReg_1024RegClassID;
+ case AMDGPU::AReg_64_Align2RegClassID:
+ return AMDGPU::AReg_64RegClassID;
+ case AMDGPU::AReg_96_Align2RegClassID:
+ return AMDGPU::AReg_96RegClassID;
+ case AMDGPU::AReg_128_Align2RegClassID:
+ return AMDGPU::AReg_128RegClassID;
+ case AMDGPU::AReg_160_Align2RegClassID:
+ return AMDGPU::AReg_160RegClassID;
+ case AMDGPU::AReg_192_Align2RegClassID:
+ return AMDGPU::AReg_192RegClassID;
+ case AMDGPU::AReg_224_Align2RegClassID:
+ return AMDGPU::AReg_224RegClassID;
+ case AMDGPU::AReg_256_Align2RegClassID:
+ return AMDGPU::AReg_256RegClassID;
+ case AMDGPU::AReg_288_Align2RegClassID:
+ return AMDGPU::AReg_288RegClassID;
+ case AMDGPU::AReg_320_Align2RegClassID:
+ return AMDGPU::AReg_320RegClassID;
+ case AMDGPU::AReg_352_Align2RegClassID:
+ return AMDGPU::AReg_352RegClassID;
+ case AMDGPU::AReg_384_Align2RegClassID:
+ return AMDGPU::AReg_384RegClassID;
+ case AMDGPU::AReg_512_Align2RegClassID:
+ return AMDGPU::AReg_512RegClassID;
+ case AMDGPU::AReg_1024_Align2RegClassID:
+ return AMDGPU::AReg_1024RegClassID;
+ case AMDGPU::AV_64_Align2RegClassID:
+ return AMDGPU::AV_64RegClassID;
+ case AMDGPU::AV_96_Align2RegClassID:
+ return AMDGPU::AV_96RegClassID;
+ case AMDGPU::AV_128_Align2RegClassID:
+ return AMDGPU::AV_128RegClassID;
+ case AMDGPU::AV_160_Align2RegClassID:
+ return AMDGPU::AV_160RegClassID;
+ case AMDGPU::AV_192_Align2RegClassID:
+ return AMDGPU::AV_192RegClassID;
+ case AMDGPU::AV_224_Align2RegClassID:
+ return AMDGPU::AV_224RegClassID;
+ case AMDGPU::AV_256_Align2RegClassID:
+ return AMDGPU::AV_256RegClassID;
+ case AMDGPU::AV_288_Align2RegClassID:
+ return AMDGPU::AV_288RegClassID;
+ case AMDGPU::AV_320_Align2RegClassID:
+ return AMDGPU::AV_320RegClassID;
+ case AMDGPU::AV_352_Align2RegClassID:
+ return AMDGPU::AV_352RegClassID;
+ case AMDGPU::AV_384_Align2RegClassID:
+ return AMDGPU::AV_384RegClassID;
+ case AMDGPU::AV_512_Align2RegClassID:
+ return AMDGPU::AV_512RegClassID;
+ case AMDGPU::AV_1024_Align2RegClassID:
+ return AMDGPU::AV_1024RegClassID;
+ case AMDGPU::VS_64_Align2RegClassID:
+ return AMDGPU::VS_64RegClassID;
+ default:
+ return -1;
+ }
+}
+
bool isInlinableLiteral64(int64_t Literal, bool HasInv2Pi) {
if (isInlinableIntLiteral(Literal))
return true;
diff --git a/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h b/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h
index d429584b00999..d4503191d3e03 100644
--- a/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h
+++ b/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h
@@ -1575,6 +1575,11 @@ unsigned getRegBitWidth(unsigned RCID);
/// Get the size in bits of a register from the register class \p RC.
unsigned getRegBitWidth(const MCRegisterClass &RC);
+/// Return the register class equivalent to the even-aligned VGPR/AGPR/AV tuple
+/// register class \p RCID with the alignment requirement relaxed, or -1 if
+/// \p RCID has no such equivalent.
+int getUnalignedEquivalentRC(unsigned RCID);
+
LLVM_READNONE
inline unsigned getOperandSize(const MCOperandInfo &OpInfo) {
switch (OpInfo.OperandType) {
More information about the llvm-branch-commits
mailing list