[clang] [llvm] [SPIRV] Handle AMDGPU buffers in AMDGCN flavoured SPIR-V (PR #199376)
via cfe-commits
cfe-commits at lists.llvm.org
Sat May 23 14:00:39 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-backend-spir-v
Author: Alex Voicu (AlexVlx)
<details>
<summary>Changes</summary>
This is a bit of a chonk because it includes a bunch of related / necessary changes (could have been a stack, but my stack-fu is weak). I'll first merge in the various bits and bobs and then un-draft it, however it seems that getting some early discussion would be very helpful.
As to the change itself: AMDGPU has fairly elaborate infrastructure around buffer ops, which is closely tied to bespoke address spaces 7 & 8 + non-trivial lowering of operations on pointers to said ASes. We want to retain the AS info in SPIR-V, so that we can work the magic when we finalise to AMDGPU. The challenge is that in SPIR-V, 7 and 8 correspond to the `Input` and, respectively, `Output` storage which have restrictive semantics (the latter is outright unavailable for `Kernel` SPIR-V). What we end up doing is:
- Retain AS7 and AS8 in IR;
- Opaquify non-constexpr AS casts to / from them, to retain generated SPIR-V validity;
- Lower constexpr AS casts into `PtrToU` followed by `UToPtr` `SpecConstantOp`s
- Lower AS7 to `DeviceOnlyINTEL` and AS8 to `HostOnlyINTEL`, which we do not use and do not plan to use at any point.
Whilst this is conceptually pretty straightforward, it turns out that a bunch of plumbing was needed, since AS7 and AS8 have somewhat oddly layouts (e.g. the pointer size is 160 and 256 bits respectively).
---
Patch is 81.17 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/199376.diff
27 Files Affected:
- (modified) clang/lib/CodeGen/CodeGenModule.cpp (+4-1)
- (modified) clang/test/CodeGen/target-data.c (+1-1)
- (modified) llvm/include/llvm/IR/IntrinsicsSPIRV.td (+5)
- (modified) llvm/lib/IR/Type.cpp (+4)
- (modified) llvm/lib/Target/SPIRV/SPIRVBuiltins.cpp (+9-7)
- (modified) llvm/lib/Target/SPIRV/SPIRVEmitIntrinsics.cpp (+8-3)
- (modified) llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp (+16-10)
- (modified) llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.h (+2-2)
- (modified) llvm/lib/Target/SPIRV/SPIRVISelLowering.cpp (+37)
- (modified) llvm/lib/Target/SPIRV/SPIRVISelLowering.h (+3)
- (modified) llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp (+27-6)
- (modified) llvm/lib/Target/SPIRV/SPIRVLegalizeZeroSizeArrays.cpp (+2-1)
- (modified) llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp (+39-18)
- (modified) llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.h (+2-1)
- (modified) llvm/lib/Target/SPIRV/SPIRVPreLegalizer.cpp (+3-2)
- (modified) llvm/lib/Target/SPIRV/SPIRVPrepareFunctions.cpp (+72-5)
- (modified) llvm/lib/Target/SPIRV/SPIRVPushConstantAccess.cpp (+2-1)
- (modified) llvm/lib/Target/SPIRV/SPIRVSubtarget.cpp (+6-3)
- (modified) llvm/lib/Target/SPIRV/SPIRVUtils.cpp (+6-2)
- (modified) llvm/lib/Target/SPIRV/SPIRVUtils.h (+6-1)
- (modified) llvm/lib/TargetParser/TargetDataLayout.cpp (+8-3)
- (added) llvm/test/CodeGen/SPIRV/pointers/amdgcnspirv-lower-buffer-fat-pointers-calls.ll (+81)
- (added) llvm/test/CodeGen/SPIRV/pointers/amdgcnspirv-lower-buffer-fat-pointers-constants.ll (+305)
- (added) llvm/test/CodeGen/SPIRV/pointers/amdgcnspirv-lower-buffer-fat-pointers-dead-intrinsics.ll (+9)
- (added) llvm/test/CodeGen/SPIRV/pointers/amdgcnspirv-lower-buffer-fat-pointers-memops.ll (+162)
- (added) llvm/test/CodeGen/SPIRV/pointers/amdgcnspirv-lower-buffer-fat-pointers-p7-in-memory.ll (+75)
- (added) llvm/test/CodeGen/SPIRV/pointers/amdgcnspirv-lower-buffer-fat-pointers-unoptimized-debug-data.ll (+142)
``````````diff
diff --git a/clang/lib/CodeGen/CodeGenModule.cpp b/clang/lib/CodeGen/CodeGenModule.cpp
index 236738e9975d3..3afeb5e4cf7c5 100644
--- a/clang/lib/CodeGen/CodeGenModule.cpp
+++ b/clang/lib/CodeGen/CodeGenModule.cpp
@@ -415,7 +415,10 @@ static void checkDataLayoutConsistency(const TargetInfo &Target,
if (Target.hasIbm128Type())
Check("__ibm128", llvm::Type::getPPC_FP128Ty(Context), Target.Ibm128Align);
- Check("void*", llvm::PointerType::getUnqual(Context), Target.PointerAlign);
+ Check("void*",
+ llvm::PointerType::get(Context,
+ Target.getTargetAddressSpace(LangAS::Default)),
+ Target.PointerAlign);
if (Target.vectorsAreElementAligned() != DL.vectorsAreElementAligned()) {
llvm::errs() << "Datalayout for target " << Triple.str()
diff --git a/clang/test/CodeGen/target-data.c b/clang/test/CodeGen/target-data.c
index 0047e1377a7f8..a73531a7714d6 100644
--- a/clang/test/CodeGen/target-data.c
+++ b/clang/test/CodeGen/target-data.c
@@ -259,7 +259,7 @@
// RUN: %clang_cc1 -triple spirv64-amd-amdhsa -o - -emit-llvm %s | \
// RUN: FileCheck %s -check-prefix=AMDGPUSPIRV64
-// AMDGPUSPIRV64: target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n32:64-S32-G1-P4-A0"
+// AMDGPUSPIRV64: target datalayout = "e-m:e-p:32:32-p1:64:64-p2:64:64-p3:32:32-p4:64:64-p5:32:32-p6:32:32-p7:160:256:256:32-p8:128:128:128:48-p9:192:256:256:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A0-G1-P4-ni:7:8:9"
// RUN: %clang_cc1 -triple spirv-unknown-vulkan -o - -emit-llvm %s | \
// RUN: FileCheck %s -check-prefix=SPIRVVULKAN
diff --git a/llvm/include/llvm/IR/IntrinsicsSPIRV.td b/llvm/include/llvm/IR/IntrinsicsSPIRV.td
index 26996774afa87..d728ed080dca4 100644
--- a/llvm/include/llvm/IR/IntrinsicsSPIRV.td
+++ b/llvm/include/llvm/IR/IntrinsicsSPIRV.td
@@ -354,6 +354,11 @@ def int_spv_rsqrt : DefaultAttrsIntrinsic<[LLVMMatchType<0>], [llvm_anyfloat_ty]
def int_spv_generic_cast_to_ptr_explicit
: DefaultAttrsIntrinsic<[llvm_anyptr_ty], [generic_ptr_ty],
[IntrNoMem, NoUndef<RetIndex>]>;
+ // Encode SPIR-V invalid but potentially target valid casts via intrinsic that
+ // always gets lowered to a function.
+ def int_spv_opaque_ptr_cast
+ : DefaultAttrsIntrinsic<[llvm_anyptr_ty], [llvm_anyptr_ty],
+ [IntrNoMem, NoUndef<RetIndex>]>;
def int_spv_unpackhalf2x16 : DefaultAttrsIntrinsic<[llvm_anyfloat_ty], [llvm_i32_ty], [IntrNoMem]>;
def int_spv_packhalf2x16 : DefaultAttrsIntrinsic<[llvm_anyint_ty], [llvm_anyfloat_ty], [IntrNoMem]>;
diff --git a/llvm/lib/IR/Type.cpp b/llvm/lib/IR/Type.cpp
index 498e78d4b0cc8..4be20db1c0d33 100644
--- a/llvm/lib/IR/Type.cpp
+++ b/llvm/lib/IR/Type.cpp
@@ -1082,6 +1082,10 @@ static TargetTypeInfo getTargetTypeInfo(const TargetExtType *Ty) {
return TargetTypeInfo(
ArrayType::get(Type::getInt8Ty(C), Ty->getIntParameter(0)),
TargetExtType::CanBeGlobal);
+ if (Name == "spirv.$TypedPointerType")
+ return TargetTypeInfo(PointerType::get(C, 4), TargetExtType::HasZeroInit,
+ TargetExtType::CanBeGlobal, TargetExtType::CanBeLocal,
+ TargetExtType::CanBeVectorElement);
if (Name.starts_with("spirv."))
return TargetTypeInfo(PointerType::get(C, 0), TargetExtType::HasZeroInit,
TargetExtType::CanBeGlobal,
diff --git a/llvm/lib/Target/SPIRV/SPIRVBuiltins.cpp b/llvm/lib/Target/SPIRV/SPIRVBuiltins.cpp
index 1febfc21ed3b0..6ef8715783748 100644
--- a/llvm/lib/Target/SPIRV/SPIRVBuiltins.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVBuiltins.cpp
@@ -510,10 +510,10 @@ static Register buildBuiltinVariableLoad(
SPIRV::LinkageType::Import}) {
Register NewRegister =
MIRBuilder.getMRI()->createVirtualRegister(&SPIRV::pIDRegClass);
- MIRBuilder.getMRI()->setType(
- NewRegister,
- LLT::pointer(storageClassToAddressSpace(SPIRV::StorageClass::Function),
- GR->getPointerSize()));
+ unsigned AS = storageClassToAddressSpace(
+ SPIRV::StorageClass::Function, GR->CurMF->getTarget().getTargetTriple());
+ MIRBuilder.getMRI()->setType(NewRegister,
+ LLT::pointer(AS, GR->getPointerSize(AS)));
SPIRVTypeInst PtrType = GR->getOrCreateSPIRVPointerType(
VariableType, MIRBuilder, SPIRV::StorageClass::Input);
GR->assignSPIRVTypeToVReg(PtrType, NewRegister, MIRBuilder.getMF());
@@ -2929,7 +2929,8 @@ static SPIRVTypeInst
getOrCreateSPIRVDeviceEventPointer(MachineIRBuilder &MIRBuilder,
SPIRVGlobalRegistry *GR) {
LLVMContext &Context = MIRBuilder.getMF().getFunction().getContext();
- unsigned SC1 = storageClassToAddressSpace(SPIRV::StorageClass::Generic);
+ unsigned SC1 = storageClassToAddressSpace(
+ SPIRV::StorageClass::Generic, GR->CurMF->getTarget().getTargetTriple());
Type *PtrType = PointerType::get(Context, SC1);
return GR->getOrCreateSPIRVType(PtrType, MIRBuilder,
SPIRV::AccessQualifier::ReadWrite, true);
@@ -2960,8 +2961,9 @@ static bool buildEnqueueKernel(const SPIRV::IncomingCall *Call,
assert(LocalSizeTy && "Local size type is expected");
const uint64_t LocalSizeNum =
cast<ArrayType>(LocalSizeTy)->getNumElements();
- unsigned SC = storageClassToAddressSpace(SPIRV::StorageClass::Generic);
- const LLT LLType = LLT::pointer(SC, GR->getPointerSize());
+ unsigned SC = storageClassToAddressSpace(
+ SPIRV::StorageClass::Generic, GR->CurMF->getTarget().getTargetTriple());
+ const LLT LLType = LLT::pointer(SC, GR->getPointerSize(SC));
const SPIRVTypeInst PointerSizeTy = GR->getOrCreateSPIRVPointerType(
Int32Ty, MIRBuilder, SPIRV::StorageClass::Function);
for (unsigned I = 0; I < LocalSizeNum; ++I) {
diff --git a/llvm/lib/Target/SPIRV/SPIRVEmitIntrinsics.cpp b/llvm/lib/Target/SPIRV/SPIRVEmitIntrinsics.cpp
index b6e71c7b76348..48f12faf8db6f 100644
--- a/llvm/lib/Target/SPIRV/SPIRVEmitIntrinsics.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVEmitIntrinsics.cpp
@@ -1638,12 +1638,17 @@ void SPIRVEmitIntrinsics::preprocessUndefs(IRBuilder<> &B) {
// pointer.
void SPIRVEmitIntrinsics::simplifyNullAddrSpaceCasts() {
for (Instruction &I : make_early_inc_range(instructions(CurrF)))
- if (auto *ASC = dyn_cast<AddrSpaceCastInst>(&I))
+ if (auto *ASC = dyn_cast<AddrSpaceCastInst>(&I)) {
if (isa<ConstantPointerNull>(ASC->getPointerOperand())) {
- ASC->replaceAllUsesWith(
- ConstantPointerNull::get(cast<PointerType>(ASC->getType())));
+ if (ASC->getType()->isVectorTy()) {
+ ASC->replaceAllUsesWith(ConstantVector::getNullValue(ASC->getType()));
+ } else {
+ ASC->replaceAllUsesWith(
+ ConstantPointerNull::get(cast<PointerType>(ASC->getType())));
+ }
ASC->eraseFromParent();
}
+ }
}
void SPIRVEmitIntrinsics::preprocessCompositeConstants(IRBuilder<> &B) {
diff --git a/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp b/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp
index 343311fb44475..06ab77566753c 100644
--- a/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp
@@ -763,7 +763,7 @@ SPIRVGlobalRegistry::getOrCreateConstNullPtr(MachineIRBuilder &MIRBuilder,
if (Res.isValid())
return Res;
- LLT LLTy = LLT::pointer(AddressSpace, getPointerSize());
+ LLT LLTy = LLT::pointer(AddressSpace, getPointerSize(AddressSpace));
Res = CurMF->getRegInfo().createGenericVirtualRegister(LLTy);
CurMF->getRegInfo().setRegClass(Res, &SPIRV::pIDRegClass);
assignSPIRVTypeToVReg(SpvType, Res, *CurMF);
@@ -863,8 +863,8 @@ Register SPIRVGlobalRegistry::buildGlobalVariable(
// Set to Reg the same type as ResVReg has.
auto MRI = MIRBuilder.getMRI();
if (Reg != ResVReg) {
- LLT RegLLTy =
- LLT::pointer(MRI->getType(ResVReg).getAddressSpace(), getPointerSize());
+ unsigned AS = MRI->getType(ResVReg).getAddressSpace();
+ LLT RegLLTy = LLT::pointer(AS, getPointerSize(AS));
MRI->setType(Reg, RegLLTy);
assignSPIRVTypeToVReg(BaseType, Reg, MIRBuilder.getMF());
} else {
@@ -2033,7 +2033,8 @@ SPIRVTypeInst SPIRVGlobalRegistry::getOrCreateSPIRVPointerTypeInternal(
SPIRVTypeInst BaseType, MachineIRBuilder &MIRBuilder,
SPIRV::StorageClass::StorageClass SC) {
const Type *PointerElementType = getTypeForSPIRVType(BaseType);
- unsigned AddressSpace = storageClassToAddressSpace(SC);
+ unsigned AddressSpace =
+ storageClassToAddressSpace(SC, CurMF->getTarget().getTargetTriple());
if (const MachineInstr *MI = findMI(PointerElementType, AddressSpace, CurMF))
return MI;
Type *Ty = TypedPointerType::get(const_cast<Type *>(PointerElementType),
@@ -2105,28 +2106,33 @@ SPIRVGlobalRegistry::getRegClass(SPIRVTypeInst SpvType) const {
return &SPIRV::iIDRegClass;
}
-inline unsigned getAS(SPIRVTypeInst SpvType) {
+inline unsigned getAS(SPIRVTypeInst SpvType, const Triple &TT) {
return storageClassToAddressSpace(
static_cast<SPIRV::StorageClass::StorageClass>(
- SpvType->getOperand(1).getImm()));
+ SpvType->getOperand(1).getImm()), TT);
}
LLT SPIRVGlobalRegistry::getRegType(SPIRVTypeInst SpvType) const {
unsigned Opcode = SpvType ? SpvType->getOpcode() : 0;
+ const Triple &TT = CurMF->getTarget().getTargetTriple();
switch (Opcode) {
case SPIRV::OpTypeInt:
case SPIRV::OpTypeFloat:
case SPIRV::OpTypeBool:
return LLT::scalar(getScalarOrVectorBitWidth(SpvType));
- case SPIRV::OpTypePointer:
- return LLT::pointer(getAS(SpvType), getPointerSize());
+ case SPIRV::OpTypePointer: {
+ unsigned AS = getAS(SpvType, TT);
+ return LLT::pointer(AS, getPointerSize(AS));
+ }
case SPIRV::OpTypeVector: {
SPIRVTypeInst ElemType = getScalarOrVectorComponentType(SpvType);
LLT ET;
switch (ElemType ? ElemType->getOpcode() : 0) {
- case SPIRV::OpTypePointer:
- ET = LLT::pointer(getAS(ElemType), getPointerSize());
+ case SPIRV::OpTypePointer: {
+ unsigned AS = getAS(ElemType, TT);
+ ET = LLT::pointer(AS, getPointerSize(AS));
break;
+ }
case SPIRV::OpTypeInt:
case SPIRV::OpTypeFloat:
case SPIRV::OpTypeBool:
diff --git a/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.h b/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.h
index f05bdc2cc9861..427aab376d69d 100644
--- a/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.h
+++ b/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.h
@@ -418,8 +418,8 @@ class SPIRVGlobalRegistry : public SPIRVIRMapping {
getPointerStorageClass(SPIRVTypeInst Type) const;
// Return the number of bits SPIR-V pointers and size_t variables require.
- unsigned getPointerSize() const {
- return DL.getPointerSizeInBits(/* AS = */ 0);
+ unsigned getPointerSize(unsigned AS = 0) const {
+ return DL.getPointerSizeInBits(AS);
}
// Returns true if two types are defined and are compatible in a sense of
diff --git a/llvm/lib/Target/SPIRV/SPIRVISelLowering.cpp b/llvm/lib/Target/SPIRV/SPIRVISelLowering.cpp
index eb16c9a314a23..c5499e9d6c1ea 100644
--- a/llvm/lib/Target/SPIRV/SPIRVISelLowering.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVISelLowering.cpp
@@ -21,6 +21,7 @@
#include "llvm/CodeGen/TargetLowering.h"
#include "llvm/IR/Instructions.h"
#include "llvm/IR/IntrinsicsSPIRV.h"
+#include "llvm/Support/AMDGPUAddrSpace.h"
#define DEBUG_TYPE "spirv-lower"
@@ -102,6 +103,12 @@ MVT SPIRVTargetLowering::getRegisterTypeForCallingConv(LLVMContext &Context,
else if (VT.getVectorElementType() == MVT::i8)
return MVT::v4i8;
}
+ // if (STI.getTargetTriple().getVendor() == Triple::VendorType::AMD) {
+ // if (VT.isVector() &&
+ // VT.getVectorElementType().isExtended())
+ // return MVT::v8i32;
+ // }
+
return getRegisterType(Context, VT);
}
@@ -684,3 +691,33 @@ SPIRVTargetLowering::shouldCastAtomicRMWIInIR(AtomicRMWInst *RMWI) const {
// SPIR-V only supports atomic exchange for integer and floating-point types.
return AtomicExpansionKind::None;
}
+
+// These are specific to AMDGCN flavoured SPIR-V, and mirror the AMDGPU
+// implementation; they are required because buffer operations and the machinery
+// around them are somewhat special.
+MVT SPIRVTargetLowering::getPointerTy(const DataLayout &DL, unsigned AS) const {
+ if (STI.getTargetTriple().getVendor() != Triple::VendorType::AMD)
+ return TargetLowering::getPointerTy(DL, AS);
+
+ if (AMDGPUAS::BUFFER_FAT_POINTER == AS && DL.getPointerSizeInBits(AS) == 160)
+ return MVT::amdgpuBufferFatPointer;
+ if (AMDGPUAS::BUFFER_STRIDED_POINTER == AS &&
+ DL.getPointerSizeInBits(AS) == 192)
+ return MVT::amdgpuBufferStridedPointer;
+
+ return TargetLowering::getPointerTy(DL, AS);
+}
+
+MVT SPIRVTargetLowering::getPointerMemTy(const DataLayout &DL,
+ unsigned AS) const {
+ if (STI.getTargetTriple().getVendor() != Triple::VendorType::AMD)
+ return TargetLowering::getPointerMemTy(DL, AS);
+
+ if ((AMDGPUAS::BUFFER_FAT_POINTER == AS &&
+ DL.getPointerSizeInBits(AS) == 160) ||
+ (AMDGPUAS::BUFFER_STRIDED_POINTER == AS &&
+ DL.getPointerSizeInBits(AS) == 192))
+ return MVT::v8i32;
+
+ return TargetLowering::getPointerMemTy(DL, AS);
+}
diff --git a/llvm/lib/Target/SPIRV/SPIRVISelLowering.h b/llvm/lib/Target/SPIRV/SPIRVISelLowering.h
index 6af0bbcc9818d..60710610c6c28 100644
--- a/llvm/lib/Target/SPIRV/SPIRVISelLowering.h
+++ b/llvm/lib/Target/SPIRV/SPIRVISelLowering.h
@@ -82,6 +82,9 @@ class SPIRVTargetLowering : public TargetLowering {
bool shouldIssueAtomicLoadForAtomicEmulationLoop() const override {
return false;
}
+
+ MVT getPointerTy(const DataLayout &DL, unsigned AS) const override;
+ MVT getPointerMemTy(const DataLayout &DL, unsigned AS) const override;
};
} // namespace llvm
diff --git a/llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp b/llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp
index 02b41b9cbd64c..df03b0cf64bf8 100644
--- a/llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp
@@ -2565,9 +2565,9 @@ SPIRVInstructionSelector::buildConstGenericPtr(MachineInstr &I, Register SrcPtr,
SPIRVTypeInst GenericPtrTy =
GR.changePointerStorageClass(SrcPtrTy, SPIRV::StorageClass::Generic, I);
Register Tmp = MRI->createVirtualRegister(&SPIRV::pIDRegClass);
- MRI->setType(Tmp, LLT::pointer(storageClassToAddressSpace(
- SPIRV::StorageClass::Generic),
- GR.getPointerSize()));
+ unsigned AS = storageClassToAddressSpace(SPIRV::StorageClass::Generic,
+ STI.getTargetTriple());
+ MRI->setType(Tmp, LLT::pointer(AS, GR.getPointerSize(AS)));
MachineFunction *MF = I.getParent()->getParent();
GR.assignSPIRVTypeToVReg(GenericPtrTy, Tmp, *MF);
MachineInstrBuilder MIB = buildSpecConstantOp(
@@ -2626,6 +2626,25 @@ bool SPIRVInstructionSelector::selectAddrSpaceCast(Register ResVReg,
I, ResVReg, MIB->getOperand(0).getReg(), getUcharPtrTypeReg(I, DstSC),
static_cast<uint32_t>(SPIRV::Opcode::GenericCastToPtr))
.constrainAllUses(TII, TRI, RBI);
+ } else if (STI.getTargetTriple().getVendor() == Triple::VendorType::AMD &&
+ isUSMStorageClass(SrcSC) && isUSMStorageClass(DstSC)) {
+ // For AMDGCN flavoured SPIR-V we repurpose DeviceOnlyINTEL and
+ // HostOnlyINTEL (which we do not use) to represent Buffer specific
+ // storage. Pointers to Buffer are essentially opaque handles, and have
+ // non trivial lowering, so we only care about retaining information about
+ // in their nature in SPIR-V, as they can only be handled properly during
+ // finalisation. Since they are inter-castable, and there is no valid
+ // SPIR-V cast between DeviceOnlyINTEL and HostOnlyIntel, we use a PtrToU
+ // followed by an UToPtr pattern to encode the cast, pattern that we match
+ // and retrieve at finalisation.
+ auto I64Ty = GR.getOrCreateSPIRVIntegerType(64, I, TII);
+ Register Tmp = createVirtualRegister(I64Ty, &GR, MRI, MRI->getMF());
+ auto MIB = buildSpecConstantOp(I, Tmp, SrcPtr, GR.getSPIRVTypeID(I64Ty),
+ SPIRV::Opcode::ConvertPtrToU);
+ MIB.constrainAllUses(TII, TRI, RBI);
+ buildSpecConstantOp(I, ResVReg, Tmp, getUcharPtrTypeReg(I, DstSC),
+ SPIRV::Opcode::ConvertUToPtr)
+ .constrainAllUses(TII, TRI, RBI);
}
return true;
}
@@ -6809,7 +6828,8 @@ bool SPIRVInstructionSelector::selectModf(Register ResVReg,
MIRBuilder.getMRI()->createVirtualRegister(&SPIRV::iIDRegClass);
MIRBuilder.getMRI()->setType(
PtrTyReg,
- LLT::pointer(storageClassToAddressSpace(SPIRV::StorageClass::Function),
+ LLT::pointer(storageClassToAddressSpace(SPIRV::StorageClass::Function,
+ STI.getTargetTriple()),
GR.getPointerSize()));
// Assign SPIR-V type of the pointer type of the alloca variable to the
@@ -6924,10 +6944,11 @@ bool SPIRVInstructionSelector::loadBuiltinInputID(
// Create new register for the input ID builtin variable.
Register NewRegister =
MIRBuilder.getMRI()->createVirtualRegister(GR.getRegClass(PtrType));
+ unsigned AS = storageClassToAddressSpace(SPIRV::StorageClass::Input,
+ STI.getTargetTriple());
MIRBuilder.getMRI()->setType(
NewRegister,
- LLT::pointer(storageClassToAddressSpace(SPIRV::StorageClass::Input),
- GR.getPointerSize()));
+ LLT::pointer(AS, GR.getPointerSize(AS)));
GR.assignSPIRVTypeToVReg(PtrType, NewRegister, MIRBuilder.getMF());
// Build global variable with the necessary decorations for the input ID
diff --git a/llvm/lib/Target/SPIRV/SPIRVLegalizeZeroSizeArrays.cpp b/llvm/lib/Target/SPIRV/SPIRVLegalizeZeroSizeArrays.cpp
index 09697b59c374a..0991ed14e7df0 100644
--- a/llvm/lib/Target/SPIRV/SPIRVLegalizeZeroSizeArrays.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVLegalizeZeroSizeArrays.cpp
@@ -119,7 +119,8 @@ Type *SPIRVLegalizeZeroSizeArraysImpl::legalizeType(Type *Ty) {
if (isa<ArrayType>(Ty)) {
LegalizedTy = PointerType::get(
Ty->getContext(),
- storageClassToAddressSpace(SPIRV::StorageClass::Generic));
+ storageClassToAddressSpace(SPIRV::StorageClass::Generic,
+ TM.getTargetTriple()));
} else if (StructType *StructTy = dyn_cast<StructType>(Ty)) {
SmallVector<Type *, 8> ElemTypes;
diff --git a/llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp b/llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp
index dd48aa3ea2f0f..bfd937d47b6f1 100644
--- a/llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp
@@ -14,6 +14,7 @@
#include "SPIRV.h"
#include "SPIRVGlobalRegistry.h"
#include "SPIRVSubtarget.h"
+#include "SPIRVTargetMachine.h"
#include "SPIRVUtils.h"
#include "llvm/CodeGen/GlobalISel/GenericMachineInstrs.h"
#include "llvm/CodeGen/GlobalISel/LegalizerHelper.h"
@@ -38,7 +39,8 @@ LegalityPredicate typeOfExtendedScalars(unsigned TypeIdx, bool IsExtendedInts) {
};
}
-SPIRVLegalizerInfo::SPIRVLegalizerInfo(const SPIRVSubtarget &ST) {
+SPIRVLegalizerInfo::SPIRVLegalizerInfo(const SPIRVSubtarget &ST,
+ const TargetMachine &TM) {
using namespace TargetOpcode;
this->ST = &ST;
@@ -81,23 +83,24 @@ SPIRVLegalizerInfo::SPIRVLegalizerInfo(const SPIRVSubtarget &ST) {
const LLT v2s8 = LLT::fixed_vector(2, 8);
const LLT v2s1 = LLT::fixed_vector(2, 1);
- const unsigned PSize = ST.getPointerSize();
- const LLT p0 = LLT::pointer(0, PSize); // Function
- const LLT p1 = LLT::pointer(1, PSize); // CrossWorkgroup
- const LLT p2 = LLT::pointer(2, PSize); // UniformConstant
- const LLT p3 = LLT::pointer(3, PSize); // Workgroup
- const LLT p4 = LLT::pointer(4, PSize); // Generic
- const LLT p5 =
- LLT::pointer(5, PSize); // Input, SPV_INTEL_usm_storage_classes (Device)
- const LLT p6 = LLT::pointer(6, PSize); // SPV_INTEL_usm_storage_classes (Host)
- const LLT p7 = LLT::pointer(7, PSize); // Input
- const LLT p8 = LL...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/199376
More information about the cfe-commits
mailing list