[llvm] [SPIRV] Cooperative matrix under the Vulkan/Shader flavor (PR #202049)
Julian Klappenbach via llvm-commits
llvm-commits at lists.llvm.org
Thu Jun 11 18:03:47 PDT 2026
https://github.com/jklappenbach updated https://github.com/llvm/llvm-project/pull/202049
>From 748b3a96da95bb8412721eaec08771c1bbe1593b Mon Sep 17 00:00:00 2001
From: Julian Klappenbach <julian at twilight.digital>
Date: Sat, 6 Jun 2026 11:14:12 -0400
Subject: [PATCH 1/3] [SPIRV] Cooperative matrix under the Vulkan/Shader flavor
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Make SPV_KHR_cooperative_matrix reachable from the Vulkan/Shader flavor (the
OpenCL __spirv_* builtin path is isShader()-gated off), mirroring the texture /
ray-query intrinsic + GlobalISel pattern:
- IntrinsicsSPIRV.td: llvm.spv.cooperative.matrix.{load,store,muladd,splat};
the opaque matrix is carried as llvm_any_ty (overloaded per concrete shape).
- SPIRVInstructionSelector.cpp: GlobalISel selection for the four ops
(load/muladd/splat via selectOpWithSrcs, store via selectCoopMatrixStore).
memory-layout/stride are <id> constant operands; float MulAdd omits the
integer-only signedness literal; splat = single-scalar OpCompositeConstruct.
- SPIRVModuleAnalysis.{cpp,h}: cooperative matrix under Shader mandates the
Vulkan memory model (spirv-val rejects Shader + CooperativeMatrixKHR under
GLSL450). Rather than guess up front, derive the model from the actual
requirement: after collectReqs, if a Shader module requires CooperativeMatrixKHR
(RequirementHandler::isCapabilityRequired, added here), upgrade GLSL450 ->
VulkanKHR (+ VulkanMemoryModelKHR capability + SPV_KHR_vulkan_memory_model
extension, OpMemoryModel Logical VulkanKHR), unless !spirv.MemoryModel set one
explicitly. Non-coop shaders keep GLSL450.
The OpTypeCooperativeMatrixKHR opaque type needs no backend change — the
existing BuiltinType machinery lowers target("spirv.CooperativeMatrixKHR", elem,
scope, rows, cols, use) flavor-agnostically.
---
llvm/include/llvm/IR/IntrinsicsSPIRV.td | 13 +++-
.../Target/SPIRV/SPIRVInstructionSelector.cpp | 30 ++++++++
llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp | 16 +++-
llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.h | 4 +
.../cooperative_matrix_kernel.ll | 74 +++++++++++++++++++
...ooperative_matrix_memory_model_negative.ll | 19 +++++
.../cooperative_matrix_ops_vulkan.ll | 51 +++++++++++++
.../cooperative_matrix_type_vulkan.ll | 22 ++++++
8 files changed, 227 insertions(+), 2 deletions(-)
create mode 100644 llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_kernel.ll
create mode 100644 llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_memory_model_negative.ll
create mode 100644 llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_ops_vulkan.ll
create mode 100644 llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_type_vulkan.ll
diff --git a/llvm/include/llvm/IR/IntrinsicsSPIRV.td b/llvm/include/llvm/IR/IntrinsicsSPIRV.td
index 1e758f9f63f49..2c465ceb3bf0d 100644
--- a/llvm/include/llvm/IR/IntrinsicsSPIRV.td
+++ b/llvm/include/llvm/IR/IntrinsicsSPIRV.td
@@ -360,5 +360,16 @@ def int_spv_rsqrt : DefaultAttrsIntrinsic<[LLVMMatchType<0>], [llvm_anyfloat_ty]
def int_spv_unpackhalf2x16 : DefaultAttrsIntrinsic<[llvm_anyfloat_ty], [llvm_i32_ty], [IntrNoMem]>;
def int_spv_packhalf2x16 : DefaultAttrsIntrinsic<[llvm_anyint_ty], [llvm_anyfloat_ty], [IntrNoMem]>;
-
+ // SPV_KHR_cooperative_matrix operations under the Vulkan/Shader flavor,
+ // reached through llvm.spv intrinsics and GlobalISel selection because the
+ // __spirv_* builtin path is OpenCL-only. The matrix value is the opaque
+ // target("spirv.CooperativeMatrixKHR", ...) type, carried as llvm_any_ty.
+ def int_spv_cooperative_matrix_load
+ : Intrinsic<[llvm_any_ty], [llvm_anyptr_ty, llvm_i32_ty, llvm_i32_ty]>;
+ def int_spv_cooperative_matrix_store
+ : Intrinsic<[], [llvm_anyptr_ty, llvm_any_ty, llvm_i32_ty, llvm_i32_ty]>;
+ def int_spv_cooperative_matrix_muladd
+ : Intrinsic<[llvm_any_ty], [llvm_any_ty, llvm_any_ty, llvm_any_ty]>;
+ def int_spv_cooperative_matrix_splat
+ : Intrinsic<[llvm_any_ty], [llvm_any_ty]>;
}
diff --git a/llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp b/llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp
index c3f21fe025bd5..8f1e098389a84 100644
--- a/llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp
@@ -444,6 +444,7 @@ class SPIRVInstructionSelector : public InstructionSelector {
bool selectGatherIntrinsic(Register &ResVReg, SPIRVTypeInst ResType,
MachineInstr &I) const;
bool selectImageWriteIntrinsic(MachineInstr &I) const;
+ bool selectCoopMatrixStore(MachineInstr &I) const;
bool selectResourceGetPointer(Register &ResVReg, SPIRVTypeInst ResType,
MachineInstr &I) const;
bool selectPushConstantGetPointer(Register &ResVReg, SPIRVTypeInst ResType,
@@ -1592,6 +1593,17 @@ bool SPIRVInstructionSelector::selectSincos(Register ResVReg,
return false;
}
+bool SPIRVInstructionSelector::selectCoopMatrixStore(MachineInstr &I) const {
+ // Operands 1..: pointer, matrix, memory_layout, stride.
+ // OpCooperativeMatrixStoreKHR has no result.
+ auto MIB = BuildMI(*I.getParent(), I, I.getDebugLoc(),
+ TII.get(SPIRV::OpCooperativeMatrixStoreKHR));
+ for (unsigned i = 1; i < I.getNumOperands(); ++i)
+ MIB.addUse(I.getOperand(i).getReg());
+ MIB.constrainAllUses(TII, TRI, RBI);
+ return true;
+}
+
bool SPIRVInstructionSelector::selectOpWithSrcs(Register ResVReg,
SPIRVTypeInst ResType,
MachineInstr &I,
@@ -4708,6 +4720,24 @@ bool SPIRVInstructionSelector::selectIntrinsic(Register ResVReg,
report_fatal_error("incompatible result and operand types in a bitcast");
return selectOpWithSrcs(ResVReg, ResType, I, {OpReg}, SPIRV::OpBitcast);
}
+ case Intrinsic::spv_cooperative_matrix_load:
+ // result = OpCooperativeMatrixLoadKHR ptr memory_layout stride
+ return selectOpWithSrcs(ResVReg, ResType, I,
+ {I.getOperand(2).getReg(), I.getOperand(3).getReg(),
+ I.getOperand(4).getReg()},
+ SPIRV::OpCooperativeMatrixLoadKHR);
+ case Intrinsic::spv_cooperative_matrix_store:
+ return selectCoopMatrixStore(I);
+ case Intrinsic::spv_cooperative_matrix_muladd:
+ // OpCooperativeMatrixMulAddKHR A B C; no operands literal for float matmul.
+ return selectOpWithSrcs(ResVReg, ResType, I,
+ {I.getOperand(2).getReg(), I.getOperand(3).getReg(),
+ I.getOperand(4).getReg()},
+ SPIRV::OpCooperativeMatrixMulAddKHR);
+ case Intrinsic::spv_cooperative_matrix_splat:
+ // OpCompositeConstruct from a scalar broadcasts the accumulator.
+ return selectOpWithSrcs(ResVReg, ResType, I, {I.getOperand(2).getReg()},
+ SPIRV::OpCompositeConstruct);
case Intrinsic::spv_unref_global:
case Intrinsic::spv_init_global: {
MachineInstr *MI = MRI->getVRegDef(I.getOperand(1).getReg());
diff --git a/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp b/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp
index 49d9dc95603ca..59de5a14623ee 100644
--- a/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp
@@ -155,7 +155,9 @@ void SPIRVModuleAnalysis::setBaseInfo(const Module &M) {
MAI.Mem =
static_cast<SPIRV::MemoryModel::MemoryModel>(getMetadataUInt(MemMD, 1));
} else {
- // TODO: Add support for VulkanMemoryModel.
+ // Default; a Shader module may be upgraded to VulkanKHR later if a
+ // capability mandates it (see runOnModule).
+ // TODO: Add general support for VulkanMemoryModel.
MAI.Mem = ST->isShader() ? SPIRV::MemoryModel::GLSL450
: SPIRV::MemoryModel::OpenCL;
if (MAI.Mem == SPIRV::MemoryModel::OpenCL) {
@@ -3023,6 +3025,18 @@ bool SPIRVModuleAnalysis::runOnModule(Module &M) {
collectReqs(M, MAI, MMI, *ST);
collectDeclarations(M);
+ // CooperativeMatrixKHR in a Shader module mandates the Vulkan memory model,
+ // so derive the model from the capability unless one was set explicitly via
+ // spirv.MemoryModel metadata.
+ if (ST->isShader() && MAI.Mem == SPIRV::MemoryModel::GLSL450 &&
+ !M.getNamedMetadata("spirv.MemoryModel") &&
+ MAI.Reqs.isCapabilityRequired(SPIRV::Capability::CooperativeMatrixKHR)) {
+ MAI.Mem = SPIRV::MemoryModel::VulkanKHR;
+ MAI.Reqs.getAndAddRequirements(SPIRV::OperandCategory::MemoryModelOperand,
+ MAI.Mem, *ST);
+ MAI.Reqs.addExtension(SPIRV::Extension::SPV_KHR_vulkan_memory_model);
+ }
+
// Number rest of registers from N+1 onwards.
numberRegistersGlobally(M);
diff --git a/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.h b/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.h
index 6559b5cfc7457..1fb16ad1af03c 100644
--- a/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.h
+++ b/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.h
@@ -119,6 +119,10 @@ struct RequirementHandler {
bool isCapabilityAvailable(Capability::Capability Cap) const {
return AvailableCaps.contains(Cap);
}
+ // True if Cap has been added to the module's required capabilities.
+ bool isCapabilityRequired(Capability::Capability Cap) const {
+ return AllCaps.contains(Cap);
+ }
// Remove capability ToRemove, but only if IfPresent is present.
void removeCapabilityIf(const Capability::Capability ToRemove,
diff --git a/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_kernel.ll b/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_kernel.ll
new file mode 100644
index 0000000000000..5eee4d187d9f3
--- /dev/null
+++ b/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_kernel.ll
@@ -0,0 +1,74 @@
+; A GLCompute kernel that runs a cooperative-matrix matmul tile against
+; descriptor-bound storage buffers under the Vulkan flavor. A and B are read-only
+; StorageBuffers and C a writable StorageBuffer, each a VulkanBuffer handle
+; decorated DescriptorSet and Binding and accessed through
+; llvm.spv.resource.handlefrombinding and getpointer. The kernel loads A (use 0)
+; and B (use 1) as cooperative matrices, mul-adds into a zero accumulator C
+; (use 2), and stores C. The module must pass spirv-val --target-env vulkan1.3.
+
+; RUN: llc -O0 -verify-machineinstrs -mtriple=spirv-unknown-vulkan1.3-compute --spirv-ext=+SPV_KHR_cooperative_matrix,+SPV_KHR_vulkan_memory_model %s -o - | FileCheck %s
+; RUN: %if spirv-tools %{ llc -O0 -mtriple=spirv-unknown-vulkan1.3-compute --spirv-ext=+SPV_KHR_cooperative_matrix,+SPV_KHR_vulkan_memory_model %s -o - -filetype=obj | spirv-val --target-env vulkan1.3 %}
+
+; CHECK-DAG: OpCapability CooperativeMatrixKHR
+; CHECK-DAG: OpExtension "SPV_KHR_cooperative_matrix"
+; CHECK-DAG: OpEntryPoint GLCompute %[[#entry:]] "main"
+; CHECK-DAG: OpDecorate %[[#A:]] DescriptorSet 0
+; CHECK-DAG: OpDecorate %[[#A]] Binding 0
+; CHECK-DAG: OpDecorate %[[#B:]] DescriptorSet 0
+; CHECK-DAG: OpDecorate %[[#B]] Binding 1
+; CHECK-DAG: OpDecorate %[[#C:]] DescriptorSet 0
+; CHECK-DAG: OpDecorate %[[#C]] Binding 2
+; CHECK: %[[#MA:]] = OpCooperativeMatrixLoadKHR
+; CHECK: %[[#MB:]] = OpCooperativeMatrixLoadKHR
+; CHECK: %[[#MC0:]] = OpCompositeConstruct
+; CHECK: %[[#MC:]] = OpCooperativeMatrixMulAddKHR %[[#]] %[[#MA]] %[[#MB]] %[[#MC0]]
+; CHECK: OpCooperativeMatrixStoreKHR %[[#]] %[[#MC]]
+
+ at .str.a = private unnamed_addr constant [2 x i8] c"a\00", align 1
+ at .str.b = private unnamed_addr constant [2 x i8] c"b\00", align 1
+ at .str.c = private unnamed_addr constant [2 x i8] c"c\00", align 1
+
+define void @main() local_unnamed_addr #0 {
+entry:
+ ; A: read-only StorageBuffer, binding 0 -> MatrixA (use 0)
+ %ha = tail call target("spirv.VulkanBuffer", [0 x float], 12, 0)
+ @llvm.spv.resource.handlefrombinding.tspirv.VulkanBuffer_a0f32_12_0t(
+ i32 0, i32 0, i32 1, i32 0, ptr nonnull @.str.a)
+ %pa = tail call ptr addrspace(11)
+ @llvm.spv.resource.getpointer.p11.tspirv.VulkanBuffer_a0f32_12_0t(
+ target("spirv.VulkanBuffer", [0 x float], 12, 0) %ha, i32 0)
+ %a = call target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 0)
+ @llvm.spv.cooperative.matrix.load(ptr addrspace(11) %pa, i32 0, i32 16)
+
+ ; B: read-only StorageBuffer, binding 1 -> MatrixB (use 1)
+ %hb = tail call target("spirv.VulkanBuffer", [0 x float], 12, 0)
+ @llvm.spv.resource.handlefrombinding.tspirv.VulkanBuffer_a0f32_12_0t(
+ i32 0, i32 1, i32 1, i32 0, ptr nonnull @.str.b)
+ %pb = tail call ptr addrspace(11)
+ @llvm.spv.resource.getpointer.p11.tspirv.VulkanBuffer_a0f32_12_0t(
+ target("spirv.VulkanBuffer", [0 x float], 12, 0) %hb, i32 0)
+ %b = call target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 1)
+ @llvm.spv.cooperative.matrix.load(ptr addrspace(11) %pb, i32 0, i32 16)
+
+ ; C = A*B + 0 (accumulator, use 2), stored to writable StorageBuffer binding 2
+ %c0 = call target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 2)
+ @llvm.spv.cooperative.matrix.splat(float 0.000000e+00)
+ %c = call target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 2)
+ @llvm.spv.cooperative.matrix.muladd(
+ target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 0) %a,
+ target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 1) %b,
+ target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 2) %c0)
+ %hc = tail call target("spirv.VulkanBuffer", [0 x float], 12, 1)
+ @llvm.spv.resource.handlefrombinding.tspirv.VulkanBuffer_a0f32_12_1t(
+ i32 0, i32 2, i32 1, i32 0, ptr nonnull @.str.c)
+ %pc = tail call ptr addrspace(11)
+ @llvm.spv.resource.getpointer.p11.tspirv.VulkanBuffer_a0f32_12_1t(
+ target("spirv.VulkanBuffer", [0 x float], 12, 1) %hc, i32 0)
+ call void @llvm.spv.cooperative.matrix.store(
+ ptr addrspace(11) %pc,
+ target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 2) %c,
+ i32 0, i32 16)
+ ret void
+}
+
+attributes #0 = { "hlsl.numthreads"="1,1,1" "hlsl.shader"="compute" }
diff --git a/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_memory_model_negative.ll b/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_memory_model_negative.ll
new file mode 100644
index 0000000000000..101a0afef0b64
--- /dev/null
+++ b/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_memory_model_negative.ll
@@ -0,0 +1,19 @@
+; Negative control for the cooperative-matrix memory-model upgrade: a plain Shader
+; compute module that does not require CooperativeMatrixKHR must keep the default
+; GLSL450 memory model, so the upgrade is conditional on the CooperativeMatrixKHR
+; requirement, not a blanket change applied to every Shader-flavor module.
+
+; RUN: llc -verify-machineinstrs -O0 -mtriple=spirv-unknown-vulkan1.3-compute %s -o - | FileCheck %s
+
+; No cooperative-matrix capability is required, so no Vulkan memory model is pulled
+; in: neither the capability nor the extension is emitted, and the module keeps the
+; default Logical GLSL450 memory model (not Logical VulkanKHR).
+; CHECK-NOT: OpCapability VulkanMemoryModelKHR
+; CHECK-NOT: OpExtension "SPV_KHR_vulkan_memory_model"
+; CHECK: OpMemoryModel Logical GLSL450
+; CHECK-NOT: OpMemoryModel Logical VulkanKHR
+
+define spir_func void @nop() {
+entry:
+ ret void
+}
diff --git a/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_ops_vulkan.ll b/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_ops_vulkan.ll
new file mode 100644
index 0000000000000..457b546fbc0b1
--- /dev/null
+++ b/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_ops_vulkan.ll
@@ -0,0 +1,51 @@
+; The cooperative-matrix operations (load, store, mul-add, splat-construct) under
+; the Vulkan/Shader flavor, reached via llvm.spv.cooperative.matrix.* intrinsics
+; and GlobalISel selection because the OpenCL __spirv_CooperativeMatrix* builtin
+; path is OpenCL-only.
+;
+; A single matmul tile fragment: C = A * B + 0. A is MatrixA (use 0), B is MatrixB
+; (use 1), C the accumulator (use 2); all 16x16 f32, Subgroup scope (3). Memory
+; layout 0 = RowMajorKHR, stride 16.
+
+; RUN: llc -verify-machineinstrs -O0 -mtriple=spirv-unknown-vulkan1.3-compute --spirv-ext=+SPV_KHR_cooperative_matrix,+SPV_KHR_vulkan_memory_model %s -o - | FileCheck %s
+
+; CHECK-DAG: OpCapability CooperativeMatrixKHR
+; CHECK-DAG: OpExtension "SPV_KHR_cooperative_matrix"
+; Requiring CooperativeMatrixKHR under the Shader flavor upgrades the default
+; GLSL450 memory model to VulkanKHR (spirv-val rejects Shader + CooperativeMatrixKHR
+; under GLSL450). This is the SPIRVModuleAnalysis derivation: once collectReqs sees
+; the CooperativeMatrixKHR capability, it pulls in the Vulkan memory model.
+; CHECK-DAG: OpCapability VulkanMemoryModelKHR
+; CHECK-DAG: OpExtension "SPV_KHR_vulkan_memory_model"
+; CHECK-DAG: OpMemoryModel Logical VulkanKHR
+; CHECK-DAG: %[[#F32:]] = OpTypeFloat 32
+; The three matrix types (A use 0, B use 1, C accumulator use 2) all lower to
+; OpTypeCooperativeMatrixKHR over the f32 component type.
+; CHECK-DAG: OpTypeCooperativeMatrixKHR %[[#F32]]
+; Data flow: mul-add consumes both loaded operands plus the splat-constructed
+; accumulator; the store consumes the mul-add result.
+; CHECK: %[[#A:]] = OpCooperativeMatrixLoadKHR
+; CHECK: %[[#B:]] = OpCooperativeMatrixLoadKHR
+; CHECK: %[[#C0:]] = OpCompositeConstruct
+; CHECK: %[[#C:]] = OpCooperativeMatrixMulAddKHR %[[#]] %[[#A]] %[[#B]] %[[#C0]]
+; CHECK: OpCooperativeMatrixStoreKHR %[[#]] %[[#C]]
+
+define spir_func void @matmul_tile(ptr %pa, ptr %pb, ptr %pc) {
+entry:
+ %a = call target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 0)
+ @llvm.spv.cooperative.matrix.load(ptr %pa, i32 0, i32 16)
+ %b = call target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 1)
+ @llvm.spv.cooperative.matrix.load(ptr %pb, i32 0, i32 16)
+ %c0 = call target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 2)
+ @llvm.spv.cooperative.matrix.splat(float 0.000000e+00)
+ %c = call target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 2)
+ @llvm.spv.cooperative.matrix.muladd(
+ target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 0) %a,
+ target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 1) %b,
+ target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 2) %c0)
+ call void @llvm.spv.cooperative.matrix.store(
+ ptr %pc,
+ target("spirv.CooperativeMatrixKHR", float, 3, 16, 16, 2) %c,
+ i32 0, i32 16)
+ ret void
+}
diff --git a/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_type_vulkan.ll b/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_type_vulkan.ll
new file mode 100644
index 0000000000000..24a8b3809ad1b
--- /dev/null
+++ b/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_type_vulkan.ll
@@ -0,0 +1,22 @@
+; Check that the opaque CooperativeMatrix type under the Vulkan/Shader flavor,
+; target("spirv.CooperativeMatrixKHR", elem, scope, rows, cols, use), lowers to
+; OpTypeCooperativeMatrixKHR, gated behind the CooperativeMatrixKHR capability and
+; the SPV_KHR_cooperative_matrix extension, and that it errors cleanly without it.
+; This is a text-emission check only; the type is forced via a by-value parameter,
+; so it is not run through spirv-val here.
+
+; RUN: not llc -O0 -mtriple=spirv-unknown-vulkan1.3-compute %s -o /dev/null 2>&1 | FileCheck %s --check-prefix=CHECK-ERROR
+; RUN: llc -verify-machineinstrs -O0 -mtriple=spirv-unknown-vulkan1.3-compute --spirv-ext=+SPV_KHR_cooperative_matrix,+SPV_KHR_vulkan_memory_model %s -o - | FileCheck %s
+
+; CHECK-ERROR: LLVM ERROR: OpTypeCooperativeMatrixKHR type requires the following SPIR-V extension: SPV_KHR_cooperative_matrix
+
+; CHECK-DAG: OpCapability CooperativeMatrixKHR
+; CHECK-DAG: OpExtension "SPV_KHR_cooperative_matrix"
+; CHECK-DAG: {{%[0-9]+}} = OpTypeCooperativeMatrixKHR
+
+; A by-value parameter of the opaque type forces OpTypeCooperativeMatrixKHR into the
+; module (referenced by OpTypeFunction — cannot be eliminated like an unused local).
+define spir_func void @use_coopmat(target("spirv.CooperativeMatrixKHR", i32, 3, 12, 12, 2) %m) {
+entry:
+ ret void
+}
>From a2722b6b395ebb9979b2f3fe5ac97950372d6bfb Mon Sep 17 00:00:00 2001
From: Julian Klappenbach <julian at twilight.digital>
Date: Thu, 11 Jun 2026 20:04:12 -0400
Subject: [PATCH 2/3] [SPIRV] Use plain ASCII in
cooperative_matrix_type_vulkan.ll comment
Replace a non-ASCII em-dash in a test comment with plain ASCII.
---
.../cooperative_matrix_type_vulkan.ll | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_type_vulkan.ll b/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_type_vulkan.ll
index 24a8b3809ad1b..e6d0aac36812f 100644
--- a/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_type_vulkan.ll
+++ b/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_type_vulkan.ll
@@ -15,7 +15,7 @@
; CHECK-DAG: {{%[0-9]+}} = OpTypeCooperativeMatrixKHR
; A by-value parameter of the opaque type forces OpTypeCooperativeMatrixKHR into the
-; module (referenced by OpTypeFunction — cannot be eliminated like an unused local).
+; module (referenced by OpTypeFunction, so it is not eliminated like an unused local).
define spir_func void @use_coopmat(target("spirv.CooperativeMatrixKHR", i32, 3, 12, 12, 2) %m) {
entry:
ret void
>From 331ccb4148f3293957fe5914da461f06187433a0 Mon Sep 17 00:00:00 2001
From: Julian Klappenbach <julian at twilight.digital>
Date: Thu, 11 Jun 2026 21:03:36 -0400
Subject: [PATCH 3/3] [SPIRV] Move cooperative-matrix Vulkan memory-model
derivation out of this PR
Drop the SPIRVModuleAnalysis change that auto-upgrades a Shader module
requiring CooperativeMatrixKHR to the Vulkan memory model; it is an
independent fix submitted separately. The cooperative-matrix tests now
request the Vulkan memory model explicitly via spirv.MemoryModel metadata,
and the memory-model negative test moves with the derivation.
---
llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp | 16 +---------------
llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.h | 4 ----
.../cooperative_matrix_kernel.ll | 4 ++++
...ooperative_matrix_memory_model_negative.ll | 19 -------------------
.../cooperative_matrix_ops_vulkan.ll | 10 ++++++----
5 files changed, 11 insertions(+), 42 deletions(-)
delete mode 100644 llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_memory_model_negative.ll
diff --git a/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp b/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp
index 3c399b1a12aff..bb6245778b2ea 100644
--- a/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp
@@ -155,9 +155,7 @@ void SPIRVModuleAnalysis::setBaseInfo(const Module &M) {
MAI.Mem =
static_cast<SPIRV::MemoryModel::MemoryModel>(getMetadataUInt(MemMD, 1));
} else {
- // Default; a Shader module may be upgraded to VulkanKHR later if a
- // capability mandates it (see runOnModule).
- // TODO: Add general support for VulkanMemoryModel.
+ // TODO: Add support for VulkanMemoryModel.
MAI.Mem = ST->isShader() ? SPIRV::MemoryModel::GLSL450
: SPIRV::MemoryModel::OpenCL;
if (MAI.Mem == SPIRV::MemoryModel::OpenCL) {
@@ -3029,18 +3027,6 @@ bool SPIRVModuleAnalysis::runOnModule(Module &M) {
collectReqs(M, MAI, MMI, *ST);
collectDeclarations(M);
- // CooperativeMatrixKHR in a Shader module mandates the Vulkan memory model,
- // so derive the model from the capability unless one was set explicitly via
- // spirv.MemoryModel metadata.
- if (ST->isShader() && MAI.Mem == SPIRV::MemoryModel::GLSL450 &&
- !M.getNamedMetadata("spirv.MemoryModel") &&
- MAI.Reqs.isCapabilityRequired(SPIRV::Capability::CooperativeMatrixKHR)) {
- MAI.Mem = SPIRV::MemoryModel::VulkanKHR;
- MAI.Reqs.getAndAddRequirements(SPIRV::OperandCategory::MemoryModelOperand,
- MAI.Mem, *ST);
- MAI.Reqs.addExtension(SPIRV::Extension::SPV_KHR_vulkan_memory_model);
- }
-
// Number rest of registers from N+1 onwards.
numberRegistersGlobally(M);
diff --git a/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.h b/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.h
index 1fb16ad1af03c..6559b5cfc7457 100644
--- a/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.h
+++ b/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.h
@@ -119,10 +119,6 @@ struct RequirementHandler {
bool isCapabilityAvailable(Capability::Capability Cap) const {
return AvailableCaps.contains(Cap);
}
- // True if Cap has been added to the module's required capabilities.
- bool isCapabilityRequired(Capability::Capability Cap) const {
- return AllCaps.contains(Cap);
- }
// Remove capability ToRemove, but only if IfPresent is present.
void removeCapabilityIf(const Capability::Capability ToRemove,
diff --git a/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_kernel.ll b/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_kernel.ll
index 5eee4d187d9f3..d968647eecc8c 100644
--- a/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_kernel.ll
+++ b/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_kernel.ll
@@ -72,3 +72,7 @@ entry:
}
attributes #0 = { "hlsl.numthreads"="1,1,1" "hlsl.shader"="compute" }
+
+; AddressingModel=Logical (0), MemoryModel=VulkanKHR (3)
+!spirv.MemoryModel = !{!0}
+!0 = !{i32 0, i32 3}
diff --git a/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_memory_model_negative.ll b/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_memory_model_negative.ll
deleted file mode 100644
index 101a0afef0b64..0000000000000
--- a/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_memory_model_negative.ll
+++ /dev/null
@@ -1,19 +0,0 @@
-; Negative control for the cooperative-matrix memory-model upgrade: a plain Shader
-; compute module that does not require CooperativeMatrixKHR must keep the default
-; GLSL450 memory model, so the upgrade is conditional on the CooperativeMatrixKHR
-; requirement, not a blanket change applied to every Shader-flavor module.
-
-; RUN: llc -verify-machineinstrs -O0 -mtriple=spirv-unknown-vulkan1.3-compute %s -o - | FileCheck %s
-
-; No cooperative-matrix capability is required, so no Vulkan memory model is pulled
-; in: neither the capability nor the extension is emitted, and the module keeps the
-; default Logical GLSL450 memory model (not Logical VulkanKHR).
-; CHECK-NOT: OpCapability VulkanMemoryModelKHR
-; CHECK-NOT: OpExtension "SPV_KHR_vulkan_memory_model"
-; CHECK: OpMemoryModel Logical GLSL450
-; CHECK-NOT: OpMemoryModel Logical VulkanKHR
-
-define spir_func void @nop() {
-entry:
- ret void
-}
diff --git a/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_ops_vulkan.ll b/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_ops_vulkan.ll
index 457b546fbc0b1..365fa2cd9a404 100644
--- a/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_ops_vulkan.ll
+++ b/llvm/test/CodeGen/SPIRV/extensions/SPV_KHR_cooperative_matrix/cooperative_matrix_ops_vulkan.ll
@@ -11,10 +11,8 @@
; CHECK-DAG: OpCapability CooperativeMatrixKHR
; CHECK-DAG: OpExtension "SPV_KHR_cooperative_matrix"
-; Requiring CooperativeMatrixKHR under the Shader flavor upgrades the default
-; GLSL450 memory model to VulkanKHR (spirv-val rejects Shader + CooperativeMatrixKHR
-; under GLSL450). This is the SPIRVModuleAnalysis derivation: once collectReqs sees
-; the CooperativeMatrixKHR capability, it pulls in the Vulkan memory model.
+; The module sets the Vulkan memory model explicitly via spirv.MemoryModel
+; metadata (spirv-val rejects Shader + CooperativeMatrixKHR under GLSL450).
; CHECK-DAG: OpCapability VulkanMemoryModelKHR
; CHECK-DAG: OpExtension "SPV_KHR_vulkan_memory_model"
; CHECK-DAG: OpMemoryModel Logical VulkanKHR
@@ -49,3 +47,7 @@ entry:
i32 0, i32 16)
ret void
}
+
+; AddressingModel=Logical (0), MemoryModel=VulkanKHR (3)
+!spirv.MemoryModel = !{!0}
+!0 = !{i32 0, i32 3}
More information about the llvm-commits
mailing list