[Mlir-commits] [mlir] [mlir][SPIR-V] Add CopyMemorySized support (PR #232190)
Alexander Shaposhnikov
llvmlistbot at llvm.org
Wed Oct 14 23:13:23 PDT 2026
https://github.com/alexander-shaposhnikov created https://github.com/llvm/llvm-project/pull/232190
Add spirv.CopyMemorySized with assembly syntax, verification, serialization, and deserialization. Allow different source and target pointee types and a scalar integer byte count. Reject zero and negative signed constant sizes, validate memory access attributes, and require SPIR-V 1.4 for the two-mask form during version, capability, and extension deduction.
>From 9488310ef7c13665753e5bfb00d877a0aa601390 Mon Sep 17 00:00:00 2001
From: Alexander Shaposhnikov <alexander.v.shaposhnikov at gmail.com>
Date: Wed, 14 Oct 2026 23:12:39 -0700
Subject: [PATCH] [mlir][SPIR-V] Add CopyMemorySized support
Add spirv.CopyMemorySized with assembly syntax, verification, serialization,
and deserialization. Allow different source and target pointee types and a
scalar integer byte count. Reject zero and negative signed constant sizes,
validate memory access attributes, and require SPIR-V 1.4 for the two-mask
form during version, capability, and extension deduction.
Share binary encoding and decoding with CopyMemory. Handle Aligned in
combined masks, diagnose malformed operands, and reject masks requiring
scope or alias-list operands that the operations cannot represent. Report
unsupported masks by name, retaining numeric values for binary diagnostics.
Preserve source-only memory access masks in both operations by emitting an
explicit None target mask in custom assembly and binary output.
Add coverage for verification, version deduction, text and binary round
trips, unsupported serialization, and malformed binaries for both opcodes.
---
.../mlir/Dialect/SPIRV/IR/SPIRVBase.td | 4 +-
.../mlir/Dialect/SPIRV/IR/SPIRVMemoryOps.td | 74 +++++++++
mlir/lib/Dialect/SPIRV/IR/MemoryOps.cpp | 90 +++++++++++
.../SPIRV/Deserialization/DeserializeOps.cpp | 135 +++++++++--------
.../SPIRV/Deserialization/Deserializer.h | 5 +
.../SPIRV/Serialization/SerializeOps.cpp | 44 ++++--
.../Target/SPIRV/Serialization/Serializer.h | 4 +
mlir/test/Dialect/SPIRV/IR/memory-ops.mlir | 141 ++++++++++++++++++
.../Transforms/copy-memory-sized-vce.mlir | 35 +++++
.../Target/SPIRV/copy-memory-unsupported.mlir | 19 +++
mlir/test/Target/SPIRV/memory-ops.mlir | 79 ++++++++++
.../Dialect/SPIRV/DeserializationTest.cpp | 115 ++++++++++++++
12 files changed, 664 insertions(+), 81 deletions(-)
create mode 100644 mlir/test/Dialect/SPIRV/Transforms/copy-memory-sized-vce.mlir
create mode 100644 mlir/test/Target/SPIRV/copy-memory-unsupported.mlir
diff --git a/mlir/include/mlir/Dialect/SPIRV/IR/SPIRVBase.td b/mlir/include/mlir/Dialect/SPIRV/IR/SPIRVBase.td
index 7e799fc0191076..32feae3c19e3a9 100644
--- a/mlir/include/mlir/Dialect/SPIRV/IR/SPIRVBase.td
+++ b/mlir/include/mlir/Dialect/SPIRV/IR/SPIRVBase.td
@@ -4535,6 +4535,7 @@ def SPIRV_OC_OpVariable : I32EnumAttrCase<"OpVariable", 59
def SPIRV_OC_OpLoad : I32EnumAttrCase<"OpLoad", 61>;
def SPIRV_OC_OpStore : I32EnumAttrCase<"OpStore", 62>;
def SPIRV_OC_OpCopyMemory : I32EnumAttrCase<"OpCopyMemory", 63>;
+def SPIRV_OC_OpCopyMemorySized : I32EnumAttrCase<"OpCopyMemorySized", 64>;
def SPIRV_OC_OpAccessChain : I32EnumAttrCase<"OpAccessChain", 65>;
def SPIRV_OC_OpInBoundsAccessChain : I32EnumAttrCase<"OpInBoundsAccessChain", 66>;
def SPIRV_OC_OpPtrAccessChain : I32EnumAttrCase<"OpPtrAccessChain", 67>;
@@ -4790,7 +4791,8 @@ def SPIRV_OpcodeAttr :
SPIRV_OC_OpSpecConstantOp, SPIRV_OC_OpFunction, SPIRV_OC_OpFunctionParameter,
SPIRV_OC_OpFunctionEnd, SPIRV_OC_OpFunctionCall, SPIRV_OC_OpVariable,
SPIRV_OC_OpLoad, SPIRV_OC_OpStore, SPIRV_OC_OpCopyMemory,
- SPIRV_OC_OpAccessChain, SPIRV_OC_OpInBoundsAccessChain, SPIRV_OC_OpPtrAccessChain,
+ SPIRV_OC_OpCopyMemorySized, SPIRV_OC_OpAccessChain,
+ SPIRV_OC_OpInBoundsAccessChain, SPIRV_OC_OpPtrAccessChain,
SPIRV_OC_OpInBoundsPtrAccessChain, SPIRV_OC_OpDecorate,
SPIRV_OC_OpMemberDecorate, SPIRV_OC_OpVectorExtractDynamic,
SPIRV_OC_OpVectorInsertDynamic, SPIRV_OC_OpVectorShuffle,
diff --git a/mlir/include/mlir/Dialect/SPIRV/IR/SPIRVMemoryOps.td b/mlir/include/mlir/Dialect/SPIRV/IR/SPIRVMemoryOps.td
index 5ae6c8af830032..c6f2411fc71381 100644
--- a/mlir/include/mlir/Dialect/SPIRV/IR/SPIRVMemoryOps.td
+++ b/mlir/include/mlir/Dialect/SPIRV/IR/SPIRVMemoryOps.td
@@ -169,6 +169,80 @@ def SPIRV_CopyMemoryOp : SPIRV_Op<"CopyMemory", [DeclareOpInterfaceMethods<Align
// -----
+def SPIRV_CopyMemorySizedOp : SPIRV_Op<"CopyMemorySized", [DeclareOpInterfaceMethods<AlignmentAttrOpInterface>]> {
+ let summary = [{
+ Copy from the memory pointed to by Source to the memory pointed to by
+ Target.
+ }];
+
+ let description = [{
+ Target and Source must have pointer types. Their pointee types may differ.
+
+ Size is the number of bytes to copy. It must have a scalar integer type.
+ If it is a constant instruction, the constant value must not be 0. It is
+ invalid for both the constant's type to have Signedness of 1 and to have
+ the sign bit set. Otherwise, as a run-time value, Size is treated as
+ unsigned, and if its value is 0, no memory access is made.
+
+ If present, any Memory Operands must begin with a memory operand
+ literal. If not present, it is the same as specifying the memory operand
+ None. Before version 1.4, at most one memory operands mask can be
+ provided. Starting with version 1.4 two masks can be provided, as
+ described in Memory Operands. If no masks or only one mask is present,
+ it applies to both Source and Target. If two masks are present, the
+ first applies to Target and must not include MakePointerVisible, and the
+ second applies to Source and must not include MakePointerAvailable.
+
+ <!-- End of AutoGen section -->
+
+ Serialization supports the Volatile, Aligned, Nontemporal, and
+ NonPrivatePointer memory operands. Masks requiring scope or alias-list
+ operands are not supported.
+
+ ```
+ copy-memory-sized-op ::= `spirv.CopyMemorySized ` ssa-use `, ` ssa-use
+ `, ` ssa-use
+ (`[` memory-access `]` (`, [` memory-access `]`)?)?
+ ` : ` spirv-pointer-type `, ` spirv-pointer-type
+ `, ` integer-type
+ ```
+
+ #### Example:
+
+ ```mlir
+ %0 = spirv.Variable : !spirv.ptr<f32, Function>
+ %1 = spirv.Variable : !spirv.ptr<i32, Function>
+ %2 = spirv.Constant 4 : i32
+ spirv.CopyMemorySized %0, %1, %2 : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.CopyMemorySized %0, %1, %2 ["Aligned", 4] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ ```
+ }];
+
+ let availability = [
+ MinVersion<SPIRV_V_1_0>,
+ MaxVersion<SPIRV_V_1_6>,
+ Extension<[]>,
+ Capability<[SPIRV_C_Addresses]>
+ ];
+
+ let arguments = (ins
+ SPIRV_AnyPtr:$target,
+ SPIRV_AnyPtr:$source,
+ SPIRV_Integer:$size,
+ OptionalAttr<SPIRV_MemoryAccessAttr>:$memory_access,
+ OptionalAttr<IntValidAlignment<I32Attr>>:$alignment,
+ OptionalAttr<SPIRV_MemoryAccessAttr>:$source_memory_access,
+ OptionalAttr<IntValidAlignment<I32Attr>>:$source_alignment
+ );
+
+ let results = (outs);
+
+ let autogenSerialization = 0;
+ let autogenAvailability = 0;
+}
+
+// -----
+
def SPIRV_InBoundsPtrAccessChainOp : SPIRV_Op<"InBoundsPtrAccessChain", [Pure]> {
let summary = [{
Has the same semantics as OpPtrAccessChain, with the addition that the
diff --git a/mlir/lib/Dialect/SPIRV/IR/MemoryOps.cpp b/mlir/lib/Dialect/SPIRV/IR/MemoryOps.cpp
index c99b6e04ef1ac9..89bb80a7b82cdc 100644
--- a/mlir/lib/Dialect/SPIRV/IR/MemoryOps.cpp
+++ b/mlir/lib/Dialect/SPIRV/IR/MemoryOps.cpp
@@ -115,6 +115,9 @@ static void printSourceMemoryAccessAttribute(
: memoryOp.getSourceMemoryAccess())) {
elidedAttrs.push_back(memoryOp.getSourceMemoryAccessAttrName());
+ // Keep the source mask in the second position even without a target mask.
+ if (!memoryOp.getMemoryAccess())
+ printer << " [\"None\"]";
printer << ", [\"" << stringifyMemoryAccess(*memAccess) << "\"";
if (spirv::bitEnumContainsAll(*memAccess, spirv::MemoryAccess::Aligned)) {
@@ -548,6 +551,93 @@ LogicalResult CopyMemoryOp::verify() {
MemoryAccessKind::Read);
}
+//===----------------------------------------------------------------------===//
+// spirv.CopyMemorySized
+//===----------------------------------------------------------------------===//
+
+void CopyMemorySizedOp::print(OpAsmPrinter &printer) {
+ printer << ' ' << getTarget() << ", " << getSource() << ", " << getSize();
+
+ SmallVector<StringRef, 4> elidedAttrs;
+ printMemoryAccessAttribute(*this, printer, elidedAttrs);
+ printSourceMemoryAccessAttribute(*this, printer, elidedAttrs,
+ getSourceMemoryAccess(),
+ getSourceAlignment());
+
+ printer.printOptionalAttrDict(
+ (*this)->getDiscardableAttrDictionary().getValue(), elidedAttrs);
+
+ printer << " : " << getTarget().getType() << ", " << getSource().getType()
+ << ", " << getSize().getType();
+}
+
+ParseResult CopyMemorySizedOp::parse(OpAsmParser &parser,
+ OperationState &result) {
+ SmallVector<OpAsmParser::UnresolvedOperand, 3> operandInfo;
+ SMLoc loc = parser.getCurrentLocation();
+ if (parser.parseOperandList(operandInfo, 3) ||
+ parseMemoryAccessAttributes<CopyMemorySizedOp>(parser, result))
+ return failure();
+
+ if (!parser.parseOptionalComma()) {
+ // Parse 2nd memory access attributes.
+ if (parseSourceMemoryAccessAttributes<CopyMemorySizedOp>(parser, result))
+ return failure();
+ }
+
+ SmallVector<Type, 3> operandTypes;
+ if (parser.parseOptionalAttrDict(result.attributes) || parser.parseColon() ||
+ parser.parseTypeList(operandTypes))
+ return failure();
+
+ return parser.resolveOperands(operandInfo, operandTypes, loc,
+ result.operands);
+}
+
+LogicalResult CopyMemorySizedOp::verify() {
+ if (auto constOp = getSize().getDefiningOp<spirv::ConstantOp>()) {
+ if (auto sizeAttr = dyn_cast<IntegerAttr>(constOp.getValue())) {
+ APInt size = sizeAttr.getValue();
+ if (size.isZero())
+ return emitOpError("size must not be a constant zero");
+ if (sizeAttr.getType().isSignedInteger() && size.isNegative())
+ return emitOpError("size must not be a negative signed constant");
+ }
+ }
+
+ // A lone mask applies to both operands. Only the first of two masks is
+ // Target-only.
+ MemoryAccessKind targetKind = getSourceMemoryAccess()
+ ? MemoryAccessKind::Write
+ : MemoryAccessKind::ReadWrite;
+ if (failed(verifyMemoryAccessAttribute(*this, targetKind)))
+ return failure();
+
+ return verifyMemoryAccessAttribute(
+ getOperation(), getSourceMemoryAccessAttr(), getSourceAlignmentAttr(),
+ MemoryAccessKind::Read);
+}
+
+std::optional<spirv::Version> CopyMemorySizedOp::getMinVersion() {
+ return getSourceMemoryAccess() ? spirv::Version::V_1_4
+ : spirv::Version::V_1_0;
+}
+
+std::optional<spirv::Version> CopyMemorySizedOp::getMaxVersion() {
+ return spirv::Version::V_1_6;
+}
+
+SmallVector<ArrayRef<spirv::Extension>, 1> CopyMemorySizedOp::getExtensions() {
+ return {};
+}
+
+SmallVector<ArrayRef<spirv::Capability>, 1>
+CopyMemorySizedOp::getCapabilities() {
+ static constexpr spirv::Capability capabilities[] = {
+ spirv::Capability::Addresses};
+ return {ArrayRef<spirv::Capability>(capabilities)};
+}
+
//===----------------------------------------------------------------------===//
// spirv.InBoundsPtrAccessChainOp
//===----------------------------------------------------------------------===//
diff --git a/mlir/lib/Target/SPIRV/Deserialization/DeserializeOps.cpp b/mlir/lib/Target/SPIRV/Deserialization/DeserializeOps.cpp
index acb43b6e04bd67..c5b74280859710 100644
--- a/mlir/lib/Target/SPIRV/Deserialization/DeserializeOps.cpp
+++ b/mlir/lib/Target/SPIRV/Deserialization/DeserializeOps.cpp
@@ -679,89 +679,96 @@ Deserializer::processOp<spirv::FunctionCallOp>(ArrayRef<uint32_t> operands) {
return success();
}
-template <>
-LogicalResult
-Deserializer::processOp<spirv::CopyMemoryOp>(ArrayRef<uint32_t> words) {
- SmallVector<Type, 1> resultTypes;
- size_t wordIndex = 0;
- SmallVector<Value, 4> operands;
- SmallVector<NamedAttribute, 4> attributes;
-
- if (wordIndex < words.size()) {
- auto arg = getValue(words[wordIndex]);
-
- if (!arg) {
- return emitError(unknownLoc, "unknown result <id> : ")
- << words[wordIndex];
- }
-
- operands.push_back(arg);
- wordIndex++;
+template <typename OpTy>
+LogicalResult Deserializer::processCopyMemoryOp(ArrayRef<uint32_t> words,
+ unsigned numValueOperands) {
+ if (words.size() < numValueOperands) {
+ return emitError(unknownLoc, "expected at least ")
+ << numValueOperands << " operands for " << OpTy::getOperationName()
+ << ", but got " << words.size();
}
- if (wordIndex < words.size()) {
- auto arg = getValue(words[wordIndex]);
-
- if (!arg) {
- return emitError(unknownLoc, "unknown result <id> : ")
- << words[wordIndex];
- }
-
- operands.push_back(arg);
- wordIndex++;
+ SmallVector<Value, 3> operands;
+ for (uint32_t operandID : words.take_front(numValueOperands)) {
+ Value operand = getValue(operandID);
+ if (!operand)
+ return emitError(unknownLoc, "unknown result <id> : ") << operandID;
+ operands.push_back(operand);
}
- bool isAlignedAttr = false;
-
- if (wordIndex < words.size()) {
- auto attrValue = words[wordIndex++];
- auto attr = opBuilder.getAttr<spirv::MemoryAccessAttr>(
- static_cast<spirv::MemoryAccess>(attrValue));
- attributes.push_back(
- opBuilder.getNamedAttr(attributeName<MemoryAccess>(), attr));
- isAlignedAttr = (attrValue == 2);
- }
-
- if (isAlignedAttr && wordIndex < words.size()) {
- attributes.push_back(opBuilder.getNamedAttr(
- "alignment", opBuilder.getI32IntegerAttr(words[wordIndex++])));
- }
-
- if (wordIndex < words.size()) {
- auto attrValue = words[wordIndex++];
- auto attr = opBuilder.getAttr<spirv::MemoryAccessAttr>(
- static_cast<spirv::MemoryAccess>(attrValue));
- attributes.push_back(opBuilder.getNamedAttr("source_memory_access", attr));
- }
-
- if (wordIndex < words.size()) {
+ SmallVector<NamedAttribute, 4> attributes;
+ size_t wordIndex = numValueOperands;
+ auto parseMemoryAccess = [&](StringRef memoryAccessName,
+ StringRef alignmentName) -> LogicalResult {
+ if (wordIndex == words.size())
+ return success();
+ auto memoryAccess = static_cast<spirv::MemoryAccess>(words[wordIndex++]);
+ // Other masks require scope or alias-list operands that these ops cannot
+ // represent. Do not mistake those operands for a second memory access mask.
+ constexpr auto supported = spirv::MemoryAccess::Volatile |
+ spirv::MemoryAccess::Aligned |
+ spirv::MemoryAccess::Nontemporal |
+ spirv::MemoryAccess::NonPrivatePointer;
+ if (static_cast<uint32_t>(memoryAccess) &
+ ~static_cast<uint32_t>(supported)) {
+ auto mask =
+ spirv::symbolizeMemoryAccess(static_cast<uint32_t>(memoryAccess));
+ return emitError(unknownLoc, "unsupported memory access mask: ")
+ << (mask ? spirv::stringifyMemoryAccess(*mask) : "unknown") << " ("
+ << static_cast<uint32_t>(memoryAccess) << ")";
+ }
attributes.push_back(opBuilder.getNamedAttr(
- "source_alignment", opBuilder.getI32IntegerAttr(words[wordIndex++])));
- }
+ memoryAccessName,
+ opBuilder.getAttr<spirv::MemoryAccessAttr>(memoryAccess)));
+ if (spirv::bitEnumContainsAll(memoryAccess, spirv::MemoryAccess::Aligned)) {
+ if (wordIndex == words.size())
+ return emitError(unknownLoc, "missing alignment literal for ")
+ << memoryAccessName;
+ attributes.push_back(opBuilder.getNamedAttr(
+ alignmentName, opBuilder.getI32IntegerAttr(words[wordIndex++])));
+ }
+ return success();
+ };
+ if (failed(parseMemoryAccess(attributeName<MemoryAccess>(), "alignment")) ||
+ failed(parseMemoryAccess("source_memory_access", "source_alignment")))
+ return failure();
if (wordIndex != words.size()) {
return emitError(unknownLoc,
- "found more operands than expected when deserializing "
- "spirv::CopyMemoryOp, only ")
- << wordIndex << " of " << words.size() << " processed";
+ "found more operands than expected when deserializing ")
+ << OpTy::getOperationName() << ", only " << wordIndex << " of "
+ << words.size() << " processed";
}
Location loc = createFileLineColLoc(opBuilder);
- spirv::CopyMemoryOp::Properties properties{};
- spirv::CopyMemoryOp::populateDefaultProperties(
- OperationName(spirv::CopyMemoryOp::getOperationName(),
- opBuilder.getContext()),
+ typename OpTy::Properties properties{};
+ OpTy::populateDefaultProperties(
+ OperationName(OpTy::getOperationName(), opBuilder.getContext()),
properties);
- if (failed(spirv::CopyMemoryOp::setPropertiesFromAttr(
+ if (failed(OpTy::setPropertiesFromAttr(
properties, opBuilder.getDictionaryAttr(attributes),
[&]() { return emitError(loc); })))
return failure();
- spirv::CopyMemoryOp::create(opBuilder, loc, resultTypes, operands, properties,
- /*discardableAttributes=*/{});
+ OpTy::create(opBuilder, loc, /*resultTypes=*/TypeRange(), operands,
+ properties, /*discardableAttributes=*/{});
return success();
}
+template <>
+LogicalResult
+Deserializer::processOp<spirv::CopyMemoryOp>(ArrayRef<uint32_t> words) {
+ return processCopyMemoryOp<spirv::CopyMemoryOp>(words,
+ /*numValueOperands=*/2);
+}
+
+template <>
+LogicalResult
+Deserializer::processOp<spirv::CopyMemorySizedOp>(ArrayRef<uint32_t> words) {
+ return processCopyMemoryOp<spirv::CopyMemorySizedOp>(words,
+ /*numValueOperands=*/3);
+}
+
template <>
LogicalResult Deserializer::processOp<spirv::GenericCastToPtrExplicitOp>(
ArrayRef<uint32_t> words) {
diff --git a/mlir/lib/Target/SPIRV/Deserialization/Deserializer.h b/mlir/lib/Target/SPIRV/Deserialization/Deserializer.h
index 543db3c99bddb0..3c3d1de90f1f94 100644
--- a/mlir/lib/Target/SPIRV/Deserialization/Deserializer.h
+++ b/mlir/lib/Target/SPIRV/Deserialization/Deserializer.h
@@ -585,6 +585,11 @@ class Deserializer {
<< OpTy::getOperationName() << " op";
}
+ /// Deserializes a memory copy with `numValueOperands` leading IDs.
+ template <typename OpTy>
+ LogicalResult processCopyMemoryOp(ArrayRef<uint32_t> words,
+ unsigned numValueOperands);
+
private:
/// The SPIR-V binary module.
ArrayRef<uint32_t> binary;
diff --git a/mlir/lib/Target/SPIRV/Serialization/SerializeOps.cpp b/mlir/lib/Target/SPIRV/Serialization/SerializeOps.cpp
index da311c386a3242..027b8e041eef99 100644
--- a/mlir/lib/Target/SPIRV/Serialization/SerializeOps.cpp
+++ b/mlir/lib/Target/SPIRV/Serialization/SerializeOps.cpp
@@ -1117,11 +1117,19 @@ Serializer::processOp<spirv::FunctionCallOp>(spirv::FunctionCallOp op) {
return success();
}
-template <>
-LogicalResult
-Serializer::processOp<spirv::CopyMemoryOp>(spirv::CopyMemoryOp op) {
+template <typename OpTy>
+LogicalResult Serializer::processCopyMemoryOp(OpTy op, spirv::Opcode opcode) {
+ constexpr auto supported =
+ spirv::MemoryAccess::Volatile | spirv::MemoryAccess::Aligned |
+ spirv::MemoryAccess::Nontemporal | spirv::MemoryAccess::NonPrivatePointer;
+ for (auto mask : {op.getMemoryAccess(), op.getSourceMemoryAccess()}) {
+ if (mask &&
+ (static_cast<uint32_t>(*mask) & ~static_cast<uint32_t>(supported)))
+ return op.emitError("unsupported memory access mask: ")
+ << spirv::stringifyMemoryAccess(*mask);
+ }
+
SmallVector<uint32_t, 4> operands;
- SmallVector<StringRef, 2> elidedAttrs;
for (Value operand : op->getOperands()) {
auto id = getValueID(operand);
@@ -1129,35 +1137,39 @@ Serializer::processOp<spirv::CopyMemoryOp>(spirv::CopyMemoryOp op) {
operands.push_back(id);
}
- StringAttr memoryAccess = op.getMemoryAccessAttrName();
if (std::optional<spirv::MemoryAccess> value = op.getMemoryAccess())
operands.push_back(static_cast<uint32_t>(*value));
+ else if (op.getSourceMemoryAccess())
+ operands.push_back(static_cast<uint32_t>(spirv::MemoryAccess::None));
- elidedAttrs.push_back(memoryAccess.strref());
-
- StringAttr alignment = op.getAlignmentAttrName();
if (std::optional<uint32_t> value = op.getAlignment())
operands.push_back(*value);
- elidedAttrs.push_back(alignment.strref());
-
- StringAttr sourceMemoryAccess = op.getSourceMemoryAccessAttrName();
if (std::optional<spirv::MemoryAccess> value = op.getSourceMemoryAccess())
operands.push_back(static_cast<uint32_t>(*value));
- elidedAttrs.push_back(sourceMemoryAccess.strref());
-
- StringAttr sourceAlignment = op.getSourceAlignmentAttrName();
if (std::optional<uint32_t> value = op.getSourceAlignment())
operands.push_back(*value);
- elidedAttrs.push_back(sourceAlignment.strref());
if (failed(emitDebugLine(functionBody, op.getLoc())))
return failure();
- encodeInstructionInto(functionBody, spirv::Opcode::OpCopyMemory, operands);
+ encodeInstructionInto(functionBody, opcode, operands);
return success();
}
+
+template <>
+LogicalResult
+Serializer::processOp<spirv::CopyMemoryOp>(spirv::CopyMemoryOp op) {
+ return processCopyMemoryOp(op, spirv::Opcode::OpCopyMemory);
+}
+
+template <>
+LogicalResult
+Serializer::processOp<spirv::CopyMemorySizedOp>(spirv::CopyMemorySizedOp op) {
+ return processCopyMemoryOp(op, spirv::Opcode::OpCopyMemorySized);
+}
+
template <>
LogicalResult Serializer::processOp<spirv::GenericCastToPtrExplicitOp>(
spirv::GenericCastToPtrExplicitOp op) {
diff --git a/mlir/lib/Target/SPIRV/Serialization/Serializer.h b/mlir/lib/Target/SPIRV/Serialization/Serializer.h
index 235c050e7dc7e0..620887a6918e9e 100644
--- a/mlir/lib/Target/SPIRV/Serialization/Serializer.h
+++ b/mlir/lib/Target/SPIRV/Serialization/Serializer.h
@@ -376,6 +376,10 @@ class Serializer {
return op.emitError("unsupported op serialization");
}
+ /// Serializes a memory copy and its optional memory access masks.
+ template <typename OpTy>
+ LogicalResult processCopyMemoryOp(OpTy op, spirv::Opcode opcode);
+
//===--------------------------------------------------------------------===//
// Utilities
//===--------------------------------------------------------------------===//
diff --git a/mlir/test/Dialect/SPIRV/IR/memory-ops.mlir b/mlir/test/Dialect/SPIRV/IR/memory-ops.mlir
index b86c769b0f8702..211d21ea08d567 100644
--- a/mlir/test/Dialect/SPIRV/IR/memory-ops.mlir
+++ b/mlir/test/Dialect/SPIRV/IR/memory-ops.mlir
@@ -814,6 +814,147 @@ func.func @copy_memory_print_maa() {
// -----
+//===----------------------------------------------------------------------===//
+// spirv.CopyMemorySized
+//===----------------------------------------------------------------------===//
+
+func.func @copy_memory_sized(%size: i64) {
+ %0 = spirv.Variable : !spirv.ptr<f32, Function>
+ %1 = spirv.Variable : !spirv.ptr<i32, Function>
+ %2 = spirv.Constant 4 : i32
+
+ // Target and Source may point to different types.
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.CopyMemorySized %0, %1, %2 : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i64
+ spirv.CopyMemorySized %0, %1, %size : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i64
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["Volatile"] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.CopyMemorySized %0, %1, %2 ["Volatile"] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["Aligned", 4] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.CopyMemorySized %0, %1, %2 ["Aligned", 4] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["Aligned", 4], ["Volatile"] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.CopyMemorySized %0, %1, %2 ["Aligned", 4], ["Volatile"] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["Aligned", 4], ["Aligned", 8] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.CopyMemorySized %0, %1, %2 ["Aligned", 4], ["Aligned", 8] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+
+ spirv.Return
+}
+
+// -----
+
+func.func @copy_memory_sized_print_maa() {
+ %0 = spirv.Variable : !spirv.ptr<f32, Function>
+ %1 = spirv.Variable : !spirv.ptr<f32, Function>
+ %2 = spirv.Constant 4 : i32
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["Volatile"] : !spirv.ptr<f32, Function>, !spirv.ptr<f32, Function>, i32
+ "spirv.CopyMemorySized"(%0, %1, %2) {memory_access=#spirv.memory_access<Volatile>} : (!spirv.ptr<f32, Function>, !spirv.ptr<f32, Function>, i32) -> ()
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["Aligned", 4], ["Aligned", 8] : !spirv.ptr<f32, Function>, !spirv.ptr<f32, Function>, i32
+ "spirv.CopyMemorySized"(%0, %1, %2) {source_memory_access=#spirv.memory_access<Aligned>, memory_access=#spirv.memory_access<Aligned>, source_alignment=8 : i32, alignment=4 : i32} : (!spirv.ptr<f32, Function>, !spirv.ptr<f32, Function>, i32) -> ()
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["MakePointerAvailable|NonPrivatePointer"], ["MakePointerVisible|NonPrivatePointer"] : !spirv.ptr<f32, Function>, !spirv.ptr<f32, Function>, i32
+ "spirv.CopyMemorySized"(%0, %1, %2) {memory_access=#spirv.memory_access<MakePointerAvailable|NonPrivatePointer>, source_memory_access=#spirv.memory_access<MakePointerVisible|NonPrivatePointer>} : (!spirv.ptr<f32, Function>, !spirv.ptr<f32, Function>, i32) -> ()
+
+ spirv.Return
+}
+
+// -----
+
+func.func @copy_memory_sized_zero_size() {
+ %0 = spirv.Variable : !spirv.ptr<f32, Function>
+ %1 = spirv.Variable : !spirv.ptr<f32, Function>
+ %2 = spirv.Constant 0 : i32
+ // expected-error @+1 {{size must not be a constant zero}}
+ spirv.CopyMemorySized %0, %1, %2 : !spirv.ptr<f32, Function>, !spirv.ptr<f32, Function>, i32
+ spirv.Return
+}
+
+// -----
+
+func.func @copy_memory_sized_negative_size() {
+ %0 = spirv.Variable : !spirv.ptr<f32, Function>
+ %1 = spirv.Variable : !spirv.ptr<f32, Function>
+ %2 = spirv.Constant -4 : si32
+ // expected-error @+1 {{size must not be a negative signed constant}}
+ spirv.CopyMemorySized %0, %1, %2 : !spirv.ptr<f32, Function>, !spirv.ptr<f32, Function>, si32
+ spirv.Return
+}
+
+// -----
+
+func.func @copy_memory_sized_vector_size(%size: vector<2xi32>) {
+ %0 = spirv.Variable : !spirv.ptr<f32, Function>
+ %1 = spirv.Variable : !spirv.ptr<f32, Function>
+ // expected-error @+1 {{op operand #2 must be 8/16/32/64-bit integer}}
+ spirv.CopyMemorySized %0, %1, %size : !spirv.ptr<f32, Function>, !spirv.ptr<f32, Function>, vector<2xi32>
+ spirv.Return
+}
+
+// -----
+
+func.func @copy_memory_sized_invalid_maa() {
+ %0 = spirv.Variable : !spirv.ptr<f32, Function>
+ %1 = spirv.Variable : !spirv.ptr<f32, Function>
+ %2 = spirv.Constant 4 : i32
+ // expected-error @+1 {{missing alignment value}}
+ "spirv.CopyMemorySized"(%0, %1, %2) {memory_access=#spirv.memory_access<Aligned>} : (!spirv.ptr<f32, Function>, !spirv.ptr<f32, Function>, i32) -> ()
+ spirv.Return
+}
+
+// -----
+
+func.func @copy_memory_sized_invalid_source_maa() {
+ %0 = spirv.Variable : !spirv.ptr<f32, Function>
+ %1 = spirv.Variable : !spirv.ptr<f32, Function>
+ %2 = spirv.Constant 4 : i32
+ // expected-error @+1 {{invalid alignment specification with non-aligned memory access specification}}
+ "spirv.CopyMemorySized"(%0, %1, %2) {source_memory_access=#spirv.memory_access<Volatile>, memory_access=#spirv.memory_access<Aligned>, source_alignment=8 : i32, alignment=4 : i32} : (!spirv.ptr<f32, Function>, !spirv.ptr<f32, Function>, i32) -> ()
+ spirv.Return
+}
+
+// -----
+
+func.func @copy_memory_sized_target_bad_operand() {
+ %0 = spirv.Variable : !spirv.ptr<f32, Function>
+ %1 = spirv.Variable : !spirv.ptr<f32, Function>
+ %2 = spirv.Constant 4 : i32
+ // expected-error @+1 {{op not compatible with memory operand 'MakePointerVisible'}}
+ "spirv.CopyMemorySized"(%0, %1, %2) {memory_access=#spirv.memory_access<MakePointerVisible|NonPrivatePointer>, source_memory_access=#spirv.memory_access<None>} : (!spirv.ptr<f32, Function>, !spirv.ptr<f32, Function>, i32) -> ()
+ spirv.Return
+}
+
+// -----
+
+func.func @copy_memory_sized_source_bad_operand() {
+ %0 = spirv.Variable : !spirv.ptr<f32, Function>
+ %1 = spirv.Variable : !spirv.ptr<f32, Function>
+ %2 = spirv.Constant 4 : i32
+ // expected-error @+1 {{op not compatible with memory operand 'MakePointerAvailable'}}
+ "spirv.CopyMemorySized"(%0, %1, %2) {source_memory_access=#spirv.memory_access<MakePointerAvailable|NonPrivatePointer>, memory_access=#spirv.memory_access<None>} : (!spirv.ptr<f32, Function>, !spirv.ptr<f32, Function>, i32) -> ()
+ spirv.Return
+}
+
+// -----
+
+// CHECK-LABEL: func @copy_memory_sized_unsigned_size
+func.func @copy_memory_sized_unsigned_size(%ptr: !spirv.ptr<i32, Function>) {
+ %size = spirv.Constant 4294967295 : ui32
+ // CHECK: spirv.CopyMemorySized
+ spirv.CopyMemorySized %ptr, %ptr, %size : !spirv.ptr<i32, Function>, !spirv.ptr<i32, Function>, ui32
+ %signless = spirv.Constant -1 : i32
+ // CHECK: spirv.CopyMemorySized
+ spirv.CopyMemorySized %ptr, %ptr, %signless : !spirv.ptr<i32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.Return
+}
+
+// -----
+
//===----------------------------------------------------------------------===//
// spirv.PtrAccessChain
//===----------------------------------------------------------------------===//
diff --git a/mlir/test/Dialect/SPIRV/Transforms/copy-memory-sized-vce.mlir b/mlir/test/Dialect/SPIRV/Transforms/copy-memory-sized-vce.mlir
new file mode 100644
index 00000000000000..bf051f51ae84dc
--- /dev/null
+++ b/mlir/test/Dialect/SPIRV/Transforms/copy-memory-sized-vce.mlir
@@ -0,0 +1,35 @@
+// RUN: mlir-opt --spirv-update-vce --split-input-file --verify-diagnostics %s | FileCheck %s
+
+// CHECK: requires #spirv.vce<v1.0, [Addresses, Kernel], []>
+spirv.module Physical64 OpenCL attributes {
+ spirv.target_env = #spirv.target_env<#spirv.vce<v1.4, [Kernel, Addresses], []>, #spirv.resource_limits<>>
+} {
+ spirv.func @copy(%ptr: !spirv.ptr<i32, Function>, %size: i32) "None" {
+ spirv.CopyMemorySized %ptr, %ptr, %size : !spirv.ptr<i32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.Return
+ }
+}
+
+// -----
+
+// CHECK: requires #spirv.vce<v1.4, [Addresses, Kernel], []>
+spirv.module Physical64 OpenCL attributes {
+ spirv.target_env = #spirv.target_env<#spirv.vce<v1.4, [Kernel, Addresses], []>, #spirv.resource_limits<>>
+} {
+ spirv.func @copy(%ptr: !spirv.ptr<i32, Function>, %size: i32) "None" {
+ spirv.CopyMemorySized %ptr, %ptr, %size ["None"], ["None"] : !spirv.ptr<i32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.Return
+ }
+}
+
+// -----
+
+spirv.module Physical64 OpenCL attributes {
+ spirv.target_env = #spirv.target_env<#spirv.vce<v1.3, [Kernel, Addresses], []>, #spirv.resource_limits<>>
+} {
+ spirv.func @copy(%ptr: !spirv.ptr<i32, Function>, %size: i32) "None" {
+ // expected-error @+1 {{'spirv.CopyMemorySized' requires min version v1.4 but target environment allows up to v1.3}}
+ spirv.CopyMemorySized %ptr, %ptr, %size ["None"], ["None"] : !spirv.ptr<i32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.Return
+ }
+}
diff --git a/mlir/test/Target/SPIRV/copy-memory-unsupported.mlir b/mlir/test/Target/SPIRV/copy-memory-unsupported.mlir
new file mode 100644
index 00000000000000..d275311af5f0ba
--- /dev/null
+++ b/mlir/test/Target/SPIRV/copy-memory-unsupported.mlir
@@ -0,0 +1,19 @@
+// RUN: mlir-translate --no-implicit-module --serialize-spirv --split-input-file --verify-diagnostics %s
+
+spirv.module Physical64 OpenCL requires #spirv.vce<v1.4, [Kernel, Addresses, Linkage], []> {
+ spirv.func @unsupported_scope(%dst: !spirv.ptr<i32, Function>, %src: !spirv.ptr<i32, Function>, %size: i32) "None" {
+ // expected-error @+1 {{unsupported memory access mask: MakePointerAvailable|NonPrivatePointer}}
+ spirv.CopyMemorySized %dst, %src, %size ["MakePointerAvailable|NonPrivatePointer"] : !spirv.ptr<i32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.Return
+ }
+}
+
+// -----
+
+spirv.module Physical64 OpenCL requires #spirv.vce<v1.4, [Kernel, Addresses, Linkage], []> {
+ spirv.func @unsupported_source_alias(%dst: !spirv.ptr<i32, Function>, %src: !spirv.ptr<i32, Function>) "None" {
+ // expected-error @+1 {{unsupported memory access mask: AliasScopeINTELMask}}
+ spirv.CopyMemory "Function" %dst, "Function" %src ["None"], ["AliasScopeINTELMask"] : i32
+ spirv.Return
+ }
+}
diff --git a/mlir/test/Target/SPIRV/memory-ops.mlir b/mlir/test/Target/SPIRV/memory-ops.mlir
index 9641d7e8ec6ed4..548ea303072135 100644
--- a/mlir/test/Target/SPIRV/memory-ops.mlir
+++ b/mlir/test/Target/SPIRV/memory-ops.mlir
@@ -1,4 +1,5 @@
// RUN: mlir-translate -no-implicit-module -test-spirv-roundtrip -split-input-file %s | FileCheck %s
+// RUN: mlir-opt -split-input-file %s | mlir-opt -split-input-file | FileCheck %s
// RUN: %if spirv-tools %{ rm -rf %t %}
// RUN: %if spirv-tools %{ mkdir %t %}
@@ -148,3 +149,81 @@ spirv.module Logical GLSL450 requires #spirv.vce<v1.4, [Shader, Linkage], []> {
spirv.Return
}
}
+
+// -----
+
+spirv.module Physical64 OpenCL requires #spirv.vce<v1.0, [Kernel, Addresses, Int64, Linkage], []> {
+ spirv.func @copy_memory_sized(%size: i64) "None" {
+ %0 = spirv.Variable : !spirv.ptr<f32, Function>
+ %1 = spirv.Variable : !spirv.ptr<i32, Function>
+ %2 = spirv.Constant 4 : i32
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.CopyMemorySized %0, %1, %2 : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i64
+ spirv.CopyMemorySized %0, %1, %size : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i64
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["Volatile"] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.CopyMemorySized %0, %1, %2 ["Volatile"] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["Aligned", 4] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.CopyMemorySized %0, %1, %2 ["Aligned", 4] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["Volatile|Aligned", 4] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.CopyMemorySized %0, %1, %2 ["Volatile|Aligned", 4] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+
+ spirv.Return
+ }
+}
+
+// -----
+
+spirv.module Physical64 OpenCL requires #spirv.vce<v1.4, [Kernel, Addresses, Linkage], []> {
+ spirv.func @copy_memory_sized_with_two_access_operands() "None" {
+ %0 = spirv.Variable : !spirv.ptr<f32, Function>
+ %1 = spirv.Variable : !spirv.ptr<i32, Function>
+ %2 = spirv.Constant 4 : i32
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["Volatile"], ["Volatile"] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.CopyMemorySized %0, %1, %2 ["Volatile"], ["Volatile"] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["Aligned", 4], ["Volatile"] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.CopyMemorySized %0, %1, %2 ["Aligned", 4], ["Volatile"] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["Volatile"], ["Aligned", 4] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.CopyMemorySized %0, %1, %2 ["Volatile"], ["Aligned", 4] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["Aligned", 8], ["Aligned", 4] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ spirv.CopyMemorySized %0, %1, %2 ["Aligned", 8], ["Aligned", 4] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["Volatile|Aligned", 8], ["Volatile|Aligned", 4]
+ spirv.CopyMemorySized %0, %1, %2 ["Volatile|Aligned", 8], ["Volatile|Aligned", 4] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+
+ // Source-only masks acquire an explicit None target mask in text and binary.
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["None"], ["Volatile"] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ "spirv.CopyMemorySized"(%0, %1, %2) {source_memory_access = #spirv.memory_access<Volatile>} : (!spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32) -> ()
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["None"], ["Volatile|Aligned", 4] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ "spirv.CopyMemorySized"(%0, %1, %2) {source_memory_access = #spirv.memory_access<Volatile|Aligned>, source_alignment = 4 : i32} : (!spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32) -> ()
+ // CHECK: spirv.CopyMemorySized %{{.*}}, %{{.*}}, %{{.*}} ["None"], ["None"] : !spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32
+ "spirv.CopyMemorySized"(%0, %1, %2) {source_memory_access = #spirv.memory_access<None>} : (!spirv.ptr<f32, Function>, !spirv.ptr<i32, Function>, i32) -> ()
+
+ spirv.Return
+ }
+}
+
+// -----
+
+spirv.module Physical64 OpenCL requires #spirv.vce<v1.4, [Kernel, Addresses, Linkage], []> {
+ spirv.func @copy_memory_combined_masks(%dst: !spirv.ptr<i32, Function>, %src: !spirv.ptr<i32, Function>) "None" {
+ // CHECK: spirv.CopyMemory "Function" %{{.*}}, "Function" %{{.*}} ["Volatile|Aligned", 4], ["Volatile|Aligned", 8] : i32
+ spirv.CopyMemory "Function" %dst, "Function" %src ["Volatile|Aligned", 4], ["Volatile|Aligned", 8] : i32
+ // CHECK: spirv.CopyMemory "Function" %{{.*}}, "Function" %{{.*}} ["None"], ["Volatile"] : i32
+ "spirv.CopyMemory"(%dst, %src) {source_memory_access = #spirv.memory_access<Volatile>} : (!spirv.ptr<i32, Function>, !spirv.ptr<i32, Function>) -> ()
+ // CHECK: spirv.CopyMemory "Function" %{{.*}}, "Function" %{{.*}} ["None"], ["Volatile|Aligned", 4] : i32
+ "spirv.CopyMemory"(%dst, %src) {source_memory_access = #spirv.memory_access<Volatile|Aligned>, source_alignment = 4 : i32} : (!spirv.ptr<i32, Function>, !spirv.ptr<i32, Function>) -> ()
+ // CHECK: spirv.CopyMemory "Function" %{{.*}}, "Function" %{{.*}} ["None"], ["None"] : i32
+ "spirv.CopyMemory"(%dst, %src) {source_memory_access = #spirv.memory_access<None>} : (!spirv.ptr<i32, Function>, !spirv.ptr<i32, Function>) -> ()
+ spirv.Return
+ }
+}
diff --git a/mlir/unittests/Dialect/SPIRV/DeserializationTest.cpp b/mlir/unittests/Dialect/SPIRV/DeserializationTest.cpp
index db19eb9e222c1b..31b722d67abd86 100644
--- a/mlir/unittests/Dialect/SPIRV/DeserializationTest.cpp
+++ b/mlir/unittests/Dialect/SPIRV/DeserializationTest.cpp
@@ -462,6 +462,121 @@ TEST_F(DeserializationTest, OpMemberNameExcessOperands) {
expectDiagnostic("unexpected trailing words in OpMemberName instruction");
}
+//===----------------------------------------------------------------------===//
+// Memory copies
+//===----------------------------------------------------------------------===//
+
+class CopyMemoryDeserializationTest
+ : public DeserializationTest,
+ public ::testing::WithParamInterface<spirv::Opcode> {
+protected:
+ void addCopyMemoryFunction() {
+ addHeader();
+ auto voidType = addVoidType();
+ auto intType = addIntType(32);
+ auto ptrType = nextID++;
+ addInstruction(spirv::Opcode::OpTypePointer,
+ {ptrType,
+ static_cast<uint32_t>(spirv::StorageClass::Function),
+ intType});
+ auto fnType = addFunctionType(voidType, {ptrType, ptrType, intType});
+ addFunction(voidType, fnType);
+ for (auto type : {ptrType, ptrType, intType}) {
+ auto id = nextID++;
+ addInstruction(spirv::Opcode::OpFunctionParameter, {type, id});
+ copyOperands.push_back(id);
+ }
+ if (GetParam() == spirv::Opcode::OpCopyMemory)
+ copyOperands.pop_back();
+ addInstruction(spirv::Opcode::OpLabel, {nextID++});
+ }
+
+ void addCopyMemory(ArrayRef<uint32_t> memoryOperands) {
+ copyOperands.append(memoryOperands.begin(), memoryOperands.end());
+ addInstruction(GetParam(), copyOperands);
+ addReturn();
+ addFunctionEnd();
+ }
+
+ SmallVector<uint32_t> copyOperands;
+};
+
+TEST_P(CopyMemoryDeserializationTest, MissingOperands) {
+ addHeader();
+ addInstruction(GetParam(), {});
+ ASSERT_FALSE(deserialize());
+ expectDiagnostic(
+ GetParam() == spirv::Opcode::OpCopyMemory
+ ? "expected at least 2 operands for spirv.CopyMemory, but got 0"
+ : "expected at least 3 operands for spirv.CopyMemorySized, but got "
+ "0");
+}
+
+TEST_P(CopyMemoryDeserializationTest, UnknownValue) {
+ addCopyMemoryFunction();
+ copyOperands[0] = 100;
+ addCopyMemory({});
+ ASSERT_FALSE(deserialize());
+ expectDiagnostic("unknown result <id> : 100");
+}
+
+TEST_P(CopyMemoryDeserializationTest, MissingTargetAlignment) {
+ addCopyMemoryFunction();
+ addCopyMemory({static_cast<uint32_t>(spirv::MemoryAccess::Volatile |
+ spirv::MemoryAccess::Aligned)});
+ ASSERT_FALSE(deserialize());
+ expectDiagnostic("missing alignment literal for memory_access");
+}
+
+TEST_P(CopyMemoryDeserializationTest, MissingSourceAlignment) {
+ addCopyMemoryFunction();
+ addCopyMemory({0, static_cast<uint32_t>(spirv::MemoryAccess::Aligned)});
+ ASSERT_FALSE(deserialize());
+ expectDiagnostic("missing alignment literal for source_memory_access");
+}
+
+TEST_P(CopyMemoryDeserializationTest, UnsupportedScopeOperand) {
+ addCopyMemoryFunction();
+ addCopyMemory(
+ {static_cast<uint32_t>(spirv::MemoryAccess::MakePointerAvailable |
+ spirv::MemoryAccess::NonPrivatePointer),
+ 1});
+ ASSERT_FALSE(deserialize());
+ expectDiagnostic("unsupported memory access mask: "
+ "MakePointerAvailable|NonPrivatePointer (40)");
+}
+
+TEST_P(CopyMemoryDeserializationTest, UnsupportedSourceAliasOperand) {
+ addCopyMemoryFunction();
+ addCopyMemory(
+ {0, static_cast<uint32_t>(spirv::MemoryAccess::AliasScopeINTELMask), 1});
+ ASSERT_FALSE(deserialize());
+ expectDiagnostic(
+ "unsupported memory access mask: AliasScopeINTELMask (65536)");
+}
+
+TEST_P(CopyMemoryDeserializationTest, UnknownMemoryAccessMask) {
+ addCopyMemoryFunction();
+ addCopyMemory({64});
+ ASSERT_FALSE(deserialize());
+ expectDiagnostic("unsupported memory access mask: unknown (64)");
+}
+
+TEST_P(CopyMemoryDeserializationTest, ExcessOperands) {
+ addCopyMemoryFunction();
+ addCopyMemory({0, 0, 0});
+ ASSERT_FALSE(deserialize());
+ expectDiagnostic(GetParam() == spirv::Opcode::OpCopyMemory
+ ? "found more operands than expected when deserializing "
+ "spirv.CopyMemory, only 4 of 5 processed"
+ : "found more operands than expected when deserializing "
+ "spirv.CopyMemorySized, only 5 of 6 processed");
+}
+
+INSTANTIATE_TEST_SUITE_P(SPIRV, CopyMemoryDeserializationTest,
+ ::testing::Values(spirv::Opcode::OpCopyMemory,
+ spirv::Opcode::OpCopyMemorySized));
+
//===----------------------------------------------------------------------===//
// Functions
//===----------------------------------------------------------------------===//
More information about the Mlir-commits
mailing list