[Mlir-commits] [mlir] [mlir][Interfaces] `CallOpInterface`: Model forwarded result + improve verification (PR #214724)
Matthias Springer
llvmlistbot at llvm.org
Fri Aug 7 06:19:09 PDT 2026
https://github.com/matthias-springer created https://github.com/llvm/llvm-project/pull/214724
`CallOpInterface` distinguishes between forwarded operands and consumed operands. This commit adds the concept of "forwarded results", making operands/results symmetric.
Forwarded operands are forwarded from the caller to the callee's block arguments. Forwarded results are forwarded from the callee to the caller's results. All other operands/results are consumed/produced by the call op.
This commit also improves verification:
- The number of forwarded operands and callee arguments must match.
- The number of forwarded results and caller results must match.
- Types must be compatible according to `CallOpInterface::areTypesCompatible`.
Note: `llvm.call` / `llvm.func` support variadic operands. These are not supported by `CallOpInterface` or `CallableOpInterface`. The `CallOpInterface` now no longer reports variadic operands as part of the forwarded operands (`getArgOperands`). The old implementation used to inconsistent: `CallOpInterface` included variadic operands, but `CallableOpInterface` did not.
>From 3b79f7b2b3cc72cc89224c8e023acd5a142a63c1 Mon Sep 17 00:00:00 2001
From: Matthias Springer <me at m-sp.org>
Date: Fri, 7 Aug 2026 11:56:09 +0000
Subject: [PATCH] [mlir][Interfaces] `CallOpInterface`: Model forwarded result
+ improve verification
---
mlir/docs/Interfaces.md | 21 ++
.../include/mlir/Dialect/Async/IR/AsyncOps.td | 5 +
mlir/include/mlir/Dialect/EmitC/IR/EmitC.td | 5 +
mlir/include/mlir/Dialect/Func/IR/FuncOps.td | 5 +
mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td | 10 +
mlir/include/mlir/Interfaces/CallInterfaces.h | 23 +++
.../include/mlir/Interfaces/CallInterfaces.td | 57 +++++-
.../Transforms/DialectInlinerInterface.td | 4 +-
mlir/lib/Analysis/DataFlow/SparseAnalysis.cpp | 22 ++-
mlir/lib/Dialect/Async/IR/Async.cpp | 23 +--
mlir/lib/Dialect/EmitC/IR/EmitC.cpp | 23 +--
mlir/lib/Dialect/Func/IR/FuncOps.cpp | 23 +--
mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp | 102 ++++++----
mlir/lib/Dialect/LLVMIR/IR/LLVMInterfaces.cpp | 12 +-
mlir/lib/Interfaces/CallInterfaces.cpp | 68 +++++++
mlir/lib/Transforms/RemoveDeadValues.cpp | 19 +-
mlir/lib/Transforms/Utils/InliningUtils.cpp | 8 +-
mlir/test/Dialect/Func/invalid.mlir | 2 +-
mlir/test/Dialect/LLVMIR/invalid.mlir | 16 +-
.../verify-call-op-interface.mlir | 186 ++++++++++++++++++
mlir/test/Transforms/remove-dead-values.mlir | 24 +++
mlir/test/lib/Dialect/Test/TestOpDefs.cpp | 64 ++++++
mlir/test/lib/Dialect/Test/TestOps.td | 40 ++++
23 files changed, 640 insertions(+), 122 deletions(-)
create mode 100644 mlir/test/Interfaces/CallInterfaces/verify-call-op-interface.mlir
diff --git a/mlir/docs/Interfaces.md b/mlir/docs/Interfaces.md
index 2de79eadd0052..9525c3e8316e3 100644
--- a/mlir/docs/Interfaces.md
+++ b/mlir/docs/Interfaces.md
@@ -840,6 +840,12 @@ interface section goes as follows:
* `CallOpInterface` - Used to represent operations like 'call'
- `CallInterfaceCallable getCallableForCallee()`
- `void setCalleeFromCallable(CallInterfaceCallable)`
+ - `Operation::operand_range getArgOperands()`
+ - `MutableOperandRange getArgOperandsMutable()`
+ - `Operation::result_range getForwardedResults()`
+ - `bool areTypesCompatible(Type, Type)`
+ - `Operation * resolveCallable()`
+ - `Operation * resolveCallableInTable(SymbolTableCollection *)`
- `ArrayAttr getArgAttrsAttr()`
- `ArrayAttr getResAttrsAttr()`
- `void setArgAttrsAttr(ArrayAttr)`
@@ -857,6 +863,21 @@ interface section goes as follows:
- `Attribute removeArgAttrsAttr()`
- `Attribute removeResAttrsAttr()`
+A call operation may have operands and results that are not part of the call
+itself; such operands are said to be *consumed* by the operation and such
+results to be *produced* by it. The operands that are passed to the callee
+(`getArgOperands`) and the results that receive the values returned by the
+callee (`getForwardedResults`) are said to be *forwarded*, and they are in a 1:1
+relationship with the arguments and results of the callee: the i-th forwarded
+operand is passed as the i-th argument of the callee and the i-th forwarded
+result receives the i-th value returned by the callee. Corresponding types need
+not be equal; `areTypesCompatible` decides which types may be paired.
+
+Variadic arguments of a call to a variadic callee (if supported by the call
+op / callee op) are consumed operands, not forwarded ones: they do not
+correspond to any argument of the callee, which reads them through dedicated
+operations instead of receiving them as block arguments.
+
##### RegionKindInterfaces
* `RegionKindInterface` - Used to describe the abstract semantics of regions.
diff --git a/mlir/include/mlir/Dialect/Async/IR/AsyncOps.td b/mlir/include/mlir/Dialect/Async/IR/AsyncOps.td
index 722370b8f3e29..39217f8710fb9 100644
--- a/mlir/include/mlir/Dialect/Async/IR/AsyncOps.td
+++ b/mlir/include/mlir/Dialect/Async/IR/AsyncOps.td
@@ -256,6 +256,11 @@ def Async_CallOp : Async_Op<"call",
return getOperandsMutable();
}
+ /// The operand and result types must match the callee exactly.
+ bool areTypesCompatible(::mlir::Type lhs, ::mlir::Type rhs) {
+ return lhs == rhs;
+ }
+
operand_iterator arg_operand_begin() { return operand_begin(); }
operand_iterator arg_operand_end() { return operand_end(); }
diff --git a/mlir/include/mlir/Dialect/EmitC/IR/EmitC.td b/mlir/include/mlir/Dialect/EmitC/IR/EmitC.td
index 49412d1dfb01c..64a748afa1f7c 100644
--- a/mlir/include/mlir/Dialect/EmitC/IR/EmitC.td
+++ b/mlir/include/mlir/Dialect/EmitC/IR/EmitC.td
@@ -789,6 +789,11 @@ def EmitC_CallOp : EmitC_Op<"call",
return getOperandsMutable();
}
+ /// The operand and result types must match the callee exactly.
+ bool areTypesCompatible(::mlir::Type lhs, ::mlir::Type rhs) {
+ return lhs == rhs;
+ }
+
operand_iterator arg_operand_begin() { return operand_begin(); }
operand_iterator arg_operand_end() { return operand_end(); }
diff --git a/mlir/include/mlir/Dialect/Func/IR/FuncOps.td b/mlir/include/mlir/Dialect/Func/IR/FuncOps.td
index a31b860276099..d286ec4f9d78c 100644
--- a/mlir/include/mlir/Dialect/Func/IR/FuncOps.td
+++ b/mlir/include/mlir/Dialect/Func/IR/FuncOps.td
@@ -105,6 +105,11 @@ def CallOp : Func_Op<"call",
return getOperandsMutable();
}
+ /// The operand and result types must match the callee exactly.
+ bool areTypesCompatible(::mlir::Type lhs, ::mlir::Type rhs) {
+ return lhs == rhs;
+ }
+
operand_iterator arg_operand_begin() { return operand_begin(); }
operand_iterator arg_operand_end() { return operand_end(); }
diff --git a/mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td b/mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td
index fc674a7afb55c..7f61e7a9b6583 100644
--- a/mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td
+++ b/mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td
@@ -767,6 +767,11 @@ def LLVM_InvokeOp
let extraClassDeclaration = [{
/// Returns the callee function type.
LLVMFunctionType getCalleeFunctionType();
+
+ /// The argument and result types must match the callee exactly.
+ bool areTypesCompatible(::mlir::Type lhs, ::mlir::Type rhs) {
+ return lhs == rhs;
+ }
}];
}
@@ -902,6 +907,11 @@ def LLVM_CallOp
let extraClassDeclaration = [{
/// Returns the callee function type.
LLVMFunctionType getCalleeFunctionType();
+
+ /// The argument and result types must match the callee exactly.
+ bool areTypesCompatible(::mlir::Type lhs, ::mlir::Type rhs) {
+ return lhs == rhs;
+ }
}];
}
diff --git a/mlir/include/mlir/Interfaces/CallInterfaces.h b/mlir/include/mlir/Interfaces/CallInterfaces.h
index 2bf3a3ca5f8a8..924dee820123e 100644
--- a/mlir/include/mlir/Interfaces/CallInterfaces.h
+++ b/mlir/include/mlir/Interfaces/CallInterfaces.h
@@ -25,6 +25,7 @@ struct CallInterfaceCallable : public PointerUnion<SymbolRefAttr, Value> {
using PointerUnion<SymbolRefAttr, Value>::PointerUnion;
};
+class CallableOpInterface;
class CallOpInterface;
namespace call_interface_impl {
@@ -36,6 +37,28 @@ namespace call_interface_impl {
Operation *resolveCallable(CallOpInterface call,
SymbolTableCollection *symbolTable = nullptr);
+/// Verify that the forwarded operands and results of `call` are in a 1:1
+/// relationship with the given argument and result types of its callee: the
+/// numbers must match and corresponding types must be compatible according to
+/// `CallOpInterface::areTypesCompatible`.
+///
+/// This overload is for call operations whose callee does not necessarily
+/// implement `CallableOpInterface`, e.g., `llvm.call`, which can also call an
+/// `llvm.mlir.alias` or an `llvm.mlir.ifunc`.
+LogicalResult verifyCallOpInterface(CallOpInterface call,
+ TypeRange argumentTypes,
+ TypeRange resultTypes);
+
+/// Same as above, taking the argument and result types from `callable`.
+LogicalResult verifyCallOpInterface(CallOpInterface call,
+ CallableOpInterface callable);
+
+/// Same as above, but resolve the callee of `call` first. Returns success if
+/// the callee cannot be resolved or does not implement `CallableOpInterface`.
+LogicalResult verifyCallOpInterface(CallOpInterface call,
+ SymbolTableCollection &symbolTable);
+LogicalResult verifyCallOpInterface(CallOpInterface call);
+
/// Parse a function or call result list.
///
/// function-result-list ::= function-result-list-parens
diff --git a/mlir/include/mlir/Interfaces/CallInterfaces.td b/mlir/include/mlir/Interfaces/CallInterfaces.td
index 19d3afe8b9243..9b431db7739e1 100644
--- a/mlir/include/mlir/Interfaces/CallInterfaces.td
+++ b/mlir/include/mlir/Interfaces/CallInterfaces.td
@@ -77,6 +77,18 @@ def CallOpInterface : OpInterface<"CallOpInterface",
another. These operations may be traditional direct calls `call @foo`, or
indirect calls to other operations `call_indirect %foo`. An operation that
uses this interface, must *not* also provide the `CallableOpInterface`.
+
+ This interface distinguishes between forwarded operands/results, and
+ consumed/produced operands/results. Forwarded operands/results are in a
+ 1:1 relationship with the arguments/results of the callee: the i-th
+ forwarded operand is passed as the i-th argument of the callee and the i-th
+ forwarded result receives the i-th value returned by the callee.
+ Corresponding types are not required to be equal; `areTypesCompatible`
+ decides which types may be paired.
+
+ Note: This interface does not model variadic operands ("argument pack").
+ Neither does the CallableOpInterface. Variadic operands should be modeled
+ as consumed operands.
}];
let cppNamespace = "::mlir";
@@ -99,17 +111,44 @@ def CallOpInterface : OpInterface<"CallOpInterface",
"void", "setCalleeFromCallable", (ins "::mlir::CallInterfaceCallable":$callee)
>,
InterfaceMethod<[{
- Returns the operands within this call that are used as arguments to the
- callee.
+ Returns the operands of this call that are used as arguments to the
+ callee ("forwarded operands").
+
+ The returned range must be a contiguous sub-range of the operation's
+ operands. The i-th forwarded operand is passed as the i-th argument of
+ the callee; the two types are not required to be equal, but must be
+ compatible according to `areTypesCompatible`. Operands that are not part
+ of the returned range are consumed by the operation itself and are not
+ passed to the callee as arguments.
}],
"::mlir::Operation::operand_range", "getArgOperands"
>,
InterfaceMethod<[{
- Returns the operands within this call that are used as arguments to the
- callee as a mutable range.
+ Returns the operands of this call that are used as arguments to the
+ callee ("forwarded operands") as a mutable range.
+
+ This must be the same range of operands as returned by `getArgOperands`.
}],
"::mlir::MutableOperandRange", "getArgOperandsMutable"
>,
+ InterfaceMethod<[{
+ Returns the results of this call that receive the values returned by the
+ callee ("forwarded results").
+
+ The returned range must be a contiguous sub-range of the operation's
+ results. The i-th forwarded result receives the i-th value returned by
+ the callee; the two types are not required to be equal, but must be
+ compatible according to `areTypesCompatible`. Results that are not part
+ of the returned range are produced by the operation itself and do not
+ originate from the callee.
+
+ By default, all results of the operation are forwarded results.
+ }],
+ "::mlir::Operation::result_range", "getForwardedResults", (ins),
+ /*methodBody=*/[{}], /*defaultImplementation=*/[{
+ return $_op->getResults();
+ }]
+ >,
InterfaceMethod<[{
Resolve the callable operation for given callee to a
CallableOpInterface, or nullptr if a valid callable was not resolved.
@@ -129,6 +168,16 @@ def CallOpInterface : OpInterface<"CallOpInterface",
/*methodBody=*/[{}], /*defaultImplementation=*/[{
return ::mlir::call_interface_impl::resolveCallable($_op);
}]
+ >,
+ InterfaceMethod<[{
+ Compares types across the call boundary for compatibility: the type of
+ a forwarded operand against the type of the corresponding callee
+ argument, and the type of a forwarded result against the type of the
+ corresponding callee result.
+ }],
+ "bool", "areTypesCompatible",
+ (ins "::mlir::Type":$callerType, "::mlir::Type":$calleeType),
+ /*methodBody=*/[{}], /*defaultImplementation=*/[{ return true; }]
>
];
}
diff --git a/mlir/include/mlir/Transforms/DialectInlinerInterface.td b/mlir/include/mlir/Transforms/DialectInlinerInterface.td
index 0975b84179d3c..e675e0165c26a 100644
--- a/mlir/include/mlir/Transforms/DialectInlinerInterface.td
+++ b/mlir/include/mlir/Transforms/DialectInlinerInterface.td
@@ -93,8 +93,8 @@ def DialectInlinerInterface : DialectInterface<"DialectInlinerInterface"> {
InterfaceMethod<[{
Handle the given inlined terminator by replacing it with a new operation
as necessary. This overload is called when the inlined region only
- contains one block. 'valuesToReplace' contains the previously returned
- values of the call site before inlining. These values must be replaced by
+ contains one block. 'valuesToReplace' contains the forwarded results
+ of the call site before inlining. These values must be replaced by
this callback if they had any users (for example for traditional function
calls, these are directly replaced with the operands of the `return`
operation). The given 'op' will be removed by the caller, after this
diff --git a/mlir/lib/Analysis/DataFlow/SparseAnalysis.cpp b/mlir/lib/Analysis/DataFlow/SparseAnalysis.cpp
index 3d9a375bfb2b7..bc61c19832d29 100644
--- a/mlir/lib/Analysis/DataFlow/SparseAnalysis.cpp
+++ b/mlir/lib/Analysis/DataFlow/SparseAnalysis.cpp
@@ -252,9 +252,23 @@ LogicalResult AbstractSparseForwardDataFlowAnalysis::visitCallOperation(
setAllToEntryStates(resultLattices);
return success();
}
+
+ // Only the forwarded results receive the values returned by the callee. Any
+ // other result is produced by the call operation itself and nothing is known
+ // about it here.
+ ResultRange forwardedResults = call.getForwardedResults();
+ unsigned firstForwarded = forwardedResults.empty()
+ ? resultLattices.size()
+ : forwardedResults[0].getResultNumber();
+ setAllToEntryStates(resultLattices.take_front(firstForwarded));
+ setAllToEntryStates(
+ resultLattices.drop_front(firstForwarded + forwardedResults.size()));
+ ArrayRef<AbstractSparseLattice *> forwardedResultLattices =
+ resultLattices.slice(firstForwarded, forwardedResults.size());
+
for (Operation *predecessor : predecessors->getKnownPredecessors())
for (auto &&[operand, resLattice] :
- llvm::zip(predecessor->getOperands(), resultLattices))
+ llvm::zip_equal(predecessor->getOperands(), forwardedResultLattices))
join(resLattice,
*getLatticeElementFor(getProgramPointAfter(call), operand));
return success();
@@ -585,8 +599,12 @@ LogicalResult AbstractSparseBackwardDataFlowAnalysis::visitCallableOperation(
getProgramPointAfter(op), getProgramPointAfter(callable));
if (callsites->allPredecessorsKnown()) {
for (Operation *call : callsites->getKnownPredecessors()) {
+ // Only the forwarded results of the call receive the values returned by
+ // the callee.
+ ResultRange forwardedResults =
+ cast<CallOpInterface>(call).getForwardedResults();
SmallVector<const AbstractSparseLattice *> callResultLattices =
- getLatticeElementsFor(getProgramPointAfter(op), call->getResults());
+ getLatticeElementsFor(getProgramPointAfter(op), forwardedResults);
for (auto [op, result] : llvm::zip(operandLattices, callResultLattices))
meet(op, *result);
}
diff --git a/mlir/lib/Dialect/Async/IR/Async.cpp b/mlir/lib/Dialect/Async/IR/Async.cpp
index c759ba04eefaf..c11400c061584 100644
--- a/mlir/lib/Dialect/Async/IR/Async.cpp
+++ b/mlir/lib/Dialect/Async/IR/Async.cpp
@@ -376,28 +376,7 @@ LogicalResult CallOp::verifySymbolUses(SymbolTableCollection &symbolTable) {
<< "' does not reference a valid async function";
// Verify that the operand and result types match the callee.
- auto fnType = fn.getFunctionType();
- if (fnType.getNumInputs() != getNumOperands())
- return emitOpError("incorrect number of operands for callee");
-
- for (unsigned i = 0, e = fnType.getNumInputs(); i != e; ++i)
- if (getOperand(i).getType() != fnType.getInput(i))
- return emitOpError("operand type mismatch: expected operand type ")
- << fnType.getInput(i) << ", but provided "
- << getOperand(i).getType() << " for operand number " << i;
-
- if (fnType.getNumResults() != getNumResults())
- return emitOpError("incorrect number of results for callee");
-
- for (unsigned i = 0, e = fnType.getNumResults(); i != e; ++i)
- if (getResult(i).getType() != fnType.getResult(i)) {
- auto diag = emitOpError("result type mismatch at index ") << i;
- diag.attachNote() << " op result types: " << getResultTypes();
- diag.attachNote() << "function result types: " << fnType.getResults();
- return diag;
- }
-
- return success();
+ return call_interface_impl::verifyCallOpInterface(*this, fn);
}
FunctionType CallOp::getCalleeType() {
diff --git a/mlir/lib/Dialect/EmitC/IR/EmitC.cpp b/mlir/lib/Dialect/EmitC/IR/EmitC.cpp
index 820c4ad578a9e..f40b6e67b30fc 100644
--- a/mlir/lib/Dialect/EmitC/IR/EmitC.cpp
+++ b/mlir/lib/Dialect/EmitC/IR/EmitC.cpp
@@ -749,28 +749,7 @@ LogicalResult CallOp::verifySymbolUses(SymbolTableCollection &symbolTable) {
<< "' does not reference a valid function";
// Verify that the operand and result types match the callee.
- auto fnType = fn.getFunctionType();
- if (fnType.getNumInputs() != getNumOperands())
- return emitOpError("incorrect number of operands for callee");
-
- for (unsigned i = 0, e = fnType.getNumInputs(); i != e; ++i)
- if (getOperand(i).getType() != fnType.getInput(i))
- return emitOpError("operand type mismatch: expected operand type ")
- << fnType.getInput(i) << ", but provided "
- << getOperand(i).getType() << " for operand number " << i;
-
- if (fnType.getNumResults() != getNumResults())
- return emitOpError("incorrect number of results for callee");
-
- for (unsigned i = 0, e = fnType.getNumResults(); i != e; ++i)
- if (getResult(i).getType() != fnType.getResult(i)) {
- auto diag = emitOpError("result type mismatch at index ") << i;
- diag.attachNote() << " op result types: " << getResultTypes();
- diag.attachNote() << "function result types: " << fnType.getResults();
- return diag;
- }
-
- return success();
+ return call_interface_impl::verifyCallOpInterface(*this, fn);
}
FunctionType CallOp::getCalleeType() {
diff --git a/mlir/lib/Dialect/Func/IR/FuncOps.cpp b/mlir/lib/Dialect/Func/IR/FuncOps.cpp
index 42925ce4fd6a0..54eb8cac737e8 100644
--- a/mlir/lib/Dialect/Func/IR/FuncOps.cpp
+++ b/mlir/lib/Dialect/Func/IR/FuncOps.cpp
@@ -71,28 +71,7 @@ LogicalResult CallOp::verifySymbolUses(SymbolTableCollection &symbolTable) {
<< "' does not reference a valid function";
// Verify that the operand and result types match the callee.
- auto fnType = fn.getFunctionType();
- if (fnType.getNumInputs() != getNumOperands())
- return emitOpError("incorrect number of operands for callee");
-
- for (unsigned i = 0, e = fnType.getNumInputs(); i != e; ++i)
- if (getOperand(i).getType() != fnType.getInput(i))
- return emitOpError("operand type mismatch: expected operand type ")
- << fnType.getInput(i) << ", but provided "
- << getOperand(i).getType() << " for operand number " << i;
-
- if (fnType.getNumResults() != getNumResults())
- return emitOpError("incorrect number of results for callee");
-
- for (unsigned i = 0, e = fnType.getNumResults(); i != e; ++i)
- if (getResult(i).getType() != fnType.getResult(i)) {
- auto diag = emitOpError("result type mismatch at index ") << i;
- diag.attachNote() << " op result types: " << getResultTypes();
- diag.attachNote() << "function result types: " << fnType.getResults();
- return diag;
- }
-
- return success();
+ return call_interface_impl::verifyCallOpInterface(*this, fn);
}
FunctionType CallOp::getCalleeType() {
diff --git a/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp b/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp
index 3bda6f5295747..374ad4a9dcb83 100644
--- a/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp
+++ b/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp
@@ -1125,13 +1125,47 @@ void CallOp::setCalleeFromCallable(CallInterfaceCallable callee) {
return setOperand(0, cast<Value>(callee));
}
+/// Return the number of leading callee operands of `callOp` that the operation
+/// consumes instead of passing them to the callee.
+template <typename OpTy>
+static unsigned getNumConsumedCalleeOperands(OpTy callOp) {
+ // The first operand is the callee if no callee attribute is present.
+ if (callOp.getCallee().has_value())
+ return 0;
+ return 1;
+}
+
+/// Return the operands of `callOp` that are passed to the callee, including the
+/// variadic arguments in case of a call to a variadic callee.
+template <typename OpTy>
+static Operation::operand_range getOperandsPassedToCallee(OpTy callOp) {
+ return callOp.getCalleeOperands().drop_front(
+ getNumConsumedCalleeOperands(callOp));
+}
+
+/// Return the operands of `callOp` that correspond to the declared parameters
+/// of the callee, i.e., its `CallOpInterface` argument operands.
+///
+/// The variadic arguments of a call to a variadic callee are *not* included:
+/// they do not correspond to any argument of the callee. The callee does not
+/// receive them as block arguments but reads them with `llvm.intr.vastart` and
+/// friends, so in terms of `CallOpInterface` they are consumed operands rather
+/// than forwarded ones.
+template <typename OpTy>
+static Operation::operand_range getArgOperandsImpl(OpTy callOp) {
+ Operation::operand_range operands = getOperandsPassedToCallee(callOp);
+ if (std::optional<LLVMFunctionType> varCalleeType = callOp.getVarCalleeType())
+ return operands.take_front(varCalleeType->getNumParams());
+ return operands;
+}
+
Operation::operand_range CallOp::getArgOperands() {
- return getCalleeOperands().drop_front(getCallee().has_value() ? 0 : 1);
+ return getArgOperandsImpl(*this);
}
MutableOperandRange CallOp::getArgOperandsMutable() {
- return MutableOperandRange(*this, getCallee().has_value() ? 0 : 1,
- getCalleeOperands().size());
+ return MutableOperandRange(*this, getNumConsumedCalleeOperands(*this),
+ getArgOperandsImpl(*this).size());
}
/// Verify that an inlinable callsite of a debug-info-bearing function in a
@@ -1163,6 +1197,11 @@ static LogicalResult verifyCallOpDebugInfo(CallOp callOp, LLVMFuncOp callee) {
/// the `callOp` argument and result types.
template <typename OpTy>
static LogicalResult verifyCallOpVarCalleeType(OpTy callOp) {
+ // An indirect call stores the callee in its first callee operand.
+ if (!callOp.getCallee().has_value() && callOp.getCalleeOperands().empty())
+ return callOp.emitOpError(
+ "must have either a `callee` attribute or at least an operand");
+
std::optional<LLVMFunctionType> varCalleeType = callOp.getVarCalleeType();
if (!varCalleeType)
return success();
@@ -1172,15 +1211,19 @@ static LogicalResult verifyCallOpVarCalleeType(OpTy callOp) {
return callOp.emitOpError(
"expected var_callee_type to be a variadic function type");
+ // Note: `getArgOperands` is derived from `var_callee_type`, so the raw callee
+ // operands are used here instead.
+ Operation::operand_range passedOperands = getOperandsPassedToCallee(callOp);
+
// Verify the variadic callee type has at most as many parameters as the call
// has argument operands.
- if (varCalleeType->getNumParams() > callOp.getArgOperands().size())
+ if (varCalleeType->getNumParams() > passedOperands.size())
return callOp.emitOpError("expected var_callee_type to have at most ")
- << callOp.getArgOperands().size() << " parameters";
+ << passedOperands.size() << " parameters";
// Verify the variadic callee type matches the call argument types.
for (auto [paramType, operand] :
- llvm::zip(varCalleeType->getParams(), callOp.getArgOperands()))
+ llvm::zip(varCalleeType->getParams(), passedOperands))
if (paramType != operand.getType())
return callOp.emitOpError()
<< "var_callee_type parameter type mismatch: " << paramType
@@ -1230,20 +1273,17 @@ LogicalResult CallOp::verifySymbolUses(SymbolTableCollection &symbolTable) {
// or indirect call.
Type fnType;
- bool isIndirect = false;
-
// If this is an indirect call, the callee attribute is missing.
FlatSymbolRefAttr calleeName = getCalleeAttr();
if (!calleeName) {
- isIndirect = true;
- if (!getNumOperands())
- return emitOpError(
- "must have either a `callee` attribute or at least an operand");
+ // Note: `verifyCallOpVarCalleeType` has already checked that there is a
+ // callee operand.
auto ptrType = llvm::dyn_cast<LLVMPointerType>(getOperand(0).getType());
if (!ptrType)
return emitOpError("indirect call expects a pointer as callee: ")
<< getOperand(0).getType();
+ // Nothing else to verify: an indirect callee cannot be resolved.
return success();
} else {
Operation *callee =
@@ -1277,27 +1317,8 @@ LogicalResult CallOp::verifySymbolUses(SymbolTableCollection &symbolTable) {
if (funcType.isVarArg() && !getVarCalleeType())
return emitOpError() << "missing var_callee_type attribute for vararg call";
- // Verify that the operand and result types match the callee.
-
- if (!funcType.isVarArg() &&
- funcType.getNumParams() != (getCalleeOperands().size() - isIndirect))
- return emitOpError() << "incorrect number of operands ("
- << (getCalleeOperands().size() - isIndirect)
- << ") for callee (expecting: "
- << funcType.getNumParams() << ")";
-
- if (funcType.getNumParams() > (getCalleeOperands().size() - isIndirect))
- return emitOpError() << "incorrect number of operands ("
- << (getCalleeOperands().size() - isIndirect)
- << ") for varargs callee (expecting at least: "
- << funcType.getNumParams() << ")";
-
- for (unsigned i = 0, e = funcType.getNumParams(); i != e; ++i)
- if (getOperand(i + isIndirect).getType() != funcType.getParamType(i))
- return emitOpError() << "operand type mismatch for operand " << i << ": "
- << getOperand(i + isIndirect).getType()
- << " != " << funcType.getParamType(i);
-
+ // Verify the result types. These checks are more specific than what
+ // `verifyCallOpInterface` can report, so they are run first.
if (getNumResults() == 0 &&
!llvm::isa<LLVM::LLVMVoidType>(funcType.getReturnType()))
return emitOpError() << "expected function call to produce a value";
@@ -1315,7 +1336,14 @@ LogicalResult CallOp::verifySymbolUses(SymbolTableCollection &symbolTable) {
return emitOpError() << "result type mismatch: " << getResult().getType()
<< " != " << funcType.getReturnType();
- return success();
+ // Verify that the operand types match the callee. Note that this does not
+ // need to special-case a variadic callee: the variadic arguments are not
+ // argument operands.
+ SmallVector<Type, 1> calleeResultTypes;
+ if (!llvm::isa<LLVM::LLVMVoidType>(funcType.getReturnType()))
+ calleeResultTypes.push_back(funcType.getReturnType());
+ return call_interface_impl::verifyCallOpInterface(*this, funcType.getParams(),
+ calleeResultTypes);
}
void CallOp::print(OpAsmPrinter &p) {
@@ -1620,12 +1648,12 @@ void InvokeOp::setCalleeFromCallable(CallInterfaceCallable callee) {
}
Operation::operand_range InvokeOp::getArgOperands() {
- return getCalleeOperands().drop_front(getCallee().has_value() ? 0 : 1);
+ return getArgOperandsImpl(*this);
}
MutableOperandRange InvokeOp::getArgOperandsMutable() {
- return MutableOperandRange(*this, getCallee().has_value() ? 0 : 1,
- getCalleeOperands().size());
+ return MutableOperandRange(*this, getNumConsumedCalleeOperands(*this),
+ getArgOperandsImpl(*this).size());
}
LogicalResult InvokeOp::verify() {
diff --git a/mlir/lib/Dialect/LLVMIR/IR/LLVMInterfaces.cpp b/mlir/lib/Dialect/LLVMIR/IR/LLVMInterfaces.cpp
index ea46b21416e9f..406ccd6849972 100644
--- a/mlir/lib/Dialect/LLVMIR/IR/LLVMInterfaces.cpp
+++ b/mlir/lib/Dialect/LLVMIR/IR/LLVMInterfaces.cpp
@@ -117,9 +117,15 @@ SmallVector<Value> mlir::LLVM::MemsetInlineOp::getAccessedOperands() {
}
SmallVector<Value> mlir::LLVM::CallOp::getAccessedOperands() {
- return llvm::filter_to_vector(getArgOperands(), [](Value arg) {
- return isa<LLVMPointerType>(arg.getType());
- });
+ // Note: This must not use `getArgOperands`, which excludes the variadic
+ // arguments of a call to a variadic callee. Those are passed to the callee
+ // and may well be accessed by it.
+ Operation::operand_range operands = getCalleeOperands();
+ // In an indirect call, the first callee operand is the callee itself.
+ if (!getCallee().has_value() && !operands.empty())
+ operands = operands.drop_front();
+ return llvm::filter_to_vector(
+ operands, [](Value arg) { return isa<LLVMPointerType>(arg.getType()); });
}
#include "mlir/Dialect/LLVMIR/LLVMInterfaces.cpp.inc"
diff --git a/mlir/lib/Interfaces/CallInterfaces.cpp b/mlir/lib/Interfaces/CallInterfaces.cpp
index e8ed4b339a0cb..8744fb5c8d074 100644
--- a/mlir/lib/Interfaces/CallInterfaces.cpp
+++ b/mlir/lib/Interfaces/CallInterfaces.cpp
@@ -187,6 +187,10 @@ Operation *
call_interface_impl::resolveCallable(CallOpInterface call,
SymbolTableCollection *symbolTable) {
CallInterfaceCallable callable = call.getCallableForCallee();
+ // Operations may not have a callee at all, e.g., when the callee is stored in
+ // an optional attribute that is missing in invalid IR.
+ if (!callable)
+ return nullptr;
if (auto symbolVal = dyn_cast<Value>(callable))
return symbolVal.getDefiningOp();
@@ -197,6 +201,70 @@ call_interface_impl::resolveCallable(CallOpInterface call,
return SymbolTable::lookupNearestSymbolFrom(call.getOperation(), symbolRef);
}
+LogicalResult call_interface_impl::verifyCallOpInterface(
+ CallOpInterface call, TypeRange argumentTypes, TypeRange resultTypes) {
+ Operation *op = call.getOperation();
+
+ // The forwarded operands are in a 1:1 relationship with the arguments of the
+ // callee.
+ OperandRange argOperands = call.getArgOperands();
+ if (argOperands.size() != argumentTypes.size())
+ return op->emitOpError("incorrect number of operands for callee: expected ")
+ << argumentTypes.size() << ", but got " << argOperands.size();
+ for (unsigned i = 0, e = argOperands.size(); i != e; ++i) {
+ Type operandType = argOperands[i].getType();
+ if (!call.areTypesCompatible(operandType, argumentTypes[i]))
+ return op->emitOpError("operand type mismatch: expected operand type ")
+ << argumentTypes[i] << ", but provided " << operandType
+ << " for operand number " << i;
+ }
+
+ // The forwarded results are in a 1:1 relationship with the results of the
+ // callee.
+ ResultRange forwardedResults = call.getForwardedResults();
+ if (forwardedResults.size() != resultTypes.size())
+ return op->emitOpError("incorrect number of results for callee: expected ")
+ << resultTypes.size() << ", but got " << forwardedResults.size();
+ for (unsigned i = 0, e = forwardedResults.size(); i != e; ++i) {
+ if (call.areTypesCompatible(forwardedResults[i].getType(), resultTypes[i]))
+ continue;
+ InFlightDiagnostic diag = op->emitOpError("result type mismatch at index ")
+ << i;
+ diag.attachNote() << " op result types: " << forwardedResults.getTypes();
+ diag.attachNote() << "callee result types: " << resultTypes;
+ return diag;
+ }
+
+ return success();
+}
+
+LogicalResult
+call_interface_impl::verifyCallOpInterface(CallOpInterface call,
+ CallableOpInterface callable) {
+ return verifyCallOpInterface(call, callable.getArgumentTypes(),
+ callable.getResultTypes());
+}
+
+/// Verify `call` against `resolved`, the operation its callee resolved to.
+/// Nothing is verified if the callee did not resolve to a callable operation.
+static LogicalResult verifyResolvedCallee(CallOpInterface call,
+ Operation *resolved) {
+ auto callable = dyn_cast_if_present<CallableOpInterface>(resolved);
+ if (!callable)
+ return success();
+ return call_interface_impl::verifyCallOpInterface(call, callable);
+}
+
+LogicalResult
+call_interface_impl::verifyCallOpInterface(CallOpInterface call,
+ SymbolTableCollection &symbolTable) {
+ return verifyResolvedCallee(call, call.resolveCallableInTable(&symbolTable));
+}
+
+LogicalResult call_interface_impl::verifyCallOpInterface(CallOpInterface call) {
+ return verifyResolvedCallee(call, call.resolveCallable());
+}
+
//===----------------------------------------------------------------------===//
// CallInterfaces
//===----------------------------------------------------------------------===//
diff --git a/mlir/lib/Transforms/RemoveDeadValues.cpp b/mlir/lib/Transforms/RemoveDeadValues.cpp
index 6e55bc390be23..ffaa0663b0457 100644
--- a/mlir/lib/Transforms/RemoveDeadValues.cpp
+++ b/mlir/lib/Transforms/RemoveDeadValues.cpp
@@ -335,8 +335,10 @@ static void processFuncOp(FunctionOpInterface funcOp,
size_t numReturns = funcOp.getNumResults();
BitVector nonLiveRets(numReturns, true);
for (Operation *callOp : users) {
- assert(isa<CallOpInterface>(callOp) && "expected a call-like user");
- BitVector liveCallRets = markLives(callOp->getResults(), nonLiveSet, la);
+ // Only the forwarded results of a call receive the values returned by the
+ // callee; any other result is produced by the call operation itself.
+ BitVector liveCallRets = markLives(
+ cast<CallOpInterface>(callOp).getForwardedResults(), nonLiveSet, la);
nonLiveRets &= liveCallRets.flip();
}
@@ -358,9 +360,16 @@ static void processFuncOp(FunctionOpInterface funcOp,
if (numReturns == 0)
return;
for (Operation *callOp : users) {
- assert(isa<CallOpInterface>(callOp) && "expected a call-like user");
- cl.results.push_back({callOp, nonLiveRets});
- collectNonLiveValues(nonLiveSet, callOp->getResults(), nonLiveRets);
+ // `nonLiveRets` is indexed by callee result. Translate it into the index
+ // space of all results of the call operation, which is what the cleanup
+ // works on.
+ ResultRange forwardedResults =
+ cast<CallOpInterface>(callOp).getForwardedResults();
+ BitVector nonLiveCallResults(callOp->getNumResults(), false);
+ for (int index : nonLiveRets.set_bits())
+ nonLiveCallResults.set(forwardedResults[index].getResultNumber());
+ cl.results.push_back({callOp, nonLiveCallResults});
+ collectNonLiveValues(nonLiveSet, callOp->getResults(), nonLiveCallResults);
}
}
diff --git a/mlir/lib/Transforms/Utils/InliningUtils.cpp b/mlir/lib/Transforms/Utils/InliningUtils.cpp
index 73107cfc36ea9..ce4d6ad71b6d3 100644
--- a/mlir/lib/Transforms/Utils/InliningUtils.cpp
+++ b/mlir/lib/Transforms/Utils/InliningUtils.cpp
@@ -487,10 +487,16 @@ LogicalResult mlir::inlineCall(
auto *entryBlock = &src->front();
ArrayRef<Type> callableResultTypes = callable.getResultTypes();
+ // Inlining replaces the call with the body of the callee, so the call must
+ // not have any results that are produced by the call operation itself: there
+ // would be nothing left to compute them.
+ if (call.getForwardedResults().size() != call->getNumResults())
+ return failure();
+
// Make sure that the number of arguments and results matchup between the call
// and the region.
SmallVector<Value, 8> callOperands(call.getArgOperands());
- SmallVector<Value, 8> callResults(call->getResults());
+ SmallVector<Value, 8> callResults(call.getForwardedResults());
if (callOperands.size() != entryBlock->getNumArguments() ||
callResults.size() != callableResultTypes.size())
return failure();
diff --git a/mlir/test/Dialect/Func/invalid.mlir b/mlir/test/Dialect/Func/invalid.mlir
index 3143bda77ebba..b3c4ec8903394 100644
--- a/mlir/test/Dialect/Func/invalid.mlir
+++ b/mlir/test/Dialect/Func/invalid.mlir
@@ -13,7 +13,7 @@ func.func private @return_i32_f32() -> (i32, f32)
func.func @call() {
// expected-error @+3 {{op result type mismatch at index 0}}
// expected-note @+2 {{op result types: 'f32', 'i32'}}
- // expected-note @+1 {{function result types: 'i32', 'f32'}}
+ // expected-note @+1 {{callee result types: 'i32', 'f32'}}
%0:2 = call @return_i32_f32() : () -> (f32, i32)
return
}
diff --git a/mlir/test/Dialect/LLVMIR/invalid.mlir b/mlir/test/Dialect/LLVMIR/invalid.mlir
index 6265f67e594d0..18cf0202203da 100644
--- a/mlir/test/Dialect/LLVMIR/invalid.mlir
+++ b/mlir/test/Dialect/LLVMIR/invalid.mlir
@@ -233,6 +233,20 @@ func.func @invalid_call() {
// -----
+llvm.func @invalid_invoke() attributes { personality = @__gxx_personality_v0 } {
+ // expected-error at +1 {{'llvm.invoke' op must have either a `callee` attribute or at least an operand}}
+ "llvm.invoke"()[^bb1, ^bb2] {op_bundle_sizes = array<i32>, operandSegmentSizes = array<i32: 0, 0, 0, 0>} : () -> ()
+^bb1:
+ llvm.return
+^bb2:
+ %0 = llvm.landingpad cleanup : !llvm.struct<(ptr, i32)>
+ llvm.return
+}
+
+llvm.func @__gxx_personality_v0(...) -> i32
+
+// -----
+
func.func @call_missing_ptr_type(%callee : !llvm.func<i8 (i8)>, %arg : i8) {
// expected-error at +1 {{expected indirect call to have 2 trailing types}}
llvm.call %callee(%arg) : (i8) -> (i8)
@@ -314,7 +328,7 @@ func.func @call_non_llvm_res(%callee : !llvm.ptr) {
llvm.func @callee_func(i8) -> ()
func.func @callee_arg_mismatch(%arg0 : i32) {
- // expected-error at +1 {{'llvm.call' op operand type mismatch for operand 0: 'i32' != 'i8'}}
+ // expected-error at +1 {{'llvm.call' op operand type mismatch: expected operand type 'i8', but provided 'i32' for operand number 0}}
llvm.call @callee_func(%arg0) : (i32) -> ()
llvm.return
}
diff --git a/mlir/test/Interfaces/CallInterfaces/verify-call-op-interface.mlir b/mlir/test/Interfaces/CallInterfaces/verify-call-op-interface.mlir
new file mode 100644
index 0000000000000..94c43fe0514d7
--- /dev/null
+++ b/mlir/test/Interfaces/CallInterfaces/verify-call-op-interface.mlir
@@ -0,0 +1,186 @@
+// RUN: mlir-opt %s -split-input-file -verify-diagnostics
+
+// Tests `call_interface_impl::verifyCallOpInterface`, i.e., the 1:1
+// relationship between the forwarded operands/results of a call operation and
+// the arguments/results of its callee.
+
+// `test.call_and_produce` produces its first result itself; only the trailing
+// results are forwarded from the callee. Neither the produced result nor a
+// non-forwarded operand takes part in the verification.
+
+func.func private @callee(i32) -> i32
+
+func.func @forwarded_operands_and_results_match(%arg0: i32) {
+ %status, %res = test.call_and_produce @callee(%arg0) : (i32) -> (i1, i32)
+ return
+}
+
+// -----
+
+func.func private @callee(i32) -> i32
+
+func.func @too_many_forwarded_operands(%arg0: i32) {
+ // expected-error @below {{incorrect number of operands for callee: expected 1, but got 2}}
+ %status, %res = test.call_and_produce @callee(%arg0, %arg0) : (i32, i32) -> (i1, i32)
+ return
+}
+
+// -----
+
+func.func private @callee(i32, i32) -> i32
+
+func.func @too_few_forwarded_operands(%arg0: i32) {
+ // expected-error @below {{incorrect number of operands for callee: expected 2, but got 1}}
+ %status, %res = test.call_and_produce @callee(%arg0) : (i32) -> (i1, i32)
+ return
+}
+
+// -----
+
+func.func private @callee(i32) -> i32
+
+func.func @too_many_forwarded_results(%arg0: i32) {
+ // expected-error @below {{incorrect number of results for callee: expected 1, but got 2}}
+ %status, %res0, %res1 = test.call_and_produce @callee(%arg0) : (i32) -> (i1, i32, i32)
+ return
+}
+
+// -----
+
+func.func private @callee(i32) -> (i32, i32)
+
+func.func @too_few_forwarded_results(%arg0: i32) {
+ // expected-error @below {{incorrect number of results for callee: expected 2, but got 1}}
+ %status, %res = test.call_and_produce @callee(%arg0) : (i32) -> (i1, i32)
+ return
+}
+
+// -----
+
+// The produced result is not counted: a callee without results is fine even
+// though the call operation has one result.
+
+func.func private @callee(i32)
+
+func.func @produced_result_only(%arg0: i32) {
+ %status = test.call_and_produce @callee(%arg0) : (i32) -> i1
+ return
+}
+
+// -----
+
+// Nothing is verified if the callee cannot be resolved.
+
+func.func @unresolvable_callee(%arg0: i32) {
+ %status, %res = test.call_and_produce @undefined_callee(%arg0) : (i32) -> (i1, f32)
+ return
+}
+
+// -----
+
+// Nothing is verified if the callee does not implement `CallableOpInterface`.
+
+memref.global "private" @not_a_callable : memref<1xi32>
+
+func.func @callee_is_not_callable(%arg0: i32) {
+ %status, %res = test.call_and_produce @not_a_callable(%arg0) : (i32) -> (i1, f32)
+ return
+}
+
+// -----
+
+// By default, all types are compatible across the call boundary.
+
+func.func private @callee(i32) -> i32
+
+func.func @types_need_not_match(%arg0: f32) {
+ %status, %res = test.call_and_produce @callee(%arg0) : (f32) -> (i1, f64)
+ return
+}
+
+// -----
+
+// `test.call_types_compat` implements `areTypesCompatible`: i32 and i64 are
+// interchangeable, everything else must match.
+
+func.func private @callee(i32) -> i32
+
+func.func @compatible_types(%arg0: i64) {
+ %res = test.call_types_compat @callee(%arg0) : (i64) -> i64
+ return
+}
+
+// -----
+
+func.func private @callee(i32) -> i32
+
+func.func @incompatible_operand_type(%arg0: f32) {
+ // expected-error @below {{operand type mismatch: expected operand type 'i32', but provided 'f32' for operand number 0}}
+ %res = test.call_types_compat @callee(%arg0) : (f32) -> i32
+ return
+}
+
+// -----
+
+// The variadic arguments of a call to a variadic callee are consumed operands,
+// not argument operands: `llvm.call` forwards only the declared parameter here,
+// so the call satisfies the 1:1 relationship and passes the shared verifier.
+
+llvm.func @printf(!llvm.ptr, ...) -> i32
+
+llvm.func @variadic_callee(%arg0: !llvm.ptr, %arg1: i32) {
+ %res = llvm.call @printf(%arg0, %arg1) vararg(!llvm.func<i32 (ptr, ...)>) : (!llvm.ptr, i32) -> i32
+ llvm.return
+}
+
+// -----
+
+// The declared parameters of a variadic callee are still checked. Here
+// `var_callee_type` declares one parameter while the callee declares two, so
+// only one operand is an argument operand and the call does not match.
+
+llvm.func @printf(!llvm.ptr, i32, ...) -> i32
+
+llvm.func @inconsistent_var_callee_type(%arg0: !llvm.ptr, %arg1: i32) {
+ // expected-error @below {{incorrect number of operands for callee: expected 2, but got 1}}
+ %res = llvm.call @printf(%arg0, %arg1) vararg(!llvm.func<i32 (ptr, ...)>) : (!llvm.ptr, i32) -> i32
+ llvm.return
+}
+
+// -----
+
+// A call operation that does not model variadic callees claims all its operands
+// as argument operands, so it does not satisfy the 1:1 relationship here.
+
+llvm.func @printf(!llvm.ptr, ...) -> i32
+
+func.func @variadic_callee_not_modelled(%arg0: !llvm.ptr, %arg1: i32) {
+ // expected-error @below {{incorrect number of operands for callee: expected 1, but got 2}}
+ %res = test.call_types_compat @printf(%arg0, %arg1) : (!llvm.ptr, i32) -> i32
+ return
+}
+
+// -----
+
+// `func.call` can never be variadic: the builtin `FunctionType` has no variadic
+// bit, so the number of forwarded operands always matches the callee exactly.
+
+func.func private @callee(i32) -> i32
+
+func.func @func_call_is_never_variadic(%arg0: i32) {
+ // expected-error @below {{incorrect number of operands for callee}}
+ %res = func.call @callee(%arg0, %arg0) : (i32, i32) -> i32
+ return
+}
+
+// -----
+
+func.func private @callee(i32) -> i32
+
+func.func @incompatible_result_type(%arg0: i32) {
+ // expected-error @+3 {{result type mismatch at index 0}}
+ // expected-note @+2 {{op result types: 'f32'}}
+ // expected-note @+1 {{callee result types: 'i32'}}
+ %res = test.call_types_compat @callee(%arg0) : (i32) -> f32
+ return
+}
diff --git a/mlir/test/Transforms/remove-dead-values.mlir b/mlir/test/Transforms/remove-dead-values.mlir
index 390a448060b7f..7ef76838b70ad 100644
--- a/mlir/test/Transforms/remove-dead-values.mlir
+++ b/mlir/test/Transforms/remove-dead-values.mlir
@@ -895,3 +895,27 @@ module @func_with_non_call_users {
}
spirv.EntryPoint "GLCompute" @callee
}
+
+// -----
+
+// A call op may have results that are produced by the call op itself instead of
+// being forwarded from the callee (`%status` below). Such results are not part
+// of the 1:1 relationship between call results and callee results, so they must
+// not shift the indices of the results that are removed.
+//
+// CHECK-LABEL: func.func private @callee_with_dead_return() {
+// CHECK-NEXT: return
+// CHECK-NEXT: }
+// CHECK: func.func @main() -> i1 {
+// CHECK-NEXT: %[[STATUS:.*]] = test.call_and_produce @callee_with_dead_return() : () -> i1
+// CHECK-NEXT: return %[[STATUS]]
+// CHECK-NEXT: }
+// CHECK-CANONICALIZE-LABEL: func.func private @callee_with_dead_return() {
+func.func private @callee_with_dead_return() -> i32 {
+ %c0 = arith.constant 0 : i32
+ return %c0 : i32
+}
+func.func @main() -> i1 {
+ %status, %non_live = test.call_and_produce @callee_with_dead_return() : () -> (i1, i32)
+ return %status : i1
+}
diff --git a/mlir/test/lib/Dialect/Test/TestOpDefs.cpp b/mlir/test/lib/Dialect/Test/TestOpDefs.cpp
index c6470da1b7852..d6310c0d8bfe7 100644
--- a/mlir/test/lib/Dialect/Test/TestOpDefs.cpp
+++ b/mlir/test/lib/Dialect/Test/TestOpDefs.cpp
@@ -1444,6 +1444,70 @@ MutableOperandRange TestCallOnDeviceOp::getArgOperandsMutable() {
return getForwardedOperandsMutable();
}
+//===----------------------------------------------------------------------===//
+// TestCallAndProduceOp
+//===----------------------------------------------------------------------===//
+
+CallInterfaceCallable TestCallAndProduceOp::getCallableForCallee() {
+ return getCallee();
+}
+
+void TestCallAndProduceOp::setCalleeFromCallable(CallInterfaceCallable callee) {
+ setCalleeAttr(cast<SymbolRefAttr>(callee));
+}
+
+Operation::operand_range TestCallAndProduceOp::getArgOperands() {
+ return getForwardedOperands();
+}
+
+MutableOperandRange TestCallAndProduceOp::getArgOperandsMutable() {
+ return getForwardedOperandsMutable();
+}
+
+Operation::result_range TestCallAndProduceOp::getForwardedResults() {
+ // The first result (`produced_status`) is produced by this operation and is
+ // not forwarded from the callee.
+ return getForwarded();
+}
+
+LogicalResult
+TestCallAndProduceOp::verifySymbolUses(SymbolTableCollection &symbolTable) {
+ return call_interface_impl::verifyCallOpInterface(*this, symbolTable);
+}
+
+//===----------------------------------------------------------------------===//
+// TestCallTypesCompatOp
+//===----------------------------------------------------------------------===//
+
+CallInterfaceCallable TestCallTypesCompatOp::getCallableForCallee() {
+ return getCallee();
+}
+
+void TestCallTypesCompatOp::setCalleeFromCallable(
+ CallInterfaceCallable callee) {
+ setCalleeAttr(cast<SymbolRefAttr>(callee));
+}
+
+Operation::operand_range TestCallTypesCompatOp::getArgOperands() {
+ return getForwardedOperands();
+}
+
+MutableOperandRange TestCallTypesCompatOp::getArgOperandsMutable() {
+ return getForwardedOperandsMutable();
+}
+
+bool TestCallTypesCompatOp::areTypesCompatible(Type lhs, Type rhs) {
+ if (lhs == rhs)
+ return true;
+ return (lhs.isInteger(32) && rhs.isInteger(64)) ||
+ (lhs.isInteger(64) && rhs.isInteger(32));
+}
+
+LogicalResult
+TestCallTypesCompatOp::verifySymbolUses(SymbolTableCollection &symbolTable) {
+ return call_interface_impl::verifyCallOpInterface(*this, symbolTable);
+}
+
//===----------------------------------------------------------------------===//
// TestStoreWithARegion
//===----------------------------------------------------------------------===//
diff --git a/mlir/test/lib/Dialect/Test/TestOps.td b/mlir/test/lib/Dialect/Test/TestOps.td
index 3b69e474d76ad..23ae00014fd55 100644
--- a/mlir/test/lib/Dialect/Test/TestOps.td
+++ b/mlir/test/lib/Dialect/Test/TestOps.td
@@ -4000,6 +4000,46 @@ def TestCallOnDeviceOp : TEST_Op<"call_on_device",
"attr-dict `:` functional-type(operands, results)";
}
+// A call-like operation whose first result is produced by the operation itself
+// and is *not* forwarded from the callee. Only the trailing results are
+// forwarded.
+def TestCallAndProduceOp : TEST_Op<"call_and_produce",
+ [DeclareOpInterfaceMethods<CallOpInterface, ["getForwardedResults"]>,
+ DeclareOpInterfaceMethods<SymbolUserOpInterface>]> {
+ let arguments = (ins
+ SymbolRefAttr:$callee,
+ Variadic<AnyType>:$forwarded_operands,
+ OptionalAttr<DictArrayAttr>:$arg_attrs,
+ OptionalAttr<DictArrayAttr>:$res_attrs
+ );
+ let results = (outs
+ I1:$produced_status,
+ Variadic<AnyType>:$forwarded
+ );
+ let assemblyFormat =
+ "$callee `(` $forwarded_operands `)` attr-dict "
+ "`:` functional-type(operands, results)";
+}
+
+// A call-like operation that overrides `areTypesCompatible`: `i32` and `i64` are
+// interchangeable across the call boundary, all other types must match.
+def TestCallTypesCompatOp : TEST_Op<"call_types_compat",
+ [DeclareOpInterfaceMethods<CallOpInterface, ["areTypesCompatible"]>,
+ DeclareOpInterfaceMethods<SymbolUserOpInterface>]> {
+ let arguments = (ins
+ SymbolRefAttr:$callee,
+ Variadic<AnyType>:$forwarded_operands,
+ OptionalAttr<DictArrayAttr>:$arg_attrs,
+ OptionalAttr<DictArrayAttr>:$res_attrs
+ );
+ let results = (outs
+ Variadic<AnyType>:$results
+ );
+ let assemblyFormat =
+ "$callee `(` $forwarded_operands `)` attr-dict "
+ "`:` functional-type(operands, results)";
+}
+
def TestStoreWithARegion : TEST_Op<"store_with_a_region",
[DeclareOpInterfaceMethods<RegionBranchOpInterface, ["getSuccessorInputs"]>,
SingleBlock]> {
More information about the Mlir-commits
mailing list