[Mlir-commits] [mlir] [mlir][Interfaces] `CallOpInterface`: Model forwarded result + improve verification (PR #214724)
llvmlistbot at llvm.org
llvmlistbot at llvm.org
Fri Aug 7 06:20:02 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-mlir-core
Author: Matthias Springer (matthias-springer)
<details>
<summary>Changes</summary>
`CallOpInterface` distinguishes between forwarded operands and consumed operands. This commit adds the concept of "forwarded results", making operands/results symmetric.
Forwarded operands are forwarded from the caller to the callee's block arguments. Forwarded results are forwarded from the callee to the caller's results. All other operands/results are consumed/produced by the call op.
This commit also improves verification:
- The number of forwarded operands and callee arguments must match.
- The number of forwarded results and caller results must match.
- Types must be compatible according to `CallOpInterface::areTypesCompatible`.
Note: `llvm.call` / `llvm.func` support variadic operands. These are not supported by `CallOpInterface` or `CallableOpInterface`. The `CallOpInterface` now no longer reports variadic operands as part of the forwarded operands (`getArgOperands`). The old implementation used to inconsistent: `CallOpInterface` included variadic operands, but `CallableOpInterface` did not.
---
Patch is 49.07 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/214724.diff
23 Files Affected:
- (modified) mlir/docs/Interfaces.md (+21)
- (modified) mlir/include/mlir/Dialect/Async/IR/AsyncOps.td (+5)
- (modified) mlir/include/mlir/Dialect/EmitC/IR/EmitC.td (+5)
- (modified) mlir/include/mlir/Dialect/Func/IR/FuncOps.td (+5)
- (modified) mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td (+10)
- (modified) mlir/include/mlir/Interfaces/CallInterfaces.h (+23)
- (modified) mlir/include/mlir/Interfaces/CallInterfaces.td (+53-4)
- (modified) mlir/include/mlir/Transforms/DialectInlinerInterface.td (+2-2)
- (modified) mlir/lib/Analysis/DataFlow/SparseAnalysis.cpp (+20-2)
- (modified) mlir/lib/Dialect/Async/IR/Async.cpp (+1-22)
- (modified) mlir/lib/Dialect/EmitC/IR/EmitC.cpp (+1-22)
- (modified) mlir/lib/Dialect/Func/IR/FuncOps.cpp (+1-22)
- (modified) mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp (+65-37)
- (modified) mlir/lib/Dialect/LLVMIR/IR/LLVMInterfaces.cpp (+9-3)
- (modified) mlir/lib/Interfaces/CallInterfaces.cpp (+68)
- (modified) mlir/lib/Transforms/RemoveDeadValues.cpp (+14-5)
- (modified) mlir/lib/Transforms/Utils/InliningUtils.cpp (+7-1)
- (modified) mlir/test/Dialect/Func/invalid.mlir (+1-1)
- (modified) mlir/test/Dialect/LLVMIR/invalid.mlir (+15-1)
- (added) mlir/test/Interfaces/CallInterfaces/verify-call-op-interface.mlir (+186)
- (modified) mlir/test/Transforms/remove-dead-values.mlir (+24)
- (modified) mlir/test/lib/Dialect/Test/TestOpDefs.cpp (+64)
- (modified) mlir/test/lib/Dialect/Test/TestOps.td (+40)
``````````diff
diff --git a/mlir/docs/Interfaces.md b/mlir/docs/Interfaces.md
index 2de79eadd0052..9525c3e8316e3 100644
--- a/mlir/docs/Interfaces.md
+++ b/mlir/docs/Interfaces.md
@@ -840,6 +840,12 @@ interface section goes as follows:
* `CallOpInterface` - Used to represent operations like 'call'
- `CallInterfaceCallable getCallableForCallee()`
- `void setCalleeFromCallable(CallInterfaceCallable)`
+ - `Operation::operand_range getArgOperands()`
+ - `MutableOperandRange getArgOperandsMutable()`
+ - `Operation::result_range getForwardedResults()`
+ - `bool areTypesCompatible(Type, Type)`
+ - `Operation * resolveCallable()`
+ - `Operation * resolveCallableInTable(SymbolTableCollection *)`
- `ArrayAttr getArgAttrsAttr()`
- `ArrayAttr getResAttrsAttr()`
- `void setArgAttrsAttr(ArrayAttr)`
@@ -857,6 +863,21 @@ interface section goes as follows:
- `Attribute removeArgAttrsAttr()`
- `Attribute removeResAttrsAttr()`
+A call operation may have operands and results that are not part of the call
+itself; such operands are said to be *consumed* by the operation and such
+results to be *produced* by it. The operands that are passed to the callee
+(`getArgOperands`) and the results that receive the values returned by the
+callee (`getForwardedResults`) are said to be *forwarded*, and they are in a 1:1
+relationship with the arguments and results of the callee: the i-th forwarded
+operand is passed as the i-th argument of the callee and the i-th forwarded
+result receives the i-th value returned by the callee. Corresponding types need
+not be equal; `areTypesCompatible` decides which types may be paired.
+
+Variadic arguments of a call to a variadic callee (if supported by the call
+op / callee op) are consumed operands, not forwarded ones: they do not
+correspond to any argument of the callee, which reads them through dedicated
+operations instead of receiving them as block arguments.
+
##### RegionKindInterfaces
* `RegionKindInterface` - Used to describe the abstract semantics of regions.
diff --git a/mlir/include/mlir/Dialect/Async/IR/AsyncOps.td b/mlir/include/mlir/Dialect/Async/IR/AsyncOps.td
index 722370b8f3e29..39217f8710fb9 100644
--- a/mlir/include/mlir/Dialect/Async/IR/AsyncOps.td
+++ b/mlir/include/mlir/Dialect/Async/IR/AsyncOps.td
@@ -256,6 +256,11 @@ def Async_CallOp : Async_Op<"call",
return getOperandsMutable();
}
+ /// The operand and result types must match the callee exactly.
+ bool areTypesCompatible(::mlir::Type lhs, ::mlir::Type rhs) {
+ return lhs == rhs;
+ }
+
operand_iterator arg_operand_begin() { return operand_begin(); }
operand_iterator arg_operand_end() { return operand_end(); }
diff --git a/mlir/include/mlir/Dialect/EmitC/IR/EmitC.td b/mlir/include/mlir/Dialect/EmitC/IR/EmitC.td
index 49412d1dfb01c..64a748afa1f7c 100644
--- a/mlir/include/mlir/Dialect/EmitC/IR/EmitC.td
+++ b/mlir/include/mlir/Dialect/EmitC/IR/EmitC.td
@@ -789,6 +789,11 @@ def EmitC_CallOp : EmitC_Op<"call",
return getOperandsMutable();
}
+ /// The operand and result types must match the callee exactly.
+ bool areTypesCompatible(::mlir::Type lhs, ::mlir::Type rhs) {
+ return lhs == rhs;
+ }
+
operand_iterator arg_operand_begin() { return operand_begin(); }
operand_iterator arg_operand_end() { return operand_end(); }
diff --git a/mlir/include/mlir/Dialect/Func/IR/FuncOps.td b/mlir/include/mlir/Dialect/Func/IR/FuncOps.td
index a31b860276099..d286ec4f9d78c 100644
--- a/mlir/include/mlir/Dialect/Func/IR/FuncOps.td
+++ b/mlir/include/mlir/Dialect/Func/IR/FuncOps.td
@@ -105,6 +105,11 @@ def CallOp : Func_Op<"call",
return getOperandsMutable();
}
+ /// The operand and result types must match the callee exactly.
+ bool areTypesCompatible(::mlir::Type lhs, ::mlir::Type rhs) {
+ return lhs == rhs;
+ }
+
operand_iterator arg_operand_begin() { return operand_begin(); }
operand_iterator arg_operand_end() { return operand_end(); }
diff --git a/mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td b/mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td
index fc674a7afb55c..7f61e7a9b6583 100644
--- a/mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td
+++ b/mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td
@@ -767,6 +767,11 @@ def LLVM_InvokeOp
let extraClassDeclaration = [{
/// Returns the callee function type.
LLVMFunctionType getCalleeFunctionType();
+
+ /// The argument and result types must match the callee exactly.
+ bool areTypesCompatible(::mlir::Type lhs, ::mlir::Type rhs) {
+ return lhs == rhs;
+ }
}];
}
@@ -902,6 +907,11 @@ def LLVM_CallOp
let extraClassDeclaration = [{
/// Returns the callee function type.
LLVMFunctionType getCalleeFunctionType();
+
+ /// The argument and result types must match the callee exactly.
+ bool areTypesCompatible(::mlir::Type lhs, ::mlir::Type rhs) {
+ return lhs == rhs;
+ }
}];
}
diff --git a/mlir/include/mlir/Interfaces/CallInterfaces.h b/mlir/include/mlir/Interfaces/CallInterfaces.h
index 2bf3a3ca5f8a8..924dee820123e 100644
--- a/mlir/include/mlir/Interfaces/CallInterfaces.h
+++ b/mlir/include/mlir/Interfaces/CallInterfaces.h
@@ -25,6 +25,7 @@ struct CallInterfaceCallable : public PointerUnion<SymbolRefAttr, Value> {
using PointerUnion<SymbolRefAttr, Value>::PointerUnion;
};
+class CallableOpInterface;
class CallOpInterface;
namespace call_interface_impl {
@@ -36,6 +37,28 @@ namespace call_interface_impl {
Operation *resolveCallable(CallOpInterface call,
SymbolTableCollection *symbolTable = nullptr);
+/// Verify that the forwarded operands and results of `call` are in a 1:1
+/// relationship with the given argument and result types of its callee: the
+/// numbers must match and corresponding types must be compatible according to
+/// `CallOpInterface::areTypesCompatible`.
+///
+/// This overload is for call operations whose callee does not necessarily
+/// implement `CallableOpInterface`, e.g., `llvm.call`, which can also call an
+/// `llvm.mlir.alias` or an `llvm.mlir.ifunc`.
+LogicalResult verifyCallOpInterface(CallOpInterface call,
+ TypeRange argumentTypes,
+ TypeRange resultTypes);
+
+/// Same as above, taking the argument and result types from `callable`.
+LogicalResult verifyCallOpInterface(CallOpInterface call,
+ CallableOpInterface callable);
+
+/// Same as above, but resolve the callee of `call` first. Returns success if
+/// the callee cannot be resolved or does not implement `CallableOpInterface`.
+LogicalResult verifyCallOpInterface(CallOpInterface call,
+ SymbolTableCollection &symbolTable);
+LogicalResult verifyCallOpInterface(CallOpInterface call);
+
/// Parse a function or call result list.
///
/// function-result-list ::= function-result-list-parens
diff --git a/mlir/include/mlir/Interfaces/CallInterfaces.td b/mlir/include/mlir/Interfaces/CallInterfaces.td
index 19d3afe8b9243..9b431db7739e1 100644
--- a/mlir/include/mlir/Interfaces/CallInterfaces.td
+++ b/mlir/include/mlir/Interfaces/CallInterfaces.td
@@ -77,6 +77,18 @@ def CallOpInterface : OpInterface<"CallOpInterface",
another. These operations may be traditional direct calls `call @foo`, or
indirect calls to other operations `call_indirect %foo`. An operation that
uses this interface, must *not* also provide the `CallableOpInterface`.
+
+ This interface distinguishes between forwarded operands/results, and
+ consumed/produced operands/results. Forwarded operands/results are in a
+ 1:1 relationship with the arguments/results of the callee: the i-th
+ forwarded operand is passed as the i-th argument of the callee and the i-th
+ forwarded result receives the i-th value returned by the callee.
+ Corresponding types are not required to be equal; `areTypesCompatible`
+ decides which types may be paired.
+
+ Note: This interface does not model variadic operands ("argument pack").
+ Neither does the CallableOpInterface. Variadic operands should be modeled
+ as consumed operands.
}];
let cppNamespace = "::mlir";
@@ -99,17 +111,44 @@ def CallOpInterface : OpInterface<"CallOpInterface",
"void", "setCalleeFromCallable", (ins "::mlir::CallInterfaceCallable":$callee)
>,
InterfaceMethod<[{
- Returns the operands within this call that are used as arguments to the
- callee.
+ Returns the operands of this call that are used as arguments to the
+ callee ("forwarded operands").
+
+ The returned range must be a contiguous sub-range of the operation's
+ operands. The i-th forwarded operand is passed as the i-th argument of
+ the callee; the two types are not required to be equal, but must be
+ compatible according to `areTypesCompatible`. Operands that are not part
+ of the returned range are consumed by the operation itself and are not
+ passed to the callee as arguments.
}],
"::mlir::Operation::operand_range", "getArgOperands"
>,
InterfaceMethod<[{
- Returns the operands within this call that are used as arguments to the
- callee as a mutable range.
+ Returns the operands of this call that are used as arguments to the
+ callee ("forwarded operands") as a mutable range.
+
+ This must be the same range of operands as returned by `getArgOperands`.
}],
"::mlir::MutableOperandRange", "getArgOperandsMutable"
>,
+ InterfaceMethod<[{
+ Returns the results of this call that receive the values returned by the
+ callee ("forwarded results").
+
+ The returned range must be a contiguous sub-range of the operation's
+ results. The i-th forwarded result receives the i-th value returned by
+ the callee; the two types are not required to be equal, but must be
+ compatible according to `areTypesCompatible`. Results that are not part
+ of the returned range are produced by the operation itself and do not
+ originate from the callee.
+
+ By default, all results of the operation are forwarded results.
+ }],
+ "::mlir::Operation::result_range", "getForwardedResults", (ins),
+ /*methodBody=*/[{}], /*defaultImplementation=*/[{
+ return $_op->getResults();
+ }]
+ >,
InterfaceMethod<[{
Resolve the callable operation for given callee to a
CallableOpInterface, or nullptr if a valid callable was not resolved.
@@ -129,6 +168,16 @@ def CallOpInterface : OpInterface<"CallOpInterface",
/*methodBody=*/[{}], /*defaultImplementation=*/[{
return ::mlir::call_interface_impl::resolveCallable($_op);
}]
+ >,
+ InterfaceMethod<[{
+ Compares types across the call boundary for compatibility: the type of
+ a forwarded operand against the type of the corresponding callee
+ argument, and the type of a forwarded result against the type of the
+ corresponding callee result.
+ }],
+ "bool", "areTypesCompatible",
+ (ins "::mlir::Type":$callerType, "::mlir::Type":$calleeType),
+ /*methodBody=*/[{}], /*defaultImplementation=*/[{ return true; }]
>
];
}
diff --git a/mlir/include/mlir/Transforms/DialectInlinerInterface.td b/mlir/include/mlir/Transforms/DialectInlinerInterface.td
index 0975b84179d3c..e675e0165c26a 100644
--- a/mlir/include/mlir/Transforms/DialectInlinerInterface.td
+++ b/mlir/include/mlir/Transforms/DialectInlinerInterface.td
@@ -93,8 +93,8 @@ def DialectInlinerInterface : DialectInterface<"DialectInlinerInterface"> {
InterfaceMethod<[{
Handle the given inlined terminator by replacing it with a new operation
as necessary. This overload is called when the inlined region only
- contains one block. 'valuesToReplace' contains the previously returned
- values of the call site before inlining. These values must be replaced by
+ contains one block. 'valuesToReplace' contains the forwarded results
+ of the call site before inlining. These values must be replaced by
this callback if they had any users (for example for traditional function
calls, these are directly replaced with the operands of the `return`
operation). The given 'op' will be removed by the caller, after this
diff --git a/mlir/lib/Analysis/DataFlow/SparseAnalysis.cpp b/mlir/lib/Analysis/DataFlow/SparseAnalysis.cpp
index 3d9a375bfb2b7..bc61c19832d29 100644
--- a/mlir/lib/Analysis/DataFlow/SparseAnalysis.cpp
+++ b/mlir/lib/Analysis/DataFlow/SparseAnalysis.cpp
@@ -252,9 +252,23 @@ LogicalResult AbstractSparseForwardDataFlowAnalysis::visitCallOperation(
setAllToEntryStates(resultLattices);
return success();
}
+
+ // Only the forwarded results receive the values returned by the callee. Any
+ // other result is produced by the call operation itself and nothing is known
+ // about it here.
+ ResultRange forwardedResults = call.getForwardedResults();
+ unsigned firstForwarded = forwardedResults.empty()
+ ? resultLattices.size()
+ : forwardedResults[0].getResultNumber();
+ setAllToEntryStates(resultLattices.take_front(firstForwarded));
+ setAllToEntryStates(
+ resultLattices.drop_front(firstForwarded + forwardedResults.size()));
+ ArrayRef<AbstractSparseLattice *> forwardedResultLattices =
+ resultLattices.slice(firstForwarded, forwardedResults.size());
+
for (Operation *predecessor : predecessors->getKnownPredecessors())
for (auto &&[operand, resLattice] :
- llvm::zip(predecessor->getOperands(), resultLattices))
+ llvm::zip_equal(predecessor->getOperands(), forwardedResultLattices))
join(resLattice,
*getLatticeElementFor(getProgramPointAfter(call), operand));
return success();
@@ -585,8 +599,12 @@ LogicalResult AbstractSparseBackwardDataFlowAnalysis::visitCallableOperation(
getProgramPointAfter(op), getProgramPointAfter(callable));
if (callsites->allPredecessorsKnown()) {
for (Operation *call : callsites->getKnownPredecessors()) {
+ // Only the forwarded results of the call receive the values returned by
+ // the callee.
+ ResultRange forwardedResults =
+ cast<CallOpInterface>(call).getForwardedResults();
SmallVector<const AbstractSparseLattice *> callResultLattices =
- getLatticeElementsFor(getProgramPointAfter(op), call->getResults());
+ getLatticeElementsFor(getProgramPointAfter(op), forwardedResults);
for (auto [op, result] : llvm::zip(operandLattices, callResultLattices))
meet(op, *result);
}
diff --git a/mlir/lib/Dialect/Async/IR/Async.cpp b/mlir/lib/Dialect/Async/IR/Async.cpp
index c759ba04eefaf..c11400c061584 100644
--- a/mlir/lib/Dialect/Async/IR/Async.cpp
+++ b/mlir/lib/Dialect/Async/IR/Async.cpp
@@ -376,28 +376,7 @@ LogicalResult CallOp::verifySymbolUses(SymbolTableCollection &symbolTable) {
<< "' does not reference a valid async function";
// Verify that the operand and result types match the callee.
- auto fnType = fn.getFunctionType();
- if (fnType.getNumInputs() != getNumOperands())
- return emitOpError("incorrect number of operands for callee");
-
- for (unsigned i = 0, e = fnType.getNumInputs(); i != e; ++i)
- if (getOperand(i).getType() != fnType.getInput(i))
- return emitOpError("operand type mismatch: expected operand type ")
- << fnType.getInput(i) << ", but provided "
- << getOperand(i).getType() << " for operand number " << i;
-
- if (fnType.getNumResults() != getNumResults())
- return emitOpError("incorrect number of results for callee");
-
- for (unsigned i = 0, e = fnType.getNumResults(); i != e; ++i)
- if (getResult(i).getType() != fnType.getResult(i)) {
- auto diag = emitOpError("result type mismatch at index ") << i;
- diag.attachNote() << " op result types: " << getResultTypes();
- diag.attachNote() << "function result types: " << fnType.getResults();
- return diag;
- }
-
- return success();
+ return call_interface_impl::verifyCallOpInterface(*this, fn);
}
FunctionType CallOp::getCalleeType() {
diff --git a/mlir/lib/Dialect/EmitC/IR/EmitC.cpp b/mlir/lib/Dialect/EmitC/IR/EmitC.cpp
index 820c4ad578a9e..f40b6e67b30fc 100644
--- a/mlir/lib/Dialect/EmitC/IR/EmitC.cpp
+++ b/mlir/lib/Dialect/EmitC/IR/EmitC.cpp
@@ -749,28 +749,7 @@ LogicalResult CallOp::verifySymbolUses(SymbolTableCollection &symbolTable) {
<< "' does not reference a valid function";
// Verify that the operand and result types match the callee.
- auto fnType = fn.getFunctionType();
- if (fnType.getNumInputs() != getNumOperands())
- return emitOpError("incorrect number of operands for callee");
-
- for (unsigned i = 0, e = fnType.getNumInputs(); i != e; ++i)
- if (getOperand(i).getType() != fnType.getInput(i))
- return emitOpError("operand type mismatch: expected operand type ")
- << fnType.getInput(i) << ", but provided "
- << getOperand(i).getType() << " for operand number " << i;
-
- if (fnType.getNumResults() != getNumResults())
- return emitOpError("incorrect number of results for callee");
-
- for (unsigned i = 0, e = fnType.getNumResults(); i != e; ++i)
- if (getResult(i).getType() != fnType.getResult(i)) {
- auto diag = emitOpError("result type mismatch at index ") << i;
- diag.attachNote() << " op result types: " << getResultTypes();
- diag.attachNote() << "function result types: " << fnType.getResults();
- return diag;
- }
-
- return success();
+ return call_interface_impl::verifyCallOpInterface(*this, fn);
}
FunctionType CallOp::getCalleeType() {
diff --git a/mlir/lib/Dialect/Func/IR/FuncOps.cpp b/mlir/lib/Dialect/Func/IR/FuncOps.cpp
index 42925ce4fd6a0..54eb8cac737e8 100644
--- a/mlir/lib/Dialect/Func/IR/FuncOps.cpp
+++ b/mlir/lib/Dialect/Func/IR/FuncOps.cpp
@@ -71,28 +71,7 @@ LogicalResult CallOp::verifySymbolUses(SymbolTableCollection &symbolTable) {
<< "' does not reference a valid function";
// Verify that the operand and result types match the callee.
- auto fnType = fn.getFunctionType();
- if (fnType.getNumInputs() != getNumOperands())
- return emitOpError("incorrect number of operands for callee");
-
- for (unsigned i = 0, e = fnType.getNumInputs(); i != e; ++i)
- if (getOperand(i).getType() != fnType.getInput(i))
- return emitOpError("operand type mismatch: expected operand type ")
- << fnType.getInput(i) << ", but provided "
- << getOperand(i).getType() << " for operand number " << i;
-
- if (fnType.getNumResults() != getNumResults())
- return emitOpError("incorrect number of results for callee");
-
- for (unsigned i = 0, e = fnType.getNumResults(); i != e; ++i)
- if (getResult(i).getType() != fnType.getResult(i)) {
- auto diag = emitOpError("result type mismatch at index ") << i;
- diag.attachNote() << " op result types: " << getResultTypes();
- diag.attachNote() << "function result types: " << fnType.getResults();
- return diag;
- }
-
- return success();
+ return call_interface_impl::verifyCallOpInterface(*this, fn);
}
FunctionType CallOp::getCalleeType() {
diff --git a/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp b/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp
index 3bda6f5295747..374ad4a9dcb83 100644
--- a/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp
+++ b/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp
@@ -1125,13 +1125,47 @@ void CallOp::setCalleeFromCallable(CallInterfaceCallable callee) {
return setOperand(0, cast<Value>(callee));
}
+/// Return the number of leading callee operands of `callOp` that the operation
+/// consumes instead of passing them to the callee.
+template <typename OpTy>
+static unsigned getNumConsumedCalleeOperands(OpTy callOp) {
+ // The first operand is the callee if no callee attribute is present.
+ if (callOp.getCallee().has_value())
+ return 0;
+ return 1;
+}
+
+/// Return the operands of `callOp` that are passed to the callee, including the
+/// variadic arguments in case of a call to a variadic callee.
+template <typename OpTy>
+sta...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/214724
More information about the Mlir-commits
mailing list