[llvm-branch-commits] [flang] [mlir] [MLIR][OpenMP] LLVM IR translation of host_eval (PR #116052)
via llvm-branch-commits
llvm-branch-commits at lists.llvm.org
Wed Nov 13 05:41:44 PST 2024
llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-mlir-llvm
Author: Sergio Afonso (skatrak)
<details>
<summary>Changes</summary>
This patch adds support for processing the `host_eval` clause of `omp.target` to populate default and runtime kernel launch attributes. Specifically, these related to the `num_teams`, `thread_limit` and `num_threads` clauses attached to operations nested inside of `omp.target`. As a result, the `thread_limit` clause of `omp.target` is also supported.
The implementation of `initTargetDefaultAttrs()` is intended to reflect clang's own processing of multiple constructs and clauses in order to define a default number of teams and threads to be used as kernel attributes and to populate global variables in the target device module.
One side effect of this change is that it is no longer possible to translate to LLVM IR target device MLIR modules unless they have a supported target triple. This is because the local `getGridValue()` function in the `OpenMPIRBuilder` only works for certain architectures, and it is called whenever the maximum number of threads has not been explicitly defined. This limitation also matches clang.
Support for evaluating the collapsed loop trip count of target SPMD kernels remains unsupported.
---
Patch is 37.90 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/116052.diff
18 Files Affected:
- (modified) flang/test/Integration/OpenMP/target-filtering.f90 (+1-1)
- (modified) flang/test/Lower/OpenMP/function-filtering-2.f90 (+3-3)
- (modified) flang/test/Lower/OpenMP/function-filtering-3.f90 (+3-3)
- (modified) flang/test/Lower/OpenMP/function-filtering.f90 (+3-3)
- (modified) mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp (+229-16)
- (modified) mlir/test/Target/LLVMIR/omptarget-byref-bycopy-generation-device.mlir (+2-2)
- (modified) mlir/test/Target/LLVMIR/omptarget-constant-alloca-raise.mlir (+2-2)
- (modified) mlir/test/Target/LLVMIR/omptarget-constant-indexing-device-region.mlir (+2-2)
- (modified) mlir/test/Target/LLVMIR/omptarget-debug.mlir (+1-1)
- (modified) mlir/test/Target/LLVMIR/omptarget-declare-target-llvm-device.mlir (+1-1)
- (modified) mlir/test/Target/LLVMIR/omptarget-parallel-llvm.mlir (+2-2)
- (modified) mlir/test/Target/LLVMIR/omptarget-region-device-llvm.mlir (+3-3)
- (modified) mlir/test/Target/LLVMIR/omptarget-target-inside-task.mlir (+2-2)
- (added) mlir/test/Target/LLVMIR/openmp-target-launch-device.mlir (+43)
- (added) mlir/test/Target/LLVMIR/openmp-target-launch-host.mlir (+31)
- (modified) mlir/test/Target/LLVMIR/openmp-target-use-device-nested.mlir (+2-2)
- (modified) mlir/test/Target/LLVMIR/openmp-task-target-device.mlir (+1-1)
- (modified) mlir/test/Target/LLVMIR/openmp-todo.mlir (+13-14)
``````````diff
diff --git a/flang/test/Integration/OpenMP/target-filtering.f90 b/flang/test/Integration/OpenMP/target-filtering.f90
index d1ab1b47e580d4..699c1040d91f9c 100644
--- a/flang/test/Integration/OpenMP/target-filtering.f90
+++ b/flang/test/Integration/OpenMP/target-filtering.f90
@@ -7,7 +7,7 @@
!===----------------------------------------------------------------------===!
!RUN: %flang_fc1 -emit-llvm -fopenmp %s -o - | FileCheck %s --check-prefixes HOST,ALL
-!RUN: %flang_fc1 -emit-llvm -fopenmp -fopenmp-is-target-device %s -o - | FileCheck %s --check-prefixes DEVICE,ALL
+!RUN: %flang_fc1 -triple amdgcn-amd-amdhsa -emit-llvm -fopenmp -fopenmp-is-target-device %s -o - | FileCheck %s --check-prefixes DEVICE,ALL
!HOST: define {{.*}}@{{.*}}before{{.*}}(
!DEVICE-NOT: define {{.*}}@before{{.*}}(
diff --git a/flang/test/Lower/OpenMP/function-filtering-2.f90 b/flang/test/Lower/OpenMP/function-filtering-2.f90
index 0c02aa223820e7..a2c5e29cfdcbf6 100644
--- a/flang/test/Lower/OpenMP/function-filtering-2.f90
+++ b/flang/test/Lower/OpenMP/function-filtering-2.f90
@@ -1,9 +1,9 @@
! RUN: %flang_fc1 -fopenmp -fopenmp-version=52 -flang-experimental-hlfir -emit-llvm %s -o - | FileCheck --check-prefixes=LLVM,LLVM-HOST %s
! RUN: %flang_fc1 -fopenmp -fopenmp-version=52 -emit-hlfir %s -o - | FileCheck --check-prefix=MLIR %s
-! RUN: %flang_fc1 -fopenmp -fopenmp-version=52 -fopenmp-is-target-device -flang-experimental-hlfir -emit-llvm %s -o - | FileCheck --check-prefixes=LLVM,LLVM-DEVICE %s
-! RUN: %flang_fc1 -fopenmp -fopenmp-version=52 -fopenmp-is-target-device -emit-hlfir %s -o - | FileCheck --check-prefix=MLIR %s
+! RUN: %flang_fc1 -triple amdgcn-amd-amdhsa -fopenmp -fopenmp-version=52 -fopenmp-is-target-device -flang-experimental-hlfir -emit-llvm %s -o - | FileCheck --check-prefixes=LLVM,LLVM-DEVICE %s
+! RUN: %flang_fc1 -triple amdgcn-amd-amdhsa -fopenmp -fopenmp-version=52 -fopenmp-is-target-device -emit-hlfir %s -o - | FileCheck --check-prefix=MLIR %s
! RUN: bbc -fopenmp -fopenmp-version=52 -emit-hlfir %s -o - | FileCheck --check-prefixes=MLIR-HOST,MLIR-ALL %s
-! RUN: bbc -fopenmp -fopenmp-version=52 -fopenmp-is-target-device -emit-hlfir %s -o - | FileCheck --check-prefixes=MLIR-DEVICE,MLIR-ALL %s
+! RUN: bbc -target amdgcn-amd-amdhsa -fopenmp -fopenmp-version=52 -fopenmp-is-target-device -emit-hlfir %s -o - | FileCheck --check-prefixes=MLIR-DEVICE,MLIR-ALL %s
! MLIR: func.func @{{.*}}implicit_invocation() attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to)>}
! MLIR: return
diff --git a/flang/test/Lower/OpenMP/function-filtering-3.f90 b/flang/test/Lower/OpenMP/function-filtering-3.f90
index a277c06d620669..4754da85de3aa7 100644
--- a/flang/test/Lower/OpenMP/function-filtering-3.f90
+++ b/flang/test/Lower/OpenMP/function-filtering-3.f90
@@ -1,9 +1,9 @@
! RUN: %flang_fc1 -fopenmp -flang-experimental-hlfir -emit-llvm %s -o - | FileCheck --check-prefixes=LLVM-HOST,LLVM-ALL %s
! RUN: %flang_fc1 -fopenmp -emit-hlfir %s -o - | FileCheck --check-prefixes=MLIR-HOST,MLIR-ALL %s
-! RUN: %flang_fc1 -fopenmp -fopenmp-is-target-device -flang-experimental-hlfir -emit-llvm %s -o - | FileCheck --check-prefixes=LLVM-DEVICE,LLVM-ALL %s
-! RUN: %flang_fc1 -fopenmp -fopenmp-is-target-device -emit-hlfir %s -o - | FileCheck --check-prefixes=MLIR-DEVICE,MLIR-ALL %s
+! RUN: %flang_fc1 -triple amdgcn-amd-amdhsa -fopenmp -fopenmp-is-target-device -flang-experimental-hlfir -emit-llvm %s -o - | FileCheck --check-prefixes=LLVM-DEVICE,LLVM-ALL %s
+! RUN: %flang_fc1 -triple amdgcn-amd-amdhsa -fopenmp -fopenmp-is-target-device -emit-hlfir %s -o - | FileCheck --check-prefixes=MLIR-DEVICE,MLIR-ALL %s
! RUN: bbc -fopenmp -emit-hlfir %s -o - | FileCheck --check-prefixes=MLIR-HOST,MLIR-ALL %s
-! RUN: bbc -fopenmp -fopenmp-is-target-device -emit-hlfir %s -o - | FileCheck --check-prefixes=MLIR-DEVICE,MLIR-ALL %s
+! RUN: bbc -target amdgcn-amd-amdhsa -fopenmp -fopenmp-is-target-device -emit-hlfir %s -o - | FileCheck --check-prefixes=MLIR-DEVICE,MLIR-ALL %s
! Check that the correct LLVM IR functions are kept for the host and device
! after running the whole set of translation and transformation passes from
diff --git a/flang/test/Lower/OpenMP/function-filtering.f90 b/flang/test/Lower/OpenMP/function-filtering.f90
index a72822503c3739..9f37e6f028d7a9 100644
--- a/flang/test/Lower/OpenMP/function-filtering.f90
+++ b/flang/test/Lower/OpenMP/function-filtering.f90
@@ -1,9 +1,9 @@
! RUN: %flang_fc1 -fopenmp -fopenmp-version=52 -flang-experimental-hlfir -emit-llvm %s -o - | FileCheck --check-prefixes=LLVM-HOST,LLVM-ALL %s
! RUN: %flang_fc1 -fopenmp -fopenmp-version=52 -emit-hlfir %s -o - | FileCheck --check-prefixes=MLIR-HOST,MLIR-ALL %s
-! RUN: %flang_fc1 -fopenmp -fopenmp-version=52 -fopenmp-is-target-device -flang-experimental-hlfir -emit-llvm %s -o - | FileCheck --check-prefixes=LLVM-DEVICE,LLVM-ALL %s
-! RUN: %flang_fc1 -fopenmp -fopenmp-version=52 -fopenmp-is-target-device -emit-hlfir %s -o - | FileCheck --check-prefixes=MLIR-DEVICE,MLIR-ALL %s
+! RUN: %flang_fc1 -triple amdgcn-amd-amdhsa -fopenmp -fopenmp-version=52 -fopenmp-is-target-device -flang-experimental-hlfir -emit-llvm %s -o - | FileCheck --check-prefixes=LLVM-DEVICE,LLVM-ALL %s
+! RUN: %flang_fc1 -triple amdgcn-amd-amdhsa -fopenmp -fopenmp-version=52 -fopenmp-is-target-device -emit-hlfir %s -o - | FileCheck --check-prefixes=MLIR-DEVICE,MLIR-ALL %s
! RUN: bbc -fopenmp -fopenmp-version=52 -emit-hlfir %s -o - | FileCheck --check-prefixes=MLIR-HOST,MLIR-ALL %s
-! RUN: bbc -fopenmp -fopenmp-version=52 -fopenmp-is-target-device -emit-hlfir %s -o - | FileCheck --check-prefixes=MLIR-DEVICE,MLIR-ALL %s
+! RUN: bbc -target amdgcn-amd-amdhsa -fopenmp -fopenmp-version=52 -fopenmp-is-target-device -emit-hlfir %s -o - | FileCheck --check-prefixes=MLIR-DEVICE,MLIR-ALL %s
! Check that the correct LLVM IR functions are kept for the host and device
! after running the whole set of translation and transformation passes from
diff --git a/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp b/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
index 9bdf3e11496f3a..9de4609ba67486 100644
--- a/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
+++ b/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
@@ -174,10 +174,6 @@ static LogicalResult checkImplementationStatus(Operation &op) {
if (op.getHint())
op.emitWarning("hint clause discarded");
};
- auto checkHostEval = [&todo](auto op, LogicalResult &result) {
- if (!op.getHostEvalVars().empty())
- result = todo("host_eval");
- };
auto checkIf = [&todo](auto op, LogicalResult &result) {
if (op.getIfExpr())
result = todo("if");
@@ -228,10 +224,6 @@ static LogicalResult checkImplementationStatus(Operation &op) {
op.getReductionSyms())
result = todo("reduction");
};
- auto checkThreadLimit = [&todo](auto op, LogicalResult &result) {
- if (op.getThreadLimit())
- result = todo("thread_limit");
- };
auto checkTaskReduction = [&todo](auto op, LogicalResult &result) {
if (!op.getTaskReductionVars().empty() || op.getTaskReductionByref() ||
op.getTaskReductionSyms())
@@ -295,7 +287,16 @@ static LogicalResult checkImplementationStatus(Operation &op) {
checkAllocate(op, result);
checkDevice(op, result);
checkHasDeviceAddr(op, result);
- checkHostEval(op, result);
+
+ // Host evaluated clauses are supported, except for target SPMD loop
+ // bounds.
+ for (BlockArgument arg :
+ cast<omp::BlockArgOpenMPOpInterface>(*op).getHostEvalBlockArgs())
+ for (Operation *user : arg.getUsers())
+ if (isa<omp::LoopNestOp>(user))
+ result = op.emitError("not yet implemented: host evaluation of "
+ "loop bounds in omp.target operation");
+
checkIf(op, result);
checkInReduction(op, result);
checkIsDevicePtr(op, result);
@@ -316,7 +317,6 @@ static LogicalResult checkImplementationStatus(Operation &op) {
"structures in omp.target operation");
}
}
- checkThreadLimit(op, result);
})
.Default([](Operation &) {
// Assume all clauses for an operation can be translated unless they are
@@ -3800,6 +3800,201 @@ createDeviceArgumentAccessor(MapInfoData &mapData, llvm::Argument &arg,
return builder.saveIP();
}
+/// Follow uses of `host_eval`-defined block arguments of the given `omp.target`
+/// operation and populate output variables with their corresponding host value
+/// (i.e. operand evaluated outside of the target region), based on their uses
+/// inside of the target region.
+///
+/// Loop bounds and steps are only optionally populated, if output vectors are
+/// provided.
+static void extractHostEvalClauses(omp::TargetOp targetOp, Value &numThreads,
+ Value &numTeamsLower, Value &numTeamsUpper,
+ Value &threadLimit) {
+ auto blockArgIface = llvm::cast<omp::BlockArgOpenMPOpInterface>(*targetOp);
+ for (auto item : llvm::zip_equal(targetOp.getHostEvalVars(),
+ blockArgIface.getHostEvalBlockArgs())) {
+ Value hostEvalVar = std::get<0>(item), blockArg = std::get<1>(item);
+
+ for (Operation *user : blockArg.getUsers()) {
+ llvm::TypeSwitch<Operation *>(user)
+ .Case([&](omp::TeamsOp teamsOp) {
+ if (teamsOp.getNumTeamsLower() == blockArg)
+ numTeamsLower = hostEvalVar;
+ else if (teamsOp.getNumTeamsUpper() == blockArg)
+ numTeamsUpper = hostEvalVar;
+ else if (teamsOp.getThreadLimit() == blockArg)
+ threadLimit = hostEvalVar;
+ else
+ llvm_unreachable("unsupported host_eval use");
+ })
+ .Case([&](omp::ParallelOp parallelOp) {
+ if (parallelOp.getNumThreads() == blockArg)
+ numThreads = hostEvalVar;
+ else
+ llvm_unreachable("unsupported host_eval use");
+ })
+ .Case([&](omp::LoopNestOp loopOp) {
+ // TODO: Extract bounds and step values.
+ })
+ .Default([](Operation *) {
+ llvm_unreachable("unsupported host_eval use");
+ });
+ }
+ }
+}
+
+/// If \p op is of the given type parameter, return it casted to that type.
+/// Otherwise, if its immediate parent operation (or some other higher-level
+/// parent, if \p immediateParent is false) is of that type, return that parent
+/// casted to the given type.
+///
+/// If \p op is \c null or neither it or its parent(s) are of the specified
+/// type, return a \c null operation.
+template <typename OpTy>
+static OpTy castOrGetParentOfType(Operation *op, bool immediateParent = false) {
+ if (!op)
+ return OpTy();
+
+ if (OpTy casted = dyn_cast<OpTy>(op))
+ return casted;
+
+ if (immediateParent)
+ return dyn_cast_if_present<OpTy>(op->getParentOp());
+
+ return op->getParentOfType<OpTy>();
+}
+
+/// Populate default `MinTeams`, `MaxTeams` and `MaxThreads` to their default
+/// values as stated by the corresponding clauses, if constant.
+///
+/// These default values must be set before the creation of the outlined LLVM
+/// function for the target region, so that they can be used to initialize the
+/// corresponding global `ConfigurationEnvironmentTy` structure.
+static void
+initTargetDefaultAttrs(omp::TargetOp targetOp,
+ llvm::OpenMPIRBuilder::TargetKernelDefaultAttrs &attrs,
+ bool isTargetDevice) {
+ Value hostNumThreads, hostNumTeamsLower, hostNumTeamsUpper, hostThreadLimit;
+ extractHostEvalClauses(targetOp, hostNumThreads, hostNumTeamsLower,
+ hostNumTeamsUpper, hostThreadLimit);
+
+ // TODO: Handle constant 'if' clauses.
+ Operation *capturedOp = targetOp.getInnermostCapturedOmpOp();
+
+ auto extractConstInteger = [](Value value) -> std::optional<int64_t> {
+ if (auto constOp =
+ dyn_cast_if_present<LLVM::ConstantOp>(value.getDefiningOp()))
+ if (auto constAttr = dyn_cast<IntegerAttr>(constOp.getValue()))
+ return constAttr.getInt();
+
+ return std::nullopt;
+ };
+
+ // Handle clauses impacting the number of teams.
+
+ int32_t minTeamsVal = 1, maxTeamsVal = -1;
+ if (castOrGetParentOfType<omp::TeamsOp>(capturedOp)) {
+ // TODO: Use `hostNumTeamsLower` to initialize `minTeamsVal`. For now, match
+ // clang and set min and max to the same value.
+ if (hostNumTeamsUpper) {
+ if (auto val = extractConstInteger(hostNumTeamsUpper))
+ minTeamsVal = maxTeamsVal = *val;
+ } else {
+ minTeamsVal = maxTeamsVal = 0;
+ }
+ } else if (castOrGetParentOfType<omp::ParallelOp>(capturedOp,
+ /*immediateParent=*/true) ||
+ castOrGetParentOfType<omp::SimdOp>(capturedOp,
+ /*immediateParent=*/true)) {
+ minTeamsVal = maxTeamsVal = 1;
+ } else {
+ minTeamsVal = maxTeamsVal = -1;
+ }
+
+ // Handle clauses impacting the number of threads.
+
+ auto setMaxValueFromClause = [&extractConstInteger](Value clauseValue,
+ int32_t &result) {
+ if (!clauseValue)
+ return;
+
+ if (auto val = extractConstInteger(clauseValue))
+ result = *val;
+
+ // Found an applicable clause, so it's not undefined. Mark as unknown
+ // because it's not constant.
+ if (result < 0)
+ result = 0;
+ };
+
+ // Extract 'thread_limit' clause from 'target' and 'teams' directives.
+ int32_t targetThreadLimitVal = -1, teamsThreadLimitVal = -1;
+ setMaxValueFromClause(targetOp.getThreadLimit(), targetThreadLimitVal);
+ setMaxValueFromClause(hostThreadLimit, teamsThreadLimitVal);
+
+ // Extract 'max_threads' clause from 'parallel' or set to 1 if it's SIMD.
+ int32_t maxThreadsVal = -1;
+ if (auto parallelOp = castOrGetParentOfType<omp::ParallelOp>(capturedOp))
+ setMaxValueFromClause(hostNumThreads, maxThreadsVal);
+ else if (castOrGetParentOfType<omp::SimdOp>(capturedOp,
+ /*immediateParent=*/true))
+ maxThreadsVal = 1;
+
+ // For max values, < 0 means unset, == 0 means set but unknown. Select the
+ // minimum value between 'max_threads' and 'thread_limit' clauses that were
+ // set.
+ int32_t combinedMaxThreadsVal = targetThreadLimitVal;
+ if (combinedMaxThreadsVal < 0 ||
+ (teamsThreadLimitVal >= 0 && teamsThreadLimitVal < combinedMaxThreadsVal))
+ combinedMaxThreadsVal = teamsThreadLimitVal;
+
+ if (combinedMaxThreadsVal < 0 ||
+ (maxThreadsVal >= 0 && maxThreadsVal < combinedMaxThreadsVal))
+ combinedMaxThreadsVal = maxThreadsVal;
+
+ // Update kernel bounds structure for the `OpenMPIRBuilder` to use.
+ attrs.MinTeams = minTeamsVal;
+ attrs.MaxTeams.front() = maxTeamsVal;
+ attrs.MinThreads = 1;
+ attrs.MaxThreads.front() = combinedMaxThreadsVal;
+}
+
+/// Gather LLVM runtime values for all clauses evaluated in the host that are
+/// passed to the kernel invocation.
+///
+/// This function must be called only when compiling for the host. Also, it will
+/// only provide correct results if it's called after the body of \c targetOp
+/// has been fully generated.
+static void
+initTargetRuntimeAttrs(llvm::IRBuilderBase &builder,
+ LLVM::ModuleTranslation &moduleTranslation,
+ omp::TargetOp targetOp,
+ llvm::OpenMPIRBuilder::TargetKernelRuntimeAttrs &attrs) {
+ Value numThreads, numTeamsLower, numTeamsUpper, teamsThreadLimit;
+ extractHostEvalClauses(targetOp, numThreads, numTeamsLower, numTeamsUpper,
+ teamsThreadLimit);
+
+ // TODO: Handle constant 'if' clauses.
+ if (Value targetThreadLimit = targetOp.getThreadLimit())
+ attrs.TargetThreadLimit.front() =
+ moduleTranslation.lookupValue(targetThreadLimit);
+
+ if (numTeamsLower)
+ attrs.MinTeams = moduleTranslation.lookupValue(numTeamsLower);
+
+ if (numTeamsUpper)
+ attrs.MaxTeams.front() = moduleTranslation.lookupValue(numTeamsUpper);
+
+ if (teamsThreadLimit)
+ attrs.TeamsThreadLimit.front() =
+ moduleTranslation.lookupValue(teamsThreadLimit);
+
+ if (numThreads)
+ attrs.MaxThreads = moduleTranslation.lookupValue(numThreads);
+
+ // TODO: Populate attrs.LoopTripCount if it is target SPMD.
+}
+
static LogicalResult
convertOmpTarget(Operation &opInst, llvm::IRBuilderBase &builder,
LLVM::ModuleTranslation &moduleTranslation) {
@@ -3809,12 +4004,13 @@ convertOmpTarget(Operation &opInst, llvm::IRBuilderBase &builder,
llvm::OpenMPIRBuilder *ompBuilder = moduleTranslation.getOpenMPBuilder();
bool isTargetDevice = ompBuilder->Config.isTargetDevice();
+
auto parentFn = opInst.getParentOfType<LLVM::LLVMFuncOp>();
+ auto blockIface = cast<omp::BlockArgOpenMPOpInterface>(opInst);
auto &targetRegion = targetOp.getRegion();
DataLayout dl = DataLayout(opInst.getParentOfType<ModuleOp>());
SmallVector<Value> mapVars = targetOp.getMapVars();
- ArrayRef<BlockArgument> mapBlockArgs =
- cast<omp::BlockArgOpenMPOpInterface>(opInst).getMapBlockArgs();
+ ArrayRef<BlockArgument> mapBlockArgs = blockIface.getMapBlockArgs();
llvm::Function *llvmOutlinedFn = nullptr;
// TODO: It can also be false if a compile-time constant `false` IF clause is
@@ -3857,7 +4053,7 @@ convertOmpTarget(Operation &opInst, llvm::IRBuilderBase &builder,
OperandRange privateVars = targetOp.getPrivateVars();
std::optional<ArrayAttr> privateSyms = targetOp.getPrivateSyms();
MutableArrayRef<BlockArgument> privateBlockArgs =
- cast<omp::BlockArgOpenMPOpInterface>(opInst).getPrivateBlockArgs();
+ blockIface.getPrivateBlockArgs();
for (auto [privVar, privatizerNameAttr, privBlockArg] :
llvm::zip_equal(privateVars, *privateSyms, privateBlockArgs)) {
@@ -3936,13 +4132,30 @@ convertOmpTarget(Operation &opInst, llvm::IRBuilderBase &builder,
allocaIP, codeGenIP);
};
- // TODO: Populate default and runtime attributes based on the construct and
- // clauses.
+ llvm::SmallVector<llvm::Value *, 4> kernelInput;
llvm::OpenMPIRBuilder::TargetKernelDefaultAttrs defaultAttrs = {
/*MaxTeams=*/{-1}, /*MinTeams=*/0, /*MaxThreads=*/{0}, /*MinThreads=*/0};
+ initTargetDefaultAttrs(targetOp, defaultAttrs, isTargetDevice);
+
+ // Collect host-evaluated values needed to properly launch the kernel from the
+ // host.
llvm::OpenMPIRBuilder::TargetKernelRuntimeAttrs runtimeAttrs;
+ if (!isTargetDevice)
+ initTargetRuntimeAttrs(builder, moduleTranslation, targetOp, runtimeAttrs);
+
+ // Pass host-evaluated values as parameters to the kernel / host fallback,
+ // except if they are constants. In any case, map the MLIR block argument to
+ // the corresponding LLVM values.
+ SmallVector<Value> hostEvalVars = targetOp.getHostEvalVars();
+ ArrayRef<BlockArgument> hostEvalBlockArgs = blockIface.getHostEvalBlockArgs();
+ for (auto [arg, var] : llvm::zip_equal(hostEvalBlockArgs, hostEvalVars)) {
+ llvm::Value *value = moduleTranslation.lookupValue(var);
+ moduleTranslation.mapValue(arg, value);
+
+ if (!llvm::isa<llvm::Constant>(value))
+ kernelInput.push_back(value);
+ }
- llvm::SmallVector<llvm::Value *, 4> kernelInput;
for (size_t i = 0; i < mapVars.size(); ++i) {
// declare target arguments are not passed to kernels as arguments
// TODO: We currently do not handle cases where a member is explicitly
diff --git a/mlir/test/Target/LLVMIR/omptarget-byref-bycopy-generation-device.mlir b/mlir/test/Target/LLVMIR/omptarget-byref-bycopy-generation-device.mlir
index 9549de1258efc5..89d8db5439bff6 100644
--- a/mlir/test/Target/LLVMIR/omptarget-byref-bycopy-generation-device.mlir
+++ b/mlir/test/Target/LLVMIR/omptarget-byref-bycopy-generation-device.mlir
@@ -1,6 +1,6 @@
// RUN: mlir-translate -mlir-to-llvmir %s | FileCheck %s
-module attributes {omp.is_target_device = true} {
+module attributes {llvm.target_triple = "amdgcn-amd-amdhsa", omp.is_target_device = true} {
llvm.func @_QQmain() attributes {fir.bindc_name = "main"} {
%0 = llvm.mlir.addressof ...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/116052
More information about the llvm-branch-commits
mailing list