[Openmp-commits] [openmp] [Flang][OpenMP][MLIR] Initial array section mapping MLIR -> LLVM-IR lowering utilising omp.bounds (PR #68689)
via Openmp-commits
openmp-commits at lists.llvm.org
Tue Oct 10 04:30:44 PDT 2023
llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-mlir-openmp
Author: None (agozillon)
<details>
<summary>Changes</summary>
This patch seeks to add initial lowering of OpenMP array sections within target region map clauses from MLIR to LLVM IR.
This patch seeks to support fixed sized arrays initially, before looking toward assumed size and shaped arrays.
Although, assumed size works in some fashion (dummy arguments) with some minor alterations to the OMPEarlyOutliner, so it is possible changes made in the IsolatedFromAbove series may allow this to work with no further required patches.
It utilises the generated omp.bounds to calculate the size of the mapped OpenMP array (both for sectioned and un-sectioned arrays) as well as the offset to be passed to the kernel argument structure.
Alongside these changes some refactoring of how map data is handled is attempted, using a new MapData structure to keep track of information utilised in the lowering of mapped values.
The initial addition of a more complex createDeviceArgumentAccessor that utilises capture kinds similarly to (and loosely based on) Clang to generate different kernel argument accesses is also added.
A similar function for altering how the kernel argument is passed to the kernel argument structure on the host is also utilised (createAlteredByCaptureMap), which allows modification of the pointer/basePointer based on their capture (and bounds information). It's of note ByRef, is the default for explicit mappings and ByCopy will be the default for implicit captures, so the former is currently tested in this patch and the latter is not for the moment.
---
Patch is 49.24 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/68689.diff
8 Files Affected:
- (modified) mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp (+372-140)
- (added) mlir/test/Target/LLVMIR/omptarget-array-sectioning-device.mlir (+78)
- (added) mlir/test/Target/LLVMIR/omptarget-array-sectioning-host.mlir (+80)
- (modified) mlir/test/Target/LLVMIR/omptarget-llvm.mlir (+43-18)
- (added) openmp/libomptarget/test/offloading/fortran/basic-target-region-1D-array-section.f90 (+27)
- (added) openmp/libomptarget/test/offloading/fortran/basic-target-region-3D-array-section.f90 (+39)
- (added) openmp/libomptarget/test/offloading/fortran/basic-target-region-3D-array.f90 (+45)
- (added) openmp/libomptarget/test/offloading/fortran/basic-target-region-array.f90 (+29)
``````````diff
diff --git a/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp b/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
index 1ec3bb8e7562a9e..e2a408ffd810e2e 100644
--- a/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
+++ b/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
@@ -1528,13 +1528,6 @@ convertOmpThreadprivate(Operation &opInst, llvm::IRBuilderBase &builder,
return success();
}
-int64_t getSizeInBytes(DataLayout &DL, const Type &type) {
- if (isa<LLVM::LLVMPointerType>(type))
- return DL.getTypeSize(cast<LLVM::LLVMPointerType>(type).getElementType());
-
- return 0;
-}
-
static llvm::OffloadEntriesInfoManager::OMPTargetDeviceClauseKind
convertToDeviceClauseKind(mlir::omp::DeclareTargetDeviceType deviceClause) {
switch (deviceClause) {
@@ -1629,13 +1622,153 @@ getRefPtrIfDeclareTarget(mlir::Value value,
return nullptr;
}
+// A small helper structure to contain data gathered
+// for map lowering and coalese it into one area and
+// avoiding extra computations such as searches in the
+// llvm module for lowered mapped varibles or checking
+// if something is declare target (and retrieving the
+// value).
+struct MapData {
+ bool isDeclareTarget = false;
+ mlir::Operation *mapClause;
+ llvm::Value *basePointer;
+ llvm::Value *pointer;
+ llvm::Value *kernelValue;
+ llvm::Type *underlyingType;
+ llvm::Value *sizeInBytes;
+};
+
+uint64_t getArrayElementSizeInBits(LLVM::LLVMArrayType arrTy, DataLayout &dl) {
+ if (auto nestedArrTy = llvm::dyn_cast_if_present<LLVM::LLVMArrayType>(
+ arrTy.getElementType()))
+ return getArrayElementSizeInBits(nestedArrTy, dl);
+ return dl.getTypeSizeInBits(arrTy.getElementType());
+}
+
+// This function calculates the size to be offloaded for a specified type, given
+// its associated map clause (which can contain bounds information which affects
+// the total size), this size is calculated based on the underlying element type
+// e.g. given a 1-D array of ints, we will calculate the size from the integer
+// type * number of elements in the array. This size can be used in other
+// calculations but is ultimately used as an argument to the OpenMP runtimes
+// kernel argument structure which is generated through the combinedInfo data
+// structures.
+// This function is somewhat equivalent to Clang's getExprTypeSize inside of
+// CGOpenMPRuntime.cpp.
+llvm::Value *getSizeInBytes(DataLayout &dl, const mlir::Type &type,
+ Operation *clauseOp, llvm::IRBuilderBase &builder,
+ LLVM::ModuleTranslation &moduleTranslation) {
+ // utilising getTypeSizeInBits instead of getTypeSize as getTypeSize gives
+ // the size in inconsistent byte or bit format.
+ uint64_t underlyingTypeSzInBits = dl.getTypeSizeInBits(type);
+ if (auto ptrTy = llvm::dyn_cast_if_present<LLVM::LLVMPointerType>(type)) {
+ underlyingTypeSzInBits = dl.getTypeSizeInBits(ptrTy.getElementType());
+ if (auto arrTy = llvm::dyn_cast_if_present<LLVM::LLVMArrayType>(
+ ptrTy.getElementType())) {
+ underlyingTypeSzInBits = getArrayElementSizeInBits(arrTy, dl);
+ }
+ }
+
+ if (auto memberClause =
+ mlir::dyn_cast_if_present<mlir::omp::MapInfoOp>(clauseOp)) {
+ // This calculates the size to transfer based on bounds and the underlying
+ // element type, provided bounds have been specified (Fortran
+ // pointers/allocatables/target and arrays that have sections specified fall
+ // into this as well).
+ if (!memberClause.getBounds().empty()) {
+ llvm::Value *elementCount = nullptr;
+ for (auto bounds : memberClause.getBounds()) {
+ if (auto boundOp = mlir::dyn_cast_if_present<mlir::omp::DataBoundsOp>(
+ bounds.getDefiningOp())) {
+
+ if (!elementCount) {
+ elementCount = builder.CreateAdd(
+ builder.CreateSub(
+ moduleTranslation.lookupValue(boundOp.getUpperBound()),
+ moduleTranslation.lookupValue(boundOp.getLowerBound())),
+ builder.getInt64(1));
+ } else {
+ elementCount = builder.CreateMul(
+ elementCount,
+ builder.CreateAdd(
+ builder.CreateSub(
+ moduleTranslation.lookupValue(boundOp.getUpperBound()),
+ moduleTranslation.lookupValue(boundOp.getLowerBound())),
+ builder.getInt64(1)));
+ }
+ }
+ }
+
+ // The size in bytes x number of elements, the sizeInBytes stored is
+ // the underyling types size, e.g. if ptr<i32>, it'll be the i32's
+ // size, so we do some on the fly runtime math to get the size in
+ // bytes from the extent (ub - lb) * sizeInBytes. NOTE: This may need
+ // some adjustment for members with more complex types.
+ return builder.CreateMul(elementCount,
+ builder.getInt64(underlyingTypeSzInBits / 8));
+ }
+ }
+
+ return builder.getInt64(underlyingTypeSzInBits / 8);
+}
+
+// Get the underlying LLVM type, this will bypass the pointer and
+// access the underlying type. Which bypasses llvm's opaque pointers
+// to get the underlying type via MLIR.
+llvm::Type *getLLVMIRType(mlir::Type inputType,
+ LLVM::ModuleTranslation &moduleTranslation) {
+ llvm::Type *type = moduleTranslation.convertType(inputType);
+ if (auto pointerType =
+ llvm::dyn_cast<mlir::omp::PointerLikeType>(inputType)) {
+ if (auto eleType = pointerType.getElementType()) {
+ type = moduleTranslation.convertType(eleType);
+ }
+ }
+ return type;
+}
+
+void collectMapDataFromMapOperands(llvm::SmallVectorImpl<MapData> &mapData,
+ llvm::SmallVectorImpl<Value> &mapOperands,
+ LLVM::ModuleTranslation &moduleTranslation,
+ DataLayout &dl,
+ llvm::IRBuilderBase &builder) {
+ auto getMapData = [&](Value val, Operation *clauseOp) -> MapData {
+ MapData map;
+ if (llvm::Value *refPtr = getRefPtrIfDeclareTarget(
+ val, moduleTranslation)) { // declare target
+ map.kernelValue = moduleTranslation.lookupValue(val);
+ map.isDeclareTarget = true;
+ map.basePointer = refPtr;
+ map.pointer = map.kernelValue;
+ } else { // regular mapped variable
+ map.kernelValue = moduleTranslation.lookupValue(val);
+ map.isDeclareTarget = false;
+ map.basePointer = map.kernelValue;
+ map.pointer = map.kernelValue;
+ }
+
+ map.sizeInBytes =
+ getSizeInBytes(dl, val.getType(), clauseOp, builder, moduleTranslation);
+ map.underlyingType = getLLVMIRType(val.getType(), moduleTranslation);
+ map.mapClause = clauseOp;
+
+ return map;
+ };
+
+ for (mlir::Value mapValue : mapOperands) {
+ if (auto mapOp = mlir::dyn_cast_if_present<mlir::omp::MapInfoOp>(
+ mapValue.getDefiningOp())) {
+ mapData.push_back(getMapData(mapOp.getVarPtr(), mapOp));
+ }
+ }
+}
+
// Generate all map related information and fill the combinedInfo.
static void genMapInfos(llvm::IRBuilderBase &builder,
LLVM::ModuleTranslation &moduleTranslation,
- DataLayout &DL,
+ DataLayout &dl,
llvm::OpenMPIRBuilder::MapInfosTy &combinedInfo,
- const SmallVector<Value> &mapOperands,
- const ArrayAttr &mapTypes,
+ llvm::SmallVectorImpl<MapData> &mapData,
const SmallVector<Value> &devPtrOperands = {},
const SmallVector<Value> &devAddrOperands = {},
bool isTargetParams = false) {
@@ -1650,58 +1783,39 @@ static void genMapInfos(llvm::IRBuilderBase &builder,
combinedInfo.Names.clear();
};
- auto findMapInfo = [&combinedInfo](llvm::Value *val, unsigned &index) {
- index = 0;
- for (auto basePtr : combinedInfo.BasePointers) {
- if (basePtr == val)
- return true;
- index++;
- }
- return false;
- };
-
- unsigned index = 0;
- for (const auto &mapOp : mapOperands) {
- // Unlike dev_ptr and dev_addr operands these map operands point
- // to a map entry operation which contains further information
- // on the variable being mapped and how it should be mapped.
- auto mapInfoOp =
- mlir::dyn_cast<mlir::omp::MapInfoOp>(mapOp.getDefiningOp());
-
- // TODO: Only LLVMPointerTypes are handled.
- if (!mapInfoOp.getType().isa<LLVM::LLVMPointerType>())
- return fail();
-
- llvm::Value *mapOpValue =
- moduleTranslation.lookupValue(mapInfoOp.getVarPtr());
-
- llvm::Value *refPtr =
- getRefPtrIfDeclareTarget(mapInfoOp.getVarPtr(), moduleTranslation);
-
- combinedInfo.BasePointers.emplace_back(refPtr ? refPtr : mapOpValue);
- combinedInfo.Pointers.emplace_back(mapOpValue);
- combinedInfo.DevicePointers.emplace_back(
- llvm::OpenMPIRBuilder::DeviceInfoTy::None);
- combinedInfo.Names.emplace_back(LLVM::createMappingInformation(
- mapInfoOp.getVarPtr().getLoc(), *ompBuilder));
-
- auto mapFlag = llvm::omp::OpenMPOffloadMappingFlags(
- mapTypes[index].cast<IntegerAttr>().getUInt());
+ for (MapData &mapValue : mapData) {
+ auto mapOp = mlir::dyn_cast<mlir::omp::MapInfoOp>(mapValue.mapClause);
+ auto mapFlag =
+ llvm::omp::OpenMPOffloadMappingFlags(mapOp.getMapType().value());
// Declare Target Mappings are excluded from being marked as
// OMP_MAP_TARGET_PARAM as they are not passed as parameters, they're marked
// with OMP_MAP_PTR_AND_OBJ instead.
- if (refPtr)
+ if (mapValue.isDeclareTarget)
mapFlag |= llvm::omp::OpenMPOffloadMappingFlags::OMP_MAP_PTR_AND_OBJ;
else if (isTargetParams)
mapFlag |= llvm::omp::OpenMPOffloadMappingFlags::OMP_MAP_TARGET_PARAM;
+ combinedInfo.BasePointers.emplace_back(mapValue.basePointer);
+ combinedInfo.Pointers.emplace_back(mapValue.pointer);
+ combinedInfo.DevicePointers.emplace_back(
+ llvm::OpenMPIRBuilder::DeviceInfoTy::None);
+ combinedInfo.Names.emplace_back(
+ LLVM::createMappingInformation(mapOp.getLoc(), *ompBuilder));
combinedInfo.Types.emplace_back(mapFlag);
- combinedInfo.Sizes.emplace_back(
- builder.getInt64(getSizeInBytes(DL, mapInfoOp.getVarPtr().getType())));
- index++;
+ combinedInfo.Sizes.emplace_back(mapValue.sizeInBytes);
}
+ auto findMapInfo = [&combinedInfo](llvm::Value *val, unsigned &index) {
+ index = 0;
+ for (llvm::Value *basePtr : combinedInfo.BasePointers) {
+ if (basePtr == val)
+ return true;
+ index++;
+ }
+ return false;
+ };
+
auto addDevInfos = [&, fail](auto devOperands, auto devOpType) -> void {
for (const auto &devOp : devOperands) {
// TODO: Only LLVMPointerTypes are handled.
@@ -1747,19 +1861,6 @@ convertOmpTargetData(Operation *op, llvm::IRBuilderBase &builder,
llvm::OpenMPIRBuilder *ompBuilder = moduleTranslation.getOpenMPBuilder();
- auto getMapTypes = [](mlir::OperandRange mapOperands,
- mlir::MLIRContext *ctx) {
- SmallVector<mlir::Attribute> mapTypes;
- for (auto mapValue : mapOperands) {
- if (mapValue.getDefiningOp()) {
- auto mapOp =
- mlir::dyn_cast<mlir::omp::MapInfoOp>(mapValue.getDefiningOp());
- mapTypes.push_back(mapOp.getMapTypeAttr());
- }
- }
- return mlir::ArrayAttr::get(ctx, mapTypes);
- };
-
LogicalResult result =
llvm::TypeSwitch<Operation *, LogicalResult>(op)
.Case([&](omp::DataOp dataOp) {
@@ -1773,8 +1874,6 @@ convertOmpTargetData(Operation *op, llvm::IRBuilderBase &builder,
deviceID = intAttr.getInt();
mapOperands = dataOp.getMapOperands();
- mapTypes =
- getMapTypes(dataOp.getMapOperands(), dataOp->getContext());
useDevPtrOperands = dataOp.getUseDevicePtr();
useDevAddrOperands = dataOp.getUseDeviceAddr();
return success();
@@ -1793,8 +1892,6 @@ convertOmpTargetData(Operation *op, llvm::IRBuilderBase &builder,
deviceID = intAttr.getInt();
RTLFn = llvm::omp::OMPRTL___tgt_target_data_begin_mapper;
mapOperands = enterDataOp.getMapOperands();
- mapTypes = getMapTypes(enterDataOp.getMapOperands(),
- enterDataOp->getContext());
return success();
})
.Case([&](omp::ExitDataOp exitDataOp) {
@@ -1812,8 +1909,6 @@ convertOmpTargetData(Operation *op, llvm::IRBuilderBase &builder,
RTLFn = llvm::omp::OMPRTL___tgt_target_data_end_mapper;
mapOperands = exitDataOp.getMapOperands();
- mapTypes = getMapTypes(exitDataOp.getMapOperands(),
- exitDataOp->getContext());
return success();
})
.Default([&](Operation *op) {
@@ -1826,17 +1921,20 @@ convertOmpTargetData(Operation *op, llvm::IRBuilderBase &builder,
using InsertPointTy = llvm::OpenMPIRBuilder::InsertPointTy;
+ llvm::SmallVector<MapData> mapData;
+ collectMapDataFromMapOperands(mapData, mapOperands, moduleTranslation, DL,
+ builder);
+
// Fill up the arrays with all the mapped variables.
llvm::OpenMPIRBuilder::MapInfosTy combinedInfo;
auto genMapInfoCB =
[&](InsertPointTy codeGenIP) -> llvm::OpenMPIRBuilder::MapInfosTy & {
builder.restoreIP(codeGenIP);
- if (auto DataOp = dyn_cast<omp::DataOp>(op)) {
- genMapInfos(builder, moduleTranslation, DL, combinedInfo, mapOperands,
- mapTypes, useDevPtrOperands, useDevAddrOperands);
+ if (auto dataOp = dyn_cast<omp::DataOp>(op)) {
+ genMapInfos(builder, moduleTranslation, DL, combinedInfo, mapData,
+ useDevPtrOperands, useDevAddrOperands);
} else {
- genMapInfos(builder, moduleTranslation, DL, combinedInfo, mapOperands,
- mapTypes);
+ genMapInfos(builder, moduleTranslation, DL, combinedInfo, mapData);
}
return combinedInfo;
};
@@ -1988,38 +2086,53 @@ static bool targetOpSupported(Operation &opInst) {
}
static void
-handleDeclareTargetMapVar(llvm::ArrayRef<Value> mapOperands,
+handleDeclareTargetMapVar(llvm::ArrayRef<MapData> mapData,
LLVM::ModuleTranslation &moduleTranslation,
llvm::IRBuilderBase &builder) {
- for (const mlir::Value &mapOp : mapOperands) {
- auto mapInfoOp =
- mlir::dyn_cast<mlir::omp::MapInfoOp>(mapOp.getDefiningOp());
- llvm::Value *mapOpValue =
- moduleTranslation.lookupValue(mapInfoOp.getVarPtr());
- if (auto *declareTarget = getRefPtrIfDeclareTarget(mapInfoOp.getVarPtr(),
- moduleTranslation)) {
- // The user's iterator will get invalidated if we modify an element,
+ for (const auto &mapValue : mapData) {
+ // In the case of declare target mapped variables, the basePointer is
+ // the reference pointer generated by the convertDeclareTargetAttr
+ // method. Whereas the kernelValue is the original variable, so for
+ // the device we must replace all uses of this original global variable
+ // (stored in kernelValue) with the reference pointer (stored in
+ // basePointer for declare target mapped variables), as for device the
+ // data is mapped into this reference pointer and should be loaded
+ // from it, the original variable is discarded. On host both exist and
+ // metadata is generated (elsewhere in the convertDeclareTargetAttr)
+ // function to link the two variables in the runtime and then both the
+ // reference pointer and the pointer are assigned in the kernel argument
+ // structure for the host.
+ if (mapValue.isDeclareTarget) {
+ // The users iterator will get invalidated if we modify an element,
// so we populate this vector of uses to alter each user on an individual
// basis to emit its own load (rather than one load for all).
llvm::SmallVector<llvm::User *> userVec;
- for (llvm::User *user : mapOpValue->users())
+ for (llvm::User *user : mapValue.kernelValue->users())
userVec.push_back(user);
- for (llvm::User *user : userVec) {
+ for (auto *user : userVec) {
if (auto *insn = dyn_cast<llvm::Instruction>(user)) {
- auto *load = builder.CreateLoad(
- moduleTranslation.convertType(mapInfoOp.getVarPtr().getType()),
- declareTarget);
+ auto *load = builder.CreateLoad(mapValue.basePointer->getType(),
+ mapValue.basePointer);
load->moveBefore(insn);
- user->replaceUsesOfWith(mapOpValue, load);
+ user->replaceUsesOfWith(mapValue.kernelValue, load);
}
}
}
}
}
+// This currently implements a very light version of Clang's
+// EmitParmDecl's handling of direct argument handling as well
+// as a portion of the argument access generation based on
+// capture types found at the end of emitOutlinedFunctionPrologue
+// in Clang. The indirect path handling of EmitParmDecl's may be
+// required for future work, but a direct 1-to-1 copy doesn't seem
+// possible as the logic is rather scattered throughout Clang's
+// lowering and perhaps we wish to deviate slightly.
static llvm::IRBuilderBase::InsertPoint
-createDeviceArgumentAccessor(llvm::Argument &arg, llvm::Value *input,
+createDeviceArgumentAccessor(llvm::SmallVectorImpl<MapData> &mapData,
+ llvm::Argument &arg, llvm::Value *input,
llvm::Value *&retVal, llvm::IRBuilderBase &builder,
llvm::OpenMPIRBuilder &ompBuilder,
LLVM::ModuleTranslation &moduleTranslation,
@@ -2027,22 +2140,152 @@ createDeviceArgumentAccessor(llvm::Argument &arg, llvm::Value *input,
llvm::IRBuilderBase::InsertPoint codeGenIP) {
builder.restoreIP(allocaIP);
- llvm::Value *addr =
+ mlir::omp::VariableCaptureKind capture =
+ mlir::omp::VariableCaptureKind::ByRef;
+ llvm::Type *inputType = input->getType();
+
+ // Find the associated MapData entry for the current input
+ for (const MapData &map : mapData) {
+ if (map.kernelValue == input) {
+ if (auto mapOp =
+ mlir::dyn_cast_if_present<mlir::omp::MapInfoOp>(map.mapClause)) {
+ capture = mapOp.getMapCaptureType().value_or(
+ mlir::omp::VariableCaptureKind::ByRef);
+ }
+
+ inputType = map.underlyingType;
+ break;
+ }
+ }
+
+ unsigned int allocaAS = ompBuilder.M.getDataLayout().getAllocaAddrSpace();
+ unsigned int defaultAS =
+ ompBuilder.M.getDataLayout().getProgramAddressSpace();
+
+ // Create the alloca for the argument the current point.
+ llvm::Value *v =
builder.CreateAlloca(arg.getType()->isPointerTy()
? arg.getType()
: llvm::Type::getInt64Ty(builder.getContext()),
- ompBuilder.M.getDataLayout().getAllocaAddrSpace());
- llvm::Value *addrAscast =
- builder.CreatePointerBitCastOrAddrSpaceCast(addr, input->getType());
- builder.CreateStore(&arg, addrAscast);
+ allocaAS);
+
+ if (allocaAS != defaultAS) {
+ v = builder.CreatePointerBitCastOrAddrSpaceCast(
+ v, arg.getType()->getPointerTo(defaultAS));
+ }
+
+ builder.CreateStore(&arg, v);
builder.restoreIP(codeGenIP);
- retVal = builder.CreateLoad(arg.getType(), addrAscast);
+ switch (capture) {
+ case mlir::omp::VariableCaptureKind::ByCopy: {
+ if (inputType->isPointerTy()) {
+ retVal = v;
+ return builder.saveIP();
+ }
+
+ // Ignore conversions like int -> uint.
+ if (v->getType() == inputType->getPointerTo()) {
+ retVal = v;
+ return builder.saveIP();
+ }
+
+ assert(false && "Currently unsupported OMPTargetVarCaptureByCopy Type");
+ break;
+ }
+ case mlir::omp::VariableCaptureKind::ByRef: {
+ retVal = builder.CreateAlignedLoad(
+ v->getType(), v,
+ ompBuilder.M.getDataLayout().get...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/68689
More information about the Openmp-commits
mailing list