[Mlir-commits] [mlir] [mlir][OpenACC] Emit multi-IV tile and element loops from ACCLoopTiling (PR #211651)
llvmlistbot at llvm.org
llvmlistbot at llvm.org
Thu Jul 23 12:48:30 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-mlir-openacc
Author: Vijay Kandiah (VijayKandiah)
<details>
<summary>Changes</summary>
`ACCLoopTiling` used to lower an N-dimensional `tile()` clause by *uncollapsing* the fused loop into a deep nest of single-IV `acc.loop`s (`tile_1 → … → tile_N → elem_1 → … → elem_N`). This discards the natural grouping of the tile clause: the 2N single-IV loops no longer express "these are the tile iterations" and "these are the in-tile iterations" as collapsible units, and gang/vector end up spread across a deep nest where only the outermost loop of each group carries the attribute. This could potentially lead to poor parallelism assignment.
With this MR:
`tileACCLoops` now rewrites the single fused `acc.loop` in-place into exactly two multi-IV loops, each carrying all the tiled induction variables:
- a **tile-group** loop whose steps are the original steps scaled by the tile sizes, and
- a nested **element-group** loop that walks the iterations within one tile, with upper bounds clamped to `min(origUB, tileStart + tileExtent)`.
Keeping each group as a single multi-IV `acc.loop` preserves the tile/element structure as two collapsible units instead of a 2N-deep single-IV nest, which is a simpler canonical form. The gang/worker/vector distribution across the two groups is unchanged from before.
This change also diagnoses `tile` + `collapse` on the same loop because these clauses give conflicting loop-association counts. The pass now emits a NYI diagnostic instead of silently dropping the `collapse` clause. Added a new invalid-input test to verify this.
With the pass no longer uncollapsing, `uncollapseLoops` and the vector-of-loops `tileACCLoops(SmallVector<LoopOp>&, …)` overload (and the now-orphaned `createInnerLoop` helper) are deleted, along with their unit tests. New unit tests are added to cover the surviving fused-loop overload.
---
Patch is 44.54 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/211651.diff
6 Files Affected:
- (modified) mlir/include/mlir/Dialect/OpenACC/OpenACCUtilsTiling.h (+24-29)
- (modified) mlir/lib/Dialect/OpenACC/Transforms/ACCLoopTiling.cpp (+14-18)
- (modified) mlir/lib/Dialect/OpenACC/Utils/OpenACCUtilsTiling.cpp (+129-216)
- (modified) mlir/test/Dialect/OpenACC/acc-loop-tiling-invalid.mlir (+23)
- (modified) mlir/test/Dialect/OpenACC/acc-loop-tiling.mlir (+28-57)
- (modified) mlir/unittests/Dialect/OpenACC/OpenACCUtilsTilingTest.cpp (+35-190)
``````````diff
diff --git a/mlir/include/mlir/Dialect/OpenACC/OpenACCUtilsTiling.h b/mlir/include/mlir/Dialect/OpenACC/OpenACCUtilsTiling.h
index 6fcb706aa3488..05ac37d1106ec 100644
--- a/mlir/include/mlir/Dialect/OpenACC/OpenACCUtilsTiling.h
+++ b/mlir/include/mlir/Dialect/OpenACC/OpenACCUtilsTiling.h
@@ -20,23 +20,20 @@
namespace mlir {
namespace acc {
-/// Uncollapse tile loops with multiple IVs and collapseCount < tileCount.
-/// This is used to prepare loops for tiling when the collapse count is less
-/// than the tile count.
+/// Tile a single fused acc.loop that carries all associated induction
+/// variables (one IV per tile dimension).
///
-/// \param origLoop The original loop operation to uncollapse.
-/// \param tileCount The number of tile dimensions.
-/// \param collapseCount The collapse count from the original loop.
-/// \param rewriter The rewriter to use for modifications.
-/// \return A vector of uncollapsed loop operations.
-llvm::SmallVector<mlir::acc::LoopOp>
-uncollapseLoops(mlir::acc::LoopOp origLoop, unsigned tileCount,
- unsigned collapseCount, mlir::RewriterBase &rewriter);
-
-/// Tile ACC loops according to the given tile sizes.
+/// This produces exactly two multi-IV loops, each carrying all of the tiled
+/// induction variables:
///
-/// Tiling a 2-level nested loop will create two 'tile' loops containing two
-/// 'element' loops. The transformation looks like:
+/// - a "tile group" loop that steps over tiles: each step is the
+/// original step multiplied by the tile size. It keeps the original loop's
+/// gang attribute.
+/// - an "element group" loop that walks the iterations inside one tile: it
+/// keeps the original step, its lower bound is the current tile's starting
+/// index, and its upper bound is clamped to min(original upper bound,
+/// tile start + tile extent). It keeps the original loop's vector (or
+/// worker) attribute.
///
/// Before Tiling:
/// \code
@@ -48,17 +45,15 @@ uncollapseLoops(mlir::acc::LoopOp origLoop, unsigned tileCount,
/// }
/// \endcode
///
-/// After Tiling:
+/// After Tiling (each group is one multi-IV loop over all tiled IVs):
/// \code
-/// for (i = lb1; i < ub1; i += (step1 * tile_size1)) { // tile loop 1
-/// for (j = lb2; j < ub2; j += (step2 * tile_size2)) { // tile loop 2
-/// for (ii = i; ii < min(ub1, (step1 * tile_size1) + i); ii += step1) {
-/// // element loop 1
-/// for (jj = j; jj < min(ub2, (step2 * tile_size2) + j); jj += step2)
-/// { // element loop 2
-/// a[ii,jj] = i + j;
-/// }
-/// }
+/// // tile group
+/// for (i = lb1; i < ub1; i += (step1 * tile_size1),
+/// j = lb2; j < ub2; j += (step2 * tile_size2)) {
+/// // element group
+/// for (ii = i; ii < min(ub1, (step1 * tile_size1) + i); ii += step1,
+/// jj = j; jj < min(ub2, (step2 * tile_size2) + j); jj += step2) {
+/// a[ii,jj] = i + j;
/// }
/// }
/// \endcode
@@ -66,13 +61,13 @@ uncollapseLoops(mlir::acc::LoopOp origLoop, unsigned tileCount,
/// Unknown tile sizes (represented as -1 in acc dialect for `tile(*)`) are
/// resolved to the provided default tile size.
///
-/// \param tileLoops The loops to tile (outermost first).
-/// \param tileSizes The tile sizes for each dimension. Values of -1 are
+/// \param tileLoop The fused loop to tile.
+/// \param tileSizes The tile sizes for each tiled dimension. Values of -1 are
/// treated as unknown and resolved to defaultTileSize.
/// \param defaultTileSize The default tile size to use for unknown (*) tiles.
/// \param rewriter The rewriter to use for modifications.
-/// \return The outermost loop after tiling.
-mlir::acc::LoopOp tileACCLoops(llvm::SmallVector<mlir::acc::LoopOp> &tileLoops,
+/// \return The tile group loop that is modified in place.
+mlir::acc::LoopOp tileACCLoops(mlir::acc::LoopOp tileLoop,
const llvm::SmallVector<mlir::Value> &tileSizes,
int32_t defaultTileSize,
mlir::RewriterBase &rewriter);
diff --git a/mlir/lib/Dialect/OpenACC/Transforms/ACCLoopTiling.cpp b/mlir/lib/Dialect/OpenACC/Transforms/ACCLoopTiling.cpp
index 6bc95ca896f37..d5865bc98727e 100644
--- a/mlir/lib/Dialect/OpenACC/Transforms/ACCLoopTiling.cpp
+++ b/mlir/lib/Dialect/OpenACC/Transforms/ACCLoopTiling.cpp
@@ -162,8 +162,16 @@ struct ACCLoopTilingImpl : public OpRewritePattern<acc::LoopOp> {
SmallVector<Value> tileSizes(origLoop.getTileValues().begin(),
origLoop.getTileValues().end());
- unsigned tileCount = tileSizes.size();
- unsigned collapseCount = origLoop.getCollapseValue().value_or(1);
+
+ // A tile clause and a collapse clause on the same loop are not supported:
+ // they give conflicting descriptions of how many loops the construct
+ // associates.
+ if (origLoop.getCollapseAttr()) {
+ accSupport.emitNYI(origLoop.getLoc(),
+ "a tile clause combined with a collapse clause on the "
+ "same loop");
+ return failure();
+ }
// Sanity check tile size types
if (failed(checkTileSizeTypes(origLoop, tileSizes)))
@@ -182,24 +190,12 @@ struct ACCLoopTilingImpl : public OpRewritePattern<acc::LoopOp> {
origLoop.removeTileOperandsDeviceTypeAttr();
rewriter.finalizeOpModification(origLoop);
- SmallVector<acc::LoopOp> loopsToTile;
- if (collapseCount < tileCount) {
- // Uncollapse tile loops before tiling if necessary
- loopsToTile =
- acc::uncollapseLoops(origLoop, tileCount, collapseCount, rewriter);
- rewriter.replaceOp(origLoop, loopsToTile[0]);
- LLVM_DEBUG(llvm::dbgs() << "\nAfter uncollapsing:\n"
- << *loopsToTile[0] << "\n");
- } else {
- loopsToTile.push_back(origLoop);
- }
-
- // loopsToTile is a vector of perfectly nested loops. The outermost loop
- // may have multiple IVs but inner loops can only have one IV.
+ // Tile the single fused loop in place: the tile iterations become one
+ // multi-IV "tile group" loop wrapping one multi-IV "element group" loop.
// The utility handles unknown tile sizes (*) by using `defaultTileSize`.
- acc::tileACCLoops(loopsToTile, tileSizes, defaultTileSize, rewriter);
+ acc::tileACCLoops(origLoop, tileSizes, defaultTileSize, rewriter);
- LLVM_DEBUG(llvm::dbgs() << "\nAfter tiling:\n " << *loopsToTile[0] << "\n");
+ LLVM_DEBUG(llvm::dbgs() << "\nAfter tiling:\n " << *origLoop << "\n");
return success();
}
diff --git a/mlir/lib/Dialect/OpenACC/Utils/OpenACCUtilsTiling.cpp b/mlir/lib/Dialect/OpenACC/Utils/OpenACCUtilsTiling.cpp
index 09e83af2bc686..32fd3957894fb 100644
--- a/mlir/lib/Dialect/OpenACC/Utils/OpenACCUtilsTiling.cpp
+++ b/mlir/lib/Dialect/OpenACC/Utils/OpenACCUtilsTiling.cpp
@@ -17,6 +17,7 @@
#include "mlir/Dialect/OpenACC/OpenACC.h"
#include "mlir/Dialect/Utils/StaticValueUtils.h"
#include "mlir/Transforms/RegionUtils.h"
+#include "llvm/ADT/STLExtras.h"
// Resolve unknown tile sizes (represented as -1 for tile(*)) to the default.
// Returns a value with the same type as targetType.
@@ -48,19 +49,13 @@ static void removeWorkerVectorFromLoop(mlir::acc::LoopOp loop) {
}
// Create a new ACC loop with new steps, lb, ub from original loop
-static mlir::acc::LoopOp
-createACCLoopFromOriginal(mlir::acc::LoopOp origLoop,
- mlir::RewriterBase &rewriter, mlir::ValueRange lb,
- mlir::ValueRange ub, mlir::ValueRange step,
- mlir::DenseBoolArrayAttr inclusiveUBAttr,
- mlir::acc::CombinedConstructsTypeAttr combinedAttr,
- mlir::Location loc, bool preserveCollapse) {
+static mlir::acc::LoopOp createACCLoopFromOriginal(
+ mlir::acc::LoopOp origLoop, mlir::RewriterBase &rewriter,
+ mlir::ValueRange lb, mlir::ValueRange ub, mlir::ValueRange step,
+ mlir::DenseBoolArrayAttr inclusiveUBAttr,
+ mlir::acc::CombinedConstructsTypeAttr combinedAttr, mlir::Location loc) {
mlir::ArrayAttr collapseAttr = mlir::ArrayAttr{};
mlir::ArrayAttr collapseDeviceTypeAttr = mlir::ArrayAttr{};
- if (preserveCollapse) {
- collapseAttr = origLoop.getCollapseAttr();
- collapseDeviceTypeAttr = origLoop.getCollapseDeviceTypeAttr();
- }
auto newLoop = mlir::acc::LoopOp::create(
rewriter, loc, origLoop->getResultTypes(), lb, ub, step, inclusiveUBAttr,
collapseAttr, collapseDeviceTypeAttr, origLoop.getGangOperands(),
@@ -78,53 +73,6 @@ createACCLoopFromOriginal(mlir::acc::LoopOp origLoop,
return newLoop;
}
-// Create inner loop inside input loop
-static mlir::acc::LoopOp
-createInnerLoop(mlir::acc::LoopOp inputLoop, mlir::RewriterBase &rewriter,
- mlir::ValueRange lb, mlir::ValueRange ub, mlir::ValueRange step,
- mlir::DenseBoolArrayAttr inclusiveUBAttr, mlir::Location loc) {
- mlir::acc::LoopOp elementLoop = createACCLoopFromOriginal(
- inputLoop, rewriter, lb, ub, step, inclusiveUBAttr,
- mlir::acc::CombinedConstructsTypeAttr{}, loc, /*preserveCollapse*/ false);
-
- // Remove gang/worker attributes from inner loops
- rewriter.startOpModification(elementLoop);
- if (inputLoop.hasGang() ||
- inputLoop.getGangValue(mlir::acc::GangArgType::Num) ||
- inputLoop.getGangValue(mlir::acc::GangArgType::Dim) ||
- inputLoop.getGangValue(mlir::acc::GangArgType::Static)) {
- elementLoop.removeGangAttr();
- elementLoop.removeGangOperandsArgTypeAttr();
- elementLoop.removeGangOperandsSegmentsAttr();
- elementLoop.removeGangOperandsDeviceTypeAttr();
- // Also drop the operand values themselves so that elementLoop does not
- // end up with a non-empty gang operand list but no corresponding
- // device-type/segment/arg-type attributes. Leaving stale operands behind
- // makes elementLoop look like it still has gang operands to later
- // queries (e.g. LoopOp::getGangValue), which then dereference the
- // now-missing device-type attribute.
- elementLoop.getGangOperandsMutable().clear();
- }
- if (inputLoop.hasVector() || inputLoop.getVectorValue()) {
- elementLoop.removeWorkerAttr();
- elementLoop.removeWorkerNumOperandsDeviceTypeAttr();
- // As above for gang, also drop the worker operand values so elementLoop
- // does not keep a dangling worker operand with no device-type attribute.
- elementLoop.getWorkerNumOperandsMutable().clear();
- }
- rewriter.finalizeOpModification(elementLoop);
-
- // Create empty block in elementLoop and add IV argument
- mlir::Block *blk = rewriter.createBlock(&elementLoop.getRegion(),
- elementLoop.getRegion().begin());
- rewriter.setInsertionPointToEnd(blk);
- mlir::acc::YieldOp::create(rewriter, loc);
- elementLoop.getBody().addArgument(
- inputLoop.getBody().getArgument(0).getType(), loc);
-
- return elementLoop;
-}
-
// Move ops from source to target Loop and replace uses of IVs
static void moveOpsAndReplaceIVs(mlir::acc::LoopOp sourceLoop,
mlir::acc::LoopOp targetLoop,
@@ -159,180 +107,145 @@ static void moveOpsAndReplaceIVs(mlir::acc::LoopOp sourceLoop,
rewriter.finalizeOpModification(op);
}
+// Create a single "element group" loop nested in `tileLoop`, carrying
+// `ivTypes.size()` induction variables so the whole element space is one
+// multi-IV loop. The element group carries vector or worker but not gang.
+static mlir::acc::LoopOp
+createElementGroupLoop(mlir::acc::LoopOp tileLoop, mlir::RewriterBase &rewriter,
+ mlir::ValueRange lbs, mlir::ValueRange ubs,
+ mlir::ValueRange steps,
+ mlir::DenseBoolArrayAttr inclusiveUBAttr,
+ llvm::ArrayRef<mlir::Type> ivTypes, mlir::Location loc) {
+ mlir::acc::LoopOp elementLoop = createACCLoopFromOriginal(
+ tileLoop, rewriter, lbs, ubs, steps, inclusiveUBAttr,
+ mlir::acc::CombinedConstructsTypeAttr{}, loc);
+
+ // Drop gang from the element group, keeping vector/worker. The operand
+ // values must be cleared too, not just the attributes.
+ rewriter.startOpModification(elementLoop);
+ if (tileLoop.hasGang() ||
+ tileLoop.getGangValue(mlir::acc::GangArgType::Num) ||
+ tileLoop.getGangValue(mlir::acc::GangArgType::Dim) ||
+ tileLoop.getGangValue(mlir::acc::GangArgType::Static)) {
+ elementLoop.removeGangAttr();
+ elementLoop.removeGangOperandsArgTypeAttr();
+ elementLoop.removeGangOperandsSegmentsAttr();
+ elementLoop.removeGangOperandsDeviceTypeAttr();
+ elementLoop.getGangOperandsMutable().clear();
+ }
+ if (tileLoop.hasVector() || tileLoop.getVectorValue()) {
+ elementLoop.removeWorkerAttr();
+ elementLoop.removeWorkerNumOperandsDeviceTypeAttr();
+ elementLoop.getWorkerNumOperandsMutable().clear();
+ }
+ rewriter.finalizeOpModification(elementLoop);
+
+ // Create the element loop body: one block argument per IV plus a terminator.
+ mlir::Block *blk = rewriter.createBlock(&elementLoop.getRegion(),
+ elementLoop.getRegion().begin());
+ rewriter.setInsertionPointToEnd(blk);
+ mlir::acc::YieldOp::create(rewriter, loc);
+ for (mlir::Type ivType : ivTypes)
+ elementLoop.getBody().addArgument(ivType, loc);
+
+ return elementLoop;
+}
+
mlir::acc::LoopOp
-mlir::acc::tileACCLoops(llvm::SmallVector<mlir::acc::LoopOp> &tileLoops,
+mlir::acc::tileACCLoops(mlir::acc::LoopOp tileLoop,
const llvm::SmallVector<mlir::Value> &tileSizes,
int32_t defaultTileSize, mlir::RewriterBase &rewriter) {
- // Tile collapsed and/or nested loops
- mlir::acc::LoopOp outerLoop = tileLoops[0];
- const mlir::Location loc = outerLoop.getLoc();
-
- mlir::acc::LoopOp innerLoop = tileLoops[tileLoops.size() - 1];
- llvm::SmallVector<mlir::Value, 3> origIVs;
- llvm::SmallVector<mlir::Value, 3> origSteps;
- llvm::SmallVector<mlir::Value, 3> origUBs;
- llvm::SmallVector<mlir::Value, 3> newSteps;
- llvm::SmallVector<mlir::Value, 3> newUBs;
- llvm::SmallVector<mlir::Value, 3> newIVs;
- size_t nOps = innerLoop.getBody().getOperations().size();
-
- // Extract original inclusiveUBs
+ // Tile a single fused acc.loop that carries all associated induction
+ // variables. This keeps the tile iterations as one multi-IV "tile group"
+ // loop and the in-tile iterations as one multi-IV "element group" loop, each
+ // spanning all of its induction variables.
+ const mlir::Location loc = tileLoop.getLoc();
+ const unsigned tileCount = tileSizes.size();
+
+ llvm::SmallVector<mlir::Value, 3> origIVs(tileLoop.getBody().getArguments());
+ llvm::SmallVector<mlir::Value, 3> origUBs(tileLoop.getUpperbound());
+ llvm::SmallVector<mlir::Value, 3> origSteps(tileLoop.getStep());
+ const unsigned numIVs = origIVs.size();
+ const size_t nOps = tileLoop.getBody().getOperations().size();
+
+ // Original inclusive-UB flags (default false when the attribute is absent).
llvm::SmallVector<bool> inclusiveUBs;
- for (auto tileLoop : tileLoops) {
- for (auto [j, step] : llvm::enumerate(tileLoop.getStep())) {
- // inclusiveUBs are present on the IR from Fortran frontend for DO loops
- // but might not be present from other frontends (python)
- // So check if it exists
- if (tileLoop.getInclusiveUpperboundAttr())
- inclusiveUBs.push_back(
- tileLoop.getInclusiveUpperboundAttr().asArrayRef()[j]);
- else
- inclusiveUBs.push_back(false);
- }
+ for (unsigned i = 0; i < numIVs; ++i) {
+ if (tileLoop.getInclusiveUpperboundAttr())
+ inclusiveUBs.push_back(
+ tileLoop.getInclusiveUpperboundAttr().asArrayRef()[i]);
+ else
+ inclusiveUBs.push_back(false);
}
- // Extract original ivs, UBs, steps, and calculate new steps
- rewriter.setInsertionPoint(outerLoop);
- for (auto [i, tileLoop] : llvm::enumerate(tileLoops)) {
- for (auto arg : tileLoop.getBody().getArguments())
- origIVs.push_back(arg);
- for (auto ub : tileLoop.getUpperbound())
- origUBs.push_back(ub);
-
- llvm::SmallVector<mlir::Value, 3> currentLoopSteps;
- for (auto [j, step] : llvm::enumerate(tileLoop.getStep())) {
- origSteps.push_back(step);
- if (i + j >= tileSizes.size()) {
- currentLoopSteps.push_back(step);
- } else {
- mlir::Value tileSize = resolveAndCastTileSize(
- tileSizes[i + j], defaultTileSize, step.getType(), rewriter, loc);
- auto newLoopStep =
- mlir::arith::MulIOp::create(rewriter, loc, step, tileSize);
- currentLoopSteps.push_back(newLoopStep);
- newSteps.push_back(newLoopStep);
- }
+ // Scale each tiled dimension's step by its tile size to form the tile group
+ // loop steps.
+ rewriter.setInsertionPoint(tileLoop);
+ llvm::SmallVector<mlir::Value, 3> scaledSteps;
+ llvm::SmallVector<mlir::Value, 3> tileLoopSteps;
+ for (unsigned i = 0; i < numIVs; ++i) {
+ if (i < tileCount) {
+ mlir::Value tileSize = resolveAndCastTileSize(
+ tileSizes[i], defaultTileSize, origSteps[i].getType(), rewriter, loc);
+ mlir::Value scaled =
+ mlir::arith::MulIOp::create(rewriter, loc, origSteps[i], tileSize);
+ scaledSteps.push_back(scaled);
+ tileLoopSteps.push_back(scaled);
+ } else {
+ tileLoopSteps.push_back(origSteps[i]);
}
-
- rewriter.startOpModification(tileLoop);
- tileLoop.getStepMutable().clear();
- tileLoop.getStepMutable().append(currentLoopSteps);
- rewriter.finalizeOpModification(tileLoop);
}
- // Calculate new upper bounds for element loops
- for (size_t i = 0; i < newSteps.size(); i++) {
- rewriter.setInsertionPoint(innerLoop.getBody().getTerminator());
- // UpperBound: min(origUB, origIV+(originalStep*tile_size))
- auto stepped =
- mlir::arith::AddIOp::create(rewriter, loc, origIVs[i], newSteps[i]);
+ // Compute the element-loop upper bounds min(origUB, origIV + scaledStep).
+ rewriter.setInsertionPoint(tileLoop.getBody().getTerminator());
+ llvm::SmallVector<mlir::Value, 3> elemLBs, elemUBs, elemSteps;
+ llvm::SmallVector<mlir::Type, 3> elemIVTypes;
+ llvm::SmallVector<bool> elemInclusiveUBs;
+ for (unsigned i = 0; i < tileCount; ++i) {
+ mlir::Value stepped =
+ mlir::arith::AddIOp::create(rewriter, loc, origIVs[i], scaledSteps[i]);
mlir::Value newUB = stepped;
if (inclusiveUBs[i]) {
- // Handle InclusiveUB
- // UpperBound: min(origUB, origIV+(originalStep*tile_size - 1))
- auto c1 = mlir::arith::ConstantOp::create(
- rewriter, loc, newSteps[i].getType(),
- rewriter.getIntegerAttr(newSteps[i].getType(), 1));
+ // Inclusive UB: min(origUB, origIV + (scaledStep - 1)).
+ mlir::Value c1 = mlir::arith::ConstantOp::create(
+ rewriter, loc, scaledSteps[i].getType(),
+ rewriter.getIntegerAttr(scaledSteps[i].getType(), 1));
newUB = mlir::arith::SubIOp::create(rewriter, loc, stepped, c1);
}
- newUBs.push_back(
+ elemUBs.push_back(
mlir::arith::MinSIOp::create(rewriter, loc, origUBs[i], newUB));
+ elemLBs.push_back(origIVs[i]);
+ elemSteps.push_back(origSteps[i]);
+ elemIVTypes.push_back(origIVs[i].getType());
+ elemInclusiveUBs.push_back(inclusiveUBs[i]);
}
- // Create and insert nested elementLoopOps before terminator of outer loopOp
- mlir::acc::LoopOp currentLoop = innerLoop;
- for (size_t i = 0; i < tileSizes.size(); i++) {
- rewriter.setInsertionPoint(currentLoop.getBody().getTerminator());
- mlir::DenseBoolArrayAttr inclusiveUBAttr = mlir::DenseBoolArrayAttr{};
- if (inclusiveUBs[i])
- inclusiveUBAttr = rewriter.getDenseBoolArrayAttr({true});
-
- mlir::acc::LoopOp elementLoop =
- createInnerLoop(innerLoop, rewriter, mlir::ValueRange{origIVs[i]},
- mlir::ValueRange{newUBs[i]},
- mlir::ValueRange{origSteps[i]}, inclusiveUBAttr, loc);
-
- // Remove vector/worker attributes from inner element loops except
- // outermost element loop
- if (i > 0) {
- rewriter.startOpModification(elementLoop);
- removeWorkerVectorFromLoop(elementLoop);
- rewriter.finalizeOpModification(elementLoop);
- }
- newIVs.push_back(elementLoop.getBody().g...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/211651
More information about the Mlir-commits
mailing list