[Mlir-commits] [mlir] [MLIR][XeGPU] Improve workgroup to subgroup distribution pattern for mulit-reduction op (PR #182178)
Artem Kroviakov
llvmlistbot at llvm.org
Sun Feb 22 05:24:31 PST 2026
================
@@ -1430,44 +1392,38 @@ struct WgToSgMultiDimReductionOp
}
// Step 2: cross-subgroup reduction using SLM
-
- // Calculate total elements in local result
- int64_t localElements = computeProduct(sgShape);
-
- // Shape cast for SLM storage - store as [1, localElements]
- SmallVector<int64_t> storeShape2D = {1, localElements};
- VectorType storeType2D = VectorType::get(storeShape2D, elemTy);
- auto storeShapeCast = vector::ShapeCastOp::create(
- rewriter, loc, storeType2D, localReductions[0]);
- Value storeData = storeShapeCast.getResult();
-
- // Calculate SLM shape - rows for sg's in reduction dims, cols for total
- // result elements across all subgroups in non-reduction dimensions
- int64_t totalReductionSubgroups = 1;
- for (int64_t dim : crossSgReductionDims) {
- totalReductionSubgroups *= sgLayout[dim];
- }
-
- // Total result elements across all subgroups in non-reduction dimensions
- int64_t totalResultElements =
- localElements * computeProduct(sgLayout) / totalReductionSubgroups;
-
- SmallVector<int64_t> slmShape2D = {totalReductionSubgroups,
- totalResultElements};
+ auto slmStoreDataShape = sgSrcShape;
+ for (int64_t dim : reductionDims)
+ slmStoreDataShape[dim] = 1;
+ VectorType slmStoreDataType = VectorType::get(slmStoreDataShape, elemTy);
+ Value slmStoreData = vector::ShapeCastOp::create(
+ rewriter, loc, slmStoreDataType, localReductions[0]);
+
+ SmallVector<int64_t> slmShape(originalSrcShape.begin(),
+ originalSrcShape.end());
+ // for reduction dimension, SLM stores partial results from each subgroup
+ for (int64_t dim : reductionDims)
+ slmShape[dim] = originalSrcShape[dim] / sgSrcShape[dim];
----------------
akroviakov wrote:
nit: curious if it would be better to use the values from `sg_layout` reduction dims here for easier understanding. But physically it is the same, so no big deal.
https://github.com/llvm/llvm-project/pull/182178
More information about the Mlir-commits
mailing list