[Openmp-commits] [llvm] [openmp] [OpenMP][offload] Size the grid of cross-team reductions by their size (PR #234190)
Robert Imschweiler via Openmp-commits
openmp-commits at lists.llvm.org
Fri Oct 30 01:19:30 PDT 2026
================
@@ -852,6 +852,65 @@ getDefaultNumBlocksGenericSPMD(const DeviceGeometryPropsTy &Props) {
return Props.NumComputeUnits * OMPX_GenericSPMDTeamsPerCU.get();
}
+/// Threshold for the reduction data size (in bytes) above which a cross-team
+/// reduction classifies as "large".
+static constexpr uint32_t LargeTeamsReductionBytes = 32;
+
+/// Get the number of blocks (each having \p NumThreads threads) to launch a
+/// kernel with a cross-team reduction, or zero to handle it like a
+/// non-reduction kernel.
+///
+/// Every block adds to the cost of combining the partial results of the blocks.
+/// For a "large" reduction (see \p LargeTeamsReductionBytes), this cost
+/// dominates -> one block / compute unit. A small/scalar reduction prefers more
+/// blocks -> as many as needed given the device's number of waves per compute
+/// unit, but at least one / compute unit. A loop too short to give every
+/// compute unit a block is handled like a non-reduction kernel.
+static uint64_t
+getXTeamReductionNumBlocks(const DeviceGeometryPropsTy &Props,
+ uint32_t NumThreads, uint64_t LoopTripCount,
+ const KernelLaunchInfoTy &KernelEnv) {
+ if (!KernelEnv.ReductionDataSize || !NumThreads)
+ return 0;
+
+ // In Generic and Generic-SPMD mode, each block runs one iteration of the
+ // distribute loop. For these modes, there currently is only performance data
+ // for large reductions.
+ bool IsLarge = KernelEnv.ReductionDataSize > LargeTeamsReductionBytes;
+ bool IsSPMD = KernelEnv.isSPMDMode();
+ if (!IsSPMD && !(IsLarge && (KernelEnv.isGenericMode() ||
+ KernelEnv.isGenericSPMDMode())))
+ return 0;
+
+ // Envar to override the device's number of waves per compute unit for a
+ // kernel with a scalar cross-team reduction. Zero keeps the device's value.
+ static UInt32Envar OMPX_TeamsReductionWavesPerCU(
+ "LIBOMPTARGET_TEAMS_REDUCTION_WAVES_PER_CU", 0);
+ uint32_t ScalarWavesPerCU = OMPX_TeamsReductionWavesPerCU.get();
+ // Measured on MI210, MI300A, MI300X, MI350X, and MI450: scalar reductions
----------------
ro-i wrote:
merging this for now, doing a follow-up for this if needed / wanted. (This way I have less trouble merging this PR with the already existing downstream code :) )
https://github.com/llvm/llvm-project/pull/234190
More information about the Openmp-commits
mailing list