[Mlir-commits] [mlir] [MLIR][XeGPU] Build float min/max neutral values the format can represent (PR #233471)
Md Abdullah Shahneous Bari
llvmlistbot at llvm.org
Mon Oct 19 08:52:36 PDT 2026
https://github.com/mshahneo created https://github.com/llvm/llvm-project/pull/233471
`createReductionNeutralValue` gives a float min/max reduction an infinity (`maximumf`, `minimumf`) or a quiet NaN (the kinds that ignore NaN). The WG-to-SG distribution and the peephole split of `vector.multi_reduction` start their new reductions from this value.
Some float formats cannot represent these values:
- A format without infinity, such as `f8E4M3FN`: `APFloat::getInf` returns NaN. So `maximumf` and `minimumf` start from NaN, and every result is NaN.
- `f4E2M1FN` has neither infinity nor NaN, so building the value aborts.
- `f8E8M0FNU` has no negative values, so building -inf for `maximumf` aborts.
This patch chooses the value by what the format supports:
- A kind that ignores NaN still uses a quiet NaN if the format has one.
- Otherwise it uses the infinity that never wins.
- A format without that infinity uses its lowest (for max) or highest (for min) value, which never wins either.
- For an unsigned format the lowest value is zero, or its smallest value if it has no zero.
IEEE formats get the same values as before.
### Testing
- New tests in `xegpu-wg-to-sg.mlir` for `f8E4M3FN`, `f4E2M1FN` and `f8E8M0FNU`. Without this patch, `mlir-opt` aborts on them (and `f8E4M3FN` `maximumf`/`minimumf` get a NaN neutral value). With it they pass. The existing neutral-value tests for `f32` are unchanged.
- `check-mlir`: one GPU integration test, `Integration/Dialect/XeVM/GPU/xevm_bitcast_shuffle.mlir`, fails both with and without this patch.
- Downstream IMEX XeGPU integration suite: same result for every test with and without this patch.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
>From 8f5c07352705309731bb49bbea30d8da1abb3cb2 Mon Sep 17 00:00:00 2001
From: "Shahneous Bari, Md Abdullah" <md.abdullah.shahneous.bari at intel.com>
Date: Mon, 19 Oct 2026 15:32:04 +0000
Subject: [PATCH] [MLIR][XeGPU] Build float min/max neutral values the format
can represent
createReductionNeutralValue gives a float min/max reduction an infinity
(maximumf, minimumf) or a quiet NaN (the kinds that ignore NaN). Some
float formats cannot represent these values:
- A format without infinity, such as f8E4M3FN: APFloat::getInf returns
NaN. So maximumf and minimumf start from NaN, and every result is NaN.
- f4E2M1FN has neither infinity nor NaN, so building the value aborts.
- f8E8M0FNU has no negative values, so building -inf for maximumf
aborts.
The WG-to-SG distribution and the peephole split of
vector.multi_reduction start their new reductions from this value.
Choose the value by what the format supports. A kind that ignores NaN
still uses a quiet NaN if the format has one. Otherwise use the infinity
that never wins. A format without that infinity uses its lowest (for
max) or highest (for min) value, which never wins either. For an
unsigned format the lowest value is zero, or its smallest value if it
has no zero.
IEEE formats get the same values as before.
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
---
mlir/lib/Dialect/XeGPU/Utils/XeGPUUtils.cpp | 53 +++++++++----
mlir/test/Dialect/XeGPU/xegpu-wg-to-sg.mlir | 88 +++++++++++++++++++++
2 files changed, 125 insertions(+), 16 deletions(-)
diff --git a/mlir/lib/Dialect/XeGPU/Utils/XeGPUUtils.cpp b/mlir/lib/Dialect/XeGPU/Utils/XeGPUUtils.cpp
index e22364df9e590..f3dc3375f50ff 100644
--- a/mlir/lib/Dialect/XeGPU/Utils/XeGPUUtils.cpp
+++ b/mlir/lib/Dialect/XeGPU/Utils/XeGPUUtils.cpp
@@ -725,6 +725,30 @@ Value xegpu::lowerCrossLaneReductionToShuffles(
return reductionResult;
}
+/// Returns the neutral value of a float min or max reduction over elements of
+/// `semantics`. Builds only values the format can represent:
+/// - A kind that ignores NaN operands uses a quiet NaN, if the format has one.
+/// - Otherwise it uses the infinity that never wins.
+/// - A format without that infinity uses its lowest (for max) or highest (for
+/// min) value. That value never wins either.
+static APFloat getMinMaxNeutral(const llvm::fltSemantics &semantics, bool isMax,
+ bool ignoresNaN) {
+ if (ignoresNaN && APFloat::semanticsHasNaN(semantics))
+ return APFloat::getQNaN(semantics);
+ if (!isMax)
+ return APFloat::semanticsHasInf(semantics) ? APFloat::getInf(semantics)
+ : APFloat::getLargest(semantics);
+ // An unsigned format has no negative values. Its lowest value is zero, or
+ // its smallest positive value if it has no zero.
+ if (!APFloat::semanticsHasSignedRepr(semantics))
+ return APFloat::semanticsHasZero(semantics)
+ ? APFloat::getZero(semantics)
+ : APFloat::getSmallest(semantics);
+ return APFloat::semanticsHasInf(semantics)
+ ? APFloat::getInf(semantics, /*Negative=*/true)
+ : APFloat::getLargest(semantics, /*Negative=*/true);
+}
+
Value xegpu::createReductionNeutralValue(OpBuilder &builder, Location loc,
Type type,
vector::CombiningKind kind) {
@@ -769,26 +793,23 @@ Value xegpu::createReductionNeutralValue(OpBuilder &builder, Location loc,
return nullptr;
case vector::CombiningKind::MINIMUMF:
- if (auto floatTy = dyn_cast<FloatType>(elemTy))
- return makeConst(builder.getFloatAttr(
- elemTy, APFloat::getInf(floatTy.getFloatSemantics())));
- return nullptr;
-
case vector::CombiningKind::MAXIMUMF:
- if (auto floatTy = dyn_cast<FloatType>(elemTy))
- return makeConst(builder.getFloatAttr(
- elemTy,
- APFloat::getInf(floatTy.getFloatSemantics(), /*Negative=*/true)));
- return nullptr;
-
case vector::CombiningKind::MINNUMF:
case vector::CombiningKind::MINIMUMNUMF:
case vector::CombiningKind::MAXNUMF:
- case vector::CombiningKind::MAXIMUMNUMF:
- if (auto floatTy = dyn_cast<FloatType>(elemTy))
- return makeConst(builder.getFloatAttr(
- elemTy, APFloat::getQNaN(floatTy.getFloatSemantics())));
- return nullptr;
+ case vector::CombiningKind::MAXIMUMNUMF: {
+ auto floatTy = dyn_cast<FloatType>(elemTy);
+ if (!floatTy)
+ return nullptr;
+ bool isMax = kind == vector::CombiningKind::MAXIMUMF ||
+ kind == vector::CombiningKind::MAXNUMF ||
+ kind == vector::CombiningKind::MAXIMUMNUMF;
+ bool ignoresNaN = kind != vector::CombiningKind::MINIMUMF &&
+ kind != vector::CombiningKind::MAXIMUMF;
+ return makeConst(builder.getFloatAttr(
+ elemTy,
+ getMinMaxNeutral(floatTy.getFloatSemantics(), isMax, ignoresNaN)));
+ }
}
return nullptr;
}
diff --git a/mlir/test/Dialect/XeGPU/xegpu-wg-to-sg.mlir b/mlir/test/Dialect/XeGPU/xegpu-wg-to-sg.mlir
index 8e53c40b2078c..0fa43bac2aa97 100644
--- a/mlir/test/Dialect/XeGPU/xegpu-wg-to-sg.mlir
+++ b/mlir/test/Dialect/XeGPU/xegpu-wg-to-sg.mlir
@@ -933,6 +933,94 @@ gpu.module @test_distribution {
gpu.return
}
+ // f8E4M3FN has no infinity: maximumf/minimumf use the lowest/highest finite value.
+ // maxnumf still uses a quiet NaN.
+ // CHECK-LABEL: gpu.func @vector_reduce_no_inf_neutral
+ // CHECK: %[[N0:.*]] = arith.constant dense<-4.480000e+02> : vector<16xf8E4M3FN>
+ // CHECK: vector.multi_reduction <maximumf>, %{{.*}}, %[[N0]] [1] : vector<16x64xf8E4M3FN> to vector<16xf8E4M3FN>
+ // CHECK: %[[N1:.*]] = arith.constant dense<4.480000e+02> : vector<16xf8E4M3FN>
+ // CHECK: vector.multi_reduction <minimumf>, %{{.*}}, %[[N1]] [1] : vector<16x64xf8E4M3FN> to vector<16xf8E4M3FN>
+ // CHECK: %[[N2:.*]] = arith.constant dense<0x7F> : vector<16xf8E4M3FN>
+ // CHECK: vector.multi_reduction <maxnumf>, %{{.*}}, %[[N2]] [1] : vector<16x64xf8E4M3FN> to vector<16xf8E4M3FN>
+ gpu.func @vector_reduce_no_inf_neutral(%src: memref<256x64xf8E4M3FN>) {
+ %cst = arith.constant dense<1.0> : vector<256xf8E4M3FN>
+ %tdesc = xegpu.create_nd_tdesc %src : memref<256x64xf8E4M3FN>
+ -> !xegpu.tensor_desc<256x64xf8E4M3FN>
+ %load = xegpu.load_nd %tdesc[0, 0] <{layout = #xegpu.layout<sg_layout = [16, 1], sg_data = [16, 64]>}>
+ : !xegpu.tensor_desc<256x64xf8E4M3FN>
+ -> vector<256x64xf8E4M3FN>
+ %r0 = vector.multi_reduction <maximumf>, %load, %cst [1]
+ : vector<256x64xf8E4M3FN> to vector<256xf8E4M3FN>
+ %a0 = xegpu.convert_layout %r0
+ <{target_layout = #xegpu.slice<#xegpu.layout<sg_layout = [16, 1], sg_data = [16, 64]>, dims = [1]>}>
+ : vector<256xf8E4M3FN>
+ %r1 = vector.multi_reduction <minimumf>, %load, %cst [1]
+ : vector<256x64xf8E4M3FN> to vector<256xf8E4M3FN>
+ %a1 = xegpu.convert_layout %r1
+ <{target_layout = #xegpu.slice<#xegpu.layout<sg_layout = [16, 1], sg_data = [16, 64]>, dims = [1]>}>
+ : vector<256xf8E4M3FN>
+ %r2 = vector.multi_reduction <maxnumf>, %load, %cst [1]
+ : vector<256x64xf8E4M3FN> to vector<256xf8E4M3FN>
+ %a2 = xegpu.convert_layout %r2
+ <{target_layout = #xegpu.slice<#xegpu.layout<sg_layout = [16, 1], sg_data = [16, 64]>, dims = [1]>}>
+ : vector<256xf8E4M3FN>
+ gpu.return
+ }
+
+ // f4E2M1FN has neither NaN nor infinity: all min/max kinds use the lowest/highest
+ // finite value.
+ // CHECK-LABEL: gpu.func @vector_reduce_no_nan_no_inf_neutral
+ // CHECK: %[[N0:.*]] = arith.constant dense<-6.000000e+00> : vector<16xf4E2M1FN>
+ // CHECK: vector.multi_reduction <maxnumf>, %{{.*}}, %[[N0]] [1] : vector<16x64xf4E2M1FN> to vector<16xf4E2M1FN>
+ // CHECK: %[[N1:.*]] = arith.constant dense<6.000000e+00> : vector<16xf4E2M1FN>
+ // CHECK: vector.multi_reduction <minimumf>, %{{.*}}, %[[N1]] [1] : vector<16x64xf4E2M1FN> to vector<16xf4E2M1FN>
+ gpu.func @vector_reduce_no_nan_no_inf_neutral(%src: memref<256x64xf4E2M1FN>) {
+ %cst = arith.constant dense<1.0> : vector<256xf4E2M1FN>
+ %tdesc = xegpu.create_nd_tdesc %src : memref<256x64xf4E2M1FN>
+ -> !xegpu.tensor_desc<256x64xf4E2M1FN>
+ %load = xegpu.load_nd %tdesc[0, 0] <{layout = #xegpu.layout<sg_layout = [16, 1], sg_data = [16, 64]>}>
+ : !xegpu.tensor_desc<256x64xf4E2M1FN>
+ -> vector<256x64xf4E2M1FN>
+ %r0 = vector.multi_reduction <maxnumf>, %load, %cst [1]
+ : vector<256x64xf4E2M1FN> to vector<256xf4E2M1FN>
+ %a0 = xegpu.convert_layout %r0
+ <{target_layout = #xegpu.slice<#xegpu.layout<sg_layout = [16, 1], sg_data = [16, 64]>, dims = [1]>}>
+ : vector<256xf4E2M1FN>
+ %r1 = vector.multi_reduction <minimumf>, %load, %cst [1]
+ : vector<256x64xf4E2M1FN> to vector<256xf4E2M1FN>
+ %a1 = xegpu.convert_layout %r1
+ <{target_layout = #xegpu.slice<#xegpu.layout<sg_layout = [16, 1], sg_data = [16, 64]>, dims = [1]>}>
+ : vector<256xf4E2M1FN>
+ gpu.return
+ }
+
+ // f8E8M0FNU is unsigned, with no zero and no infinity: maximumf uses the smallest
+ // value, minimumf the largest finite value.
+ // CHECK-LABEL: gpu.func @vector_reduce_unsigned_neutral
+ // CHECK: %[[N0:.*]] = arith.constant dense<5.877470e-39> : vector<16xf8E8M0FNU>
+ // CHECK: vector.multi_reduction <maximumf>, %{{.*}}, %[[N0]] [1] : vector<16x64xf8E8M0FNU> to vector<16xf8E8M0FNU>
+ // CHECK: %[[N1:.*]] = arith.constant dense<1.701410e+38> : vector<16xf8E8M0FNU>
+ // CHECK: vector.multi_reduction <minimumf>, %{{.*}}, %[[N1]] [1] : vector<16x64xf8E8M0FNU> to vector<16xf8E8M0FNU>
+ gpu.func @vector_reduce_unsigned_neutral(%src: memref<256x64xf8E8M0FNU>) {
+ %cst = arith.constant dense<1.0> : vector<256xf8E8M0FNU>
+ %tdesc = xegpu.create_nd_tdesc %src : memref<256x64xf8E8M0FNU>
+ -> !xegpu.tensor_desc<256x64xf8E8M0FNU>
+ %load = xegpu.load_nd %tdesc[0, 0] <{layout = #xegpu.layout<sg_layout = [16, 1], sg_data = [16, 64]>}>
+ : !xegpu.tensor_desc<256x64xf8E8M0FNU>
+ -> vector<256x64xf8E8M0FNU>
+ %r0 = vector.multi_reduction <maximumf>, %load, %cst [1]
+ : vector<256x64xf8E8M0FNU> to vector<256xf8E8M0FNU>
+ %a0 = xegpu.convert_layout %r0
+ <{target_layout = #xegpu.slice<#xegpu.layout<sg_layout = [16, 1], sg_data = [16, 64]>, dims = [1]>}>
+ : vector<256xf8E8M0FNU>
+ %r1 = vector.multi_reduction <minimumf>, %load, %cst [1]
+ : vector<256x64xf8E8M0FNU> to vector<256xf8E8M0FNU>
+ %a1 = xegpu.convert_layout %r1
+ <{target_layout = #xegpu.slice<#xegpu.layout<sg_layout = [16, 1], sg_data = [16, 64]>, dims = [1]>}>
+ : vector<256xf8E8M0FNU>
+ gpu.return
+ }
+
// NaN-propagating kinds have +/-inf as neutral element.
// CHECK-LABEL: gpu.func @vector_reduce_nan_propagating_neutral
gpu.func @vector_reduce_nan_propagating_neutral(%src: memref<256x64xf32>) {
More information about the Mlir-commits
mailing list