[Mlir-commits] [mlir] [MLIR][XeVM] XeVM to LLVM: Use fp conversion spirv builtins (PR #232463)
Sang Ik Lee
llvmlistbot at llvm.org
Mon Oct 19 11:57:22 PDT 2026
https://github.com/silee2 updated https://github.com/llvm/llvm-project/pull/232463
>From a6ea01e2a814556769138dc1ab8b2bb651f3391a Mon Sep 17 00:00:00 2001
From: "Lee, Sang Ik" <sang.ik.lee at intel.com>
Date: Sat, 17 Oct 2026 04:27:47 +0000
Subject: [PATCH 1/4] [mlir][XeVM] Fix the fp8 conversions, which named two
builtins that do not exist
TruncfToOCLPattern and ExtfToOCLPattern named IGC's internal fp8 builtins
directly, pasting the vector length onto the symbol. Two of those names do not
exist. IGC spells E4M3 as e4m3 in the conversion builtins, not hf8, so
f16 -> f8 asked for __builtin_IB_hftohf8_N where IGC defines
__builtin_IB_hftoe4m3_N, and f8 -> f16 asked for __builtin_IB_hf8tohf_N where
it defines __builtin_IB_e4m3tohf_N. Only E5M2 uses the bf8 spelling. The
asymmetry is easy to miss because the scaled DPAS builtin does spell E4M3 as
hf8 (bdpas_f_f_hf8_hf8_8_8), which is correct and unchanged here.
Since createDeviceFunctionCall emits these names unmangled, the nonexistent
symbol linked silently and the call returned undef: A reached the DPAS as 0xCC
fill bytes, which decode in E4M3 as -1.5 x 2^2 = -6, so a K=32 dot product of
ones produced -192 instead of 32.
Call the SPV_INTEL_fp_conversions wrappers instead:
charN __builtin_spirv_ConvertFP16ToE5M2EXT(halfN)
charN __builtin_spirv_ConvertFP16ToE4M3EXT(halfN)
charN __builtin_spirv_ConvertBF16ToE5M2EXT(shortN)
charN __builtin_spirv_ConvertBF16ToE4M3EXT(shortN)
halfN __builtin_spirv_ConvertE5M2ToFP16EXT(charN)
halfN __builtin_spirv_ConvertE4M3ToFP16EXT(charN)
shortN __builtin_spirv_ConvertE5M2ToBF16EXT(charN)
shortN __builtin_spirv_ConvertE4M3ToBF16EXT(charN)
They are declared overloadable, so the existing mangle() helper derives the
length from the argument type rather than a suffix pasted onto the name, and a
name or signature mismatch fails to link instead of binding to nothing. As the
note above these patterns says, they are the experimental SPIR-V extension
implementation rather than an official OpenCL API, so the TODO to move once one
exists still stands.
The bf16 forms take and return the bits as shortN and route through f32 and f16
internally. That retires the explicit bftof / convert_half and convert_float /
ftobf round trips, so each bf16 conversion is one call rather than three, and
lenSuffix is gone from both patterns.
Two smaller corrections fall out of using mangle() here. src_etype and dst_etype
are what say how to read the f16/bf16 side, and the verifier only constrains its
bitwidth, so f16 and bf16 are interchangeable to it and either may arrive under
either attribute. The operand is now bitcast to what the selected wrapper takes,
rather than being handed to an unmangled declaration whose parameter type was
whatever turned up, which also keeps bf16 out of mangle(), where there is no
code for it. On the extf side the result is cast to the type the op actually
produces, which additionally fixes a dst_etype and result type disagreement that
previously failed to legalize.
The fp4 lowering is left alone. Its builtins take and return one i4 per value,
which SPIR-V can only carry with SPV_INTEL_int4 and which the LLVM SPIR-V
backend cannot currently lower through a call signature, so moving fp4 onto the
vector wrappers is a separate question from this fix.
Length coverage in xevm_fp_conversion_lengths.mlir is completed for fp8 while
the expected symbols are being updated: E4M3 gains lengths other than 16 in both
directions, extf gains its only odd length at 3, and bf16 -> E5M2 moves off
being tested at 16 alone.
---
mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp | 182 ++++++++----------
.../xevm_fp_conversion_lengths.mlir | 114 ++++++++---
.../XeVMToLLVM/xevm_mx-to-llvm.mlir | 76 +++-----
3 files changed, 199 insertions(+), 173 deletions(-)
diff --git a/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp b/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp
index f3bf1a5fc7866..ecf7d2b3e1b08 100644
--- a/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp
+++ b/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp
@@ -1351,62 +1351,62 @@ class TruncfToOCLPattern : public OpConversionPattern<TruncfOp> {
}
// Handle the case where dst type is fp8.
- // The fp8 conversions come as one builtin per vector length, so the length
- // is simply appended to the builtin name.
- std::string lenSuffix = std::to_string(numElements);
- // BF16 type needs some preprocessing before conversion,
- // First extended to F32 and then truncated to F16.
- if (srcEtype == TruncfSrcElemTypes::BF16) {
- // Step 1: Extend to F32
- // Use floatN __builtin_IB_bftof_N(shortN)
- src = LLVM::BitcastOp::create(
- rewriter, op.getLoc(),
- VectorType::get(vecSrcTy.getShape(), rewriter.getI16Type()), src);
- std::string fnName = "__builtin_IB_bftof_" + lenSuffix;
- SmallVector<Type> argTypes{src.getType()};
- SmallVector<Value> args{src};
- Type resTy = VectorType::get(vecSrcTy.getShape(), rewriter.getF32Type());
- src = createDeviceFunctionCall(rewriter, fnName, resTy, argTypes, args,
- {}, funcAttrs, op.getOperation())
- ->getResult(0);
- // Step 2: Truncf to F16
- // Use halfN convert_halfN(floatN)
- std::string truncFnName = "convert_half" + lenSuffix;
- SmallVector<Type> truncArgTypes{src.getType()};
- SmallVector<Value> truncArgs{src};
- truncFnName = mangle(truncFnName, truncArgTypes);
- resTy = VectorType::get(vecSrcTy.getShape(), rewriter.getF16Type());
- src =
- createDeviceFunctionCall(rewriter, truncFnName, resTy, truncArgTypes,
- truncArgs, {}, funcAttrs, op.getOperation())
- ->getResult(0);
- }
+ //
+ // The conversions come from the SPV_INTEL_fp_conversions half of the IGC
+ // BiFModule, which overloads one wrapper per vector length:
+ // charN __builtin_spirv_ConvertFP16ToE5M2EXT(halfN)
+ // charN __builtin_spirv_ConvertFP16ToE4M3EXT(halfN)
+ // charN __builtin_spirv_ConvertBF16ToE5M2EXT(shortN)
+ // charN __builtin_spirv_ConvertBF16ToE4M3EXT(shortN)
+ //
+ // These wrap the __builtin_IB_* builtins this used to name directly. Two
+ // of those names do not exist: IGC spells E4M3 as `e4m3` in the conversion
+ // builtins, not `hf8`, so `__builtin_IB_hftohf8_N` was never defined -
+ // even though the scaled DPAS builtin does use `hf8`
+ // (bdpas_f_f_hf8_hf8_8_8), which is correct and unchanged here. Because
+ // createDeviceFunctionCall emits these names unmangled, the nonexistent
+ // symbol linked silently and the call returned undef.
+ //
+ // The wrappers are declared `overloadable`, so the existing mangle() helper
+ // derives the vector length from the argument type instead of pasting a
+ // suffix onto the name, and a name or signature mismatch fails to link
+ // rather than binding to nothing. As the note above these patterns says,
+ // they are the experimental SPIR-V extension implementation rather than an
+ // official OpenCL API, so the TODO to move once one exists still stands.
+ //
+ // A bf16 source takes the bf16 form, which accepts the bits as shortN and
+ // routes through f32 and f16 internally. That retires the explicit
+ // __builtin_IB_bftof plus convert_half round trip, so each bf16 conversion
+ // is one call rather than three.
+ //
+ // `src_etype` is what says how to read the source, and the verifier only
+ // constrains the operand's bitwidth, so f16 and bf16 are interchangeable to
+ // it and either may arrive under either attribute. Coerce the operand to
+ // what the selected wrapper takes, which also keeps bf16 out of the
+ // mangling, where there is no code for it.
+ bool srcIsBF16 = srcEtype == TruncfSrcElemTypes::BF16;
+ Type argElemTy = srcIsBF16 ? cast<Type>(rewriter.getI16Type())
+ : cast<Type>(rewriter.getF16Type());
+ src = castIfNeeded(rewriter, loc,
+ VectorType::get(vecSrcTy.getShape(), argElemTy), src);
+ StringRef convertName;
if (dstEtype == TruncfDstElemTypes::BF8) { // Float8E5M2Type
- // Use charN __builtin_IB_hftobf8_N(halfN)
- std::string fnName = "__builtin_IB_hftobf8_" + lenSuffix;
- SmallVector<Type> argTypes{src.getType()};
- SmallVector<Value> args{src};
- Value result =
- createDeviceFunctionCall(rewriter, fnName, dstTy, argTypes, args, {},
- funcAttrs, op.getOperation())
- ->getResult(0);
-
- rewriter.replaceOp(op, result);
+ convertName = srcIsBF16 ? "__builtin_spirv_ConvertBF16ToE5M2EXT"
+ : "__builtin_spirv_ConvertFP16ToE5M2EXT";
} else if (dstEtype == TruncfDstElemTypes::F8) { // Float8E4M3FNType
- // Use charN __builtin_IB_hftohf8_N(halfN)
- std::string fnName = "__builtin_IB_hftohf8_" + lenSuffix;
- SmallVector<Type> argTypes{src.getType()};
- SmallVector<Value> args{src};
- Value result =
- createDeviceFunctionCall(rewriter, fnName, dstTy, argTypes, args, {},
- funcAttrs, op.getOperation())
- ->getResult(0);
-
- rewriter.replaceOp(op, result);
+ convertName = srcIsBF16 ? "__builtin_spirv_ConvertBF16ToE4M3EXT"
+ : "__builtin_spirv_ConvertFP16ToE4M3EXT";
} else {
return rewriter.notifyMatchFailure(
op, "Unsupported src, dst element type pair.");
}
+ SmallVector<Type> argTypes{src.getType()};
+ SmallVector<Value> args{src};
+ Value result = createDeviceFunctionCall(
+ rewriter, mangle(convertName, argTypes), dstTy, argTypes,
+ args, {}, funcAttrs, op.getOperation())
+ ->getResult(0);
+ rewriter.replaceOp(op, result);
return success();
}
};
@@ -1507,55 +1507,43 @@ class ExtfToOCLPattern : public OpConversionPattern<ExtfOp> {
if (!vecSrcTy || vecSrcTy.getNumElements() != numElements)
return rewriter.notifyMatchFailure(
op, "fp8 src and dst must have the same number of elements");
- std::string lenSuffix = std::to_string(numElements);
-
- // Step 1: Extend fp8 (bf8/hf8) to F16.
- // bf8 -> half: halfN __builtin_IB_bf8tohf_N(charN)
- // hf8 -> half: halfN __builtin_IB_hf8tohf_N(charN)
- std::string fnName = (srcEtype == ExtfSrcElemTypes::BF8)
- ? "__builtin_IB_bf8tohf_"
- : "__builtin_IB_hf8tohf_";
- fnName += lenSuffix;
- Type f16Ty = VectorType::get(vecSrcTy.getShape(), rewriter.getF16Type());
- SmallVector<Type> argTypes{src.getType()};
- SmallVector<Value> args{src};
- Value result =
- createDeviceFunctionCall(rewriter, fnName, f16Ty, argTypes, args, {},
- funcAttrs, op.getOperation())
- ->getResult(0);
- // When the destination is F16, we are done.
- if (dstEtype == ExtfDstElemTypes::F16) {
- rewriter.replaceOp(op, result);
- return success();
+ // Mangled SPV_INTEL_fp_conversions wrappers, for the reasons given in the
+ // truncf lowering above:
+ // halfN __builtin_spirv_ConvertE5M2ToFP16EXT(charN)
+ // halfN __builtin_spirv_ConvertE4M3ToFP16EXT(charN)
+ // shortN __builtin_spirv_ConvertE5M2ToBF16EXT(charN)
+ // shortN __builtin_spirv_ConvertE4M3ToBF16EXT(charN)
+ //
+ // As on the truncf side, `__builtin_IB_hf8tohf_N` never existed - IGC
+ // defines `__builtin_IB_e4m3tohf_N` - so the f8 -> f16 call linked to
+ // nothing and returned undef.
+ //
+ // A bf16 destination takes the bf16 form, which returns the bits as shortN,
+ // retiring the convert_float plus __builtin_IB_ftobf round trip: one call
+ // instead of three.
+ bool dstIsBF16 = dstEtype == ExtfDstElemTypes::BF16;
+ StringRef convertName;
+ if (srcEtype == ExtfSrcElemTypes::BF8) { // Float8E5M2Type
+ convertName = dstIsBF16 ? "__builtin_spirv_ConvertE5M2ToBF16EXT"
+ : "__builtin_spirv_ConvertE5M2ToFP16EXT";
+ } else { // Float8E4M3FNType
+ convertName = dstIsBF16 ? "__builtin_spirv_ConvertE4M3ToBF16EXT"
+ : "__builtin_spirv_ConvertE4M3ToFP16EXT";
}
-
- // BF16 destination needs some postprocessing.
- // First extend F16 to F32 and then truncate to BF16.
- // Step 2: Extend to F32.
- // Use floatN convert_floatN(halfN)
- std::string convFnName = "convert_float" + lenSuffix;
- SmallVector<Type> convArgTypes{result.getType()};
- SmallVector<Value> convArgs{result};
- convFnName = mangle(convFnName, convArgTypes);
- Type f32Ty = VectorType::get(vecSrcTy.getShape(), rewriter.getF32Type());
- result =
- createDeviceFunctionCall(rewriter, convFnName, f32Ty, convArgTypes,
- convArgs, {}, funcAttrs, op.getOperation())
- ->getResult(0);
- // Step 3: Truncate F32 to BF16.
- // Use shortN __builtin_IB_ftobf_N(floatN)
- std::string ftobfFnName = "__builtin_IB_ftobf_" + lenSuffix;
- SmallVector<Type> ftobfArgTypes{result.getType()};
- SmallVector<Value> ftobfArgs{result};
- Type i16Ty = VectorType::get(vecSrcTy.getShape(), rewriter.getI16Type());
- result =
- createDeviceFunctionCall(rewriter, ftobfFnName, i16Ty, ftobfArgTypes,
- ftobfArgs, {}, funcAttrs, op.getOperation())
- ->getResult(0);
- // The builtin returns the bf16 bits as i16, bitcast to the bf16 dst type.
- result = LLVM::BitcastOp::create(rewriter, op.getLoc(), vecDstTy, result);
- rewriter.replaceOp(op, result);
+ Type resElemTy = dstIsBF16 ? cast<Type>(rewriter.getI16Type())
+ : cast<Type>(rewriter.getF16Type());
+ Type resTy = VectorType::get(vecSrcTy.getShape(), resElemTy);
+ SmallVector<Type> argTypes{src.getType()};
+ SmallVector<Value> args{src};
+ Value result = createDeviceFunctionCall(
+ rewriter, mangle(convertName, argTypes), resTy, argTypes,
+ args, {}, funcAttrs, op.getOperation())
+ ->getResult(0);
+ // The bf16 wrapper returns the bf16 bits as i16. As on the truncf side,
+ // `dst_etype` may disagree with the result's element type, so cast to what
+ // the op actually produces rather than to what the attribute implies.
+ rewriter.replaceOp(op, castIfNeeded(rewriter, loc, vecDstTy, result));
return success();
}
};
diff --git a/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir b/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir
index fa2183f358798..a6ad1b8c8f680 100644
--- a/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir
+++ b/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir
@@ -4,11 +4,11 @@
// This file covers the SPIR-V vector lengths other than 16, which
// xevm_mx-to-llvm.mlir covers.
-// CHECK: llvm.func spir_funccc @__builtin_IB_hftobf8_2(vector<2xf16>) -> vector<2xi8>
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE5M2EXTDv2_Dh(vector<2xf16>) -> vector<2xi8>
// CHECK-LABEL: llvm.func @truncf_f16_to_bf8_v2
// CHECK-SAME: %[[ARG0:.*]]: vector<2xf16>
llvm.func @truncf_f16_to_bf8_v2(%src: vector<2xf16>) -> vector<2xi8> {
- // CHECK: %[[RES:.*]] = llvm.call spir_funccc @__builtin_IB_hftobf8_2(%[[ARG0]])
+ // CHECK: %[[RES:.*]] = llvm.call spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE5M2EXTDv2_Dh(%[[ARG0]])
// CHECK-SAME: : (vector<2xf16>) -> vector<2xi8>
%dst = xevm.truncf %src { src_etype = f16, dst_etype = bf8 } : (vector<2xf16>) -> vector<2xi8>
llvm.return %dst : vector<2xi8>
@@ -16,65 +16,127 @@ llvm.func @truncf_f16_to_bf8_v2(%src: vector<2xf16>) -> vector<2xi8> {
// -----
-// A 3 element vector is a SPIR-V vector length, and the builtins provide it.
-// CHECK: llvm.func spir_funccc @__builtin_IB_hftohf8_3(vector<3xf16>) -> vector<3xi8>
-// CHECK: llvm.func spir_funccc @_Z13convert_half3Dv3_f(vector<3xf32>) -> vector<3xf16>
-// CHECK: llvm.func spir_funccc @__builtin_IB_bftof_3(vector<3xi16>) -> vector<3xf32>
+// A 3 element vector is a SPIR-V vector length, and the wrappers overload it.
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertBF16ToE4M3EXTDv3_s(vector<3xi16>) -> vector<3xi8>
// CHECK-LABEL: llvm.func @truncf_bf16_to_hf8_v3
// CHECK-SAME: %[[ARG0:.*]]: vector<3xbf16>
llvm.func @truncf_bf16_to_hf8_v3(%src: vector<3xbf16>) -> vector<3xi8> {
// CHECK: %[[BC:.*]] = llvm.bitcast %[[ARG0]] : vector<3xbf16> to vector<3xi16>
- // CHECK: %[[F32:.*]] = llvm.call spir_funccc @__builtin_IB_bftof_3(%[[BC]])
- // CHECK-SAME: : (vector<3xi16>) -> vector<3xf32>
- // CHECK: %[[F16:.*]] = llvm.call spir_funccc @_Z13convert_half3Dv3_f(%[[F32]])
- // CHECK-SAME: : (vector<3xf32>) -> vector<3xf16>
- // CHECK: %[[RES:.*]] = llvm.call spir_funccc @__builtin_IB_hftohf8_3(%[[F16]])
- // CHECK-SAME: : (vector<3xf16>) -> vector<3xi8>
+ // CHECK: %[[RES:.*]] = llvm.call spir_funccc @_Z36__builtin_spirv_ConvertBF16ToE4M3EXTDv3_s(%[[BC]])
+ // CHECK-SAME: : (vector<3xi16>) -> vector<3xi8>
%dst = xevm.truncf %src { src_etype = bf16, dst_etype = f8 } : (vector<3xbf16>) -> vector<3xi8>
llvm.return %dst : vector<3xi8>
}
// -----
-// CHECK: llvm.func spir_funccc @__builtin_IB_hftobf8_8(vector<8xf16>) -> vector<8xi8>
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE5M2EXTDv8_Dh(vector<8xf16>) -> vector<8xi8>
// CHECK-LABEL: llvm.func @truncf_f16_to_bf8_v8
llvm.func @truncf_f16_to_bf8_v8(%src: vector<8xf16>) -> vector<8xi8> {
- // CHECK: llvm.call spir_funccc @__builtin_IB_hftobf8_8({{.*}}) {{.*}} : (vector<8xf16>) -> vector<8xi8>
+ // CHECK: llvm.call spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE5M2EXTDv8_Dh({{.*}}) {{.*}} : (vector<8xf16>) -> vector<8xi8>
%dst = xevm.truncf %src { src_etype = f16, dst_etype = bf8 } : (vector<8xf16>) -> vector<8xi8>
llvm.return %dst : vector<8xi8>
}
// -----
-// CHECK: llvm.func spir_funccc @__builtin_IB_bf8tohf_2(vector<2xi8>) -> vector<2xf16>
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertE5M2ToFP16EXTDv2_c(vector<2xi8>) -> vector<2xf16>
// CHECK-LABEL: llvm.func @extf_bf8_to_f16_v2
llvm.func @extf_bf8_to_f16_v2(%src: vector<2xi8>) -> vector<2xf16> {
- // CHECK: llvm.call spir_funccc @__builtin_IB_bf8tohf_2({{.*}}) {{.*}} : (vector<2xi8>) -> vector<2xf16>
+ // CHECK: llvm.call spir_funccc @_Z36__builtin_spirv_ConvertE5M2ToFP16EXTDv2_c({{.*}}) {{.*}} : (vector<2xi8>) -> vector<2xf16>
%dst = xevm.extf %src { src_etype = bf8, dst_etype = f16 } : (vector<2xi8>) -> vector<2xf16>
llvm.return %dst : vector<2xf16>
}
// -----
-// CHECK: llvm.func spir_funccc @__builtin_IB_ftobf_4(vector<4xf32>) -> vector<4xi16>
-// CHECK: llvm.func spir_funccc @_Z14convert_float4Dv4_Dh(vector<4xf16>) -> vector<4xf32>
-// CHECK: llvm.func spir_funccc @__builtin_IB_bf8tohf_4(vector<4xi8>) -> vector<4xf16>
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertE5M2ToBF16EXTDv4_c(vector<4xi8>) -> vector<4xi16>
// CHECK-LABEL: llvm.func @extf_bf8_to_bf16_v4
// CHECK-SAME: %[[ARG0:.*]]: vector<4xi8>
llvm.func @extf_bf8_to_bf16_v4(%src: vector<4xi8>) -> vector<4xbf16> {
- // CHECK: %[[F16:.*]] = llvm.call spir_funccc @__builtin_IB_bf8tohf_4(%[[ARG0]])
- // CHECK-SAME: : (vector<4xi8>) -> vector<4xf16>
- // CHECK: %[[F32:.*]] = llvm.call spir_funccc @_Z14convert_float4Dv4_Dh(%[[F16]])
- // CHECK-SAME: : (vector<4xf16>) -> vector<4xf32>
- // CHECK: %[[BF:.*]] = llvm.call spir_funccc @__builtin_IB_ftobf_4(%[[F32]])
- // CHECK-SAME: : (vector<4xf32>) -> vector<4xi16>
- // CHECK: %[[RES:.*]] = llvm.bitcast %[[BF]] : vector<4xi16> to vector<4xbf16>
+ // CHECK: %[[F16:.*]] = llvm.call spir_funccc @_Z36__builtin_spirv_ConvertE5M2ToBF16EXTDv4_c(%[[ARG0]])
+ // CHECK-SAME: : (vector<4xi8>) -> vector<4xi16>
+ // CHECK: %[[RES:.*]] = llvm.bitcast %[[F16]] : vector<4xi16> to vector<4xbf16>
%dst = xevm.extf %src { src_etype = bf8, dst_etype = bf16 } : (vector<4xi8>) -> vector<4xbf16>
llvm.return %dst : vector<4xbf16>
}
// -----
+// f16 to E4M3 away from length 16. Note IGC spells E4M3 `e4m3` in the
+// conversion builtins, not `hf8`.
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE4M3EXTDv4_Dh(vector<4xf16>) -> vector<4xi8>
+// CHECK-LABEL: llvm.func @truncf_f16_to_hf8_v4
+// CHECK-SAME: %[[ARG0:.*]]: vector<4xf16>
+llvm.func @truncf_f16_to_hf8_v4(%src: vector<4xf16>) -> vector<4xi8> {
+ // CHECK: llvm.call spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE4M3EXTDv4_Dh(%[[ARG0]])
+ // CHECK-SAME: : (vector<4xf16>) -> vector<4xi8>
+ %dst = xevm.truncf %src { src_etype = f16, dst_etype = f8 } : (vector<4xf16>) -> vector<4xi8>
+ llvm.return %dst : vector<4xi8>
+}
+
+// -----
+
+// E4M3 to f16 away from length 16.
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertE4M3ToFP16EXTDv8_c(vector<8xi8>) -> vector<8xf16>
+// CHECK-LABEL: llvm.func @extf_hf8_to_f16_v8
+// CHECK-SAME: %[[ARG0:.*]]: vector<8xi8>
+llvm.func @extf_hf8_to_f16_v8(%src: vector<8xi8>) -> vector<8xf16> {
+ // CHECK: llvm.call spir_funccc @_Z36__builtin_spirv_ConvertE4M3ToFP16EXTDv8_c(%[[ARG0]])
+ // CHECK-SAME: : (vector<8xi8>) -> vector<8xf16>
+ %dst = xevm.extf %src { src_etype = f8, dst_etype = f16 } : (vector<8xi8>) -> vector<8xf16>
+ llvm.return %dst : vector<8xf16>
+}
+
+// -----
+
+// `src_etype` is authoritative, and the verifier only constrains the operand's
+// bitwidth, so a bf16 vector can arrive under `src_etype = f16`. The operand is
+// reinterpreted as what the selected wrapper takes rather than reaching the
+// mangler as a bf16 type, which it has no code for.
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE5M2EXTDv2_Dh(vector<2xf16>) -> vector<2xi8>
+// CHECK-LABEL: llvm.func @truncf_etype_f16_operand_bf16
+// CHECK-SAME: %[[ARG0:.*]]: vector<2xbf16>
+llvm.func @truncf_etype_f16_operand_bf16(%src: vector<2xbf16>) -> vector<2xi8> {
+ // CHECK: %[[SRC:.*]] = llvm.bitcast %[[ARG0]] : vector<2xbf16> to vector<2xf16>
+ // CHECK: llvm.call spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE5M2EXTDv2_Dh(%[[SRC]])
+ // CHECK-SAME: : (vector<2xf16>) -> vector<2xi8>
+ %dst = xevm.truncf %src { src_etype = f16, dst_etype = bf8 } : (vector<2xbf16>) -> vector<2xi8>
+ llvm.return %dst : vector<2xi8>
+}
+
+// -----
+
+// The only odd length for an fp8 conversion, where one value per byte means the
+// source and the destination have the same length.
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertE5M2ToBF16EXTDv3_c(vector<3xi8>) -> vector<3xi16>
+// CHECK-LABEL: llvm.func @extf_bf8_to_bf16_v3
+// CHECK-SAME: %[[ARG0:.*]]: vector<3xi8>
+llvm.func @extf_bf8_to_bf16_v3(%src: vector<3xi8>) -> vector<3xbf16> {
+ // CHECK: %[[CONV:.*]] = llvm.call spir_funccc @_Z36__builtin_spirv_ConvertE5M2ToBF16EXTDv3_c(%[[ARG0]])
+ // CHECK-SAME: : (vector<3xi8>) -> vector<3xi16>
+ // CHECK: llvm.bitcast %[[CONV]] : vector<3xi16> to vector<3xbf16>
+ %dst = xevm.extf %src { src_etype = bf8, dst_etype = bf16 } : (vector<3xi8>) -> vector<3xbf16>
+ llvm.return %dst : vector<3xbf16>
+}
+
+// -----
+
+// bf16 to E5M2 away from length 16. The bf16 wrapper takes the bits as shortN,
+// so the only preprocessing is the bitcast.
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertBF16ToE5M2EXTDv4_s(vector<4xi16>) -> vector<4xi8>
+// CHECK-LABEL: llvm.func @truncf_bf16_to_bf8_v4
+// CHECK-SAME: %[[ARG0:.*]]: vector<4xbf16>
+llvm.func @truncf_bf16_to_bf8_v4(%src: vector<4xbf16>) -> vector<4xi8> {
+ // CHECK: %[[SRC:.*]] = llvm.bitcast %[[ARG0]] : vector<4xbf16> to vector<4xi16>
+ // CHECK: llvm.call spir_funccc @_Z36__builtin_spirv_ConvertBF16ToE5M2EXTDv4_s(%[[SRC]])
+ // CHECK-SAME: : (vector<4xi16>) -> vector<4xi8>
+ %dst = xevm.truncf %src { src_etype = bf16, dst_etype = bf8 } : (vector<4xbf16>) -> vector<4xi8>
+ llvm.return %dst : vector<4xi8>
+}
+
+// -----
+
// Two fp4 values pack into a single byte. SPIR-V has no vector of length one,
// so the packed side is a scalar, and only the low byte of the dnscl result is
// kept.
diff --git a/mlir/test/Conversion/XeVMToLLVM/xevm_mx-to-llvm.mlir b/mlir/test/Conversion/XeVMToLLVM/xevm_mx-to-llvm.mlir
index bc92009292959..d9e5640a6a8c1 100644
--- a/mlir/test/Conversion/XeVMToLLVM/xevm_mx-to-llvm.mlir
+++ b/mlir/test/Conversion/XeVMToLLVM/xevm_mx-to-llvm.mlir
@@ -1,18 +1,18 @@
// RUN: mlir-opt --convert-xevm-to-llvm --split-input-file %s | FileCheck %s
-// CHECK: llvm.func spir_funccc @__builtin_IB_hftobf8_16(vector<16xf16>) -> vector<16xi8>
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE5M2EXTDv16_Dh(vector<16xf16>) -> vector<16xi8>
// CHECK-SAME: attributes {convergent, memory_effects = #llvm.memory_effects<other = none,
// CHECK-SAME: argMem = none, inaccessibleMem = none, errnoMem = none,
// CHECK-SAME: targetMem0 = none, targetMem1 = none>, no_unwind, will_return}
// CHECK-LABEL: llvm.func @truncf_f16_to_bf8
// CHECK-SAME: %[[ARG0:.*]]: vector<16xf16>
llvm.func @truncf_f16_to_bf8(%src: vector<16xf16>) -> vector<16xi8> {
- // CHECK: %[[VAR0:.*]] = llvm.call spir_funccc @__builtin_IB_hftobf8_16(%[[ARG0]])
+ // CHECK: %[[VAR0:.*]] = llvm.call spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE5M2EXTDv16_Dh(%[[ARG0]])
// CHECK-SAME: {convergent, function_type = !llvm.func<vector<16xi8> (vector<16xf16>)>,
// CHECK-SAME: linkage = #llvm.linkage<external>, memory_effects = #llvm.memory_effects<other = none,
// CHECK-SAME: argMem = none, inaccessibleMem = none, errnoMem = none,
// CHECK-SAME: targetMem0 = none, targetMem1 = none>,
- // CHECK-SAME: no_unwind, sym_name = "__builtin_IB_hftobf8_16",
+ // CHECK-SAME: no_unwind, sym_name = "_Z36__builtin_spirv_ConvertFP16ToE5M2EXTDv16_Dh",
// CHECK-SAME: visibility_ = 0 : i64, will_return} :
// CHECK-SAME: (vector<16xf16>) -> vector<16xi8>
%dst = xevm.truncf %src { src_etype = f16, dst_etype = bf8 } : (vector<16xf16>) -> vector<16xi8>
@@ -21,19 +21,19 @@ llvm.func @truncf_f16_to_bf8(%src: vector<16xf16>) -> vector<16xi8> {
// -----
-// CHECK: llvm.func spir_funccc @__builtin_IB_hftohf8_16(vector<16xf16>) -> vector<16xi8>
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE4M3EXTDv16_Dh(vector<16xf16>) -> vector<16xi8>
// CHECK-SAME: attributes {convergent, memory_effects = #llvm.memory_effects<other = none,
// CHECK-SAME: argMem = none, inaccessibleMem = none, errnoMem = none,
// CHECK-SAME: targetMem0 = none, targetMem1 = none>, no_unwind, will_return}
// CHECK-LABEL: llvm.func @truncf_f16_to_hf8
// CHECK-SAME: %[[ARG0:.*]]: vector<16xf16>
llvm.func @truncf_f16_to_hf8(%src: vector<16xf16>) -> vector<16xi8> {
- // CHECK: %[[VAR0:.*]] = llvm.call spir_funccc @__builtin_IB_hftohf8_16(%[[ARG0]])
+ // CHECK: %[[VAR0:.*]] = llvm.call spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE4M3EXTDv16_Dh(%[[ARG0]])
// CHECK-SAME: {convergent, function_type = !llvm.func<vector<16xi8> (vector<16xf16>)>,
// CHECK-SAME: linkage = #llvm.linkage<external>, memory_effects = #llvm.memory_effects<other = none,
// CHECK-SAME: argMem = none, inaccessibleMem = none, errnoMem = none,
// CHECK-SAME: targetMem0 = none, targetMem1 = none>,
- // CHECK-SAME: no_unwind, sym_name = "__builtin_IB_hftohf8_16",
+ // CHECK-SAME: no_unwind, sym_name = "_Z36__builtin_spirv_ConvertFP16ToE4M3EXTDv16_Dh",
// CHECK-SAME: visibility_ = 0 : i64, will_return} :
// CHECK-SAME: (vector<16xf16>) -> vector<16xi8>
%dst = xevm.truncf %src { src_etype = f16, dst_etype = f8 } : (vector<16xf16>) -> vector<16xi8>
@@ -42,38 +42,26 @@ llvm.func @truncf_f16_to_hf8(%src: vector<16xf16>) -> vector<16xi8> {
// -----
-// CHECK: llvm.func spir_funccc @__builtin_IB_hftobf8_16(vector<16xf16>) -> vector<16xi8>
-// CHECK: llvm.func spir_funccc @_Z14convert_half16Dv16_f(vector<16xf32>) -> vector<16xf16>
-// CHECK: llvm.func spir_funccc @__builtin_IB_bftof_16(vector<16xi16>) -> vector<16xf32>
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertBF16ToE5M2EXTDv16_s(vector<16xi16>) -> vector<16xi8>
// CHECK-LABEL: llvm.func @truncf_bf16_to_bf8
// CHECK-SAME: %[[ARG0:.*]]: vector<16xbf16>
llvm.func @truncf_bf16_to_bf8(%src: vector<16xbf16>) -> vector<16xi8> {
// CHECK: %[[VAR0:.*]] = llvm.bitcast %[[ARG0]] : vector<16xbf16> to vector<16xi16>
- // CHECK: %[[VAR1:.*]] = llvm.call spir_funccc @__builtin_IB_bftof_16(%[[VAR0]])
- // CHECK-SAME: : (vector<16xi16>) -> vector<16xf32>
- // CHECK: %[[VAR2:.*]] = llvm.call spir_funccc @_Z14convert_half16Dv16_f(%[[VAR1]])
- // CHECK-SAME: : (vector<16xf32>) -> vector<16xf16>
- // CHECK: %[[VAR3:.*]] = llvm.call spir_funccc @__builtin_IB_hftobf8_16(%[[VAR2]])
- // CHECK-SAME: : (vector<16xf16>) -> vector<16xi8>
+ // CHECK: %[[VAR3:.*]] = llvm.call spir_funccc @_Z36__builtin_spirv_ConvertBF16ToE5M2EXTDv16_s(%[[VAR0]])
+ // CHECK-SAME: : (vector<16xi16>) -> vector<16xi8>
%dst = xevm.truncf %src { src_etype = bf16, dst_etype = bf8 } : (vector<16xbf16>) -> vector<16xi8>
llvm.return %dst : vector<16xi8>
}
// -----
-// CHECK: llvm.func spir_funccc @__builtin_IB_hftohf8_16(vector<16xf16>) -> vector<16xi8>
-// CHECK: llvm.func spir_funccc @_Z14convert_half16Dv16_f(vector<16xf32>) -> vector<16xf16>
-// CHECK: llvm.func spir_funccc @__builtin_IB_bftof_16(vector<16xi16>) -> vector<16xf32>
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertBF16ToE4M3EXTDv16_s(vector<16xi16>) -> vector<16xi8>
// CHECK-LABEL: llvm.func @truncf_bf16_to_hf8
// CHECK-SAME: %[[ARG0:.*]]: vector<16xbf16>
llvm.func @truncf_bf16_to_hf8(%src: vector<16xbf16>) -> vector<16xi8> {
// CHECK: %[[VAR0:.*]] = llvm.bitcast %[[ARG0]] : vector<16xbf16> to vector<16xi16>
- // CHECK: %[[VAR1:.*]] = llvm.call spir_funccc @__builtin_IB_bftof_16(%[[VAR0]])
- // CHECK-SAME: : (vector<16xi16>) -> vector<16xf32>
- // CHECK: %[[VAR2:.*]] = llvm.call spir_funccc @_Z14convert_half16Dv16_f(%[[VAR1]])
- // CHECK-SAME: : (vector<16xf32>) -> vector<16xf16>
- // CHECK: %[[VAR3:.*]] = llvm.call spir_funccc @__builtin_IB_hftohf8_16(%[[VAR2]])
- // CHECK-SAME: : (vector<16xf16>) -> vector<16xi8>
+ // CHECK: %[[VAR3:.*]] = llvm.call spir_funccc @_Z36__builtin_spirv_ConvertBF16ToE4M3EXTDv16_s(%[[VAR0]])
+ // CHECK-SAME: : (vector<16xi16>) -> vector<16xi8>
%dst = xevm.truncf %src { src_etype = bf16, dst_etype = f8 } : (vector<16xbf16>) -> vector<16xi8>
llvm.return %dst : vector<16xi8>
}
@@ -162,19 +150,19 @@ llvm.func @truncf_bf16_to_e2m1(%src: vector<16xbf16>) -> vector<8xi8> {
// -----
-// CHECK: llvm.func spir_funccc @__builtin_IB_bf8tohf_16(vector<16xi8>) -> vector<16xf16>
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertE5M2ToFP16EXTDv16_c(vector<16xi8>) -> vector<16xf16>
// CHECK-SAME: attributes {convergent, memory_effects = #llvm.memory_effects<other = none,
// CHECK-SAME: argMem = none, inaccessibleMem = none, errnoMem = none,
// CHECK-SAME: targetMem0 = none, targetMem1 = none>, no_unwind, will_return}
// CHECK-LABEL: llvm.func @extf_bf8_to_f16
// CHECK-SAME: %[[ARG0:.*]]: vector<16xi8>
llvm.func @extf_bf8_to_f16(%src: vector<16xi8>) -> vector<16xf16> {
- // CHECK: %[[VAR0:.*]] = llvm.call spir_funccc @__builtin_IB_bf8tohf_16(%[[ARG0]])
+ // CHECK: %[[VAR0:.*]] = llvm.call spir_funccc @_Z36__builtin_spirv_ConvertE5M2ToFP16EXTDv16_c(%[[ARG0]])
// CHECK-SAME: {convergent, function_type = !llvm.func<vector<16xf16> (vector<16xi8>)>,
// CHECK-SAME: linkage = #llvm.linkage<external>, memory_effects = #llvm.memory_effects<other = none,
// CHECK-SAME: argMem = none, inaccessibleMem = none, errnoMem = none,
// CHECK-SAME: targetMem0 = none, targetMem1 = none>,
- // CHECK-SAME: no_unwind, sym_name = "__builtin_IB_bf8tohf_16",
+ // CHECK-SAME: no_unwind, sym_name = "_Z36__builtin_spirv_ConvertE5M2ToFP16EXTDv16_c",
// CHECK-SAME: visibility_ = 0 : i64, will_return} :
// CHECK-SAME: (vector<16xi8>) -> vector<16xf16>
%dst = xevm.extf %src { src_etype = bf8, dst_etype = f16 } : (vector<16xi8>) -> vector<16xf16>
@@ -183,19 +171,19 @@ llvm.func @extf_bf8_to_f16(%src: vector<16xi8>) -> vector<16xf16> {
// -----
-// CHECK: llvm.func spir_funccc @__builtin_IB_hf8tohf_16(vector<16xi8>) -> vector<16xf16>
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertE4M3ToFP16EXTDv16_c(vector<16xi8>) -> vector<16xf16>
// CHECK-SAME: attributes {convergent, memory_effects = #llvm.memory_effects<other = none,
// CHECK-SAME: argMem = none, inaccessibleMem = none, errnoMem = none,
// CHECK-SAME: targetMem0 = none, targetMem1 = none>, no_unwind, will_return}
// CHECK-LABEL: llvm.func @extf_f8_to_f16
// CHECK-SAME: %[[ARG0:.*]]: vector<16xi8>
llvm.func @extf_f8_to_f16(%src: vector<16xi8>) -> vector<16xf16> {
- // CHECK: %[[VAR0:.*]] = llvm.call spir_funccc @__builtin_IB_hf8tohf_16(%[[ARG0]])
+ // CHECK: %[[VAR0:.*]] = llvm.call spir_funccc @_Z36__builtin_spirv_ConvertE4M3ToFP16EXTDv16_c(%[[ARG0]])
// CHECK-SAME: {convergent, function_type = !llvm.func<vector<16xf16> (vector<16xi8>)>,
// CHECK-SAME: linkage = #llvm.linkage<external>, memory_effects = #llvm.memory_effects<other = none,
// CHECK-SAME: argMem = none, inaccessibleMem = none, errnoMem = none,
// CHECK-SAME: targetMem0 = none, targetMem1 = none>,
- // CHECK-SAME: no_unwind, sym_name = "__builtin_IB_hf8tohf_16",
+ // CHECK-SAME: no_unwind, sym_name = "_Z36__builtin_spirv_ConvertE4M3ToFP16EXTDv16_c",
// CHECK-SAME: visibility_ = 0 : i64, will_return} :
// CHECK-SAME: (vector<16xi8>) -> vector<16xf16>
%dst = xevm.extf %src { src_etype = f8, dst_etype = f16 } : (vector<16xi8>) -> vector<16xf16>
@@ -204,38 +192,26 @@ llvm.func @extf_f8_to_f16(%src: vector<16xi8>) -> vector<16xf16> {
// -----
-// CHECK: llvm.func spir_funccc @__builtin_IB_ftobf_16(vector<16xf32>) -> vector<16xi16>
-// CHECK: llvm.func spir_funccc @_Z15convert_float16Dv16_Dh(vector<16xf16>) -> vector<16xf32>
-// CHECK: llvm.func spir_funccc @__builtin_IB_bf8tohf_16(vector<16xi8>) -> vector<16xf16>
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertE5M2ToBF16EXTDv16_c(vector<16xi8>) -> vector<16xi16>
// CHECK-LABEL: llvm.func @extf_bf8_to_bf16
// CHECK-SAME: %[[ARG0:.*]]: vector<16xi8>
llvm.func @extf_bf8_to_bf16(%src: vector<16xi8>) -> vector<16xbf16> {
- // CHECK: %[[VAR0:.*]] = llvm.call spir_funccc @__builtin_IB_bf8tohf_16(%[[ARG0]])
- // CHECK-SAME: : (vector<16xi8>) -> vector<16xf16>
- // CHECK: %[[VAR1:.*]] = llvm.call spir_funccc @_Z15convert_float16Dv16_Dh(%[[VAR0]])
- // CHECK-SAME: : (vector<16xf16>) -> vector<16xf32>
- // CHECK: %[[VAR2:.*]] = llvm.call spir_funccc @__builtin_IB_ftobf_16(%[[VAR1]])
- // CHECK-SAME: : (vector<16xf32>) -> vector<16xi16>
- // CHECK: %[[VAR3:.*]] = llvm.bitcast %[[VAR2]] : vector<16xi16> to vector<16xbf16>
+ // CHECK: %[[VAR0:.*]] = llvm.call spir_funccc @_Z36__builtin_spirv_ConvertE5M2ToBF16EXTDv16_c(%[[ARG0]])
+ // CHECK-SAME: : (vector<16xi8>) -> vector<16xi16>
+ // CHECK: %[[VAR3:.*]] = llvm.bitcast %[[VAR0]] : vector<16xi16> to vector<16xbf16>
%dst = xevm.extf %src { src_etype = bf8, dst_etype = bf16 } : (vector<16xi8>) -> vector<16xbf16>
llvm.return %dst : vector<16xbf16>
}
// -----
-// CHECK: llvm.func spir_funccc @__builtin_IB_ftobf_16(vector<16xf32>) -> vector<16xi16>
-// CHECK: llvm.func spir_funccc @_Z15convert_float16Dv16_Dh(vector<16xf16>) -> vector<16xf32>
-// CHECK: llvm.func spir_funccc @__builtin_IB_hf8tohf_16(vector<16xi8>) -> vector<16xf16>
+// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertE4M3ToBF16EXTDv16_c(vector<16xi8>) -> vector<16xi16>
// CHECK-LABEL: llvm.func @extf_f8_to_bf16
// CHECK-SAME: %[[ARG0:.*]]: vector<16xi8>
llvm.func @extf_f8_to_bf16(%src: vector<16xi8>) -> vector<16xbf16> {
- // CHECK: %[[VAR0:.*]] = llvm.call spir_funccc @__builtin_IB_hf8tohf_16(%[[ARG0]])
- // CHECK-SAME: : (vector<16xi8>) -> vector<16xf16>
- // CHECK: %[[VAR1:.*]] = llvm.call spir_funccc @_Z15convert_float16Dv16_Dh(%[[VAR0]])
- // CHECK-SAME: : (vector<16xf16>) -> vector<16xf32>
- // CHECK: %[[VAR2:.*]] = llvm.call spir_funccc @__builtin_IB_ftobf_16(%[[VAR1]])
- // CHECK-SAME: : (vector<16xf32>) -> vector<16xi16>
- // CHECK: %[[VAR3:.*]] = llvm.bitcast %[[VAR2]] : vector<16xi16> to vector<16xbf16>
+ // CHECK: %[[VAR0:.*]] = llvm.call spir_funccc @_Z36__builtin_spirv_ConvertE4M3ToBF16EXTDv16_c(%[[ARG0]])
+ // CHECK-SAME: : (vector<16xi8>) -> vector<16xi16>
+ // CHECK: %[[VAR3:.*]] = llvm.bitcast %[[VAR0]] : vector<16xi16> to vector<16xbf16>
%dst = xevm.extf %src { src_etype = f8, dst_etype = bf16 } : (vector<16xi8>) -> vector<16xbf16>
llvm.return %dst : vector<16xbf16>
}
>From 8fb7762bad01043f5238bd801fc2204d36a797c1 Mon Sep 17 00:00:00 2001
From: "Lee, Sang Ik" <sang.ik.lee at intel.com>
Date: Sat, 17 Oct 2026 05:01:46 +0000
Subject: [PATCH 2/4] [mlir][XeVM] Trim the fp8 conversion comments
The comment blocks added with the fp8 fix restated the commit message at
length. Keep what is not evident from the code - that two of the old
__builtin_IB_* names did not exist and why that failed silently, that the
wrappers are overloadable so mangle() supplies the length, that the bf16 forms
take shortN, and that the element type attribute need not agree with the operand
type - and drop the rest. Same in the three test comments.
No functional change.
---
mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp | 70 ++++++-------------
.../xevm_fp_conversion_lengths.mlir | 16 ++---
2 files changed, 28 insertions(+), 58 deletions(-)
diff --git a/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp b/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp
index ecf7d2b3e1b08..c9d0ab8ecdaf5 100644
--- a/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp
+++ b/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp
@@ -1350,40 +1350,23 @@ class TruncfToOCLPattern : public OpConversionPattern<TruncfOp> {
return success();
}
- // Handle the case where dst type is fp8.
+ // Handle the case where dst type is fp8, via the SPV_INTEL_fp_conversions
+ // wrappers in IGC's BiFModule:
+ // charN __builtin_spirv_Convert{FP16,BF16}To{E5M2,E4M3}EXT(halfN/shortN)
//
- // The conversions come from the SPV_INTEL_fp_conversions half of the IGC
- // BiFModule, which overloads one wrapper per vector length:
- // charN __builtin_spirv_ConvertFP16ToE5M2EXT(halfN)
- // charN __builtin_spirv_ConvertFP16ToE4M3EXT(halfN)
- // charN __builtin_spirv_ConvertBF16ToE5M2EXT(shortN)
- // charN __builtin_spirv_ConvertBF16ToE4M3EXT(shortN)
+ // These replace the __builtin_IB_* names used before, two of which did not
+ // exist: IGC spells E4M3 as `e4m3` in the conversion builtins, not `hf8`.
+ // createDeviceFunctionCall emits names unmangled, so the missing symbol
+ // linked silently and the call returned undef. The wrappers are
+ // `overloadable`, so mangle() takes the length from the argument type and a
+ // mismatch fails to link instead. The bf16 forms take the bits as shortN
+ // and route through f32 and f16 internally, replacing the explicit
+ // bftof/convert_half round trip.
//
- // These wrap the __builtin_IB_* builtins this used to name directly. Two
- // of those names do not exist: IGC spells E4M3 as `e4m3` in the conversion
- // builtins, not `hf8`, so `__builtin_IB_hftohf8_N` was never defined -
- // even though the scaled DPAS builtin does use `hf8`
- // (bdpas_f_f_hf8_hf8_8_8), which is correct and unchanged here. Because
- // createDeviceFunctionCall emits these names unmangled, the nonexistent
- // symbol linked silently and the call returned undef.
- //
- // The wrappers are declared `overloadable`, so the existing mangle() helper
- // derives the vector length from the argument type instead of pasting a
- // suffix onto the name, and a name or signature mismatch fails to link
- // rather than binding to nothing. As the note above these patterns says,
- // they are the experimental SPIR-V extension implementation rather than an
- // official OpenCL API, so the TODO to move once one exists still stands.
- //
- // A bf16 source takes the bf16 form, which accepts the bits as shortN and
- // routes through f32 and f16 internally. That retires the explicit
- // __builtin_IB_bftof plus convert_half round trip, so each bf16 conversion
- // is one call rather than three.
- //
- // `src_etype` is what says how to read the source, and the verifier only
- // constrains the operand's bitwidth, so f16 and bf16 are interchangeable to
- // it and either may arrive under either attribute. Coerce the operand to
- // what the selected wrapper takes, which also keeps bf16 out of the
- // mangling, where there is no code for it.
+ // `src_etype` says how to read the source and the verifier only checks its
+ // bitwidth, so either element type may arrive under either attribute.
+ // Coerce to what the wrapper takes, which also keeps bf16 away from
+ // mangle(), which has no code for it.
bool srcIsBF16 = srcEtype == TruncfSrcElemTypes::BF16;
Type argElemTy = srcIsBF16 ? cast<Type>(rewriter.getI16Type())
: cast<Type>(rewriter.getF16Type());
@@ -1508,20 +1491,12 @@ class ExtfToOCLPattern : public OpConversionPattern<ExtfOp> {
return rewriter.notifyMatchFailure(
op, "fp8 src and dst must have the same number of elements");
- // Mangled SPV_INTEL_fp_conversions wrappers, for the reasons given in the
- // truncf lowering above:
- // halfN __builtin_spirv_ConvertE5M2ToFP16EXT(charN)
- // halfN __builtin_spirv_ConvertE4M3ToFP16EXT(charN)
- // shortN __builtin_spirv_ConvertE5M2ToBF16EXT(charN)
- // shortN __builtin_spirv_ConvertE4M3ToBF16EXT(charN)
- //
- // As on the truncf side, `__builtin_IB_hf8tohf_N` never existed - IGC
- // defines `__builtin_IB_e4m3tohf_N` - so the f8 -> f16 call linked to
- // nothing and returned undef.
+ // The matching wrappers, for the reasons given in the truncf lowering:
+ // halfN/shortN __builtin_spirv_Convert{E5M2,E4M3}To{FP16,BF16}EXT(charN)
//
- // A bf16 destination takes the bf16 form, which returns the bits as shortN,
- // retiring the convert_float plus __builtin_IB_ftobf round trip: one call
- // instead of three.
+ // `__builtin_IB_hf8tohf_N`, used before, never existed either. A bf16
+ // destination returns the bits as shortN, replacing the
+ // convert_float/ftobf round trip.
bool dstIsBF16 = dstEtype == ExtfDstElemTypes::BF16;
StringRef convertName;
if (srcEtype == ExtfSrcElemTypes::BF8) { // Float8E5M2Type
@@ -1540,9 +1515,8 @@ class ExtfToOCLPattern : public OpConversionPattern<ExtfOp> {
rewriter, mangle(convertName, argTypes), resTy, argTypes,
args, {}, funcAttrs, op.getOperation())
->getResult(0);
- // The bf16 wrapper returns the bf16 bits as i16. As on the truncf side,
- // `dst_etype` may disagree with the result's element type, so cast to what
- // the op actually produces rather than to what the attribute implies.
+ // The bf16 wrapper returns the bits as i16, and `dst_etype` may disagree
+ // with the result's element type, so cast to what the op produces.
rewriter.replaceOp(op, castIfNeeded(rewriter, loc, vecDstTy, result));
return success();
}
diff --git a/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir b/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir
index a6ad1b8c8f680..2ba5e23298c05 100644
--- a/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir
+++ b/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir
@@ -63,8 +63,7 @@ llvm.func @extf_bf8_to_bf16_v4(%src: vector<4xi8>) -> vector<4xbf16> {
// -----
-// f16 to E4M3 away from length 16. Note IGC spells E4M3 `e4m3` in the
-// conversion builtins, not `hf8`.
+// f16 to E4M3 away from length 16.
// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE4M3EXTDv4_Dh(vector<4xf16>) -> vector<4xi8>
// CHECK-LABEL: llvm.func @truncf_f16_to_hf8_v4
// CHECK-SAME: %[[ARG0:.*]]: vector<4xf16>
@@ -90,10 +89,8 @@ llvm.func @extf_hf8_to_f16_v8(%src: vector<8xi8>) -> vector<8xf16> {
// -----
-// `src_etype` is authoritative, and the verifier only constrains the operand's
-// bitwidth, so a bf16 vector can arrive under `src_etype = f16`. The operand is
-// reinterpreted as what the selected wrapper takes rather than reaching the
-// mangler as a bf16 type, which it has no code for.
+// The verifier only constrains the operand's bitwidth, so a bf16 vector can
+// arrive under `src_etype = f16`. It is reinterpreted as what the wrapper takes.
// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE5M2EXTDv2_Dh(vector<2xf16>) -> vector<2xi8>
// CHECK-LABEL: llvm.func @truncf_etype_f16_operand_bf16
// CHECK-SAME: %[[ARG0:.*]]: vector<2xbf16>
@@ -107,8 +104,8 @@ llvm.func @truncf_etype_f16_operand_bf16(%src: vector<2xbf16>) -> vector<2xi8> {
// -----
-// The only odd length for an fp8 conversion, where one value per byte means the
-// source and the destination have the same length.
+// The only odd length an fp8 conversion has: one value per byte, so the source
+// and the destination are the same length.
// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertE5M2ToBF16EXTDv3_c(vector<3xi8>) -> vector<3xi16>
// CHECK-LABEL: llvm.func @extf_bf8_to_bf16_v3
// CHECK-SAME: %[[ARG0:.*]]: vector<3xi8>
@@ -122,8 +119,7 @@ llvm.func @extf_bf8_to_bf16_v3(%src: vector<3xi8>) -> vector<3xbf16> {
// -----
-// bf16 to E5M2 away from length 16. The bf16 wrapper takes the bits as shortN,
-// so the only preprocessing is the bitcast.
+// bf16 to E5M2 away from length 16.
// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertBF16ToE5M2EXTDv4_s(vector<4xi16>) -> vector<4xi8>
// CHECK-LABEL: llvm.func @truncf_bf16_to_bf8_v4
// CHECK-SAME: %[[ARG0:.*]]: vector<4xbf16>
>From 6d87ceadfd785c5fdc2334cd90ab1fb78e1cd1e1 Mon Sep 17 00:00:00 2001
From: "Lee, Sang Ik" <sang.ik.lee at intel.com>
Date: Mon, 19 Oct 2026 18:25:37 +0000
Subject: [PATCH 3/4] [mlir][XeVM] Update two stale fp8 conversion test
comments
xevm_truncf_extf_roundtrip_bf16_lengths.mlir still said a bf16 round trip exercises __builtin_IB_bftof_N and convert_halfN on the way down and convert_floatN and __builtin_IB_ftobf_N on the way back up. Those four calls are gone: the bf16 wrappers take and return the bits as shortN and route through f32 and f16 inside IGC.
Also note the element type attribute case in the header of xevm_fp_conversion_lengths.mlir, which covers more than vector lengths now.
No functional change.
---
.../Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir | 3 ++-
.../XeVM/GPU/xevm_truncf_extf_roundtrip_bf16_lengths.mlir | 7 ++++---
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir b/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir
index 2ba5e23298c05..51fce2e38ab63 100644
--- a/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir
+++ b/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir
@@ -2,7 +2,8 @@
// The fp8 and fp4 conversions are provided as one builtin per vector length.
// This file covers the SPIR-V vector lengths other than 16, which
-// xevm_mx-to-llvm.mlir covers.
+// xevm_mx-to-llvm.mlir covers, and the case where the element type attribute
+// and the operand's element type disagree.
// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE5M2EXTDv2_Dh(vector<2xf16>) -> vector<2xi8>
// CHECK-LABEL: llvm.func @truncf_f16_to_bf8_v2
diff --git a/mlir/test/Integration/Dialect/XeVM/GPU/xevm_truncf_extf_roundtrip_bf16_lengths.mlir b/mlir/test/Integration/Dialect/XeVM/GPU/xevm_truncf_extf_roundtrip_bf16_lengths.mlir
index 81366afd50098..20a793bc81d2c 100644
--- a/mlir/test/Integration/Dialect/XeVM/GPU/xevm_truncf_extf_roundtrip_bf16_lengths.mlir
+++ b/mlir/test/Integration/Dialect/XeVM/GPU/xevm_truncf_extf_roundtrip_bf16_lengths.mlir
@@ -12,9 +12,10 @@
// f8E5M2) format and a bf16 source and destination, at the SPIR-V vector
// lengths other than 16, which xevm_truncf_extf_roundtrip_bf16.mlir covers.
//
-// A bf16 operand is converted through f32 and f16 rather than directly, so this
-// exercises __builtin_IB_bftof_N and convert_halfN on the way down, and
-// convert_floatN and __builtin_IB_ftobf_N on the way back up, at each length.
+// A bf16 operand takes the bf16 form of the conversion wrappers, which pass the
+// bits as shortN and route through f32 and f16 inside IGC, so this exercises
+// __builtin_spirv_ConvertBF16ToE5M2EXT on the way down and
+// __builtin_spirv_ConvertE5M2ToBF16EXT on the way back up, at each length.
//
// Each of the 16 lanes owns one row of 32 bf16 values and converts four slices
// of it, of 2, 3, 4 and 8 elements. Every value used is exactly representable in
>From 74da40b96d5fe50b3366c2ea0afd414f3e8a29f6 Mon Sep 17 00:00:00 2001
From: "Lee, Sang Ik" <sang.ik.lee at intel.com>
Date: Mon, 19 Oct 2026 18:57:08 +0000
Subject: [PATCH 4/4] [mlir][XeVM] Drop the truncf operand coercion
The fp8 truncf lowering bitcast its operand to whatever src_etype named, so a bf16 vector arriving under src_etype = f16 was reinterpreted as f16 and converted as such. That turns invalid IR into silently wrong numerics: a bf16 1.0 converts as the f16 1.875.
Drive the bitcast off src_etype alone, as the wrapper selection already is, which is only the bf16 to shortN step the builtin's signature needs.
The disagreement itself is not this lowering's problem. src_etype and the operand's element type carry the same information, the op accepts f16 and bf16 only, and XeGPUToXeVM derives the attribute from the type, so a disagreement is invalid IR that TruncfOp::verify should reject and does not yet. Until it does, mangle() asserts on bf16, which is a clearer outcome than a wrong conversion and no worse than main, where the operand type reaches an unmangled __builtin_IB_hftobf8_N declaration that does not exist. The verifier rule is a follow up.
Removes the test for the coerced case with it.
---
mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp | 17 +++++++++--------
.../XeVMToLLVM/xevm_fp_conversion_lengths.mlir | 18 +-----------------
2 files changed, 10 insertions(+), 25 deletions(-)
diff --git a/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp b/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp
index c9d0ab8ecdaf5..a45566220e729 100644
--- a/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp
+++ b/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp
@@ -1363,15 +1363,16 @@ class TruncfToOCLPattern : public OpConversionPattern<TruncfOp> {
// and route through f32 and f16 internally, replacing the explicit
// bftof/convert_half round trip.
//
- // `src_etype` says how to read the source and the verifier only checks its
- // bitwidth, so either element type may arrive under either attribute.
- // Coerce to what the wrapper takes, which also keeps bf16 away from
- // mangle(), which has no code for it.
+ // `src_etype` is what says how to read the source, so the bf16 bitcast is
+ // driven by it. An operand whose element type disagrees with it is invalid
+ // IR that `TruncfOp::verify` does not reject yet, and mangle() has no code
+ // for bf16, so it asserts there. A verifier rule for that belongs with the
+ // op, not here.
bool srcIsBF16 = srcEtype == TruncfSrcElemTypes::BF16;
- Type argElemTy = srcIsBF16 ? cast<Type>(rewriter.getI16Type())
- : cast<Type>(rewriter.getF16Type());
- src = castIfNeeded(rewriter, loc,
- VectorType::get(vecSrcTy.getShape(), argElemTy), src);
+ if (srcIsBF16)
+ src = LLVM::BitcastOp::create(
+ rewriter, loc,
+ VectorType::get(vecSrcTy.getShape(), rewriter.getI16Type()), src);
StringRef convertName;
if (dstEtype == TruncfDstElemTypes::BF8) { // Float8E5M2Type
convertName = srcIsBF16 ? "__builtin_spirv_ConvertBF16ToE5M2EXT"
diff --git a/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir b/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir
index 51fce2e38ab63..50be4b4753cc1 100644
--- a/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir
+++ b/mlir/test/Conversion/XeVMToLLVM/xevm_fp_conversion_lengths.mlir
@@ -2,8 +2,7 @@
// The fp8 and fp4 conversions are provided as one builtin per vector length.
// This file covers the SPIR-V vector lengths other than 16, which
-// xevm_mx-to-llvm.mlir covers, and the case where the element type attribute
-// and the operand's element type disagree.
+// xevm_mx-to-llvm.mlir covers.
// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE5M2EXTDv2_Dh(vector<2xf16>) -> vector<2xi8>
// CHECK-LABEL: llvm.func @truncf_f16_to_bf8_v2
@@ -90,21 +89,6 @@ llvm.func @extf_hf8_to_f16_v8(%src: vector<8xi8>) -> vector<8xf16> {
// -----
-// The verifier only constrains the operand's bitwidth, so a bf16 vector can
-// arrive under `src_etype = f16`. It is reinterpreted as what the wrapper takes.
-// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE5M2EXTDv2_Dh(vector<2xf16>) -> vector<2xi8>
-// CHECK-LABEL: llvm.func @truncf_etype_f16_operand_bf16
-// CHECK-SAME: %[[ARG0:.*]]: vector<2xbf16>
-llvm.func @truncf_etype_f16_operand_bf16(%src: vector<2xbf16>) -> vector<2xi8> {
- // CHECK: %[[SRC:.*]] = llvm.bitcast %[[ARG0]] : vector<2xbf16> to vector<2xf16>
- // CHECK: llvm.call spir_funccc @_Z36__builtin_spirv_ConvertFP16ToE5M2EXTDv2_Dh(%[[SRC]])
- // CHECK-SAME: : (vector<2xf16>) -> vector<2xi8>
- %dst = xevm.truncf %src { src_etype = f16, dst_etype = bf8 } : (vector<2xbf16>) -> vector<2xi8>
- llvm.return %dst : vector<2xi8>
-}
-
-// -----
-
// The only odd length an fp8 conversion has: one value per byte, so the source
// and the destination are the same length.
// CHECK: llvm.func spir_funccc @_Z36__builtin_spirv_ConvertE5M2ToBF16EXTDv3_c(vector<3xi8>) -> vector<3xi16>
More information about the Mlir-commits
mailing list