[Mlir-commits] [mlir] [MLIR][XeGPU][XeVM] Lower xegpu.lane_shuffle to xevm.bitcast_shuffle (PR #215306)
llvmlistbot at llvm.org
llvmlistbot at llvm.org
Mon Aug 10 13:53:23 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-mlir-gpu
Author: Sang Ik Lee (silee2)
<details>
<summary>Changes</summary>
Add a `convert-xegpu-to-xevm` pattern that lowers `xegpu.lane_shuffle` onto
`xevm.bitcast_shuffle`.
`xevm.bitcast_shuffle` concatenates the components of its operand across the
subgroup, the first component of every lane first, and then hands chunks the
size of a result component back out to the lanes in order. Numbering the
elements of a `vector<NxT>` fragment held by lane `i` of a subgroup of size `S`
by their logical position, that concatenation is exactly the `pack` mode input
numbering `j * S + i`. Taking the result as a single `N * width(T)` bit scalar
then hands lane `i` the logical positions `i * N .. i * N + N - 1`, which is the
`pack` mode output numbering.
So `pack` becomes a vector-to-scalar `xevm.bitcast_shuffle` followed by a
bitcast back to the fragment type, and `unpack`, being its exact inverse,
becomes a bitcast to the scalar followed by a scalar-to-vector
`xevm.bitcast_shuffle`.
The shuffle redistributes whole bytes between the lanes, so sub-byte element
types cannot be shuffled and the pattern reports a conversion failure for them.
This covers fp4 fragments. Fragments that are not 8, 16, 32 or 64 bits wide are
rejected the same way, as there is no integer type to pack them into. The op
definition and verifier are left unchanged; the restriction is specific to this
lowering.
Element types without an LLVM representation, the fp8 formats in particular, are
converted to a same-width integer by the type converter before reaching the
pattern, and are shuffled as such.
---
Patch is 36.00 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/215306.diff
10 Files Affected:
- (modified) mlir/include/mlir/Dialect/LLVMIR/XeVMOps.td (+66)
- (modified) mlir/lib/Conversion/XeGPUToXeVM/XeGPUToXeVM.cpp (+52)
- (modified) mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp (+80-26)
- (modified) mlir/lib/Dialect/LLVMIR/IR/XeVMDialect.cpp (+16)
- (modified) mlir/test/Conversion/XeGPUToXeVM/failed_conversion.mlir (+26)
- (added) mlir/test/Conversion/XeGPUToXeVM/lane_shuffle.mlir (+76)
- (modified) mlir/test/Conversion/XeVMToLLVM/xevm-to-llvm.mlir (+67)
- (modified) mlir/test/Dialect/LLVMIR/invalid.mlir (+25)
- (modified) mlir/test/Dialect/LLVMIR/xevm.mlir (+32)
- (added) mlir/test/Integration/Dialect/XeVM/GPU/xevm_bitcast_shuffle.mlir (+219)
``````````diff
diff --git a/mlir/include/mlir/Dialect/LLVMIR/XeVMOps.td b/mlir/include/mlir/Dialect/LLVMIR/XeVMOps.td
index 8c4e409e9c395..1ab08c04bd903 100644
--- a/mlir/include/mlir/Dialect/LLVMIR/XeVMOps.td
+++ b/mlir/include/mlir/Dialect/LLVMIR/XeVMOps.td
@@ -740,6 +740,72 @@ def XeVM_MMAMxOp
let hasVerifier = 1;
}
+def XeVM_BitcastShuffleElemType
+ : AnyTypeOf<[I8, I16, I32, I64, BF16, F16, F32, F64]>;
+
+def XeVM_BitcastShuffleOp
+ : XeVM_Op<"bitcast_shuffle">,
+ Results<(outs AnyTypeOf<
+ [XeVM_BitcastShuffleElemType,
+ FixedVectorOfRankAndType<[1],
+ [XeVM_BitcastShuffleElemType]>]>:$res)>,
+ Arguments<(ins AnyTypeOf<
+ [XeVM_BitcastShuffleElemType,
+ FixedVectorOfRankAndType<[1],
+ [XeVM_BitcastShuffleElemType]>]>:$src)> {
+
+ let summary = "Subgroup bit-preserving conversion and shuffle";
+
+ let description = [{
+ The `xevm.bitcast_shuffle` operation performs a bit-preserving type
+ conversion and shuffle of `src` among the invocations in a subgroup. It is
+ a cooperative operation: all invocations in the subgroup participate. This
+ operation may execute more efficiently than a traditional bitcast.
+
+ Both `src` and `res` must be a scalar or a 1D vector of numerical type, and
+ they must have different types. The total number of bits of `res` must equal
+ the total number of bits of `src`. The shuffle is performed at byte
+ granularity, so element types narrower than 8 bits, such as f4 and i4, are
+ not supported. Floating point formats that have no LLVM representation, such
+ as the f8 formats, are expected to already have been converted to a
+ same-width integer type.
+
+ If `res` has the same number of components as `src`, the behavior is the
+ same as a traditional bitcast.
+
+ Otherwise, the bitcast and shuffle is performed as follows. Take the first
+ component of `src` for each invocation in the subgroup and concatenate those
+ components together, ordered by subgroup local invocation ID. Then repeat
+ for the next component of `src` of each invocation, until all of `src` has
+ been concatenated. This forms an `M * N` bit number, where `M` is the size
+ in bits of `src` and `N` is the subgroup size.
+
+ Now take the first `C` bits of the concatenated number and assign them to
+ the first component of `res` of the first invocation in the subgroup, where
+ `C` is the size in bits of each component of `res` when it is a vector type,
+ or the size in bits of `res` when it is a scalar type. Assign the next `C`
+ bits to the first component of `res` of the next invocation, and so on,
+ ordered by subgroup local invocation ID. Once `C` bits have been assigned to
+ the first component of all invocations in the subgroup, repeat for the
+ second component of `res`, and so on, until all bits have been assigned.
+
+ Like a traditional bitcast, the operation is reversible: an
+ `xevm.bitcast_shuffle` to one type followed by another `xevm.bitcast_shuffle`
+ back to the original type yields the original source data.
+
+ Example:
+ ```mlir
+ %res = xevm.bitcast_shuffle %src : (vector<8xi16>) -> vector<4xi32>
+ ```
+ }];
+
+ let assemblyFormat = [{
+ $src attr-dict `:` functional-type(operands, results)
+ }];
+
+ let hasVerifier = 1;
+}
+
//===----------------------------------------------------------------------===//
// XeVM target attribute.
//===----------------------------------------------------------------------===//
diff --git a/mlir/lib/Conversion/XeGPUToXeVM/XeGPUToXeVM.cpp b/mlir/lib/Conversion/XeGPUToXeVM/XeGPUToXeVM.cpp
index 78d99cf88b768..681dbd86234b0 100644
--- a/mlir/lib/Conversion/XeGPUToXeVM/XeGPUToXeVM.cpp
+++ b/mlir/lib/Conversion/XeGPUToXeVM/XeGPUToXeVM.cpp
@@ -1341,6 +1341,57 @@ class TruncfToXeVMPattern : public OpConversionPattern<arith::TruncFOp> {
}
};
+// Lowers `xegpu.lane_shuffle` to `xevm.bitcast_shuffle`.
+//
+// `xevm.bitcast_shuffle` concatenates the components of its operand across the
+// subgroup, the first component of every lane first, and then hands chunks the
+// size of a result component back out to the lanes in order. Numbering the
+// elements of a `vector<NxT>` fragment held by lane `i` of a subgroup of size
+// `S` by their logical position, that concatenation is exactly the `pack` mode
+// input numbering `j * S + i`. Taking the result as a single `N * width(T)` bit
+// scalar then hands lane `i` the logical positions `i * N .. i * N + N - 1`,
+// which is the `pack` mode output numbering.
+//
+// So `pack` is a vector-to-scalar `xevm.bitcast_shuffle` followed by a bitcast
+// back to the fragment type, and `unpack`, being its inverse, is a bitcast to
+// the scalar followed by a scalar-to-vector `xevm.bitcast_shuffle`.
+class LaneShuffleToXeVMPattern
+ : public OpConversionPattern<xegpu::LaneShuffleOp> {
+ using OpConversionPattern::OpConversionPattern;
+ LogicalResult
+ matchAndRewrite(xegpu::LaneShuffleOp op, OpAdaptor adaptor,
+ ConversionPatternRewriter &rewriter) const override {
+ auto vecTy = dyn_cast<VectorType>(adaptor.getSource().getType());
+ if (!vecTy)
+ return rewriter.notifyMatchFailure(op, "Expected a vector fragment.");
+ // The shuffle redistributes whole bytes between the lanes, so sub-byte
+ // element types, fp4 in particular, cannot be shuffled.
+ unsigned elemBits = vecTy.getElementTypeBitWidth();
+ if (elemBits < 8)
+ return rewriter.notifyMatchFailure(
+ op, "Sub-byte element types are not supported.");
+ int64_t fragmentBits = vecTy.getNumElements() * elemBits;
+ if (fragmentBits > 64 || !llvm::isPowerOf2_64(fragmentBits))
+ return rewriter.notifyMatchFailure(
+ op, "Expected a fragment of 8, 16, 32 or 64 bits.");
+
+ Location loc = op.getLoc();
+ Type packedTy = rewriter.getIntegerType(fragmentBits);
+ Value res;
+ if (op.getMode() == xegpu::LaneShuffleMode::Pack) {
+ res = xevm::BitcastShuffleOp::create(rewriter, loc, packedTy,
+ adaptor.getSource());
+ res = LLVM::BitcastOp::create(rewriter, loc, vecTy, res);
+ } else {
+ Value packed =
+ LLVM::BitcastOp::create(rewriter, loc, packedTy, adaptor.getSource());
+ res = xevm::BitcastShuffleOp::create(rewriter, loc, vecTy, packed);
+ }
+ rewriter.replaceOp(op, res);
+ return success();
+ }
+};
+
//===----------------------------------------------------------------------===//
// Pass Definition
//===----------------------------------------------------------------------===//
@@ -1664,4 +1715,5 @@ void mlir::populateXeGPUToXeVMConversionPatterns(
patterns.add<DpasMxToXeVMPattern>(typeConverter, patterns.getContext());
patterns.add<ExtfToXeVMPattern, TruncfToXeVMPattern>(typeConverter,
patterns.getContext());
+ patterns.add<LaneShuffleToXeVMPattern>(typeConverter, patterns.getContext());
}
diff --git a/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp b/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp
index c51da4d5d4d3d..9757ed4cef658 100644
--- a/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp
+++ b/mlir/lib/Conversion/XeVMToLLVM/XeVMToLLVM.cpp
@@ -99,6 +99,27 @@ std::string mangle(StringRef baseName, ArrayRef<Type> types,
return os.str();
}
+// Returns the mangling of `ty` used to name an overloaded `llvm.genx.GenISA.*`
+// intrinsic: `i32`, `f16`, `v8i16`, ... Note that this is IGC's own scheme for
+// its intrinsics, not the Itanium mangling used for the SPIR-V friendly and OCL
+// builtins that `mangle` above produces.
+std::string getGenISATypeMangling(Type ty) {
+ return TypeSwitch<Type, std::string>(ty)
+ .Case([](VectorType ty) -> std::string {
+ return "v" + std::to_string(ty.getNumElements()) +
+ getGenISATypeMangling(ty.getElementType());
+ })
+ .Case([](IntegerType ty) -> std::string {
+ return "i" + std::to_string(ty.getWidth());
+ })
+ // Floats of the same width share a spelling, so both bfloat16 and half
+ // mangle as `f16`.
+ .Case([](FloatType ty) -> std::string {
+ return "f" + std::to_string(ty.getWidth());
+ })
+ .DefaultUnreachable("unhandled type for GenISA mangling");
+}
+
std::string builtinElemType(ElemType elemType) {
switch (elemType) {
case ElemType::BF8:
@@ -1520,6 +1541,39 @@ class MMAMxToOCLPattern : public OpConversionPattern<MMAMxOp> {
}
};
+// Lowers `xevm.bitcast_shuffle` to a call to the IGC intrinsic
+// `llvm.genx.GenISA.SubgroupBitcastShuffle`, which is overloaded on both the
+// result and the operand type. E.g. a `vector<4xi8>` -> `vector<2xi16>` shuffle
+// becomes a call to
+// `llvm.genx.GenISA.SubgroupBitcastShuffle.v2i16.v4i8`.
+//
+// Note that the overload suffix spells every float of a given width the same
+// way, so a bf16 operand is passed through unchanged but named `f16`, as IGC
+// expects. The suffix therefore does not identify the overload on its own; the
+// declaration carries the actual types.
+class BitcastShuffleToGenISAPattern
+ : public OpConversionPattern<BitcastShuffleOp> {
+ using OpConversionPattern::OpConversionPattern;
+ LogicalResult
+ matchAndRewrite(BitcastShuffleOp op, BitcastShuffleOp::Adaptor adaptor,
+ ConversionPatternRewriter &rewriter) const override {
+ Type srcTy = op.getSrc().getType();
+ Type resTy = op.getRes().getType();
+
+ std::string fnName = "llvm.genx.GenISA.SubgroupBitcastShuffle." +
+ getGenISATypeMangling(resTy) + "." +
+ getGenISATypeMangling(srcTy);
+
+ Value result = createDeviceFunctionCall(
+ rewriter, fnName, resTy, {srcTy}, {adaptor.getSrc()}, {},
+ convergentNoUnwindWillReturnAttrs, op.getOperation())
+ ->getResult(0);
+
+ rewriter.replaceOp(op, result);
+ return success();
+ }
+};
+
class AllocaToGlobalPattern : public OpConversionPattern<LLVM::AllocaOp> {
using OpConversionPattern::OpConversionPattern;
LogicalResult
@@ -1787,30 +1841,30 @@ void ::mlir::populateXeVMToLLVMConversionPatterns(ConversionTarget &target,
return !op->hasAttr("cache_control");
});
target.addIllegalDialect<XeVMDialect>();
- patterns
- .add<LoadStorePrefetchToOCLPattern<BlockLoad2dOp>,
- LoadStorePrefetchToOCLPattern<BlockStore2dOp>,
- LoadStorePrefetchToOCLPattern<BlockPrefetch2dOp>, MMAToOCLPattern,
- MemfenceToOCLPattern, PrefetchToOCLPattern,
- LLVMLoadStoreToOCLPattern<LLVM::LoadOp>,
- LLVMLoadStoreToOCLPattern<LLVM::StoreOp>,
- BlockLoadStore1DToOCLPattern<BlockLoadOp>,
- BlockLoadStore1DToOCLPattern<BlockStoreOp>,
- LaunchConfigOpToOCLPattern<WorkitemIdXOp>,
- LaunchConfigOpToOCLPattern<WorkitemIdYOp>,
- LaunchConfigOpToOCLPattern<WorkitemIdZOp>,
- LaunchConfigOpToOCLPattern<WorkgroupDimXOp>,
- LaunchConfigOpToOCLPattern<WorkgroupDimYOp>,
- LaunchConfigOpToOCLPattern<WorkgroupDimZOp>,
- LaunchConfigOpToOCLPattern<WorkgroupIdXOp>,
- LaunchConfigOpToOCLPattern<WorkgroupIdYOp>,
- LaunchConfigOpToOCLPattern<WorkgroupIdZOp>,
- LaunchConfigOpToOCLPattern<GridDimXOp>,
- LaunchConfigOpToOCLPattern<GridDimYOp>,
- LaunchConfigOpToOCLPattern<GridDimZOp>,
- SubgroupOpWorkitemOpToOCLPattern<LaneIdOp>,
- SubgroupOpWorkitemOpToOCLPattern<SubgroupIdOp>,
- SubgroupOpWorkitemOpToOCLPattern<SubgroupSizeOp>, TruncfToOCLPattern,
- ExtfToOCLPattern, MMAMxToOCLPattern, AllocaToGlobalPattern>(
- patterns.getContext());
+ patterns.add<LoadStorePrefetchToOCLPattern<BlockLoad2dOp>,
+ LoadStorePrefetchToOCLPattern<BlockStore2dOp>,
+ LoadStorePrefetchToOCLPattern<BlockPrefetch2dOp>,
+ MMAToOCLPattern, MemfenceToOCLPattern, PrefetchToOCLPattern,
+ LLVMLoadStoreToOCLPattern<LLVM::LoadOp>,
+ LLVMLoadStoreToOCLPattern<LLVM::StoreOp>,
+ BlockLoadStore1DToOCLPattern<BlockLoadOp>,
+ BlockLoadStore1DToOCLPattern<BlockStoreOp>,
+ LaunchConfigOpToOCLPattern<WorkitemIdXOp>,
+ LaunchConfigOpToOCLPattern<WorkitemIdYOp>,
+ LaunchConfigOpToOCLPattern<WorkitemIdZOp>,
+ LaunchConfigOpToOCLPattern<WorkgroupDimXOp>,
+ LaunchConfigOpToOCLPattern<WorkgroupDimYOp>,
+ LaunchConfigOpToOCLPattern<WorkgroupDimZOp>,
+ LaunchConfigOpToOCLPattern<WorkgroupIdXOp>,
+ LaunchConfigOpToOCLPattern<WorkgroupIdYOp>,
+ LaunchConfigOpToOCLPattern<WorkgroupIdZOp>,
+ LaunchConfigOpToOCLPattern<GridDimXOp>,
+ LaunchConfigOpToOCLPattern<GridDimYOp>,
+ LaunchConfigOpToOCLPattern<GridDimZOp>,
+ SubgroupOpWorkitemOpToOCLPattern<LaneIdOp>,
+ SubgroupOpWorkitemOpToOCLPattern<SubgroupIdOp>,
+ SubgroupOpWorkitemOpToOCLPattern<SubgroupSizeOp>,
+ TruncfToOCLPattern, ExtfToOCLPattern, MMAMxToOCLPattern,
+ BitcastShuffleToGenISAPattern, AllocaToGlobalPattern>(
+ patterns.getContext());
}
diff --git a/mlir/lib/Dialect/LLVMIR/IR/XeVMDialect.cpp b/mlir/lib/Dialect/LLVMIR/IR/XeVMDialect.cpp
index e8b3c7065f880..1249cc34866cc 100644
--- a/mlir/lib/Dialect/LLVMIR/IR/XeVMDialect.cpp
+++ b/mlir/lib/Dialect/LLVMIR/IR/XeVMDialect.cpp
@@ -384,6 +384,22 @@ LogicalResult ExtfOp::verify() {
return success();
}
+LogicalResult BitcastShuffleOp::verify() {
+ Type srcTy = getSrc().getType();
+ Type resTy = getRes().getType();
+ if (srcTy == resTy)
+ return emitOpError("src and res types must be different");
+
+ auto getTotalBitWidth = [](Type ty) -> unsigned {
+ if (auto vecTy = dyn_cast<VectorType>(ty))
+ return vecTy.getNumElements() * vecTy.getElementTypeBitWidth();
+ return ty.getIntOrFloatBitWidth();
+ };
+ if (getTotalBitWidth(srcTy) != getTotalBitWidth(resTy))
+ return emitOpError("src and res types must have the same total bit width");
+ return success();
+}
+
LogicalResult
XeVMTargetAttr::verify(function_ref<InFlightDiagnostic()> emitError, int O,
StringRef triple, StringRef chip, DictionaryAttr flags,
diff --git a/mlir/test/Conversion/XeGPUToXeVM/failed_conversion.mlir b/mlir/test/Conversion/XeGPUToXeVM/failed_conversion.mlir
index cabc65aa0e41d..93bda1dfc0dca 100644
--- a/mlir/test/Conversion/XeGPUToXeVM/failed_conversion.mlir
+++ b/mlir/test/Conversion/XeGPUToXeVM/failed_conversion.mlir
@@ -28,3 +28,29 @@ gpu.module @test_kernel {
gpu.return
}
}
+
+// -----
+
+// Verify that xegpu.lane_shuffle of a sub-byte element type is rejected: the
+// shuffle redistributes whole bytes between the lanes, so fp4 fragments cannot
+// be shuffled.
+
+gpu.module @test_kernel {
+ gpu.func @lane_shuffle_f4(%a: vector<4xf4E2M1FN>) {
+ // expected-error at +1 {{failed to legalize operation 'xegpu.lane_shuffle' that was explicitly marked illegal}}
+ %0 = xegpu.lane_shuffle %a pack : vector<4xf4E2M1FN>
+ gpu.return
+ }
+}
+
+// -----
+
+// Verify that a xegpu.lane_shuffle fragment wider than 64 bits is rejected.
+
+gpu.module @test_kernel {
+ gpu.func @lane_shuffle_too_wide(%a: vector<4xi32>) {
+ // expected-error at +1 {{failed to legalize operation 'xegpu.lane_shuffle' that was explicitly marked illegal}}
+ %0 = xegpu.lane_shuffle %a pack : vector<4xi32>
+ gpu.return
+ }
+}
diff --git a/mlir/test/Conversion/XeGPUToXeVM/lane_shuffle.mlir b/mlir/test/Conversion/XeGPUToXeVM/lane_shuffle.mlir
new file mode 100644
index 0000000000000..e0ff25c34fb99
--- /dev/null
+++ b/mlir/test/Conversion/XeGPUToXeVM/lane_shuffle.mlir
@@ -0,0 +1,76 @@
+// RUN: mlir-opt -convert-xegpu-to-xevm -split-input-file %s | FileCheck %s
+
+gpu.module @test {
+// CHECK-LABEL: gpu.func @lane_shuffle_pack_i16
+// CHECK-SAME: %[[SRC:.*]]: vector<2xi16>
+gpu.func @lane_shuffle_pack_i16(%a: vector<2xi16>) -> vector<2xi16> {
+ // CHECK: %[[PACKED:.*]] = xevm.bitcast_shuffle %[[SRC]] : (vector<2xi16>) -> i32
+ // CHECK: %[[RES:.*]] = llvm.bitcast %[[PACKED]] : i32 to vector<2xi16>
+ // CHECK: gpu.return %[[RES]]
+ %0 = xegpu.lane_shuffle %a pack : vector<2xi16>
+ gpu.return %0 : vector<2xi16>
+}
+}
+
+// -----
+
+gpu.module @test {
+// CHECK-LABEL: gpu.func @lane_shuffle_unpack_i16
+// CHECK-SAME: %[[SRC:.*]]: vector<2xi16>
+gpu.func @lane_shuffle_unpack_i16(%a: vector<2xi16>) -> vector<2xi16> {
+ // CHECK: %[[PACKED:.*]] = llvm.bitcast %[[SRC]] : vector<2xi16> to i32
+ // CHECK: %[[RES:.*]] = xevm.bitcast_shuffle %[[PACKED]] : (i32) -> vector<2xi16>
+ // CHECK: gpu.return %[[RES]]
+ %0 = xegpu.lane_shuffle %a unpack : vector<2xi16>
+ gpu.return %0 : vector<2xi16>
+}
+}
+
+// -----
+
+gpu.module @test {
+// CHECK-LABEL: gpu.func @lane_shuffle_pack_f16
+gpu.func @lane_shuffle_pack_f16(%a: vector<4xf16>) -> vector<4xf16> {
+ // CHECK: %[[PACKED:.*]] = xevm.bitcast_shuffle %{{.*}} : (vector<4xf16>) -> i64
+ // CHECK: llvm.bitcast %[[PACKED]] : i64 to vector<4xf16>
+ %0 = xegpu.lane_shuffle %a pack : vector<4xf16>
+ gpu.return %0 : vector<4xf16>
+}
+}
+
+// -----
+
+gpu.module @test {
+// CHECK-LABEL: gpu.func @lane_shuffle_pack_bf16
+gpu.func @lane_shuffle_pack_bf16(%a: vector<2xbf16>) -> vector<2xbf16> {
+ // CHECK: %[[PACKED:.*]] = xevm.bitcast_shuffle %{{.*}} : (vector<2xbf16>) -> i32
+ // CHECK: llvm.bitcast %[[PACKED]] : i32 to vector<2xbf16>
+ %0 = xegpu.lane_shuffle %a pack : vector<2xbf16>
+ gpu.return %0 : vector<2xbf16>
+}
+}
+
+// -----
+
+// The f8 element type is converted to a same-width integer before the shuffle.
+gpu.module @test {
+// CHECK-LABEL: gpu.func @lane_shuffle_unpack_f8
+gpu.func @lane_shuffle_unpack_f8(%a: vector<4xf8E5M2>) -> vector<4xf8E5M2> {
+ // CHECK: %[[PACKED:.*]] = llvm.bitcast %{{.*}} : vector<4xi8> to i32
+ // CHECK: xevm.bitcast_shuffle %[[PACKED]] : (i32) -> vector<4xi8>
+ %0 = xegpu.lane_shuffle %a unpack : vector<4xf8E5M2>
+ gpu.return %0 : vector<4xf8E5M2>
+}
+}
+
+// -----
+
+gpu.module @test {
+// CHECK-LABEL: gpu.func @lane_shuffle_pack_i8
+gpu.func @lane_shuffle_pack_i8(%a: vector<8xi8>) -> vector<8xi8> {
+ // CHECK: %[[PACKED:.*]] = xevm.bitcast_shuffle %{{.*}} : (vector<8xi8>) -> i64
+ // CHECK: llvm.bitcast %[[PACKED]] : i64 to vector<8xi8>
+ %0 = xegpu.lane_shuffle %a pack : vector<8xi8>
+ gpu.return %0 : vector<8xi8>
+}
+}
diff --git a/mlir/test/Conversion/XeVMToLLVM/xevm-to-llvm.mlir b/mlir/test/Conversion/XeVMToLLVM/xevm-to-llvm.mlir
index 37064b1a9be9c..2d61f38ea0163 100644
--- a/mlir/test/Conversion/XeVMToLLVM/xevm-to-llvm.mlir
+++ b/mlir/test/Conversion/XeVMToLLVM/xevm-to-llvm.mlir
@@ -569,3 +569,70 @@ llvm.func @subgroup_id() -> i32 {
%1 = xevm.subgroup_id : i32
llvm.return %1 : i32
}
+
+// -----
+// CHECK-LABEL: llvm.func spir_funccc @llvm.genx.GenISA.SubgroupBitcastShuffle.v4i32.v8i16(vector<8xi16>) -> vector<4xi32>
+// CHECK-SAME: attributes {convergent, no_unwind, will_return}
+llvm.func @bitcast_shuffle(%a: vector<8xi16>) -> vector<4xi32> {
+ // CHECK: %[[VAR0:.*]] = llvm.call spir_funccc @llvm.genx.GenISA.SubgroupBitcastShuffle.v4i32.v8i16(%[[ARG0:.*]]) {convergent,
+ // CHECK-SAME: function_type = !llvm.func<vector<4xi32> (vector<8xi16>)>, linkage = #llvm.linkage<external>,
+ // CHECK-SAME: no_unwind, sym_name = "llvm.genx.GenISA.SubgroupBitcastShuffle.v4i32.v8i16", visibility_ = 0 : i64, will_return}
+ // CHECK-SAME: : (vector<8xi16>) -> vector<4xi32>
+ %0 = xevm.bitcast_shuffle %a : (vector<8xi16>) -> vector<4xi32>
+ llvm.return %0 : vector<4xi32>
+}
+
+// -----
+// CHECK-LABEL: llvm.func spir_funccc @llvm.genx.GenISA.SubgroupBitcastShuffle.v2f16.f32(f32) -> vector<2xf16>
+llvm.func @bitcast_shuffle_scalar_src(%a: f32) -> vector<2xf16> {
+ // CHECK: llvm.call spir_funccc @llvm.genx.GenISA.SubgroupBitcastShuffle.v2f16.f32({{.*}}) {{{.*}}} : (f32) -> vector<2xf16>
+ %0 = xevm.bitcast_shuffle %a : (f32) -> vector<2xf16>
+ llvm.return %0 : vector<2xf16>
+}
+
+// -----
+// CHECK-LABEL: llvm.func spir_funccc @llvm.genx.GenISA.SubgroupBitcastShuffle.i16.v2i8(vector<2xi8>) -> i16
+llvm.func @bitcast_shuffle_scalar_res(%a: vector<2xi8>...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/215306
More information about the Mlir-commits
mailing list