[llvm-branch-commits] [clang] [llvm] [mlir] [mlir] Migrate AMDGPU/ROCDL to targets, not chipset versions (PR #220105)
Krzysztof Drewniak via llvm-branch-commits
llvm-branch-commits at lists.llvm.org
Mon Aug 31 17:59:10 PDT 2026
https://github.com/krzysz00 updated https://github.com/llvm/llvm-project/pull/220105
>From 2b2e72c3d72ddd4152c020455c1c550b40b23712 Mon Sep 17 00:00:00 2001
From: Krzysztof Drewniak <Krzysztof.Drewniak at amd.com>
Date: Mon, 31 Aug 2026 19:32:37 +0000
Subject: [PATCH] [mlir] Migrate AMDGPU/ROCDL to targets, not chipset versions
**migration tl;dr:** `chipset=` becomes `triple=`, migrate off of
`amdgpu::Chipset` to `ROCDL::TargetInfo`, and eventually change
`gfxXYZ` to `amdgpuX.YZ-amd-amdhsa` in that `triple` argument.
`amdgpu::Chipset` was an awkward hack that was hard to keep up to date
with changes in the compiler/new architectures, and didn't properly
support generic targets (and has been strongly disfavored by the
compiler team).
This PR replaces `amdgpu::Chipset` with `ROCDL::TargetInfo`, a
structure that uses LLVM's TargetParser and the underlying LLVM
features tables to get the real nature of the target being compiled
for.
This also helps MLIR move to
new-style (`-mtriple=amdgpuX.YZ-amd-amdhsa`) over "old
style" (`-mtriple=amdgcn-amd-amdhsa -mcpu=gfxXYZ`) triples.
The utility structure is moved from AMDGPU to ROCDL, both because it's
tied to LLVM rather directly and because projects like Triton should
be able to use these feature tests without pulling in the AMDGPU
dialect and its memref dependencies.
This migration also fixes a few correctness issues:
- Atomic emulation was producing floating-point additions that don't
exist on gfx90c (even though it's "after" gfx90a) and gfx908's more
precise about where emulation is needed.
- gfx90c was also being handed a bare `s_barrier`, but it has no
hardware barrier back-off, so it needs the inline asm workaround.
- gfx950 won't allow xf32 MFMAs anymore.
- permlane_swap forms that don't exist on some architectures no
longer lower.
- gfx11.7 is now listed as an OCP FP8-having target.
Some checks still need to check for a generation (ex. when encoding
s_waitcnt or what the semantics of a WMMA are) go through an
`isGeneration(N)` method, which checks for having instructions from
generation N but not N+1.
(The barrier lowering has been reordered to account for gfx13 not
having, but also not needing, BackOffBarrier.)
This migration changes the `chipset` options on most passes to triple,
chip, features of argument. The `triple` option can, in addition to
being an actual triple, a `gfxXYZ` name for compatibility. The same
rename happens to the IR-visible `chipset` attribute on
`transform.apply_conversion_patterns.gpu.gpu_to_rocdl` and
`transform.apply_patterns.gpu.gpu_shuffle_to_amdgpu`, which does break
existing transform scripts.
Chipset is deprecated instead of being removed so that folks have time
to migrate.
AI disclosure: I steered this, Claude wrote the code, I tried to clean
up the docs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply at anthropic.com>
---
clang/test/CodeGen/link-builtin-bitcode.c | 6 +-
llvm/lib/Target/AMDGPU/AMDGPU.td | 14 +-
mlir/docs/ReleaseNotes.md | 34 +
.../Conversion/AMDGPUToROCDL/AMDGPUToROCDL.h | 4 +-
.../Conversion/ArithToAMDGPU/ArithToAMDGPU.h | 4 +-
.../Conversion/GPUToROCDL/GPUToROCDLPass.h | 7 +-
.../mlir/Conversion/MathToROCDL/MathToROCDL.h | 8 +-
mlir/include/mlir/Conversion/Passes.td | 50 +-
.../mlir/Dialect/AMDGPU/Transforms/Passes.h | 4 +-
.../mlir/Dialect/AMDGPU/Transforms/Passes.td | 18 +-
.../mlir/Dialect/AMDGPU/Utils/Chipset.h | 12 +
.../GPU/TransformOps/GPUTransformOps.td | 27 +-
.../mlir/Dialect/GPU/Transforms/Passes.h | 15 +-
.../mlir/Dialect/LLVMIR/ROCDLTargetInfo.h | 126 ++++
.../AMDGPUToROCDL/AMDGPUToROCDL.cpp | 636 +++++++++---------
.../ArithToAMDGPU/ArithToAMDGPU.cpp | 47 +-
.../GPUToROCDL/LowerGpuOpsToROCDLOps.cpp | 60 +-
.../Conversion/MathToROCDL/MathToROCDL.cpp | 33 +-
.../Dialect/AMDGPU/Transforms/CMakeLists.txt | 1 +
.../AMDGPU/Transforms/EmulateAtomics.cpp | 85 +--
.../GPU/Pipelines/GPUToROCDLPipeline.cpp | 30 +-
.../GPU/TransformOps/GPUTransformOps.cpp | 53 +-
.../GPU/Transforms/PromoteShuffleToAMDGPU.cpp | 8 +-
.../GPU/Transforms/SubgroupReduceLowering.cpp | 34 +-
mlir/lib/Dialect/LLVMIR/CMakeLists.txt | 2 +
.../lib/Dialect/LLVMIR/IR/ROCDLTargetInfo.cpp | 180 +++++
.../8-bit-floats-ocp-gfx1170.mlir | 28 -
.../AMDGPUToROCDL/8-bit-floats-ocp.mlir | 6 +-
.../AMDGPUToROCDL/8-bit-floats.mlir | 2 +-
.../AMDGPUToROCDL/amdgpu-to-rocdl.mlir | 14 +-
.../Conversion/AMDGPUToROCDL/dot-gfx11.mlir | 2 +-
.../Conversion/AMDGPUToROCDL/dot-gfx12.mlir | 2 +-
.../Conversion/AMDGPUToROCDL/dot-gfx9.mlir | 2 +-
.../Conversion/AMDGPUToROCDL/dot-invalid.mlir | 4 +-
mlir/test/Conversion/AMDGPUToROCDL/dpp.mlir | 6 +-
.../Conversion/AMDGPUToROCDL/gfx1250.mlir | 2 +-
.../AMDGPUToROCDL/global-prefetch.mlir | 2 +-
.../AMDGPUToROCDL/global_transpose_load.mlir | 6 +-
.../AMDGPUToROCDL/lds-barrier-gfx90c.mlir | 13 +-
.../AMDGPUToROCDL/load_lds-gfx950.mlir | 4 +-
.../Conversion/AMDGPUToROCDL/load_lds.mlir | 4 +-
.../AMDGPUToROCDL/memory_counter_wait.mlir | 8 +-
.../memory_counter_wait_tensor.mlir | 2 +-
.../memory_counter_wait_unsupported.mlir | 6 +-
.../AMDGPUToROCDL/mfma-fp8-invalid.mlir | 21 +
.../Conversion/AMDGPUToROCDL/mfma-gfx950.mlir | 16 +-
.../mfma-reduce-precision-invalid.mlir | 23 +
mlir/test/Conversion/AMDGPUToROCDL/mfma.mlir | 2 +-
.../Conversion/AMDGPUToROCDL/packed-ext.mlir | 2 +-
.../AMDGPUToROCDL/packed-trunc-invalid.mlir | 2 +-
.../AMDGPUToROCDL/packed-trunc.mlir | 2 +-
.../permlane-gfx1200-invalid.mlir | 21 +
.../permlane-gfx1250-invalid.mlir | 11 +
.../AMDGPUToROCDL/permlane-gfx1250.mlir | 12 +
.../AMDGPUToROCDL/permlane-var.mlir | 2 +-
.../Conversion/AMDGPUToROCDL/permlane.mlir | 6 +-
.../AMDGPUToROCDL/sparse-mfma-gfx950.mlir | 2 +-
.../Conversion/AMDGPUToROCDL/sparse-mfma.mlir | 2 +-
.../Conversion/AMDGPUToROCDL/swizzle.mlir | 2 +-
.../AMDGPUToROCDL/swmmac-gfx12.mlir | 2 +-
.../AMDGPUToROCDL/swmmac-gfx1250.mlir | 2 +-
.../AMDGPUToROCDL/transpose_load.mlir | 4 +-
.../AMDGPUToROCDL/transpose_load_gfx1250.mlir | 2 +-
.../transpose_load_gfx1250_invalid.mlir | 2 +-
.../transpose_load_gfx950_invalid.mlir | 2 +-
.../AMDGPUToROCDL/transpose_load_reject.mlir | 2 +-
.../Conversion/AMDGPUToROCDL/wmma-gfx11.mlir | 2 +-
.../Conversion/AMDGPUToROCDL/wmma-gfx12.mlir | 2 +-
.../AMDGPUToROCDL/wmma-gfx1250.mlir | 2 +-
.../ArithToAMDGPU/16-bit-floats.mlir | 2 +-
.../8-bit-float-saturation-ocp.mlir | 4 +-
.../ArithToAMDGPU/8-bit-float-saturation.mlir | 2 +-
.../ArithToAMDGPU/8-bit-floats-ocp.mlir | 4 +-
.../ArithToAMDGPU/8-bit-floats.mlir | 2 +-
.../ArithToAMDGPU/scaling-extf.mlir | 4 +-
.../ArithToAMDGPU/scaling-truncf-tensor.mlir | 2 +-
.../ArithToAMDGPU/scaling-truncf.mlir | 4 +-
.../Conversion/GPUCommon/lower-global-id.mlir | 2 +-
.../GPUCommon/lower-memory-space-attrs.mlir | 2 +-
.../GPUCommon/memory-attrbution.mlir | 2 +-
.../GPUCommon/memref-arg-attrs.mlir | 2 +-
.../GPUCommon/memref-arg-noalias-attrs.mlir | 2 +-
.../GPUCommon/memref-arg-noalias-warning.mlir | 2 +-
.../GPUToROCDL/constant-address-space.mlir | 2 +-
.../GPUToROCDL/gpu-to-rocdl-barrier.mlir | 4 +-
.../gpu-to-rocdl-barriers-gfx12.mlir | 2 +-
.../GPUToROCDL/gpu-to-rocdl-hip.mlir | 2 +-
.../gpu-to-rocdl-invalid-ballot.mlir | 2 +-
.../gpu-to-rocdl-invalid-dialect.mlir | 2 +-
.../gpu-to-rocdl-invalid-named-barrier.mlir | 2 +-
.../gpu-to-rocdl-named-barrier-non-const.mlir | 2 +-
.../GPUToROCDL/gpu-to-rocdl-opencl.mlir | 2 +-
.../GPUToROCDL/gpu-to-rocdl-subgroup-id.mlir | 4 +-
.../Conversion/GPUToROCDL/gpu-to-rocdl.mlir | 6 +-
mlir/test/Conversion/GPUToROCDL/memref.mlir | 4 +-
.../Conversion/MathToROCDL/math-to-rocdl.mlir | 4 +-
.../AMDGPU/amdgpu-emulate-atomics.mlir | 73 +-
.../GPU/promote-shuffle-amdgpu-invalid.mlir | 38 ++
.../Dialect/GPU/promote-shuffle-amdgpu.mlir | 2 +-
.../Integration/GPU/ROCM/gpu-to-hsaco.mlir | 2 +-
mlir/test/Integration/GPU/ROCM/printf.mlir | 2 +-
.../Integration/GPU/ROCM/two-modules.mlir | 2 +-
mlir/test/Integration/GPU/ROCM/vecadd.mlir | 2 +-
.../GPU/ROCM/vector-transferops.mlir | 2 +-
mlir/test/lib/Dialect/GPU/TestGpuRewrite.cpp | 16 +-
.../Dialect/AMDGPU/AMDGPUUtilsTest.cpp | 8 +
mlir/unittests/Dialect/LLVMIR/CMakeLists.txt | 2 +
.../Dialect/LLVMIR/ROCDLTargetInfoTest.cpp | 246 +++++++
108 files changed, 1518 insertions(+), 710 deletions(-)
create mode 100644 mlir/include/mlir/Dialect/LLVMIR/ROCDLTargetInfo.h
create mode 100644 mlir/lib/Dialect/LLVMIR/IR/ROCDLTargetInfo.cpp
delete mode 100644 mlir/test/Conversion/AMDGPUToROCDL/8-bit-floats-ocp-gfx1170.mlir
create mode 100644 mlir/test/Conversion/AMDGPUToROCDL/mfma-fp8-invalid.mlir
create mode 100644 mlir/test/Conversion/AMDGPUToROCDL/mfma-reduce-precision-invalid.mlir
create mode 100644 mlir/test/Conversion/AMDGPUToROCDL/permlane-gfx1200-invalid.mlir
create mode 100644 mlir/test/Conversion/AMDGPUToROCDL/permlane-gfx1250-invalid.mlir
create mode 100644 mlir/test/Conversion/AMDGPUToROCDL/permlane-gfx1250.mlir
create mode 100644 mlir/test/Dialect/GPU/promote-shuffle-amdgpu-invalid.mlir
create mode 100644 mlir/unittests/Dialect/LLVMIR/ROCDLTargetInfoTest.cpp
diff --git a/clang/test/CodeGen/link-builtin-bitcode.c b/clang/test/CodeGen/link-builtin-bitcode.c
index 2cae7c027a4f3..9fcd0dccad12f 100644
--- a/clang/test/CodeGen/link-builtin-bitcode.c
+++ b/clang/test/CodeGen/link-builtin-bitcode.c
@@ -44,6 +44,6 @@ int bar() { return no_attr() + attr_in_target() + attr_not_in_target() + attr_in
// CHECK-SAME: () #[[ATTR_INCOMPATIBLE:[0-9]+]] {
// CHECK: attributes #[[ATTR_BAR]] = { {{.*}} "no-trapping-math"="true" {{.*}} }
-// CHECK: attributes #[[ATTR_COMPATIBLE]] = { {{.*}} "target-features"="+16-bit-insts,+atomic-buffer-global-pk-add-f16-insts,+atomic-fadd-rtn-insts,+atomic-fmin-fmax-global-f64,+ci-insts,+cube-insts,+cvt-pknorm-vop2-insts,+dl-insts,+dot1-insts,+dot10-insts,+dot2-insts,+dot3-insts,+dot4-insts,+dot5-insts,+dot6-insts,+dot7-insts,+dpp,+flat-buffer-global-fadd-f64-inst,+flat-global-insts,+gfx8-insts,+gfx9-insts,+gfx90a-insts,+gws,+image-insts,+lerp-inst,+mai-insts,+mqsad-insts,+mqsad-pk-insts,+msad-insts,+qsad-insts,+s-memrealtime,+s-memtime-inst,+sad-insts,+vmem-to-lds-load-insts,+wavefrontsize64" }
-// CHECK: attributes #[[ATTR_EXTEND]] = { {{.*}} "target-features"="+16-bit-insts,+atomic-buffer-global-pk-add-f16-insts,+atomic-fadd-rtn-insts,+atomic-fmin-fmax-global-f64,+ci-insts,+cube-insts,+cvt-pknorm-vop2-insts,+dl-insts,+dot1-insts,+dot10-insts,+dot2-insts,+dot3-insts,+dot4-insts,+dot5-insts,+dot6-insts,+dot7-insts,+dot8-insts,+dpp,+flat-buffer-global-fadd-f64-inst,+flat-global-insts,+gfx8-insts,+gfx9-insts,+gfx90a-insts,+gws,+image-insts,+lerp-inst,+mai-insts,+mqsad-insts,+mqsad-pk-insts,+msad-insts,+qsad-insts,+s-memrealtime,+s-memtime-inst,+sad-insts,+vmem-to-lds-load-insts,+wavefrontsize64" }
-// CHECK: attributes #[[ATTR_INCOMPATIBLE]] = { {{.*}} "target-features"="+16-bit-insts,+atomic-buffer-global-pk-add-f16-insts,+atomic-fadd-rtn-insts,+atomic-fmin-fmax-global-f64,+ci-insts,+cube-insts,+cvt-pknorm-vop2-insts,+dl-insts,+dot1-insts,+dot10-insts,+dot2-insts,+dot3-insts,+dot4-insts,+dot5-insts,+dot6-insts,+dot7-insts,+dpp,+flat-buffer-global-fadd-f64-inst,+flat-global-insts,+gfx8-insts,+gfx90a-insts,+gws,+image-insts,+lerp-inst,+mai-insts,+mqsad-insts,+mqsad-pk-insts,+msad-insts,+qsad-insts,+s-memrealtime,+s-memtime-inst,+sad-insts,+vmem-to-lds-load-insts,+wavefrontsize64,-gfx9-insts" }
+// CHECK: attributes #[[ATTR_COMPATIBLE]] = { {{.*}} "target-features"="+16-bit-insts,+atomic-buffer-global-pk-add-f16-insts,+atomic-fadd-no-rtn-insts,+atomic-fadd-rtn-insts,+atomic-fmin-fmax-global-f64,+back-off-barrier,+ci-insts,+cube-insts,+cvt-pknorm-vop2-insts,+dl-insts,+dot1-insts,+dot10-insts,+dot2-insts,+dot3-insts,+dot4-insts,+dot5-insts,+dot6-insts,+dot7-insts,+dpp,+flat-buffer-global-fadd-f64-inst,+flat-global-insts,+gfx8-insts,+gfx9-insts,+gfx90a-insts,+gws,+image-insts,+lerp-inst,+mai-insts,+mqsad-insts,+mqsad-pk-insts,+msad-insts,+qsad-insts,+s-memrealtime,+s-memtime-inst,+sad-insts,+vmem-to-lds-load-insts,+wavefrontsize64" }
+// CHECK: attributes #[[ATTR_EXTEND]] = { {{.*}} "target-features"="+16-bit-insts,+atomic-buffer-global-pk-add-f16-insts,+atomic-fadd-no-rtn-insts,+atomic-fadd-rtn-insts,+atomic-fmin-fmax-global-f64,+back-off-barrier,+ci-insts,+cube-insts,+cvt-pknorm-vop2-insts,+dl-insts,+dot1-insts,+dot10-insts,+dot2-insts,+dot3-insts,+dot4-insts,+dot5-insts,+dot6-insts,+dot7-insts,+dot8-insts,+dpp,+flat-buffer-global-fadd-f64-inst,+flat-global-insts,+gfx8-insts,+gfx9-insts,+gfx90a-insts,+gws,+image-insts,+lerp-inst,+mai-insts,+mqsad-insts,+mqsad-pk-insts,+msad-insts,+qsad-insts,+s-memrealtime,+s-memtime-inst,+sad-insts,+vmem-to-lds-load-insts,+wavefrontsize64" }
+// CHECK: attributes #[[ATTR_INCOMPATIBLE]] = { {{.*}} "target-features"="+16-bit-insts,+atomic-buffer-global-pk-add-f16-insts,+atomic-fadd-no-rtn-insts,+atomic-fadd-rtn-insts,+atomic-fmin-fmax-global-f64,+back-off-barrier,+ci-insts,+cube-insts,+cvt-pknorm-vop2-insts,+dl-insts,+dot1-insts,+dot10-insts,+dot2-insts,+dot3-insts,+dot4-insts,+dot5-insts,+dot6-insts,+dot7-insts,+dpp,+flat-buffer-global-fadd-f64-inst,+flat-global-insts,+gfx8-insts,+gfx90a-insts,+gws,+image-insts,+lerp-inst,+mai-insts,+mqsad-insts,+mqsad-pk-insts,+msad-insts,+qsad-insts,+s-memrealtime,+s-memtime-inst,+sad-insts,+vmem-to-lds-load-insts,+wavefrontsize64,-gfx9-insts" }
diff --git a/llvm/lib/Target/AMDGPU/AMDGPU.td b/llvm/lib/Target/AMDGPU/AMDGPU.td
index 65fb323a0b10e..6a6ba9f6dbe1d 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPU.td
+++ b/llvm/lib/Target/AMDGPU/AMDGPU.td
@@ -3202,8 +3202,9 @@ def AMDGPUFrontendVisibleFeatures {
Feature16BitInsts, FeatureAddMinMaxInsts, FeatureAshrPkInsts,
FeatureAsyncLoadToLDSInsts, FeatureAsyncStoreFromLDSInsts, FeatureAsynccnt,
FeatureAtomicBufferGlobalPkAddF16Insts, FeatureAtomicBufferPkAddBF16Inst, FeatureAtomicDsPkAdd16Insts,
- FeatureAtomicFMinFMaxF32GlobalInsts, FeatureAtomicFMinFMaxF64GlobalInsts, FeatureAtomicFaddRtnInsts,
- FeatureAtomicFlatPkAdd16Insts, FeatureAtomicGlobalPkAddBF16Inst, FeatureBF16ConversionInsts,
+ FeatureAtomicFMinFMaxF32GlobalInsts, FeatureAtomicFMinFMaxF64GlobalInsts, FeatureAtomicFaddNoRtnInsts,
+ FeatureAtomicFaddRtnInsts, FeatureAtomicFlatPkAdd16Insts, FeatureAtomicGlobalPkAddBF16Inst,
+ FeatureBF16ConversionInsts,
FeatureBF16PackedInsts, FeatureBF16TransInsts, FeatureBF8ConversionScaleInsts,
FeatureBVHRayTracingInsts, FeatureBitOp3Insts, FeatureCIInsts,
FeatureClusters, FeatureCubeInsts, FeatureCvtPkNormVOP2Insts,
@@ -3221,9 +3222,10 @@ def AMDGPUFrontendVisibleFeatures {
FeatureGFX1250Insts, FeatureGFX1251GEMMInsts, FeatureGFX12Insts,
FeatureGFX13Insts, FeatureGFX8Insts, FeatureGFX90AInsts,
FeatureGFX940Insts, FeatureGFX950Insts, FeatureGFX9Insts,
- FeatureGWS, FeatureImageInsts, FeatureLerpInst,
- FeatureMAIInsts, FeatureMcastLoadInsts, FeatureMqsadInsts,
- FeatureMqsadPkInsts, FeatureMsadInsts, FeaturePermlane16Swap,
+ FeatureGWS, FeatureImageInsts, FeatureLdsBarrierArriveAtomic,
+ FeatureLerpInst, FeatureMAIInsts, FeatureMcastLoadInsts,
+ FeatureMqsadInsts, FeatureMqsadPkInsts, FeatureMsadInsts,
+ FeatureOCPFP8ConversionInsts, FeaturePermlane16Insts, FeaturePermlane16Swap,
FeaturePermlane32Swap, FeaturePkAddMinMaxInsts, FeaturePrngInst,
FeatureQsadInsts, FeatureSMemRealTime, FeatureSMemTimeInst,
FeatureSWMMACGfx1200Insts, FeatureSWMMACGfx1250Insts, FeatureSWakeupBarrier,
@@ -3235,7 +3237,7 @@ def AMDGPUFrontendVisibleFeatures {
FeatureSupportsWave32, FeatureFastFMAF32, FeatureFastDenormalF32,
FeatureSupportsXNACK, FeatureSupportsSRAMECC, FeatureXNACKOnOffModes,
FeatureSGPRInitBug, FeatureApertureRegs, FeatureGetDoorbellID,
- FeatureAGPRAlloc, Feature1536VGPRs,
+ FeatureAGPRAlloc, Feature1536VGPRs, FeatureBackOffBarrier
];
}
diff --git a/mlir/docs/ReleaseNotes.md b/mlir/docs/ReleaseNotes.md
index 16b93c8909670..6dec8346f965f 100644
--- a/mlir/docs/ReleaseNotes.md
+++ b/mlir/docs/ReleaseNotes.md
@@ -8,6 +8,40 @@ specifically, it is a snapshot of the MLIR development at the time of the releas
[TOC]
+## LLVM 24
+
+### GPU/AMDGPU Changes
+
+- `mlir::amdgpu::Chipset` is deprecated in favour of `mlir::ROCDL::TargetInfo`,
+ which describes a target by its triple subarch plus the resolved set of
+ frontend-visible target features from LLVM's own tables. Lowerings should ask
+ whether a target has a feature rather than compare gfx version numbers, which
+ relates GPUs across families that share no instructions. `TargetInfo` also
+ represents generic targets such as `gfx9-4-generic`, rejects well-formed but
+ nonexistent names such as `gfx999`, and carries the wavefront size, which a
+ gfx version alone cannot answer for the targets that support both.
+- Accordingly, the `chipset` option on `convert-amdgpu-to-rocdl`,
+ `convert-gpu-to-rocdl`, `convert-arith-to-amdgpu`, `convert-math-to-rocdl` and
+ `amdgpu-emulate-atomics` is replaced by `triple`, `chip` and `features`,
+ matching `rocdl-attach-target` and `#rocdl.target`. `triple` accepts either a
+ triple (`amdgpu9.42-amd-amdhsa`) or a bare GPU name (`gfx942`), so existing
+ invocations can migrate by renaming the option alone.
+- `triple` has no usable default: it is `invalid`, so a target must be passed
+ explicitly. The old `chipset` default of `gfx000` parsed successfully into a
+ target that then failed every capability check, causing silent failures.
+- The IR-visible `chipset` attribute on
+ `transform.apply_conversion_patterns.gpu.gpu_to_rocdl` and
+ `transform.apply_patterns.gpu.gpu_shuffle_to_amdgpu` is likewise replaced by
+ `triple`, `chip` and `features`, now spelled as a property dictionary so that
+ further target knobs don't each need their own keyword:
+
+ ```mlir
+ transform.apply_patterns.gpu.gpu_shuffle_to_amdgpu <triple = "gfx950">
+ ```
+
+ This breaks existing transform scripts, which have to be updated by hand;
+ `triple` accepts a bare GPU name, so no target spelling has to change.
+
## LLVM 21
### GPU/NVVM Changes
diff --git a/mlir/include/mlir/Conversion/AMDGPUToROCDL/AMDGPUToROCDL.h b/mlir/include/mlir/Conversion/AMDGPUToROCDL/AMDGPUToROCDL.h
index 393658652dbac..861bbe0cc8f75 100644
--- a/mlir/include/mlir/Conversion/AMDGPUToROCDL/AMDGPUToROCDL.h
+++ b/mlir/include/mlir/Conversion/AMDGPUToROCDL/AMDGPUToROCDL.h
@@ -8,7 +8,7 @@
#ifndef MLIR_CONVERSION_AMDGPUTOROCDL_AMDGPUTOROCDL_H_
#define MLIR_CONVERSION_AMDGPUTOROCDL_AMDGPUTOROCDL_H_
-#include "mlir/Dialect/AMDGPU/Utils/Chipset.h"
+#include "mlir/Dialect/LLVMIR/ROCDLTargetInfo.h"
#include <memory>
#include <string>
@@ -27,7 +27,7 @@ class Pass;
/// populateAMDGPUTypeAndAttributeConversions().
void populateAMDGPUToROCDLConversionPatterns(LLVMTypeConverter &converter,
RewritePatternSet &patterns,
- amdgpu::Chipset chipset);
+ const ROCDL::TargetInfo &target);
namespace amdgpu {
/// Remap common GPU memory spaces (Workgroup, Private, etc) to LLVM address
diff --git a/mlir/include/mlir/Conversion/ArithToAMDGPU/ArithToAMDGPU.h b/mlir/include/mlir/Conversion/ArithToAMDGPU/ArithToAMDGPU.h
index fd144edf77452..e6e137759a76a 100644
--- a/mlir/include/mlir/Conversion/ArithToAMDGPU/ArithToAMDGPU.h
+++ b/mlir/include/mlir/Conversion/ArithToAMDGPU/ArithToAMDGPU.h
@@ -9,7 +9,7 @@
#ifndef MLIR_CONVERSION_ARITHTOAMDGPU_ARITHTOAMDGPU_H
#define MLIR_CONVERSION_ARITHTOAMDGPU_ARITHTOAMDGPU_H
-#include "mlir/Dialect/AMDGPU/Utils/Chipset.h"
+#include "mlir/Dialect/LLVMIR/ROCDLTargetInfo.h"
#include "mlir/IR/PatternMatch.h"
#include <memory>
#include <string>
@@ -31,7 +31,7 @@ namespace arith {
void populateArithToAMDGPUConversionPatterns(
RewritePatternSet &patterns, bool convertFP8Arithmetic,
bool saturateFP8Truncf, bool allowPackedF16Rtz, bool supportsScaledExtTrunc,
- amdgpu::Chipset chipset, PatternBenefit benefit = 1);
+ const ROCDL::TargetInfo &target, PatternBenefit benefit = 1);
} // namespace arith
} // namespace mlir
diff --git a/mlir/include/mlir/Conversion/GPUToROCDL/GPUToROCDLPass.h b/mlir/include/mlir/Conversion/GPUToROCDL/GPUToROCDLPass.h
index 220da0ad3c08f..494ee8cfa11ab 100644
--- a/mlir/include/mlir/Conversion/GPUToROCDL/GPUToROCDLPass.h
+++ b/mlir/include/mlir/Conversion/GPUToROCDL/GPUToROCDLPass.h
@@ -10,6 +10,7 @@
#include "mlir/Conversion/GPUToROCDL/Runtimes.h"
#include "mlir/Conversion/LLVMCommon/LoweringOptions.h"
+#include "mlir/Dialect/LLVMIR/ROCDLTargetInfo.h"
#include <memory>
namespace mlir {
@@ -21,10 +22,6 @@ class RewritePatternSet;
template <typename OpT>
class OperationPass;
-namespace amdgpu {
-struct Chipset;
-} // namespace amdgpu
-
namespace gpu {
class GPUModuleOp;
} // namespace gpu
@@ -38,7 +35,7 @@ class GPUModuleOp;
void populateGpuToROCDLConversionPatterns(const LLVMTypeConverter &converter,
RewritePatternSet &patterns,
gpu::amd::Runtime runtime,
- amdgpu::Chipset chipset);
+ const ROCDL::TargetInfo &target);
/// Configure target to convert from the GPU dialect to ROCDL.
void configureGpuToROCDLConversionLegality(ConversionTarget &target);
diff --git a/mlir/include/mlir/Conversion/MathToROCDL/MathToROCDL.h b/mlir/include/mlir/Conversion/MathToROCDL/MathToROCDL.h
index 60f1888569362..8ba104972abff 100644
--- a/mlir/include/mlir/Conversion/MathToROCDL/MathToROCDL.h
+++ b/mlir/include/mlir/Conversion/MathToROCDL/MathToROCDL.h
@@ -9,7 +9,7 @@
#define MLIR_CONVERSION_MATHTOROCDL_MATHTOROCDL_H_
#include "mlir/Conversion/LLVMCommon/TypeConverter.h"
-#include "mlir/Dialect/AMDGPU/Utils/Chipset.h"
+#include "mlir/Dialect/LLVMIR/ROCDLTargetInfo.h"
#include "mlir/IR/PatternMatch.h"
#include <memory>
@@ -20,11 +20,11 @@ class Pass;
#include "mlir/Conversion/Passes.h.inc"
/// Populate the given list with patterns that convert from Math to ROCDL calls.
-// `chipset` specifies the AMDGPU chipset to target. If `std::nullopt`,
-// none of the chipset dependent patterns are added.
+// `target` describes the AMDGPU target. If `std::nullopt`, none of the
+// target-dependent patterns are added.
void populateMathToROCDLConversionPatterns(
const LLVMTypeConverter &converter, RewritePatternSet &patterns,
- std::optional<amdgpu::Chipset> chipset);
+ std::optional<ROCDL::TargetInfo> target);
} // namespace mlir
#endif // MLIR_CONVERSION_MATHTOROCDL_MATHTOROCDL_H_
diff --git a/mlir/include/mlir/Conversion/Passes.td b/mlir/include/mlir/Conversion/Passes.td
index f13cb9a801139..44c318a537b79 100644
--- a/mlir/include/mlir/Conversion/Passes.td
+++ b/mlir/include/mlir/Conversion/Passes.td
@@ -130,9 +130,15 @@ def ConvertAMDGPUToROCDLPass : Pass<"convert-amdgpu-to-rocdl"> {
"LLVM::LLVMDialect",
"ROCDL::ROCDLDialect",
];
- let options = [Option<"chipset", "chipset", "std::string",
- /*default=*/"\"gfx000\"",
- "Chipset that these operations will run on">];
+ let options = [
+ Option<"triple", "triple", "std::string",
+ /*default=*/"\"invalid\"",
+ "Target triple (e.g. amdgpu9.42-amd-amdhsa) or GPU name (e.g. gfx942). Defaults to an invalid target so that one must be given explicitly">,
+ Option<"chip", "chip", "std::string", /*default=*/"\"\"",
+ "Target GPU, if not given by the triple (e.g. gfx942)">,
+ Option<"features", "features", "std::string", /*default=*/"\"\"",
+ "Target features (e.g. +wavefrontsize64)">,
+ ];
}
//===----------------------------------------------------------------------===//
@@ -150,9 +156,13 @@ def ArithToAMDGPUConversionPass : Pass<"convert-arith-to-amdgpu"> {
let dependentDialects = ["amdgpu::AMDGPUDialect", "vector::VectorDialect"];
let options = [
- Option<"chipset", "chipset", "std::string",
- /*default=*/"\"gfx000\"",
- "Chipset that these operations will run on">,
+ Option<"triple", "triple", "std::string",
+ /*default=*/"\"invalid\"",
+ "Target triple (e.g. amdgpu9.42-amd-amdhsa) or GPU name (e.g. gfx942). Defaults to an invalid target so that one must be given explicitly">,
+ Option<"chip", "chip", "std::string", /*default=*/"\"\"",
+ "Target GPU, if not given by the triple (e.g. gfx942)">,
+ Option<"features", "features", "std::string", /*default=*/"\"\"",
+ "Target features (e.g. +wavefrontsize64)">,
Option<"saturateFP8Truncf", "saturate-fp8-truncf", "bool",
/*default=*/"false",
"Use saturating truncation for 8-bit float types">,
@@ -690,9 +700,13 @@ def ConvertGpuOpsToROCDLOps : Pass<"convert-gpu-to-rocdl", "gpu::GPUModuleOp"> {
"memref::MemRefDialect",
];
let options = [
- Option<"chipset", "chipset", "std::string",
- /*default=*/"\"gfx000\"",
- "Chipset that these operations will run on">,
+ Option<"triple", "triple", "std::string",
+ /*default=*/"\"invalid\"",
+ "Target triple (e.g. amdgpu9.42-amd-amdhsa) or GPU name (e.g. gfx942). Defaults to an invalid target so that one must be given explicitly">,
+ Option<"chip", "chip", "std::string", /*default=*/"\"\"",
+ "Target GPU, if not given by the triple (e.g. gfx942)">,
+ Option<"features", "features", "std::string", /*default=*/"\"\"",
+ "Target features (e.g. +wavefrontsize64)">,
Option<"indexBitwidth", "index-bitwidth", "unsigned",
/*default=kDeriveIndexBitwidthFromDataLayout*/ "0",
"Bitwidth of the index type, 0 to use size of machine word">,
@@ -857,9 +871,9 @@ def ConvertMathToROCDL : Pass<"convert-math-to-rocdl", "ModuleOp"> {
let description = [{
This pass converts supported Math ops to ROCDL library calls.
- The chipset option specifies the target AMDGPU architecture. If the chipset
- is empty, none of the chipset-dependent patterns are added, and the pass
- will not attempt to parse the chipset.
+ The triple option specifies the target AMDGPU architecture. If it is empty,
+ none of the target-dependent patterns are added and the pass does not
+ resolve a target; a chip or feature list without a triple is rejected.
}];
let dependentDialects = [
"arith::ArithDialect",
@@ -867,9 +881,15 @@ def ConvertMathToROCDL : Pass<"convert-math-to-rocdl", "ModuleOp"> {
"ROCDL::ROCDLDialect",
"vector::VectorDialect",
];
- let options = [Option<"chipset", "chipset", "std::string",
- /*default=*/"\"\"",
- "Chipset that these operations will run on">];
+ let options = [
+ Option<"triple", "triple", "std::string", /*default=*/"\"\"",
+ "Target triple (e.g. amdgpu9.42-amd-amdhsa) or GPU name (e.g. gfx942). "
+ "If empty, no target-dependent patterns are added">,
+ Option<"chip", "chip", "std::string", /*default=*/"\"\"",
+ "Target GPU, if not given by the triple (e.g. gfx942)">,
+ Option<"features", "features", "std::string", /*default=*/"\"\"",
+ "Target features (e.g. +wavefrontsize64)">,
+ ];
}
//===----------------------------------------------------------------------===//
diff --git a/mlir/include/mlir/Dialect/AMDGPU/Transforms/Passes.h b/mlir/include/mlir/Dialect/AMDGPU/Transforms/Passes.h
index 48e7658568f86..c8ec368231f83 100644
--- a/mlir/include/mlir/Dialect/AMDGPU/Transforms/Passes.h
+++ b/mlir/include/mlir/Dialect/AMDGPU/Transforms/Passes.h
@@ -13,7 +13,7 @@
#ifndef MLIR_DIALECT_AMDGPU_TRANSFORMS_PASSES_H_
#define MLIR_DIALECT_AMDGPU_TRANSFORMS_PASSES_H_
-#include "mlir/Dialect/AMDGPU/Utils/Chipset.h"
+#include "mlir/Dialect/LLVMIR/ROCDLTargetInfo.h"
#include "mlir/IR/PatternMatch.h"
#include "mlir/Pass/Pass.h"
@@ -29,7 +29,7 @@ namespace amdgpu {
void populateAmdgpuEmulateAtomicsPatterns(ConversionTarget &target,
RewritePatternSet &patterns,
- Chipset chipset,
+ const ROCDL::TargetInfo &targetInfo,
PatternBenefit benefit = 1);
void populateAmdgpuResolveStridedMetadataPatterns(RewritePatternSet &patterns,
diff --git a/mlir/include/mlir/Dialect/AMDGPU/Transforms/Passes.td b/mlir/include/mlir/Dialect/AMDGPU/Transforms/Passes.td
index 7dd7ac750a9eb..036d4178ab54c 100644
--- a/mlir/include/mlir/Dialect/AMDGPU/Transforms/Passes.td
+++ b/mlir/include/mlir/Dialect/AMDGPU/Transforms/Passes.td
@@ -16,19 +16,25 @@
include "mlir/Pass/PassBase.td"
def AmdgpuEmulateAtomicsPass : Pass<"amdgpu-emulate-atomics"> {
- let summary = "Emulate atomic operations on chipsets that do not support them";
+ let summary = "Emulate atomic operations the target does not support";
let description = [{
- This pass rewrites any AMDGPU-specific atomic operation that is not supported
- on the given `chipset` into a compare-and-swap loop.
+ This pass rewrites any AMDGPU-specific atomic operation that the target does
+ not support into a compare-and-swap loop.
}];
let dependentDialects = [
"cf::ControlFlowDialect",
"arith::ArithDialect",
"vector::VectorDialect"
];
- let options = [Option<"chipset", "chipset", "std::string",
- /*default=*/"\"gfx000\"",
- "Chipset that these operations will run on">];
+ let options = [
+ Option<"triple", "triple", "std::string",
+ /*default=*/"\"invalid\"",
+ "Target triple (e.g. amdgpu9.42-amd-amdhsa) or GPU name (e.g. gfx942). Defaults to an invalid target so that one must be given explicitly">,
+ Option<"chip", "chip", "std::string", /*default=*/"\"\"",
+ "Target GPU, if not given by the triple (e.g. gfx942)">,
+ Option<"features", "features", "std::string", /*default=*/"\"\"",
+ "Target features (e.g. +wavefrontsize64)">,
+ ];
}
def AmdgpuResolveStridedMetadataPass : Pass<"amdgpu-resolve-strided-metadata"> {
diff --git a/mlir/include/mlir/Dialect/AMDGPU/Utils/Chipset.h b/mlir/include/mlir/Dialect/AMDGPU/Utils/Chipset.h
index 256065537aeb9..1a964c27a6333 100644
--- a/mlir/include/mlir/Dialect/AMDGPU/Utils/Chipset.h
+++ b/mlir/include/mlir/Dialect/AMDGPU/Utils/Chipset.h
@@ -9,6 +9,7 @@
#define MLIR_DIALECT_AMDGPU_UTILS_CHIPSET_H_
#include "mlir/Support/LLVM.h"
+#include "llvm/Support/Compiler.h"
#include <tuple>
namespace mlir::amdgpu {
@@ -19,6 +20,10 @@ namespace mlir::amdgpu {
/// gfx942 --> major = 9, minor = 0x4, stepping = 0x2
/// gfx90a --> major = 9, minor = 0x0, stepping = 0xa
/// gfx1103 --> major = 11, minor = 0x0, stepping = 0x3
+///
+/// \deprecated Use `mlir::ROCDL::TargetInfo` instead, and rely on target
+/// features rather than about version numbers. Will be removed after one
+/// release.
struct Chipset {
unsigned majorVersion = 0; // The major version (decimal).
unsigned minorVersion = 0; // The minor version (hexadecimal).
@@ -30,6 +35,9 @@ struct Chipset {
/// Parses the chipset version string and returns the chipset on success, and
/// failure otherwise.
+ ///
+ /// \deprecated Use `ROCDL::TargetInfo::get`.
+ LLVM_DEPRECATED("use ROCDL::TargetInfo::get instead", "")
static FailureOr<Chipset> parse(StringRef name);
std::tuple<unsigned, unsigned, unsigned> asTuple() const {
@@ -49,6 +57,10 @@ struct Chipset {
#undef DEFINE_COMP_OPERATOR
};
+/// \deprecated Test `llvm::AMDGPU::FEAT_OCP_FP8_CONVERSION_INSTS` on a
+/// `ROCDL::TargetInfo` instead. This misses gfx11.7, which does have the OCP
+/// fp8 conversions.
+LLVM_DEPRECATED("test FEAT_OCP_FP8_CONVERSION_INSTS on a ROCDL::TargetInfo", "")
inline bool hasOcpFp8(const Chipset &chipset) {
return (chipset.majorVersion == 9 && chipset.minorVersion >= 5) ||
chipset.majorVersion >= 12;
diff --git a/mlir/include/mlir/Dialect/GPU/TransformOps/GPUTransformOps.td b/mlir/include/mlir/Dialect/GPU/TransformOps/GPUTransformOps.td
index 3a8caf8aa42e2..d7b6924b4bb33 100644
--- a/mlir/include/mlir/Dialect/GPU/TransformOps/GPUTransformOps.td
+++ b/mlir/include/mlir/Dialect/GPU/TransformOps/GPUTransformOps.td
@@ -61,11 +61,16 @@ def ApplyGPUToROCDLConversionPatternsOp : Op<Transform_Dialect,
let description = [{
Collects patterns that convert GPU dialect ops to ROCDL dialect ops. These
patterns require an "LLVMTypeConverter".
+
+ `triple` names the target, either as a triple ("amdgpu9.42-amd-amdhsa") or
+ as a bare GPU name ("gfx942", "gfx9-4-generic"). `chip` plays the `-mcpu`
+ role and must be compatible with the triple; `features` is a `-mattr`-style
+ comma-separated list of "+feature"/"-feature" modifiers.
}];
- let arguments = (ins StrAttr:$chipset);
- let assemblyFormat = [{
- `chipset` `=` $chipset attr-dict
- }];
+ let arguments = (ins StrAttr:$triple,
+ OptionalAttr<StrAttr>:$chip,
+ OptionalAttr<StrAttr>:$features);
+ let assemblyFormat = "prop-dict attr-dict";
}
//===----------------------------------------------------------------------===//
@@ -330,11 +335,17 @@ def ApplyGPUPromoteShuffleToAMDGPUPatternsOp : Op<Transform_Dialect,
let description = [{
Collects patterns that are tryin to promote `gpu.shuffle`s to specialized
AMDGPU intrinsics.
+
+ `triple` names the target, either as a triple ("amdgpu9.50-amd-amdhsa") or
+ as a bare GPU name ("gfx950"). Omitting it collects only the patterns that
+ hold on every target. `chip` and `features` play the `-mcpu` and `-mattr`
+ roles and may only be given alongside a `triple`.
}];
- let arguments = (ins OptionalAttr<StrAttr>:$chipset);
- let assemblyFormat = [{
- (`chipset` `=` $chipset^)? attr-dict
- }];
+ let arguments = (ins OptionalAttr<StrAttr>:$triple,
+ OptionalAttr<StrAttr>:$chip,
+ OptionalAttr<StrAttr>:$features);
+ let assemblyFormat = "prop-dict attr-dict";
+ let hasVerifier = 1;
}
diff --git a/mlir/include/mlir/Dialect/GPU/Transforms/Passes.h b/mlir/include/mlir/Dialect/GPU/Transforms/Passes.h
index d5c253d6c9c08..6657ff819dea5 100644
--- a/mlir/include/mlir/Dialect/GPU/Transforms/Passes.h
+++ b/mlir/include/mlir/Dialect/GPU/Transforms/Passes.h
@@ -13,9 +13,9 @@
#ifndef MLIR_DIALECT_GPU_TRANSFORMS_PASSES_H_
#define MLIR_DIALECT_GPU_TRANSFORMS_PASSES_H_
-#include "mlir/Dialect/AMDGPU/Utils/Chipset.h"
#include "mlir/Dialect/GPU/IR/GPUDialect.h"
#include "mlir/Dialect/GPU/Utils/GPUUtils.h"
+#include "mlir/Dialect/LLVMIR/ROCDLTargetInfo.h"
#include "mlir/IR/PatternMatch.h"
#include "mlir/Pass/Pass.h"
#include <optional>
@@ -76,16 +76,15 @@ void populateGpuLowerClusteredSubgroupReduceToShufflePatterns(
/// Collect a set of patterns to lower `gpu.subgroup_reduce` into `amdgpu.dpp`
/// ops over scalar types. Assumes that the subgroup has
/// `subgroupSize` lanes. Applicable only to AMD GPUs.
-void populateGpuLowerSubgroupReduceToDPPPatterns(RewritePatternSet &patterns,
- unsigned subgroupSize,
- amdgpu::Chipset chipset,
- PatternBenefit benefit = 1);
+void populateGpuLowerSubgroupReduceToDPPPatterns(
+ RewritePatternSet &patterns, unsigned subgroupSize,
+ const ROCDL::TargetInfo &target, PatternBenefit benefit = 1);
/// Disjoint counterpart of `populateGpuLowerSubgroupReduceToDPPPatterns`
/// that only matches `gpu.subgroup_reduce` ops with a `cluster_size`.
void populateGpuLowerClusteredSubgroupReduceToDPPPatterns(
- RewritePatternSet &patterns, unsigned subgroupSize, amdgpu::Chipset chipset,
- PatternBenefit benefit = 1);
+ RewritePatternSet &patterns, unsigned subgroupSize,
+ const ROCDL::TargetInfo &target, PatternBenefit benefit = 1);
/// Collect all patterns to rewrite ops within the GPU dialect.
inline void populateGpuRewritePatterns(RewritePatternSet &patterns) {
@@ -115,7 +114,7 @@ void populateGpuEliminateBarriersPatterns(RewritePatternSet &patterns);
/// Tries to promote `gpu.shuffle`s to specialized AMDGPU intrinsics.
void populateGpuPromoteShuffleToAMDGPUPatterns(
- RewritePatternSet &patterns, std::optional<amdgpu::Chipset> maybeChipset);
+ RewritePatternSet &patterns, std::optional<ROCDL::TargetInfo> target);
/// Generate the code for registering passes.
#define GEN_PASS_REGISTRATION
diff --git a/mlir/include/mlir/Dialect/LLVMIR/ROCDLTargetInfo.h b/mlir/include/mlir/Dialect/LLVMIR/ROCDLTargetInfo.h
new file mode 100644
index 0000000000000..d8f5c9a16094e
--- /dev/null
+++ b/mlir/include/mlir/Dialect/LLVMIR/ROCDLTargetInfo.h
@@ -0,0 +1,126 @@
+//===- ROCDLTargetInfo.h - AMDGPU target description ------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+#ifndef MLIR_DIALECT_LLVMIR_ROCDLTARGETINFO_H_
+#define MLIR_DIALECT_LLVMIR_ROCDLTARGETINFO_H_
+
+#include "mlir/IR/Diagnostics.h"
+#include "mlir/Support/LLVM.h"
+#include "llvm/TargetParser/AMDGPUTargetParser.h"
+#include "llvm/TargetParser/Triple.h"
+#include <optional>
+
+namespace mlir::ROCDL {
+
+/// Describes the AMDGPU target a lowering is producing code for: the triple's
+/// subarch (which identifies the GPU) together with the resolved set of
+/// frontend-visible target features.
+///
+/// Lowerings should gate on features (`has(FEAT_...)`) rather than on ISA
+/// version arithmetic, and add features if necessary.
+class TargetInfo {
+public:
+ using Feature = ::llvm::AMDGPU::AMDGPUFeature;
+
+ /// Constructs an unknown target: no subarch, and every feature query answers
+ /// false.
+ TargetInfo() = default;
+
+ /// Resolves a target description.
+ ///
+ /// \p tripleOrChip is either a triple ("amdgpu9.42-amd-amdhsa", or the legacy
+ /// subarch-less "amdgcn-amd-amdhsa") or a bare GPU name ("gfx942",
+ /// "gfx9-4-generic"). \p chip plays the role of `-mcpu`: when given alongside
+ /// a triple it names the exact GPU and must be compatible with the triple's
+ /// subarch. \p features is an `-mattr`-style "+a,-b" list applied on top of
+ /// the GPU's default features.
+ ///
+ /// Diagnostics are emitted via `emitError`.
+ static FailureOr<TargetInfo>
+ get(StringRef tripleOrChip, StringRef chip = "", StringRef features = "",
+ function_ref<InFlightDiagnostic()> emitError = nullptr);
+
+ /// Returns whether the target has \p feature.
+ bool has(Feature feature) const { return featureBits.test(feature); }
+
+ /// Returns whether the target's fp8 conversions exist and use the OCP formats
+ /// (E4M3FN/E5M2) rather than the FNUZ ones.
+ bool hasOcpFp8() const {
+ return has(::llvm::AMDGPU::FEAT_OCP_FP8_CONVERSION_INSTS);
+ }
+
+ /// Returns whether the target has fp8 conversions that use the FNUZ formats
+ /// (E4M3FNUZ/E5M2FNUZ).
+ bool hasFnuzFp8() const {
+ return has(::llvm::AMDGPU::FEAT_FP8_CONVERSION_INSTS) && !hasOcpFp8();
+ }
+
+ /// Returns whether the target belongs to gfx generation \p major (9 for any
+ /// gfx9xx, 12 for any gfx12xx, ...).
+ ///
+ /// Prefer `has()` where a feature expresses the condition; this is used when
+ /// no feature exists and the property being checked is a function of the
+ /// major ISA generation (such as the details of buffer encoding).
+ bool isGeneration(unsigned major) const;
+
+ /// Returns the width in bits of the num_records field of the buffer resource
+ /// (V#), or nullopt for an unknown target. This is a descriptor layout width
+ /// rather than a capability, so it is a number: asking "does it have 45-bit
+ /// num_records" only works while there are exactly two widths. There is no
+ /// safe default, so a lowering that needs it must bail out when it is absent
+ /// rather than guess.
+ std::optional<unsigned> getBufferResourceNumRecordsWidth() const;
+
+ /// Returns the maximum LDS in bytes a single workgroup can address, or
+ /// nullopt for an unknown target. This is a fixed hardware cap and does not
+ /// depend on how many SIMDs a workgroup runs on.
+ std::optional<unsigned> getMaxAddressableLocalMemorySize() const;
+
+ /// Returns the wavefront size, or nullopt for an unknown target. Targets that
+ /// support both sizes report 32 unless "+wavefrontsize64" was requested.
+ std::optional<unsigned> getWavefrontSize() const;
+
+ /// Returns whether the GPU runs at either wavefront size, so that the choice
+ /// comes from the features rather than from the GPU. This is the one thing a
+ /// triple alone cannot express, and it is not recoverable from the resolved
+ /// features, which always name a size.
+ bool supportsBothWavefrontSizes() const { return dualWavefrontSize; }
+
+ /// Returns the ISA version. For a generic target this is the floor of the
+ /// family it covers (gfx9-4-generic reports 9.4.0), so it must not be used to
+ /// decide whether an instruction is available.
+ ::llvm::AMDGPU::IsaVersion getIsaVersion() const;
+
+ ::llvm::Triple::SubArchType getSubArch() const { return subArch; }
+ ::llvm::AMDGPU::GPUKind getGPUKind() const { return kind; }
+
+ /// Returns the canonical GPU name ("gfx942", "gfx9-4-generic"), or "" if the
+ /// target is unknown.
+ StringRef getArchName() const;
+
+ /// Returns whether this is a "gfxN-generic" target, which carries only the
+ /// features common to every GPU it covers.
+ bool isGeneric() const;
+
+ /// Returns whether no GPU was identified, in which case every feature query
+ /// answers false.
+ bool isUnknown() const { return kind == ::llvm::AMDGPU::GK_NONE; }
+
+ const ::llvm::AMDGPU::AMDGPUFeatureBitset &getFeatures() const {
+ return featureBits;
+ }
+
+private:
+ ::llvm::Triple::SubArchType subArch = ::llvm::Triple::NoSubArch;
+ ::llvm::AMDGPU::GPUKind kind = ::llvm::AMDGPU::GK_NONE;
+ ::llvm::AMDGPU::AMDGPUFeatureBitset featureBits;
+ bool dualWavefrontSize = false;
+};
+
+} // namespace mlir::ROCDL
+
+#endif // MLIR_DIALECT_LLVMIR_ROCDLTARGETINFO_H_
diff --git a/mlir/lib/Conversion/AMDGPUToROCDL/AMDGPUToROCDL.cpp b/mlir/lib/Conversion/AMDGPUToROCDL/AMDGPUToROCDL.cpp
index 2a846339b562e..b4aec12e76347 100644
--- a/mlir/lib/Conversion/AMDGPUToROCDL/AMDGPUToROCDL.cpp
+++ b/mlir/lib/Conversion/AMDGPUToROCDL/AMDGPUToROCDL.cpp
@@ -14,7 +14,6 @@
#include "mlir/Conversion/LLVMCommon/TypeConverter.h"
#include "mlir/Dialect/AMDGPU/IR/AMDGPUDialect.h"
#include "mlir/Dialect/AMDGPU/IR/AMDGPUEnums.h"
-#include "mlir/Dialect/AMDGPU/Utils/Chipset.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/LLVMIR/LLVMTypes.h"
#include "mlir/Dialect/LLVMIR/ROCDLDialect.h"
@@ -32,6 +31,7 @@
#include "llvm/Support/AMDGPUAddrSpace.h"
#include "llvm/Support/Casting.h"
#include "llvm/Support/ErrorHandling.h"
+#include "llvm/Support/MathExtras.h"
#include <cstdint>
#include <optional>
@@ -43,72 +43,6 @@ namespace mlir {
using namespace mlir;
using namespace mlir::amdgpu;
-// Define commonly used chipsets versions for convenience.
-constexpr Chipset kGfx908 = Chipset(9, 0, 8);
-constexpr Chipset kGfx90a = Chipset(9, 0, 0xa);
-constexpr Chipset kGfx942 = Chipset(9, 4, 2);
-constexpr Chipset kGfx950 = Chipset(9, 5, 0);
-constexpr Chipset kGfx1200 = Chipset(12, 0, 0);
-constexpr Chipset kGfx1250 = Chipset(12, 5, 0);
-
-// Predicates mirroring the LLVM AMDGPU `HasDot{N}Insts` features that gate
-// the `v_dot*` instructions consumed by the `amdgpu.dot` lowering.
-static bool hasDot1Insts(const Chipset &chipset) {
- if (chipset.majorVersion == 9)
- return chipset >= Chipset(9, 0, 6);
- if (chipset.majorVersion == 10) {
- if (chipset.minorVersion == 1)
- return chipset.steppingVersion == 1u || chipset.steppingVersion == 2u;
- return chipset.minorVersion >= 3u;
- }
- return false;
-}
-
-static bool hasDot2Insts(const Chipset &chipset) {
- return hasDot1Insts(chipset);
-}
-
-static bool hasDot7Insts(const Chipset &chipset) {
- return chipset.majorVersion >= 11 || hasDot1Insts(chipset);
-}
-
-static bool hasDot8Insts(const Chipset &chipset) {
- return chipset.majorVersion >= 11;
-}
-
-static bool hasDot9Insts(const Chipset &chipset) {
- if (chipset.majorVersion == 11)
- return true;
- return chipset.majorVersion == 12 && chipset.minorVersion == 0;
-}
-
-static bool hasDot10Insts(const Chipset &chipset) {
- if (chipset.majorVersion == 11)
- return true;
- if (chipset.majorVersion == 12)
- return chipset.minorVersion == 0;
- return hasDot1Insts(chipset);
-}
-
-static bool hasDot11Insts(const Chipset &chipset) {
- if (chipset.majorVersion == 11)
- return chipset.minorVersion == 7u;
- return chipset.majorVersion == 12 && chipset.minorVersion == 0;
-}
-
-static bool hasDot12Insts(const Chipset &chipset) {
- if (chipset == Chipset(9, 5, 0))
- return true;
- if (chipset.majorVersion == 11)
- return true;
- return chipset.majorVersion == 12 && chipset.minorVersion == 0;
-}
-
-static bool has45BitNumRecordsBufferResource(const Chipset &chipset) {
- return chipset.majorVersion > 12 ||
- (chipset.majorVersion == 12 && chipset.minorVersion >= 5);
-}
-
/// Zero-extend or truncate the unsigned number `val` to `width` bits.
static Value convertUnsignedToInt(ConversionPatternRewriter &rewriter,
Location loc, Value val, unsigned width) {
@@ -172,10 +106,9 @@ static Value getNumRecords(ConversionPatternRewriter &rewriter, Location loc,
MemRefType memrefType,
MemRefDescriptor &memrefDescriptor,
ArrayRef<int64_t> strides, int64_t elementByteWidth,
- amdgpu::Chipset chipset, bool boundsCheck) {
- if (has45BitNumRecordsBufferResource(chipset) && !boundsCheck) {
- constexpr int64_t first45bits = (1ll << 45) - 1;
- return createI64Constant(rewriter, loc, first45bits);
+ unsigned numRecordsWidth, bool boundsCheck) {
+ if (numRecordsWidth > 32 && !boundsCheck) {
+ return createI64Constant(rewriter, loc, llvm::maxUIntN(numRecordsWidth));
}
if (memrefType.hasStaticShape() &&
!llvm::any_of(strides, ShapedType::isDynamic)) {
@@ -202,7 +135,8 @@ static Value getNumRecords(ConversionPatternRewriter &rewriter, Location loc,
static Value makeBufferRsrc(ConversionPatternRewriter &rewriter, Location loc,
Value basePointer, Value numRecords,
- bool boundsCheck, amdgpu::Chipset chipset,
+ bool boundsCheck, const ROCDL::TargetInfo &target,
+ unsigned numRecordsWidth,
Value cacheSwizzleStride = nullptr,
unsigned addressSpace = 8) {
// The stride value is generally 0. However, on MI-300 and onward, you can
@@ -210,7 +144,7 @@ static Value makeBufferRsrc(ConversionPatternRewriter &rewriter, Location loc,
// and setting that stride to a cache stride.
Type i16 = rewriter.getI16Type();
Value stride;
- if (chipset.majorVersion == 9 && chipset >= kGfx942 && cacheSwizzleStride) {
+ if (target.has(llvm::AMDGPU::FEAT_GFX940_INSTS) && cacheSwizzleStride) {
Value cacheStrideZext =
LLVM::ZExtOp::create(rewriter, loc, i16, cacheSwizzleStride);
Value swizzleBit = LLVM::ConstantOp::create(
@@ -223,7 +157,7 @@ static Value makeBufferRsrc(ConversionPatternRewriter &rewriter, Location loc,
}
uint32_t flags = 0;
- if (chipset >= kGfx1250) {
+ if (target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS)) {
// Flag word:
// bit 0: swizzle
// bit 1: 0 means (total_offset + payload > numRecords)
@@ -249,16 +183,14 @@ static Value makeBufferRsrc(ConversionPatternRewriter &rewriter, Location loc,
// none, 3 = either swizzles or testing against offset field) RDNA only
// bits 30-31: Type (must be 0)
flags |= (7 << 12) | (4 << 15);
- if (chipset.majorVersion >= 10) {
+ if (target.has(llvm::AMDGPU::FEAT_GFX10_INSTS)) {
flags |= (1 << 24);
uint32_t oob = boundsCheck ? 3 : 2;
flags |= (oob << 28);
}
}
Value flagsConst = createI32Constant(rewriter, loc, flags);
- numRecords =
- convertUnsignedToInt(rewriter, loc, numRecords,
- has45BitNumRecordsBufferResource(chipset) ? 45 : 32);
+ numRecords = convertUnsignedToInt(rewriter, loc, numRecords, numRecordsWidth);
Type rsrcType =
LLVM::LLVMPointerType::get(rewriter.getContext(), addressSpace);
Value resource = rewriter.createOrFold<ROCDL::MakeBufferRsrcOp>(
@@ -269,11 +201,11 @@ static Value makeBufferRsrc(ConversionPatternRewriter &rewriter, Location loc,
namespace {
struct FatRawBufferCastLowering
: public ConvertOpToLLVMPattern<FatRawBufferCastOp> {
- FatRawBufferCastLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<FatRawBufferCastOp>(converter),
- chipset(chipset) {}
+ FatRawBufferCastLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<FatRawBufferCastOp>(converter), target(target) {}
- Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(FatRawBufferCastOp op, FatRawBufferCastOpAdaptor adaptor,
@@ -293,11 +225,17 @@ struct FatRawBufferCastLowering
if (failed(memrefType.getStridesAndOffset(strideVals, unusedOffset)))
return op.emitOpError("Can't lower non-stride-offset memrefs");
+ std::optional<unsigned> numRecordsWidth =
+ target.getBufferResourceNumRecordsWidth();
+ if (!numRecordsWidth)
+ return op.emitOpError(
+ "buffer resource num_records width is unknown for this target");
+
Value numRecords = adaptor.getValidBytes();
if (!numRecords)
- numRecords =
- getNumRecords(rewriter, loc, memrefType, descriptor, strideVals,
- elementByteWidth, chipset, adaptor.getBoundsCheck());
+ numRecords = getNumRecords(rewriter, loc, memrefType, descriptor,
+ strideVals, elementByteWidth, *numRecordsWidth,
+ adaptor.getBoundsCheck());
Value basePointer =
adaptor.getResetOffset()
@@ -324,7 +262,8 @@ struct FatRawBufferCastLowering
Value fatPtr = makeBufferRsrc(
rewriter, loc, basePointer, numRecords, adaptor.getBoundsCheck(),
- chipset, adaptor.getCacheSwizzleStride(), /*addressSpace=*/7);
+ target, *numRecordsWidth, adaptor.getCacheSwizzleStride(),
+ /*addressSpace=*/7);
Value result = MemRefDescriptor::poison(
rewriter, loc,
@@ -349,10 +288,11 @@ struct FatRawBufferCastLowering
/// Define lowering patterns for raw buffer ops
template <typename GpuOp, typename Intrinsic>
struct RawBufferOpLowering : public ConvertOpToLLVMPattern<GpuOp> {
- RawBufferOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<GpuOp>(converter), chipset(chipset) {}
+ RawBufferOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<GpuOp>(converter), target(target) {}
- Chipset chipset;
+ ROCDL::TargetInfo target;
static constexpr uint32_t maxVectorOpWidth = 128;
LogicalResult
@@ -363,7 +303,7 @@ struct RawBufferOpLowering : public ConvertOpToLLVMPattern<GpuOp> {
Value unconvertedMemref = gpuOp.getMemref();
MemRefType memrefType = cast<MemRefType>(unconvertedMemref.getType());
- if (chipset.majorVersion < 9)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX9_INSTS))
return gpuOp.emitOpError("raw buffer ops require GCN or higher");
Value storeData = adaptor.getODSOperands(0)[0];
@@ -468,11 +408,18 @@ struct RawBufferOpLowering : public ConvertOpToLLVMPattern<GpuOp> {
Value ptr = memrefDescriptor.bufferPtr(
rewriter, loc, *this->getTypeConverter(), memrefType);
- Value numRecords =
- getNumRecords(rewriter, loc, memrefType, memrefDescriptor, strides,
- elementByteWidth, chipset, adaptor.getBoundsCheck());
- Value resource = makeBufferRsrc(rewriter, loc, ptr, numRecords,
- adaptor.getBoundsCheck(), chipset);
+ std::optional<unsigned> numRecordsWidth =
+ target.getBufferResourceNumRecordsWidth();
+ if (!numRecordsWidth)
+ return gpuOp.emitOpError(
+ "buffer resource num_records width is unknown for this target");
+
+ Value numRecords = getNumRecords(
+ rewriter, loc, memrefType, memrefDescriptor, strides, elementByteWidth,
+ *numRecordsWidth, adaptor.getBoundsCheck());
+ Value resource =
+ makeBufferRsrc(rewriter, loc, ptr, numRecords, adaptor.getBoundsCheck(),
+ target, *numRecordsWidth);
args.push_back(resource);
// Indexing (voffset)
@@ -526,15 +473,16 @@ struct RawBufferOpLowering : public ConvertOpToLLVMPattern<GpuOp> {
/// Lgkmcnt = Waitcnt[11:8] (pre-gfx10)
/// Lgkmcnt = Waitcnt[13:8] (gfx10)
/// Lgkmcnt = Waitcnt[9:4] (gfx11)
-static FailureOr<unsigned> encodeWaitcnt(Chipset chipset, unsigned vmcnt,
- unsigned expcnt, unsigned lgkmcnt) {
- if (chipset.majorVersion < 9) {
+static FailureOr<unsigned> encodeWaitcnt(const ROCDL::TargetInfo &target,
+ unsigned vmcnt, unsigned expcnt,
+ unsigned lgkmcnt) {
+ if (!target.has(llvm::AMDGPU::FEAT_GFX9_INSTS)) {
vmcnt = std::min(15u, vmcnt);
expcnt = std::min(7u, expcnt);
lgkmcnt = std::min(15u, lgkmcnt);
return vmcnt | (expcnt << 4) | (lgkmcnt << 8);
}
- if (chipset.majorVersion == 9) {
+ if (target.isGeneration(9)) {
vmcnt = std::min(63u, vmcnt);
expcnt = std::min(7u, expcnt);
lgkmcnt = std::min(15u, lgkmcnt);
@@ -543,7 +491,7 @@ static FailureOr<unsigned> encodeWaitcnt(Chipset chipset, unsigned vmcnt,
unsigned otherCnts = (expcnt << 4) | (lgkmcnt << 8);
return lowBits | highBits | otherCnts;
}
- if (chipset.majorVersion == 10) {
+ if (target.isGeneration(10)) {
vmcnt = std::min(63u, vmcnt);
expcnt = std::min(7u, expcnt);
lgkmcnt = std::min(63u, lgkmcnt);
@@ -552,7 +500,7 @@ static FailureOr<unsigned> encodeWaitcnt(Chipset chipset, unsigned vmcnt,
unsigned otherCnts = (expcnt << 4) | (lgkmcnt << 8);
return lowBits | highBits | otherCnts;
}
- if (chipset.majorVersion == 11) {
+ if (target.isGeneration(11)) {
vmcnt = std::min(63u, vmcnt);
expcnt = std::min(7u, expcnt);
lgkmcnt = std::min(63u, lgkmcnt);
@@ -564,16 +512,16 @@ static FailureOr<unsigned> encodeWaitcnt(Chipset chipset, unsigned vmcnt,
struct MemoryCounterWaitOpLowering
: public ConvertOpToLLVMPattern<MemoryCounterWaitOp> {
MemoryCounterWaitOpLowering(const LLVMTypeConverter &converter,
- Chipset chipset)
- : ConvertOpToLLVMPattern<MemoryCounterWaitOp>(converter),
- chipset(chipset) {}
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<MemoryCounterWaitOp>(converter), target(target) {
+ }
- Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(MemoryCounterWaitOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset.majorVersion >= 12) {
+ if (target.has(llvm::AMDGPU::FEAT_GFX12_INSTS)) {
Location loc = op.getLoc();
if (std::optional<int> ds = adaptor.getDs())
ROCDL::WaitDscntOp::create(rewriter, loc, *ds);
@@ -618,7 +566,7 @@ struct MemoryCounterWaitOpLowering
vmcnt = getVal(store);
}
- FailureOr<unsigned> waitcnt = encodeWaitcnt(chipset, vmcnt, exp, ds);
+ FailureOr<unsigned> waitcnt = encodeWaitcnt(target, vmcnt, exp, ds);
if (failed(waitcnt))
return op.emitOpError("unsupported chipset");
@@ -628,18 +576,23 @@ struct MemoryCounterWaitOpLowering
};
struct LDSBarrierOpLowering : public ConvertOpToLLVMPattern<LDSBarrierOp> {
- LDSBarrierOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<LDSBarrierOp>(converter), chipset(chipset) {}
+ LDSBarrierOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<LDSBarrierOp>(converter), target(target) {}
- Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(LDSBarrierOp op, LDSBarrierOp::Adaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
Location loc = op.getLoc();
- // This ensures that waits on global memory aren't introduced on
- // chips that don't have the BackOffBarrier feature enabled in LLVM.
- bool requiresInlineAsm = chipset < kGfx90a;
+ // Targets with split barriers never emit s_barrier, so they need neither
+ // the hardware back-off nor the workaround below. Of the rest, those
+ // without FeatureBackOffBarrier would have waits on global memory
+ // introduced around s_barrier, which the inline asm avoids.
+ bool hasSplitBarriers = target.has(llvm::AMDGPU::FEAT_GFX12_INSTS);
+ bool requiresInlineAsm =
+ !hasSplitBarriers && !target.has(llvm::AMDGPU::FEAT_BACK_OFF_BARRIER);
Attribute mmra =
rewriter.getAttr<LLVM::MMRATagAttr>("amdgpu-synchronize-as", "local");
@@ -668,7 +621,7 @@ struct LDSBarrierOpLowering : public ConvertOpToLLVMPattern<LDSBarrierOp> {
/*is_align_stack=*/false, LLVM::TailCallKind::None,
/*asm_dialect=*/asmDialectAttr,
/*operand_attrs=*/ArrayAttr());
- } else if (chipset.majorVersion < 12) {
+ } else if (!hasSplitBarriers) {
ROCDL::SBarrierOp::create(rewriter, loc);
} else {
ROCDL::BarrierSignalOp::create(rewriter, loc, -1);
@@ -684,10 +637,11 @@ struct LDSBarrierOpLowering : public ConvertOpToLLVMPattern<LDSBarrierOp> {
};
struct SchedBarrierOpLowering : public ConvertOpToLLVMPattern<SchedBarrierOp> {
- SchedBarrierOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<SchedBarrierOp>(converter), chipset(chipset) {}
+ SchedBarrierOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<SchedBarrierOp>(converter), target(target) {}
- Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(SchedBarrierOp op, SchedBarrierOp::Adaptor adaptor,
@@ -903,31 +857,34 @@ static void wmmaPushOutputOperand(ConversionPatternRewriter &rewriter,
}
/// Return true if `type` is the E5M2 variant of an 8-bit float that is
-/// supported by the `_bf8` instructions on the given `chipset`.
-static bool typeIsExpectedBf8ForChipset(Chipset chipset, Type type) {
- return (chipset == kGfx942 && isa<Float8E5M2FNUZType>(type)) ||
- (hasOcpFp8(chipset) && isa<Float8E5M2Type>(type));
+/// supported by the `_bf8` instructions on `target`.
+static bool typeIsExpectedBf8ForTarget(const ROCDL::TargetInfo &target,
+ Type type) {
+ return (target.hasFnuzFp8() && isa<Float8E5M2FNUZType>(type)) ||
+ (target.hasOcpFp8() && isa<Float8E5M2Type>(type));
}
/// Return true if `type` is the E4M3FN variant of an 8-bit float that is
-/// supported by the `_fp8` instructions on the given `chipset`.
-static bool typeIsExpectedFp8ForChipset(Chipset chipset, Type type) {
- return (chipset == kGfx942 && isa<Float8E4M3FNUZType>(type)) ||
- (hasOcpFp8(chipset) && isa<Float8E4M3FNType>(type));
+/// supported by the `_fp8` instructions on `target`.
+static bool typeIsExpectedFp8ForTarget(const ROCDL::TargetInfo &target,
+ Type type) {
+ return (target.hasFnuzFp8() && isa<Float8E4M3FNUZType>(type)) ||
+ (target.hasOcpFp8() && isa<Float8E4M3FNType>(type));
}
/// Return the `rocdl` intrinsic corresponding to a MFMA operation `mfma`
/// if one exists. This includes checking to ensure the intrinsic is supported
/// on the architecture you are compiling for.
-static std::optional<StringRef> mfmaOpToIntrinsic(MFMAOp mfma,
- Chipset chipset) {
+static std::optional<StringRef>
+mfmaOpToIntrinsic(MFMAOp mfma, const ROCDL::TargetInfo &target) {
uint32_t m = mfma.getM(), n = mfma.getN(), k = mfma.getK(),
b = mfma.getBlocks();
Type sourceElem = getElementTypeOrSelf(mfma.getSourceA().getType());
Type destElem = getElementTypeOrSelf(mfma.getDestC().getType());
if (sourceElem.isF32() && destElem.isF32()) {
- if (mfma.getReducePrecision() && chipset >= kGfx942) {
+ if (mfma.getReducePrecision() &&
+ target.has(llvm::AMDGPU::FEAT_XF32_INSTS)) {
if (m == 32 && n == 32 && k == 4 && b == 1)
return ROCDL::mfma_f32_32x32x4_xf32::getOperationName();
if (m == 16 && n == 16 && k == 8 && b == 1)
@@ -946,7 +903,7 @@ static std::optional<StringRef> mfmaOpToIntrinsic(MFMAOp mfma,
}
if (sourceElem.isF16() && destElem.isF32()) {
- if (chipset >= kGfx950) {
+ if (target.has(llvm::AMDGPU::FEAT_GFX950_INSTS)) {
if (m == 32 && n == 32 && k == 16 && b == 1)
return ROCDL::mfma_f32_32x32x16_f16::getOperationName();
if (m == 16 && n == 16 && k == 32 && b == 1)
@@ -965,13 +922,13 @@ static std::optional<StringRef> mfmaOpToIntrinsic(MFMAOp mfma,
}
if (sourceElem.isBF16() && destElem.isF32()) {
- if (chipset >= kGfx950) {
+ if (target.has(llvm::AMDGPU::FEAT_GFX950_INSTS)) {
if (m == 32 && n == 32 && k == 16 && b == 1)
return ROCDL::mfma_f32_32x32x16_bf16::getOperationName();
if (m == 16 && n == 16 && k == 32 && b == 1)
return ROCDL::mfma_f32_16x16x32_bf16::getOperationName();
}
- if (chipset >= kGfx90a) {
+ if (target.has(llvm::AMDGPU::FEAT_GFX90A_INSTS)) {
if (m == 32 && n == 32 && k == 4 && b == 2)
return ROCDL::mfma_f32_32x32x4bf16_1k::getOperationName();
if (m == 16 && n == 16 && k == 4 && b == 4)
@@ -996,7 +953,7 @@ static std::optional<StringRef> mfmaOpToIntrinsic(MFMAOp mfma,
}
if (sourceElem.isInteger(8) && destElem.isInteger(32)) {
- if (chipset >= kGfx950) {
+ if (target.has(llvm::AMDGPU::FEAT_GFX950_INSTS)) {
if (m == 32 && n == 32 && k == 32 && b == 1)
return ROCDL::mfma_i32_32x32x32_i8::getOperationName();
if (m == 16 && n == 16 && k == 64 && b == 1)
@@ -1012,51 +969,54 @@ static std::optional<StringRef> mfmaOpToIntrinsic(MFMAOp mfma,
return ROCDL::mfma_i32_32x32x8i8::getOperationName();
if (m == 16 && n == 16 && k == 16 && b == 1)
return ROCDL::mfma_i32_16x16x16i8::getOperationName();
- if (m == 32 && n == 32 && k == 16 && b == 1 && chipset >= kGfx942)
+ if (m == 32 && n == 32 && k == 16 && b == 1 &&
+ target.has(llvm::AMDGPU::FEAT_GFX940_INSTS))
return ROCDL::mfma_i32_32x32x16_i8::getOperationName();
- if (m == 16 && n == 16 && k == 32 && b == 1 && chipset >= kGfx942)
+ if (m == 16 && n == 16 && k == 32 && b == 1 &&
+ target.has(llvm::AMDGPU::FEAT_GFX940_INSTS))
return ROCDL::mfma_i32_16x16x32_i8::getOperationName();
}
- if (sourceElem.isF64() && destElem.isF64() && chipset >= kGfx90a) {
+ if (sourceElem.isF64() && destElem.isF64() &&
+ target.has(llvm::AMDGPU::FEAT_GFX90A_INSTS)) {
if (m == 16 && n == 16 && k == 4 && b == 1)
return ROCDL::mfma_f64_16x16x4f64::getOperationName();
if (m == 4 && n == 4 && k == 4 && b == 4)
return ROCDL::mfma_f64_4x4x4f64::getOperationName();
}
- if (destElem.isF32() && typeIsExpectedBf8ForChipset(chipset, sourceElem)) {
+ if (destElem.isF32() && typeIsExpectedBf8ForTarget(target, sourceElem)) {
// Known to be correct because there are no scalar f8 instructions and
// because a length mismatch will have been caught by the verifier.
Type sourceBElem =
cast<VectorType>(mfma.getSourceB().getType()).getElementType();
if (m == 16 && n == 16 && k == 32 && b == 1) {
- if (typeIsExpectedBf8ForChipset(chipset, sourceBElem))
+ if (typeIsExpectedBf8ForTarget(target, sourceBElem))
return ROCDL::mfma_f32_16x16x32_bf8_bf8::getOperationName();
- if (typeIsExpectedFp8ForChipset(chipset, sourceBElem))
+ if (typeIsExpectedFp8ForTarget(target, sourceBElem))
return ROCDL::mfma_f32_16x16x32_bf8_fp8::getOperationName();
}
if (m == 32 && n == 32 && k == 16 && b == 1) {
- if (typeIsExpectedBf8ForChipset(chipset, sourceBElem))
+ if (typeIsExpectedBf8ForTarget(target, sourceBElem))
return ROCDL::mfma_f32_32x32x16_bf8_bf8::getOperationName();
- if (typeIsExpectedFp8ForChipset(chipset, sourceBElem))
+ if (typeIsExpectedFp8ForTarget(target, sourceBElem))
return ROCDL::mfma_f32_32x32x16_bf8_fp8::getOperationName();
}
}
- if (destElem.isF32() && typeIsExpectedFp8ForChipset(chipset, sourceElem)) {
+ if (destElem.isF32() && typeIsExpectedFp8ForTarget(target, sourceElem)) {
Type sourceBElem =
cast<VectorType>(mfma.getSourceB().getType()).getElementType();
if (m == 16 && n == 16 && k == 32 && b == 1) {
- if (typeIsExpectedBf8ForChipset(chipset, sourceBElem))
+ if (typeIsExpectedBf8ForTarget(target, sourceBElem))
return ROCDL::mfma_f32_16x16x32_fp8_bf8::getOperationName();
- if (typeIsExpectedFp8ForChipset(chipset, sourceBElem))
+ if (typeIsExpectedFp8ForTarget(target, sourceBElem))
return ROCDL::mfma_f32_16x16x32_fp8_fp8::getOperationName();
}
if (m == 32 && n == 32 && k == 16 && b == 1) {
- if (typeIsExpectedBf8ForChipset(chipset, sourceBElem))
+ if (typeIsExpectedBf8ForTarget(target, sourceBElem))
return ROCDL::mfma_f32_32x32x16_fp8_bf8::getOperationName();
- if (typeIsExpectedFp8ForChipset(chipset, sourceBElem))
+ if (typeIsExpectedFp8ForTarget(target, sourceBElem))
return ROCDL::mfma_f32_32x32x16_fp8_fp8::getOperationName();
}
}
@@ -1088,12 +1048,13 @@ using ScaledMFMAIntrinsic =
static std::optional<ScaledMFMAIntrinsic>
mfmaOpToScaledIntrinsic(Type aType, Type bType, Type destType, uint32_t m,
- uint32_t n, uint32_t k, uint32_t b, Chipset chipset) {
+ uint32_t n, uint32_t k, uint32_t b,
+ const ROCDL::TargetInfo &target) {
aType = getElementTypeOrSelf(aType);
bType = getElementTypeOrSelf(bType);
destType = getElementTypeOrSelf(destType);
- if (chipset < kGfx950)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX950_INSTS))
return std::nullopt;
if (!isa<Float32Type>(destType))
return std::nullopt;
@@ -1117,19 +1078,19 @@ mfmaOpToScaledIntrinsic(Type aType, Type bType, Type destType, uint32_t m,
}
static std::optional<ScaledMFMAIntrinsic>
-mfmaOpToScaledIntrinsic(MFMAOp mfma, Chipset chipset) {
+mfmaOpToScaledIntrinsic(MFMAOp mfma, const ROCDL::TargetInfo &target) {
return mfmaOpToScaledIntrinsic(
mfma.getSourceA().getType(), mfma.getSourceB().getType(),
mfma.getDestC().getType(), mfma.getM(), mfma.getN(), mfma.getK(),
- mfma.getBlocks(), chipset);
+ mfma.getBlocks(), target);
}
static std::optional<ScaledMFMAIntrinsic>
-mfmaOpToScaledIntrinsic(ScaledMFMAOp smfma, Chipset chipset) {
+mfmaOpToScaledIntrinsic(ScaledMFMAOp smfma, const ROCDL::TargetInfo &target) {
return mfmaOpToScaledIntrinsic(smfma.getSourceA().getType(),
smfma.getSourceB().getType(),
smfma.getDestC().getType(), smfma.getM(),
- smfma.getN(), smfma.getK(), 1u, chipset);
+ smfma.getN(), smfma.getK(), 1u, target);
}
/// Returns the `rocdl` intrinsic corresponding to a WMMA operation `wmma`
@@ -1284,11 +1245,11 @@ static std::optional<StringRef> wmmaOpToIntrinsicGfx1250(Type elemSourceType,
/// Returns the `rocdl` intrinsic corresponding to a SparseMFMA (smfmac)
/// operation if one exists. This includes checking to ensure the intrinsic is
/// supported on the architecture you are compiling for.
-static std::optional<StringRef> smfmacOpToIntrinsic(SparseMFMAOp op,
- Chipset chipset) {
- bool isGfx950 = chipset >= kGfx950;
- auto isFp8 = [&](Type t) { return typeIsExpectedFp8ForChipset(chipset, t); };
- auto isBf8 = [&](Type t) { return typeIsExpectedBf8ForChipset(chipset, t); };
+static std::optional<StringRef>
+smfmacOpToIntrinsic(SparseMFMAOp op, const ROCDL::TargetInfo &target) {
+ bool isGfx950 = target.has(llvm::AMDGPU::FEAT_GFX950_INSTS);
+ auto isFp8 = [&](Type t) { return typeIsExpectedFp8ForTarget(target, t); };
+ auto isBf8 = [&](Type t) { return typeIsExpectedBf8ForTarget(target, t); };
uint32_t m = op.getM(), n = op.getN(), k = op.getK();
Type sourceAElem = getElementTypeOrSelf(op.getSourceA().getType());
@@ -1383,8 +1344,8 @@ static std::optional<StringRef> smfmacOpToIntrinsic(SparseMFMAOp op,
/// Returns the `rocdl` intrinsic corresponding to a WMMA operation `wmma`
/// if one exists. This includes checking to ensure the intrinsic is supported
/// on the architecture you are compiling for.
-static std::optional<StringRef> wmmaOpToIntrinsic(WMMAOp wmma,
- Chipset chipset) {
+static std::optional<StringRef>
+wmmaOpToIntrinsic(WMMAOp wmma, const ROCDL::TargetInfo &target) {
auto sourceVectorType = cast<VectorType>(wmma.getSourceA().getType());
auto sourceBVectorType = cast<VectorType>(wmma.getSourceB().getType());
auto destVectorType = cast<VectorType>(wmma.getDestC().getType());
@@ -1393,8 +1354,9 @@ static std::optional<StringRef> wmmaOpToIntrinsic(WMMAOp wmma,
Type elemDestType = destVectorType.getElementType();
const uint32_t k = wmma.getK();
- const bool isRDNA3 = chipset.majorVersion == 11;
- const bool isRDNA4 = chipset.majorVersion == 12 && chipset.minorVersion == 0;
+ const bool isRDNA3 = target.isGeneration(11);
+ const bool isRDNA4 =
+ target.isGeneration(12) && !target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS);
// Handle RDNA3 and RDNA4.
if (isRDNA3 || isRDNA4)
@@ -1402,7 +1364,7 @@ static std::optional<StringRef> wmmaOpToIntrinsic(WMMAOp wmma,
k, isRDNA3);
// Handle gfx1250.
- if (chipset == kGfx1250)
+ if (target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS))
return wmmaOpToIntrinsicGfx1250(elemSourceType, elemBSourceType,
elemDestType, k);
@@ -1420,7 +1382,7 @@ struct SparseWMMAOpInfo {
};
static std::optional<SparseWMMAOpInfo>
-sparseWMMAOpToIntrinsic(SparseWMMAOp swmmac, Chipset chipset) {
+sparseWMMAOpToIntrinsic(SparseWMMAOp swmmac, const ROCDL::TargetInfo &target) {
Type sourceAElem = getElementTypeOrSelf(swmmac.getSourceA().getType());
Type sourceBElem = getElementTypeOrSelf(swmmac.getSourceB().getType());
Type destElem = getElementTypeOrSelf(swmmac.getDestC().getType());
@@ -1430,7 +1392,8 @@ sparseWMMAOpToIntrinsic(SparseWMMAOp swmmac, Chipset chipset) {
if ((m != 16) || (n != 16))
return std::nullopt;
- const bool isRDNA4 = chipset.majorVersion == 12 && chipset.minorVersion == 0;
+ const bool isRDNA4 =
+ target.isGeneration(12) && !target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS);
if (isRDNA4) {
if (k == 32) {
if (destElem.isF32() && sourceAElem.isF16() && sourceBElem.isF16())
@@ -1488,7 +1451,7 @@ sparseWMMAOpToIntrinsic(SparseWMMAOp swmmac, Chipset chipset) {
}
}
- const bool isGFX1250 = chipset == kGfx1250;
+ const bool isGFX1250 = target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS);
const bool isWavesize64 = swmmac.getWave64();
if (isGFX1250 && !isWavesize64) {
if (k == 64) {
@@ -1566,10 +1529,11 @@ sparseWMMAOpToIntrinsic(SparseWMMAOp swmmac, Chipset chipset) {
namespace {
struct MFMAOpLowering : public ConvertOpToLLVMPattern<MFMAOp> {
- MFMAOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<MFMAOp>(converter), chipset(chipset) {}
+ MFMAOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<MFMAOp>(converter), target(target) {}
- Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(MFMAOp op, MFMAOpAdaptor adaptor,
@@ -1582,18 +1546,18 @@ struct MFMAOpLowering : public ConvertOpToLLVMPattern<MFMAOp> {
if (outVecType.getElementType().isBF16())
intrinsicOutType = outVecType.clone(rewriter.getI16Type());
- if (chipset.majorVersion != 9 || chipset < kGfx908)
+ if (!target.has(llvm::AMDGPU::FEAT_MAI_INSTS))
return op->emitOpError("MFMA only supported on gfx908+");
uint32_t getBlgpField = static_cast<uint32_t>(op.getBlgp());
if (op.getNegateA() || op.getNegateB() || op.getNegateC()) {
- if (chipset < kGfx942)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX940_INSTS))
return op.emitOpError("negation unsupported on older than gfx942");
getBlgpField |=
op.getNegateA() | (op.getNegateB() << 1) | (op.getNegateC() << 2);
}
- std::optional<StringRef> maybeIntrinsic = mfmaOpToIntrinsic(op, chipset);
+ std::optional<StringRef> maybeIntrinsic = mfmaOpToIntrinsic(op, target);
std::optional<ScaledMFMAIntrinsic> maybeScaledIntrinsic =
- mfmaOpToScaledIntrinsic(op, chipset);
+ mfmaOpToScaledIntrinsic(op, target);
if (!maybeIntrinsic.has_value() && !maybeScaledIntrinsic.has_value())
return op.emitOpError("no intrinsic matching MFMA size on given chipset");
@@ -1611,7 +1575,7 @@ struct MFMAOpLowering : public ConvertOpToLLVMPattern<MFMAOp> {
// Determine if we can use bf16 in the intrinsic. Newer MFMAs in gfx950+
// allows bf16 as the input. For reference check IntrinsicsAMDGPU.td file.
bool allowBf16 = [&]() {
- if (chipset < kGfx950)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX950_INSTS))
return false;
if (isScaled)
return true;
@@ -1659,10 +1623,11 @@ struct MFMAOpLowering : public ConvertOpToLLVMPattern<MFMAOp> {
};
struct ScaledMFMAOpLowering : public ConvertOpToLLVMPattern<ScaledMFMAOp> {
- ScaledMFMAOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern(converter), chipset(chipset) {}
+ ScaledMFMAOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern(converter), target(target) {}
- Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(ScaledMFMAOp op, ScaledMFMAOpAdaptor adaptor,
@@ -1670,10 +1635,10 @@ struct ScaledMFMAOpLowering : public ConvertOpToLLVMPattern<ScaledMFMAOp> {
Location loc = op.getLoc();
Type intrinsicOutType = typeConverter->convertType(op.getDestD().getType());
- if (chipset.majorVersion != 9 || chipset < kGfx950)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX950_INSTS))
return op->emitOpError("scaled MFMA only supported on gfx908+");
std::optional<ScaledMFMAIntrinsic> maybeScaledIntrinsic =
- mfmaOpToScaledIntrinsic(op, chipset);
+ mfmaOpToScaledIntrinsic(op, target);
if (!maybeScaledIntrinsic.has_value())
return op.emitOpError(
"no intrinsic matching scaled MFMA size on given chipset");
@@ -1705,10 +1670,11 @@ struct ScaledMFMAOpLowering : public ConvertOpToLLVMPattern<ScaledMFMAOp> {
};
struct SparseMFMAOpLowering : public ConvertOpToLLVMPattern<SparseMFMAOp> {
- SparseMFMAOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<SparseMFMAOp>(converter), chipset(chipset) {}
+ SparseMFMAOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<SparseMFMAOp>(converter), target(target) {}
- Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(SparseMFMAOp op, SparseMFMAOpAdaptor adaptor,
@@ -1720,10 +1686,10 @@ struct SparseMFMAOpLowering : public ConvertOpToLLVMPattern<SparseMFMAOp> {
return rewriter.notifyMatchFailure(op, "type conversion failed");
// smfmac is supported on gfx942 and gfx950.
- if (chipset.majorVersion != 9 || chipset < kGfx942)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX940_INSTS))
return op->emitOpError("sparse MFMA (smfmac) only supported on gfx942+");
- std::optional<StringRef> maybeIntrinsic = smfmacOpToIntrinsic(op, chipset);
+ std::optional<StringRef> maybeIntrinsic = smfmacOpToIntrinsic(op, target);
if (!maybeIntrinsic.has_value())
return op.emitOpError(
"no intrinsic matching sparse MFMA on the given chipset");
@@ -1732,7 +1698,8 @@ struct SparseMFMAOpLowering : public ConvertOpToLLVMPattern<SparseMFMAOp> {
ROCDL::smfmac_f32_16x16x32_bf16::getOperationName() ||
*maybeIntrinsic ==
ROCDL::smfmac_f32_32x32x16_bf16::getOperationName());
- bool isGfx950 = (chipset >= kGfx950) && !isGfx942BF16;
+ bool isGfx950 =
+ (target.has(llvm::AMDGPU::FEAT_GFX950_INSTS)) && !isGfx942BF16;
Value a = convertPackedVectorOperand(rewriter, loc, adaptor.getSourceA(),
isGfx950);
@@ -1760,10 +1727,11 @@ struct SparseMFMAOpLowering : public ConvertOpToLLVMPattern<SparseMFMAOp> {
};
struct WMMAOpLowering : public ConvertOpToLLVMPattern<WMMAOp> {
- WMMAOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<WMMAOp>(converter), chipset(chipset) {}
+ WMMAOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<WMMAOp>(converter), target(target) {}
- Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(WMMAOp op, WMMAOpAdaptor adaptor,
@@ -1774,10 +1742,10 @@ struct WMMAOpLowering : public ConvertOpToLLVMPattern<WMMAOp> {
if (!outType)
return rewriter.notifyMatchFailure(op, "type conversion failed");
- if (chipset.majorVersion != 11 && chipset.majorVersion != 12)
+ if (!target.isGeneration(11) && !target.isGeneration(12))
return op->emitOpError("WMMA only supported on gfx11 and gfx12");
- bool isGFX1250 = chipset >= kGfx1250;
+ bool isGFX1250 = target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS);
// The WMMA operations represent vectors of bf16s as vectors of i16s
// (except on gfx1250), so we need to bitcast bfloats to i16 and then
@@ -1805,12 +1773,13 @@ struct WMMAOpLowering : public ConvertOpToLLVMPattern<WMMAOp> {
destC = LLVM::BitcastOp::create(
rewriter, loc, destCType.clone(rewriter.getI16Type()), destC);
- std::optional<StringRef> maybeIntrinsic = wmmaOpToIntrinsic(op, chipset);
+ std::optional<StringRef> maybeIntrinsic = wmmaOpToIntrinsic(op, target);
if (!maybeIntrinsic.has_value())
return op.emitOpError("no intrinsic matching WMMA on the given chipset");
- if (chipset.majorVersion >= 12 && op.getSubwordOffset() != 0)
+ if (target.has(llvm::AMDGPU::FEAT_GFX12_INSTS) &&
+ op.getSubwordOffset() != 0)
return op.emitOpError("subwordOffset not supported on gfx12+");
SmallVector<Value, 4> operands;
@@ -1849,7 +1818,7 @@ enum class DotFamily {
};
static std::optional<std::pair<StringRef, DotFamily>>
-dotOpToIntrinsic(DotOp op, Chipset chipset) {
+dotOpToIntrinsic(DotOp op, const ROCDL::TargetInfo &target) {
Type aElem = cast<VectorType>(op.getSourceA().getType()).getElementType();
Type bElem = cast<VectorType>(op.getSourceB().getType()).getElementType();
Type dest = op.getDestC().getType();
@@ -1858,18 +1827,18 @@ dotOpToIntrinsic(DotOp op, Chipset chipset) {
// f16 x f16 -> f32 / f16.
if (aElem.isF16() && bElem.isF16()) {
- if (dest.isF32() && hasDot10Insts(chipset))
+ if (dest.isF32() && target.has(llvm::AMDGPU::FEAT_DOT10_INSTS))
return {{ROCDL::fdot2::getOperationName(), DotFamily::Clamp}};
- if (dest.isF16() && hasDot9Insts(chipset))
+ if (dest.isF16() && target.has(llvm::AMDGPU::FEAT_DOT9_INSTS))
return {{ROCDL::fdot2_f16_f16::getOperationName(), DotFamily::NoClamp}};
return std::nullopt;
}
// bf16 x bf16 -> f32 / bf16.
if (aElem.isBF16() && bElem.isBF16()) {
- if (dest.isF32() && hasDot12Insts(chipset))
+ if (dest.isF32() && target.has(llvm::AMDGPU::FEAT_DOT12_INSTS))
return {{ROCDL::fdot2_f32_bf16::getOperationName(), DotFamily::Clamp}};
- if (dest.isBF16() && hasDot9Insts(chipset))
+ if (dest.isBF16() && target.has(llvm::AMDGPU::FEAT_DOT9_INSTS))
return {{ROCDL::fdot2_bf16_bf16::getOperationName(), DotFamily::NoClamp}};
return std::nullopt;
}
@@ -1881,7 +1850,7 @@ dotOpToIntrinsic(DotOp op, Chipset chipset) {
unsigned elemWidth = aElem.getIntOrFloatBitWidth();
if (mixedSign) {
- if (!hasDot8Insts(chipset))
+ if (!target.has(llvm::AMDGPU::FEAT_DOT8_INSTS))
return std::nullopt;
StringRef name;
switch (elemWidth) {
@@ -1901,19 +1870,21 @@ dotOpToIntrinsic(DotOp op, Chipset chipset) {
bool supported = false;
switch (elemWidth) {
case 16:
- supported = hasDot2Insts(chipset);
+ supported = target.has(llvm::AMDGPU::FEAT_DOT2_INSTS);
name = uA ? ROCDL::udot2::getOperationName()
: ROCDL::sdot2::getOperationName();
break;
case 8:
- supported = uA ? hasDot7Insts(chipset)
- : hasDot1Insts(chipset) || hasDot8Insts(chipset);
+ supported = uA ? target.has(llvm::AMDGPU::FEAT_DOT7_INSTS)
+ : target.has(llvm::AMDGPU::FEAT_DOT1_INSTS) ||
+ target.has(llvm::AMDGPU::FEAT_DOT8_INSTS);
name = uA ? ROCDL::udot4::getOperationName()
: ROCDL::sdot4::getOperationName();
break;
case 4:
- supported = uA ? hasDot7Insts(chipset)
- : hasDot1Insts(chipset) || hasDot8Insts(chipset);
+ supported = uA ? target.has(llvm::AMDGPU::FEAT_DOT7_INSTS)
+ : target.has(llvm::AMDGPU::FEAT_DOT1_INSTS) ||
+ target.has(llvm::AMDGPU::FEAT_DOT8_INSTS);
name = uA ? ROCDL::udot8::getOperationName()
: ROCDL::sdot8::getOperationName();
break;
@@ -1931,7 +1902,7 @@ dotOpToIntrinsic(DotOp op, Chipset chipset) {
bool bIsFp8 = isa<Float8E4M3FNType>(bElem);
bool bIsBf8 = isa<Float8E5M2Type>(bElem);
if ((aIsFp8 || aIsBf8) && (bIsFp8 || bIsBf8) && dest.isF32()) {
- if (!hasDot11Insts(chipset))
+ if (!target.has(llvm::AMDGPU::FEAT_DOT11_INSTS))
return std::nullopt;
StringRef name;
if (aIsFp8 && bIsFp8)
@@ -1949,10 +1920,11 @@ dotOpToIntrinsic(DotOp op, Chipset chipset) {
}
struct DotOpLowering : public ConvertOpToLLVMPattern<DotOp> {
- DotOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<DotOp>(converter), chipset(chipset) {}
+ DotOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<DotOp>(converter), target(target) {}
- Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(DotOp op, DotOpAdaptor adaptor,
@@ -1960,7 +1932,7 @@ struct DotOpLowering : public ConvertOpToLLVMPattern<DotOp> {
Location loc = op.getLoc();
std::optional<std::pair<StringRef, DotFamily>> maybeIntrinsic =
- dotOpToIntrinsic(op, chipset);
+ dotOpToIntrinsic(op, target);
if (!maybeIntrinsic)
return op.emitOpError("no intrinsic matching dot on the given chipset: ")
<< op.getSourceA().getType() << " * " << op.getSourceB().getType()
@@ -1997,10 +1969,11 @@ struct DotOpLowering : public ConvertOpToLLVMPattern<DotOp> {
};
struct SparseWMMAOpLowering : public ConvertOpToLLVMPattern<SparseWMMAOp> {
- SparseWMMAOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<SparseWMMAOp>(converter), chipset(chipset) {}
+ SparseWMMAOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<SparseWMMAOp>(converter), target(target) {}
- Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(SparseWMMAOp op, SparseWMMAOpAdaptor adaptor,
@@ -2012,7 +1985,7 @@ struct SparseWMMAOpLowering : public ConvertOpToLLVMPattern<SparseWMMAOp> {
return rewriter.notifyMatchFailure(op, "type conversion failed");
std::optional<SparseWMMAOpInfo> maybeIntrinsic =
- sparseWMMAOpToIntrinsic(op, chipset);
+ sparseWMMAOpToIntrinsic(op, target);
if (!maybeIntrinsic.has_value())
return op.emitOpError(
@@ -2044,8 +2017,7 @@ struct SparseWMMAOpLowering : public ConvertOpToLLVMPattern<SparseWMMAOp> {
if (intrinsic.useClamp && op.getClampAttr())
attrs.push_back({"clamp", op.getClampAttr()});
- const bool isGFX1250orHigher =
- chipset.majorVersion == 12 && chipset.minorVersion >= 5;
+ const bool isGFX1250orHigher = target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS);
Value a = convertPackedVectorOperand(rewriter, loc, adaptor.getSourceA(),
isGFX1250orHigher);
Value b = convertPackedVectorOperand(rewriter, loc, adaptor.getSourceB(),
@@ -2078,10 +2050,11 @@ struct SparseWMMAOpLowering : public ConvertOpToLLVMPattern<SparseWMMAOp> {
};
struct ScaledWMMAOpLowering : public ConvertOpToLLVMPattern<ScaledWMMAOp> {
- ScaledWMMAOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<ScaledWMMAOp>(converter), chipset(chipset) {}
+ ScaledWMMAOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<ScaledWMMAOp>(converter), target(target) {}
- Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(ScaledWMMAOp op, ScaledWMMAOpAdaptor adaptor,
@@ -2092,7 +2065,7 @@ struct ScaledWMMAOpLowering : public ConvertOpToLLVMPattern<ScaledWMMAOp> {
if (!outType)
return rewriter.notifyMatchFailure(op, "type conversion failed");
- if (chipset < kGfx1250)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS))
return op->emitOpError("WMMA scale only supported on gfx1250+");
int64_t m = op.getM();
@@ -2198,15 +2171,17 @@ struct ScaledWMMAOpLowering : public ConvertOpToLLVMPattern<ScaledWMMAOp> {
struct TransposeLoadOpLowering
: public ConvertOpToLLVMPattern<TransposeLoadOp> {
- TransposeLoadOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<TransposeLoadOp>(converter), chipset(chipset) {}
+ TransposeLoadOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<TransposeLoadOp>(converter), target(target) {}
- Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(TransposeLoadOp op, TransposeLoadOpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset != kGfx950 && chipset < kGfx1250)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX950_INSTS) &&
+ !target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS))
return op.emitOpError(
"transpose_load is only supported on gfx950 and gfx1250+");
@@ -2247,7 +2222,7 @@ struct TransposeLoadOpLowering
};
Value intrinsic;
- if (chipset >= kGfx1250) {
+ if (target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS)) {
switch (elementTypeSize) {
case 4: {
if (numElements != 16)
@@ -2336,17 +2311,17 @@ struct TransposeLoadOpLowering
struct GlobalTransposeLoadOpLowering
: public ConvertOpToLLVMPattern<GlobalTransposeLoadOp> {
GlobalTransposeLoadOpLowering(const LLVMTypeConverter &converter,
- Chipset chipset)
+ const ROCDL::TargetInfo &target)
: ConvertOpToLLVMPattern<GlobalTransposeLoadOp>(converter),
- chipset(chipset) {}
+ target(target) {}
- Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(GlobalTransposeLoadOp op,
GlobalTransposeLoadOpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset < kGfx1200)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX12_INSTS))
return op.emitOpError(
"global_transpose_load is only supported on gfx1200+");
@@ -2374,7 +2349,7 @@ struct GlobalTransposeLoadOpLowering
switch (elementTypeSize) {
case 4: {
assert(numElements == 16);
- if (chipset < kGfx1250)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS))
return op.emitOpError("4-bit global_transpose_load requires gfx1250+");
auto rocdlOp = ROCDL::GlobalLoadTr4_B64::create(rewriter, loc,
rocdlResultType, srcPtr);
@@ -2383,7 +2358,7 @@ struct GlobalTransposeLoadOpLowering
}
case 6: {
assert(numElements == 16);
- if (chipset < kGfx1250)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS))
return op.emitOpError("6-bit global_transpose_load requires gfx1250+");
auto rocdlOp = ROCDL::GlobalLoadTr6_B96::create(rewriter, loc,
rocdlResultType, srcPtr);
@@ -2412,15 +2387,16 @@ struct GlobalTransposeLoadOpLowering
};
struct GatherToLDSOpLowering : public ConvertOpToLLVMPattern<GatherToLDSOp> {
- GatherToLDSOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<GatherToLDSOp>(converter), chipset(chipset) {}
+ GatherToLDSOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<GatherToLDSOp>(converter), target(target) {}
- Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(GatherToLDSOp op, GatherToLDSOpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset.majorVersion < 9 || chipset.majorVersion > 10)
+ if (!target.has(llvm::AMDGPU::FEAT_VMEM_TO_LDS_LOAD_INSTS))
return op.emitOpError("pre-gfx9 and post-gfx10 not supported");
Location loc = op.getLoc();
@@ -2445,7 +2421,8 @@ struct GatherToLDSOpLowering : public ConvertOpToLLVMPattern<GatherToLDSOp> {
if (!llvm::is_contained({1, 2, 4, 12, 16}, loadWidth))
return op.emitOpError("chipset unsupported element size");
- if (chipset != kGfx950 && llvm::is_contained({12, 16}, loadWidth))
+ if (!target.has(llvm::AMDGPU::FEAT_GFX950_INSTS) &&
+ llvm::is_contained({12, 16}, loadWidth))
return op.emitOpError("Gather to LDS instructions with 12-byte and "
"16-byte load widths are only supported on gfx950");
@@ -2477,17 +2454,17 @@ struct GatherToLDSOpLowering : public ConvertOpToLLVMPattern<GatherToLDSOp> {
struct GlobalLoadAsyncToLDSOpLowering
: public ConvertOpToLLVMPattern<GlobalLoadAsyncToLDSOp> {
GlobalLoadAsyncToLDSOpLowering(const LLVMTypeConverter &converter,
- Chipset chipset)
+ const ROCDL::TargetInfo &target)
: ConvertOpToLLVMPattern<GlobalLoadAsyncToLDSOp>(converter),
- chipset(chipset) {}
+ target(target) {}
- Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(GlobalLoadAsyncToLDSOp op,
GlobalLoadAsyncToLDSOpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset < kGfx1250)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS))
return op.emitOpError(
"global_load_async_to_lds is only supported on gfx1250+");
@@ -2554,10 +2531,11 @@ struct GlobalLoadAsyncToLDSOpLowering
namespace {
struct ExtPackedFp8OpLowering final
: public ConvertOpToLLVMPattern<ExtPackedFp8Op> {
- ExtPackedFp8OpLowering(const LLVMTypeConverter &converter, Chipset chipset)
+ ExtPackedFp8OpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
: ConvertOpToLLVMPattern<amdgpu::ExtPackedFp8Op>(converter),
- chipset(chipset) {}
- Chipset chipset;
+ target(target) {}
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(ExtPackedFp8Op op, ExtPackedFp8OpAdaptor adaptor,
@@ -2567,10 +2545,10 @@ struct ExtPackedFp8OpLowering final
struct ScaledExtPackedMatrixOpLowering final
: public ConvertOpToLLVMPattern<ScaledExtPackedMatrixOp> {
ScaledExtPackedMatrixOpLowering(const LLVMTypeConverter &converter,
- Chipset chipset)
+ const ROCDL::TargetInfo &target)
: ConvertOpToLLVMPattern<amdgpu::ScaledExtPackedMatrixOp>(converter),
- chipset(chipset) {}
- Chipset chipset;
+ target(target) {}
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(ScaledExtPackedMatrixOp op,
@@ -2581,10 +2559,10 @@ struct ScaledExtPackedMatrixOpLowering final
struct PackedTrunc2xFp8OpLowering final
: public ConvertOpToLLVMPattern<PackedTrunc2xFp8Op> {
PackedTrunc2xFp8OpLowering(const LLVMTypeConverter &converter,
- Chipset chipset)
+ const ROCDL::TargetInfo &target)
: ConvertOpToLLVMPattern<amdgpu::PackedTrunc2xFp8Op>(converter),
- chipset(chipset) {}
- Chipset chipset;
+ target(target) {}
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(PackedTrunc2xFp8Op op, PackedTrunc2xFp8OpAdaptor adaptor,
@@ -2594,10 +2572,10 @@ struct PackedTrunc2xFp8OpLowering final
struct PackedStochRoundFp8OpLowering final
: public ConvertOpToLLVMPattern<PackedStochRoundFp8Op> {
PackedStochRoundFp8OpLowering(const LLVMTypeConverter &converter,
- Chipset chipset)
+ const ROCDL::TargetInfo &target)
: ConvertOpToLLVMPattern<amdgpu::PackedStochRoundFp8Op>(converter),
- chipset(chipset) {}
- Chipset chipset;
+ target(target) {}
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(PackedStochRoundFp8Op op,
@@ -2607,10 +2585,11 @@ struct PackedStochRoundFp8OpLowering final
struct ScaledExtPackedOpLowering final
: public ConvertOpToLLVMPattern<ScaledExtPackedOp> {
- ScaledExtPackedOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
+ ScaledExtPackedOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
: ConvertOpToLLVMPattern<amdgpu::ScaledExtPackedOp>(converter),
- chipset(chipset) {}
- Chipset chipset;
+ target(target) {}
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(ScaledExtPackedOp op, ScaledExtPackedOpAdaptor adaptor,
@@ -2620,10 +2599,10 @@ struct ScaledExtPackedOpLowering final
struct PackedScaledTruncOpLowering final
: public ConvertOpToLLVMPattern<PackedScaledTruncOp> {
PackedScaledTruncOpLowering(const LLVMTypeConverter &converter,
- Chipset chipset)
+ const ROCDL::TargetInfo &target)
: ConvertOpToLLVMPattern<amdgpu::PackedScaledTruncOp>(converter),
- chipset(chipset) {}
- Chipset chipset;
+ target(target) {}
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(PackedScaledTruncOp op, PackedScaledTruncOpAdaptor adaptor,
@@ -2636,7 +2615,7 @@ LogicalResult ExtPackedFp8OpLowering::matchAndRewrite(
ExtPackedFp8Op op, ExtPackedFp8OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
Location loc = op.getLoc();
- if (!(chipset == kGfx942 || hasOcpFp8(chipset)))
+ if (!target.has(llvm::AMDGPU::FEAT_FP8_CONVERSION_INSTS))
return rewriter.notifyMatchFailure(
loc, "Fp8 conversion instructions are not available on target "
"architecture and their emulation is not implemented");
@@ -2667,18 +2646,18 @@ LogicalResult ExtPackedFp8OpLowering::matchAndRewrite(
}
Value i32Source = LLVM::BitcastOp::create(rewriter, loc, i32, source);
if (resultVecType) {
- if (typeIsExpectedBf8ForChipset(chipset, sourceElemType)) {
+ if (typeIsExpectedBf8ForTarget(target, sourceElemType)) {
rewriter.replaceOpWithNewOp<ROCDL::CvtPkF32Bf8Op>(op, f32, i32Source,
op.getIndex());
- } else if (typeIsExpectedFp8ForChipset(chipset, sourceElemType)) {
+ } else if (typeIsExpectedFp8ForTarget(target, sourceElemType)) {
rewriter.replaceOpWithNewOp<ROCDL::CvtPkF32Fp8Op>(op, f32, i32Source,
op.getIndex());
}
} else {
- if (typeIsExpectedBf8ForChipset(chipset, sourceElemType)) {
+ if (typeIsExpectedBf8ForTarget(target, sourceElemType)) {
rewriter.replaceOpWithNewOp<ROCDL::CvtF32Bf8Op>(op, f32, i32Source,
op.getIndex());
- } else if (typeIsExpectedFp8ForChipset(chipset, sourceElemType)) {
+ } else if (typeIsExpectedFp8ForTarget(target, sourceElemType)) {
rewriter.replaceOpWithNewOp<ROCDL::CvtF32Fp8Op>(op, f32, i32Source,
op.getIndex());
}
@@ -2786,7 +2765,7 @@ LogicalResult ScaledExtPackedMatrixOpLowering::matchAndRewrite(
using fp6 = Float6E2M3FNType;
using bf6 = Float6E3M2FNType;
Location loc = op.getLoc();
- if (chipset != kGfx1250) {
+ if (!target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS)) {
return rewriter.notifyMatchFailure(
loc,
"Scaled fp packed conversion instructions are not available on target "
@@ -2857,7 +2836,7 @@ LogicalResult ScaledExtPackedOpLowering::matchAndRewrite(
ScaledExtPackedOp op, ScaledExtPackedOpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
Location loc = op.getLoc();
- if (chipset != kGfx950)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX950_INSTS))
return rewriter.notifyMatchFailure(
loc, "Scaled fp conversion instructions are not available on target "
"architecture and their emulation is not implemented");
@@ -2937,7 +2916,7 @@ LogicalResult PackedScaledTruncOpLowering::matchAndRewrite(
PackedScaledTruncOp op, PackedScaledTruncOpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
Location loc = op.getLoc();
- if (chipset != kGfx950)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX950_INSTS))
return rewriter.notifyMatchFailure(
loc, "Scaled fp conversion instructions are not available on target "
"architecture and their emulation is not implemented");
@@ -3019,7 +2998,7 @@ LogicalResult PackedTrunc2xFp8OpLowering::matchAndRewrite(
PackedTrunc2xFp8Op op, PackedTrunc2xFp8OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
Location loc = op.getLoc();
- if (!(chipset == kGfx942 || hasOcpFp8(chipset)))
+ if (!target.has(llvm::AMDGPU::FEAT_FP8_CONVERSION_INSTS))
return rewriter.notifyMatchFailure(
loc, "Fp8 conversion instructions are not available on target "
"architecture and their emulation is not implemented");
@@ -3039,10 +3018,10 @@ LogicalResult PackedTrunc2xFp8OpLowering::matchAndRewrite(
existing = LLVM::UndefOp::create(rewriter, loc, i32);
Value result;
- if (typeIsExpectedBf8ForChipset(chipset, resultElemType))
+ if (typeIsExpectedBf8ForTarget(target, resultElemType))
result = ROCDL::CvtPkBf8F32Op::create(rewriter, loc, i32, sourceA, sourceB,
existing, op.getWordIndex());
- else if (typeIsExpectedFp8ForChipset(chipset, resultElemType))
+ else if (typeIsExpectedFp8ForTarget(target, resultElemType))
result = ROCDL::CvtPkFp8F32Op::create(rewriter, loc, i32, sourceA, sourceB,
existing, op.getWordIndex());
else
@@ -3058,7 +3037,7 @@ LogicalResult PackedStochRoundFp8OpLowering::matchAndRewrite(
PackedStochRoundFp8Op op, PackedStochRoundFp8OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
Location loc = op.getLoc();
- if (!(chipset == kGfx942 || hasOcpFp8(chipset)))
+ if (!target.has(llvm::AMDGPU::FEAT_FP8_CONVERSION_INSTS))
return rewriter.notifyMatchFailure(
loc, "Fp8 conversion instructions are not available on target "
"architecture and their emulation is not implemented");
@@ -3076,10 +3055,10 @@ LogicalResult PackedStochRoundFp8OpLowering::matchAndRewrite(
existing = LLVM::UndefOp::create(rewriter, loc, i32);
Value result;
- if (typeIsExpectedBf8ForChipset(chipset, resultElemType))
+ if (typeIsExpectedBf8ForTarget(target, resultElemType))
result = ROCDL::CvtSrBf8F32Op::create(rewriter, loc, i32, source, stoch,
existing, op.getStoreIndex());
- else if (typeIsExpectedFp8ForChipset(chipset, resultElemType))
+ else if (typeIsExpectedFp8ForTarget(target, resultElemType))
result = ROCDL::CvtSrFp8F32Op::create(rewriter, loc, i32, source, stoch,
existing, op.getStoreIndex());
else
@@ -3094,9 +3073,10 @@ LogicalResult PackedStochRoundFp8OpLowering::matchAndRewrite(
// Implement the AMDGPU_DPPLowering class that will convert the amdgpu.dpp
// operation into the corresponding ROCDL instructions.
struct AMDGPUDPPLowering : public ConvertOpToLLVMPattern<DPPOp> {
- AMDGPUDPPLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<DPPOp>(converter), chipset(chipset) {}
- Chipset chipset;
+ AMDGPUDPPLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<DPPOp>(converter), target(target) {}
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(DPPOp DppOp, DPPOp::Adaptor adaptor,
@@ -3280,20 +3260,25 @@ struct AMDGPUSwizzleBitModeLowering
struct AMDGPUPermlaneLowering : public ConvertOpToLLVMPattern<PermlaneSwapOp> {
using ConvertOpToLLVMPattern::ConvertOpToLLVMPattern;
- AMDGPUPermlaneLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<PermlaneSwapOp>(converter), chipset(chipset) {}
- Chipset chipset;
+ AMDGPUPermlaneLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<PermlaneSwapOp>(converter), target(target) {}
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(PermlaneSwapOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset < kGfx950)
- return op->emitOpError("permlane_swap is only supported on gfx950+");
+ unsigned rowLength = op.getRowLength();
+ bool supported = rowLength == 16
+ ? target.has(llvm::AMDGPU::FEAT_PERMLANE16_SWAP)
+ : target.has(llvm::AMDGPU::FEAT_PERMLANE32_SWAP);
+ if (!supported)
+ return op->emitOpError("permlane_swap of row length ")
+ << rowLength << " is not supported on " << target.getArchName();
Location loc = op.getLoc();
Type i32 = rewriter.getI32Type();
Value src = adaptor.getSrc();
- unsigned rowLength = op.getRowLength();
bool fi = op.getFetchInactive();
bool boundctrl = op.getBoundCtrl();
@@ -3340,14 +3325,15 @@ struct AMDGPUPermlaneVarLowering
: public ConvertOpToLLVMPattern<PermlaneVarOp> {
using ConvertOpToLLVMPattern::ConvertOpToLLVMPattern;
- AMDGPUPermlaneVarLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<PermlaneVarOp>(converter), chipset(chipset) {}
- Chipset chipset;
+ AMDGPUPermlaneVarLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<PermlaneVarOp>(converter), target(target) {}
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(PermlaneVarOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset < kGfx1200)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX12_INSTS))
return op->emitOpError("permlane_var is only supported on GFX12+");
Location loc = op.getLoc();
@@ -3397,15 +3383,16 @@ constexpr int32_t kDsBarrierPendingCountMask =
struct DsBarrierInitOpLowering
: public ConvertOpToLLVMPattern<DsBarrierInitOp> {
- Chipset chipset;
+ ROCDL::TargetInfo target;
- DsBarrierInitOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<DsBarrierInitOp>(converter), chipset(chipset) {}
+ DsBarrierInitOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<DsBarrierInitOp>(converter), target(target) {}
LogicalResult
matchAndRewrite(DsBarrierInitOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset < kGfx1250)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS))
return op->emitOpError("only supported on gfx1250+");
Location loc = op.getLoc();
@@ -3450,17 +3437,17 @@ struct DsBarrierInitOpLowering
struct DsBarrierPollStateOpLowering
: public ConvertOpToLLVMPattern<DsBarrierPollStateOp> {
- Chipset chipset;
+ ROCDL::TargetInfo target;
DsBarrierPollStateOpLowering(const LLVMTypeConverter &converter,
- Chipset chipset)
+ const ROCDL::TargetInfo &target)
: ConvertOpToLLVMPattern<DsBarrierPollStateOp>(converter),
- chipset(chipset) {}
+ target(target) {}
LogicalResult
matchAndRewrite(DsBarrierPollStateOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset < kGfx1250)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS))
return op->emitOpError("only supported on gfx1250+");
Location loc = op.getLoc();
@@ -3483,17 +3470,17 @@ struct DsBarrierPollStateOpLowering
struct DsAsyncBarrierArriveOpLowering
: public ConvertOpToLLVMPattern<DsAsyncBarrierArriveOp> {
- Chipset chipset;
+ ROCDL::TargetInfo target;
DsAsyncBarrierArriveOpLowering(const LLVMTypeConverter &converter,
- Chipset chipset)
+ const ROCDL::TargetInfo &target)
: ConvertOpToLLVMPattern<DsAsyncBarrierArriveOp>(converter),
- chipset(chipset) {}
+ target(target) {}
LogicalResult
matchAndRewrite(DsAsyncBarrierArriveOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset < kGfx1250)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS))
return op->emitOpError("only supported on gfx1250+");
Location loc = op.getLoc();
@@ -3511,16 +3498,16 @@ struct DsAsyncBarrierArriveOpLowering
struct DsBarrierArriveOpLowering
: public ConvertOpToLLVMPattern<DsBarrierArriveOp> {
- Chipset chipset;
+ ROCDL::TargetInfo target;
- DsBarrierArriveOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<DsBarrierArriveOp>(converter), chipset(chipset) {
- }
+ DsBarrierArriveOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<DsBarrierArriveOp>(converter), target(target) {}
LogicalResult
matchAndRewrite(DsBarrierArriveOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset < kGfx1250)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS))
return op->emitOpError("only supported on gfx1250+");
Location loc = op.getLoc();
@@ -3653,14 +3640,15 @@ struct AMDGPUMakeDmaBaseLowering : public ConvertOpToLLVMPattern<BaseOp> {
using ConvertOpToLLVMPattern<BaseOp>::ConvertOpToLLVMPattern;
using Adaptor = typename ConvertOpToLLVMPattern<BaseOp>::OpAdaptor;
- AMDGPUMakeDmaBaseLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<BaseOp>(converter), chipset(chipset) {}
- Chipset chipset;
+ AMDGPUMakeDmaBaseLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<BaseOp>(converter), target(target) {}
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(BaseOp op, Adaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset < kGfx1250)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS))
return op->emitOpError("make_dma_base is only supported on gfx1250");
Location loc = op.getLoc();
@@ -3743,9 +3731,10 @@ struct AMDGPULowerDescriptor : public ConvertOpToLLVMPattern<DescriptorOp> {
using ConvertOpToLLVMPattern<DescriptorOp>::ConvertOpToLLVMPattern;
using OpAdaptor = typename ConvertOpToLLVMPattern<DescriptorOp>::OpAdaptor;
- AMDGPULowerDescriptor(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<DescriptorOp>(converter), chipset(chipset) {}
- Chipset chipset;
+ AMDGPULowerDescriptor(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<DescriptorOp>(converter), target(target) {}
+ ROCDL::TargetInfo target;
Value getDGroup0(OpAdaptor adaptor) const { return adaptor.getBase(); }
@@ -4428,7 +4417,7 @@ struct AMDGPULowerDescriptor : public ConvertOpToLLVMPattern<DescriptorOp> {
LogicalResult
matchAndRewrite(DescriptorOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset < kGfx1250)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS))
return op->emitOpError(
"make_dma_descriptor is only supported on gfx1250");
@@ -4454,14 +4443,14 @@ struct AMDGPUTensorLoadStoreOpLowering
using ConvertOpToLLVMPattern<SourceOp>::ConvertOpToLLVMPattern;
using Adaptor = typename ConvertOpToLLVMPattern<SourceOp>::OneToNOpAdaptor;
AMDGPUTensorLoadStoreOpLowering(const LLVMTypeConverter &converter,
- Chipset chipset)
- : ConvertOpToLLVMPattern<SourceOp>(converter), chipset(chipset) {}
- Chipset chipset;
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<SourceOp>(converter), target(target) {}
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(SourceOp op, Adaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset < kGfx1250)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS))
return op->emitOpError("is only supported on gfx1250");
ValueRange desc = adaptor.getDesc();
@@ -4481,13 +4470,14 @@ struct AMDGPUTensorLoadStoreOpLowering
struct GlobalPrefetchOpLowering
: public ConvertOpToLLVMPattern<GlobalPrefetchOp> {
- GlobalPrefetchOpLowering(const LLVMTypeConverter &converter, Chipset chipset)
- : ConvertOpToLLVMPattern<GlobalPrefetchOp>(converter), chipset(chipset) {}
+ GlobalPrefetchOpLowering(const LLVMTypeConverter &converter,
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<GlobalPrefetchOp>(converter), target(target) {}
LogicalResult
matchAndRewrite(GlobalPrefetchOp op, GlobalPrefetchOpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset < kGfx1250)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS))
return op->emitOpError("is only supported on gfx1250+");
const bool isSpeculative = op.getSpeculative();
@@ -4517,7 +4507,7 @@ struct GlobalPrefetchOpLowering
}
private:
- Chipset chipset;
+ ROCDL::TargetInfo target;
};
struct ConvertAMDGPUToROCDLPass
@@ -4526,16 +4516,16 @@ struct ConvertAMDGPUToROCDLPass
void runOnOperation() override {
MLIRContext *ctx = &getContext();
- FailureOr<Chipset> maybeChipset = Chipset::parse(chipset);
- if (failed(maybeChipset)) {
- emitError(UnknownLoc::get(ctx), "Invalid chipset name: " + chipset);
+ FailureOr<ROCDL::TargetInfo> targetInfo =
+ ROCDL::TargetInfo::get(triple, chip, features,
+ [&] { return emitError(UnknownLoc::get(ctx)); });
+ if (failed(targetInfo))
return signalPassFailure();
- }
RewritePatternSet patterns(ctx);
LLVMTypeConverter converter(ctx);
- populateAMDGPUToROCDLConversionPatterns(converter, patterns, *maybeChipset);
+ populateAMDGPUToROCDLConversionPatterns(converter, patterns, *targetInfo);
amdgpu::populateCommonGPUTypeAndAttributeConversions(converter);
LLVMConversionTarget target(getContext());
target.addIllegalDialect<::mlir::amdgpu::AMDGPUDialect>();
@@ -4627,9 +4617,9 @@ void mlir::populateAMDGPUTypeAndAttributeConversions(
typeConverter.addTargetMaterialization(addUnrealizedCast);
}
-void mlir::populateAMDGPUToROCDLConversionPatterns(LLVMTypeConverter &converter,
- RewritePatternSet &patterns,
- Chipset chipset) {
+void mlir::populateAMDGPUToROCDLConversionPatterns(
+ LLVMTypeConverter &converter, RewritePatternSet &patterns,
+ const ROCDL::TargetInfo &target) {
populateAMDGPUTypeAndAttributeConversions(converter);
patterns
.add<FatRawBufferCastLowering,
@@ -4664,7 +4654,7 @@ void mlir::populateAMDGPUToROCDLConversionPatterns(LLVMTypeConverter &converter,
ROCDL::TensorStoreFromLDSOp>,
DsBarrierInitOpLowering, DsBarrierPollStateOpLowering,
DsAsyncBarrierArriveOpLowering, DsBarrierArriveOpLowering,
- GlobalPrefetchOpLowering>(converter, chipset);
+ GlobalPrefetchOpLowering>(converter, target);
patterns.add<AMDGPUSwizzleBitModeLowering, DsBarrierStatePhaseOpLowering,
DsBarrierStatePendingCountOpLowering,
DsBarrierStateInitCountOpLowering,
diff --git a/mlir/lib/Conversion/ArithToAMDGPU/ArithToAMDGPU.cpp b/mlir/lib/Conversion/ArithToAMDGPU/ArithToAMDGPU.cpp
index cf44c2a0033ac..604cc35260bb7 100644
--- a/mlir/lib/Conversion/ArithToAMDGPU/ArithToAMDGPU.cpp
+++ b/mlir/lib/Conversion/ArithToAMDGPU/ArithToAMDGPU.cpp
@@ -9,11 +9,11 @@
#include "mlir/Conversion/ArithToAMDGPU/ArithToAMDGPU.h"
#include "mlir/Dialect/AMDGPU/IR/AMDGPUDialect.h"
-#include "mlir/Dialect/AMDGPU/Utils/Chipset.h"
#include "mlir/Dialect/Arith/IR/Arith.h"
#include "mlir/Dialect/Arith/Utils/Utils.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/LLVMIR/ROCDLDialect.h"
+#include "mlir/Dialect/LLVMIR/ROCDLTargetInfo.h"
#include "mlir/Dialect/Utils/IndexingUtils.h"
#include "mlir/Dialect/Vector/IR/VectorOps.h"
#include "mlir/Dialect/Vector/Transforms/VectorRewritePatterns.h"
@@ -33,9 +33,6 @@ using namespace mlir;
using namespace mlir::amdgpu;
namespace {
-// Define commonly used chipsets versions for convenience.
-constexpr Chipset kGfx942 = Chipset(9, 4, 2);
-constexpr Chipset kGfx950 = Chipset(9, 5, 0);
struct ArithToAMDGPUConversionPass final
: impl::ArithToAMDGPUConversionPassBase<ArithToAMDGPUConversionPass> {
@@ -48,10 +45,10 @@ struct ArithToAMDGPUConversionPass final
struct ExtFOnFloat8RewritePattern final : OpRewritePattern<arith::ExtFOp> {
using Base::Base;
- Chipset chipset;
- ExtFOnFloat8RewritePattern(MLIRContext *ctx, Chipset chipset,
+ ROCDL::TargetInfo target;
+ ExtFOnFloat8RewritePattern(MLIRContext *ctx, const ROCDL::TargetInfo &target,
PatternBenefit benefit)
- : OpRewritePattern::OpRewritePattern(ctx, benefit), chipset(chipset) {}
+ : OpRewritePattern::OpRewritePattern(ctx, benefit), target(target) {}
LogicalResult matchAndRewrite(arith::ExtFOp op,
PatternRewriter &rewriter) const override;
@@ -60,10 +57,11 @@ struct ExtFOnFloat8RewritePattern final : OpRewritePattern<arith::ExtFOp> {
struct TruncFToFloat8RewritePattern final : OpRewritePattern<arith::TruncFOp> {
bool saturateFP8 = false;
TruncFToFloat8RewritePattern(MLIRContext *ctx, bool saturateFP8,
- Chipset chipset, PatternBenefit benefit)
+ const ROCDL::TargetInfo &target,
+ PatternBenefit benefit)
: OpRewritePattern::OpRewritePattern(ctx, benefit),
- saturateFP8(saturateFP8), chipset(chipset) {}
- Chipset chipset;
+ saturateFP8(saturateFP8), target(target) {}
+ ROCDL::TargetInfo target;
LogicalResult matchAndRewrite(arith::TruncFOp op,
PatternRewriter &rewriter) const override;
@@ -96,10 +94,10 @@ struct ScalingTruncFRewritePattern final
} // end namespace
-static bool isSupportedF8(Type elementType, Chipset chipset) {
- if (chipset == kGfx942)
+static bool isSupportedF8(Type elementType, const ROCDL::TargetInfo &target) {
+ if (target.hasFnuzFp8())
return isa<Float8E4M3FNUZType, Float8E5M2FNUZType>(elementType);
- if (hasOcpFp8(chipset))
+ if (target.hasOcpFp8())
return isa<Float8E4M3FNType, Float8E5M2Type>(elementType);
return false;
}
@@ -126,7 +124,7 @@ ExtFOnFloat8RewritePattern::matchAndRewrite(arith::ExtFOp op,
return failure();
inType = inVecType.getElementType();
}
- if (!isSupportedF8(inType, chipset))
+ if (!isSupportedF8(inType, target))
return failure();
Location loc = op.getLoc();
@@ -275,7 +273,7 @@ TruncFToFloat8RewritePattern::matchAndRewrite(arith::TruncFOp op,
// Conversion between 8-bit floats is not supported with truncation enabled.
return failure();
- if (!isSupportedF8(outType, chipset))
+ if (!isSupportedF8(outType, target))
return failure();
Location loc = op.getLoc();
@@ -697,13 +695,13 @@ ScalingTruncFRewritePattern::matchAndRewrite(arith::ScalingTruncFOp op,
void mlir::arith::populateArithToAMDGPUConversionPatterns(
RewritePatternSet &patterns, bool convertFP8Arithmetic,
bool saturateFP8Truncf, bool allowPackedF16Rtz, bool supportsScaledExtTrunc,
- Chipset chipset, PatternBenefit benefit) {
+ const ROCDL::TargetInfo &target, PatternBenefit benefit) {
if (convertFP8Arithmetic) {
- patterns.add<ExtFOnFloat8RewritePattern>(patterns.getContext(), chipset,
+ patterns.add<ExtFOnFloat8RewritePattern>(patterns.getContext(), target,
benefit);
patterns.add<TruncFToFloat8RewritePattern>(
- patterns.getContext(), saturateFP8Truncf, chipset, benefit);
+ patterns.getContext(), saturateFP8Truncf, target, benefit);
}
if (allowPackedF16Rtz)
patterns.add<TruncfToFloat16RewritePattern>(patterns.getContext(), benefit);
@@ -718,18 +716,19 @@ void ArithToAMDGPUConversionPass::runOnOperation() {
Operation *op = getOperation();
MLIRContext *ctx = &getContext();
RewritePatternSet patterns(op->getContext());
- FailureOr<amdgpu::Chipset> maybeChipset = amdgpu::Chipset::parse(chipset);
- if (failed(maybeChipset)) {
- emitError(UnknownLoc::get(ctx), "Invalid chipset name: " + chipset);
+ FailureOr<ROCDL::TargetInfo> targetInfo = ROCDL::TargetInfo::get(
+ triple, chip, features, [&] { return emitError(UnknownLoc::get(ctx)); });
+ if (failed(targetInfo)) {
return signalPassFailure();
}
bool convertFP8Arithmetic =
- *maybeChipset == kGfx942 || hasOcpFp8(*maybeChipset);
- bool supportsScaledExtTrunc = *maybeChipset == kGfx950;
+ targetInfo->has(llvm::AMDGPU::FEAT_FP8_CONVERSION_INSTS);
+ bool supportsScaledExtTrunc =
+ targetInfo->has(llvm::AMDGPU::FEAT_GFX950_INSTS);
arith::populateArithToAMDGPUConversionPatterns(
patterns, convertFP8Arithmetic, saturateFP8Truncf, allowPackedF16Rtz,
- supportsScaledExtTrunc, *maybeChipset);
+ supportsScaledExtTrunc, *targetInfo);
if (failed(applyPatternsGreedily(op, std::move(patterns))))
return signalPassFailure();
}
diff --git a/mlir/lib/Conversion/GPUToROCDL/LowerGpuOpsToROCDLOps.cpp b/mlir/lib/Conversion/GPUToROCDL/LowerGpuOpsToROCDLOps.cpp
index a3819df4f8a84..17cc67fb3c345 100644
--- a/mlir/lib/Conversion/GPUToROCDL/LowerGpuOpsToROCDLOps.cpp
+++ b/mlir/lib/Conversion/GPUToROCDL/LowerGpuOpsToROCDLOps.cpp
@@ -241,18 +241,18 @@ struct GPUSubgroupSizeOpToROCDL : ConvertOpToLLVMPattern<gpu::SubgroupSizeOp> {
using ConvertOpToLLVMPattern::ConvertOpToLLVMPattern;
GPUSubgroupSizeOpToROCDL(const LLVMTypeConverter &converter,
- amdgpu::Chipset chipset)
- : ConvertOpToLLVMPattern<gpu::SubgroupSizeOp>(converter),
- chipset(chipset) {}
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<gpu::SubgroupSizeOp>(converter), target(target) {
+ }
LogicalResult
matchAndRewrite(gpu::SubgroupSizeOp op, gpu::SubgroupSizeOp::Adaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
LLVM::ConstantRangeAttr bounds = nullptr;
- bool isBeforeGfx10 = chipset.majorVersion < 10;
+ bool isWave64 = target.getWavefrontSize() == 64;
if (auto upperBoundAttr = op.getUpperBoundAttr()) {
bounds = rewriter.getAttr<LLVM::ConstantRangeAttr>(
- /*bitWidth=*/32, /*lower=*/isBeforeGfx10 ? 64 : 32,
+ /*bitWidth=*/32, /*lower=*/isWave64 ? 64 : 32,
/*upper=*/op.getUpperBoundAttr().getInt() + 1);
}
Value wavefrontOp = ROCDL::WavefrontSizeOp::create(
@@ -263,16 +263,15 @@ struct GPUSubgroupSizeOpToROCDL : ConvertOpToLLVMPattern<gpu::SubgroupSizeOp> {
return success();
}
- const amdgpu::Chipset chipset;
+ const ROCDL::TargetInfo target;
};
struct GPUSubgroupIdOpToROCDL : ConvertOpToLLVMPattern<gpu::SubgroupIdOp> {
using ConvertOpToLLVMPattern::ConvertOpToLLVMPattern;
GPUSubgroupIdOpToROCDL(const LLVMTypeConverter &converter,
- amdgpu::Chipset chipset)
- : ConvertOpToLLVMPattern<gpu::SubgroupIdOp>(converter), chipset(chipset) {
- }
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<gpu::SubgroupIdOp>(converter), target(target) {}
LogicalResult
matchAndRewrite(gpu::SubgroupIdOp op, gpu::SubgroupIdOp::Adaptor adaptor,
@@ -281,7 +280,7 @@ struct GPUSubgroupIdOpToROCDL : ConvertOpToLLVMPattern<gpu::SubgroupIdOp> {
auto int32Type = rewriter.getI32Type();
Value subgroupId;
- if (chipset.majorVersion >= 12) {
+ if (target.has(llvm::AMDGPU::FEAT_GFX12_INSTS)) {
// For gfx12+, use the hardware wave.id register directly.
LLVM::ConstantRangeAttr bounds;
if (auto upperBoundAttr = op.getUpperBoundAttr())
@@ -345,7 +344,7 @@ struct GPUSubgroupIdOpToROCDL : ConvertOpToLLVMPattern<gpu::SubgroupIdOp> {
return success();
}
- const amdgpu::Chipset chipset;
+ const ROCDL::TargetInfo target;
};
static bool isSupportedReadLaneType(Type type) {
@@ -567,10 +566,10 @@ static constexpr int32_t kWholeClusterBarrierId = -3;
static constexpr int32_t kWholeWorkgroupBarrierId = -1;
struct GPUBarrierOpLowering final : ConvertOpToLLVMPattern<gpu::BarrierOp> {
GPUBarrierOpLowering(const LLVMTypeConverter &converter,
- amdgpu::Chipset chipset)
- : ConvertOpToLLVMPattern<gpu::BarrierOp>(converter), chipset(chipset) {}
+ const ROCDL::TargetInfo &target)
+ : ConvertOpToLLVMPattern<gpu::BarrierOp>(converter), target(target) {}
- amdgpu::Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(gpu::BarrierOp op, gpu::BarrierOp::Adaptor adaptor,
@@ -591,7 +590,7 @@ struct GPUBarrierOpLowering final : ConvertOpToLLVMPattern<gpu::BarrierOp> {
// Cluster scope: gfx1250+ only, signal/wait with constant -3.
if (scope == gpu::BarrierScope::Cluster) {
- if (chipset < amdgpu::Chipset(12, 5, 0))
+ if (!target.has(llvm::AMDGPU::FEAT_GFX1250_INSTS))
return op.emitOpError("cluster scope barriers require gfx1250+");
emitFences(op.getAddressSpaces(), rewriter, loc, "cluster",
/*before=*/true);
@@ -609,7 +608,7 @@ struct GPUBarrierOpLowering final : ConvertOpToLLVMPattern<gpu::BarrierOp> {
// Named barrier path.
if (Value namedBarrier = adaptor.getNamedBarrier()) {
- if (chipset.majorVersion < 12)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX12_INSTS))
return op.emitOpError("named barriers require gfx12+");
emitFences(op.getAddressSpaces(), rewriter, loc, "workgroup",
@@ -631,7 +630,7 @@ struct GPUBarrierOpLowering final : ConvertOpToLLVMPattern<gpu::BarrierOp> {
// Regular workgroup barrier.
emitFences(op.getAddressSpaces(), rewriter, loc, "workgroup",
/*before=*/true);
- if (chipset.majorVersion < 12) {
+ if (!target.has(llvm::AMDGPU::FEAT_GFX12_INSTS)) {
ROCDL::SBarrierOp::create(rewriter, loc);
} else {
ROCDL::BarrierSignalOp::create(rewriter, loc, kWholeWorkgroupBarrierId);
@@ -648,17 +647,17 @@ struct GPUBarrierOpLowering final : ConvertOpToLLVMPattern<gpu::BarrierOp> {
struct GPUInitializeNamedBarrierOpLowering final
: ConvertOpToLLVMPattern<gpu::InitializeNamedBarrierOp> {
GPUInitializeNamedBarrierOpLowering(const LLVMTypeConverter &converter,
- amdgpu::Chipset chipset)
+ const ROCDL::TargetInfo &target)
: ConvertOpToLLVMPattern<gpu::InitializeNamedBarrierOp>(converter),
- chipset(chipset) {}
+ target(target) {}
- amdgpu::Chipset chipset;
+ ROCDL::TargetInfo target;
LogicalResult
matchAndRewrite(gpu::InitializeNamedBarrierOp op,
gpu::InitializeNamedBarrierOp::Adaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
- if (chipset.majorVersion < 12)
+ if (!target.has(llvm::AMDGPU::FEAT_GFX12_INSTS))
return op.emitOpError("named barriers require gfx12+");
Location loc = op.getLoc();
@@ -749,9 +748,10 @@ struct LowerGpuOpsToROCDLOpsPass final
UnitAttr::get(ctx));
}
- FailureOr<amdgpu::Chipset> maybeChipset = amdgpu::Chipset::parse(chipset);
- if (failed(maybeChipset)) {
- emitError(UnknownLoc::get(ctx), "Invalid chipset name: " + chipset);
+ FailureOr<ROCDL::TargetInfo> targetInfo =
+ ROCDL::TargetInfo::get(triple, chip, features,
+ [&] { return emitError(UnknownLoc::get(ctx)); });
+ if (failed(targetInfo)) {
return signalPassFailure();
}
@@ -784,7 +784,7 @@ struct LowerGpuOpsToROCDLOpsPass final
{
RewritePatternSet patterns(ctx);
populateGpuRewritePatterns(patterns);
- populateGpuPromoteShuffleToAMDGPUPatterns(patterns, maybeChipset);
+ populateGpuPromoteShuffleToAMDGPUPatterns(patterns, *targetInfo);
(void)applyPatternsGreedily(m, std::move(patterns));
}
@@ -820,9 +820,9 @@ struct LowerGpuOpsToROCDLOpsPass final
}
populateAMDGPUToROCDLConversionPatterns(converter, llvmPatterns,
- *maybeChipset);
+ *targetInfo);
populateGpuToROCDLConversionPatterns(converter, llvmPatterns, runtime,
- *maybeChipset);
+ *targetInfo);
configureGpuToROCDLConversionLegality(target);
if (failed(applyPartialConversion(m, target, std::move(llvmPatterns))))
signalPassFailure();
@@ -870,7 +870,7 @@ void mlir::configureGpuToROCDLConversionLegality(ConversionTarget &target) {
void mlir::populateGpuToROCDLConversionPatterns(
const LLVMTypeConverter &converter, RewritePatternSet &patterns,
- mlir::gpu::amd::Runtime runtime, amdgpu::Chipset chipset) {
+ mlir::gpu::amd::Runtime runtime, const ROCDL::TargetInfo &target) {
using gpu::index_lowering::IndexKind;
using gpu::index_lowering::IntrType;
using mlir::gpu::amd::Runtime;
@@ -909,7 +909,7 @@ void mlir::populateGpuToROCDLConversionPatterns(
GPUSubgroupBroadcastOpToROCDL, GPUBallotOpToROCDL>(converter);
patterns.add<GPUSubgroupIdOpToROCDL, GPUSubgroupSizeOpToROCDL,
GPUBarrierOpLowering, GPUInitializeNamedBarrierOpLowering>(
- converter, chipset);
+ converter, target);
- populateMathToROCDLConversionPatterns(converter, patterns, chipset);
+ populateMathToROCDLConversionPatterns(converter, patterns, target);
}
diff --git a/mlir/lib/Conversion/MathToROCDL/MathToROCDL.cpp b/mlir/lib/Conversion/MathToROCDL/MathToROCDL.cpp
index a922338176f11..a15271190bc70 100644
--- a/mlir/lib/Conversion/MathToROCDL/MathToROCDL.cpp
+++ b/mlir/lib/Conversion/MathToROCDL/MathToROCDL.cpp
@@ -11,10 +11,10 @@
#include "mlir/Conversion/LLVMCommon/LoweringOptions.h"
#include "mlir/Conversion/LLVMCommon/TypeConverter.h"
#include "mlir/Conversion/LLVMCommon/VectorPattern.h"
-#include "mlir/Dialect/AMDGPU/Utils/Chipset.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/LLVMIR/ROCDLDialect.h"
+#include "mlir/Dialect/LLVMIR/ROCDLTargetInfo.h"
#include "mlir/Dialect/Math/IR/Math.h"
#include "mlir/Dialect/Vector/IR/VectorOps.h"
#include "mlir/IR/BuiltinDialect.h"
@@ -84,7 +84,7 @@ struct ClampFOpConversion final
void mlir::populateMathToROCDLConversionPatterns(
const LLVMTypeConverter &converter, RewritePatternSet &patterns,
- std::optional<amdgpu::Chipset> chipset) {
+ std::optional<ROCDL::TargetInfo> target) {
// Handled by mathToLLVM: math::AbsIOp
// Handled by mathToLLVM: math::AbsFOp
// Handled by mathToLLVM: math::CopySignOp
@@ -160,10 +160,10 @@ void mlir::populateMathToROCDLConversionPatterns(
populateOpPatterns<arith::RemFOp>(converter, patterns, "__ocml_fmod_f32",
"__ocml_fmod_f64", "__ocml_fmod_f16");
- if (chipset.has_value() && chipset->majorVersion >= 9) {
+ if (target && target->has(llvm::AMDGPU::FEAT_GFX9_INSTS)) {
patterns.add<ClampFOpConversion>(converter);
} else {
- LDBG() << "Chipset dependent patterns were not added";
+ LDBG() << "Target dependent patterns were not added";
}
}
@@ -183,15 +183,26 @@ void ConvertMathToROCDLPass::runOnOperation() {
LowerToLLVMOptions options(ctx, DataLayout(m));
LLVMTypeConverter converter(ctx, options);
- FailureOr<amdgpu::Chipset> maybeChipset;
- if (!chipset.empty()) {
- maybeChipset = amdgpu::Chipset::parse(chipset);
- if (failed(maybeChipset))
+ // An empty triple means "no target", in which case the target-dependent
+ // patterns are simply not added. A chip or feature list without a triple is
+ // a mistake, though.
+ std::optional<ROCDL::TargetInfo> resolved;
+ if (triple.empty()) {
+ if (!chip.empty() || !features.empty()) {
+ emitError(UnknownLoc::get(&getContext()))
+ << "'chip' and 'features' need a 'triple' to apply to";
return signalPassFailure();
+ }
+ } else {
+ FailureOr<ROCDL::TargetInfo> targetInfo =
+ ROCDL::TargetInfo::get(triple, chip, features, [&] {
+ return emitError(UnknownLoc::get(&getContext()));
+ });
+ if (failed(targetInfo))
+ return signalPassFailure();
+ resolved = *targetInfo;
}
- populateMathToROCDLConversionPatterns(
- converter, patterns,
- succeeded(maybeChipset) ? std::optional(*maybeChipset) : std::nullopt);
+ populateMathToROCDLConversionPatterns(converter, patterns, resolved);
ConversionTarget target(getContext());
target
diff --git a/mlir/lib/Dialect/AMDGPU/Transforms/CMakeLists.txt b/mlir/lib/Dialect/AMDGPU/Transforms/CMakeLists.txt
index 29baef635ec80..28fa958181d03 100644
--- a/mlir/lib/Dialect/AMDGPU/Transforms/CMakeLists.txt
+++ b/mlir/lib/Dialect/AMDGPU/Transforms/CMakeLists.txt
@@ -16,6 +16,7 @@ add_mlir_dialect_library(MLIRAMDGPUTransforms
MLIRAffineUtils
MLIRArithDialect
MLIRMemRefDialect
+ MLIRROCDLDialect
MLIRSCFDialect
MLIRVectorDialect
MLIRControlFlowDialect
diff --git a/mlir/lib/Dialect/AMDGPU/Transforms/EmulateAtomics.cpp b/mlir/lib/Dialect/AMDGPU/Transforms/EmulateAtomics.cpp
index 332ac8cdb60c0..ca6f4f731b346 100644
--- a/mlir/lib/Dialect/AMDGPU/Transforms/EmulateAtomics.cpp
+++ b/mlir/lib/Dialect/AMDGPU/Transforms/EmulateAtomics.cpp
@@ -9,9 +9,9 @@
#include "mlir/Dialect/AMDGPU/Transforms/Passes.h"
#include "mlir/Dialect/AMDGPU/IR/AMDGPUDialect.h"
-#include "mlir/Dialect/AMDGPU/Utils/Chipset.h"
#include "mlir/Dialect/Arith/IR/Arith.h"
#include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h"
+#include "mlir/Dialect/LLVMIR/ROCDLTargetInfo.h"
#include "mlir/Dialect/Vector/IR/VectorOps.h"
#include "mlir/IR/BuiltinAttributes.h"
#include "mlir/IR/TypeUtilities.h"
@@ -164,43 +164,49 @@ LogicalResult RawBufferAtomicByCasPattern<AtomicOp, ArithOp>::matchAndRewrite(
return success();
}
+/// Returns whether \p op can be lowered to a native buffer atomic fadd on
+/// \p target.
+static bool isFaddNativelySupported(const ROCDL::TargetInfo &target,
+ RawBufferAtomicFaddOp op) {
+ namespace AMDGPU = ::llvm::AMDGPU;
+
+ // A target with only the no-return form can still perform the atomic, it
+ // just cannot report the old value, so it suffices when the result is dead.
+ bool hasFadd = target.has(AMDGPU::FEAT_ATOMIC_FADD_RTN_INSTS) ||
+ (target.has(AMDGPU::FEAT_ATOMIC_FADD_NO_RTN_INSTS) &&
+ op.getOldValue().use_empty());
+ if (!hasFadd)
+ return false;
+
+ // The packed 16-bit forms are separate instructions with their own features.
+ Type elemType = getElementTypeOrSelf(op.getValue().getType());
+ if (isa<Float16Type>(elemType))
+ return target.has(AMDGPU::FEAT_ATOMIC_BUFFER_GLOBAL_PK_ADD_F16_INSTS);
+ if (isa<BFloat16Type>(elemType))
+ return target.has(AMDGPU::FEAT_ATOMIC_BUFFER_PK_ADD_BF16_INST);
+ return true;
+}
+
void mlir::amdgpu::populateAmdgpuEmulateAtomicsPatterns(
- ConversionTarget &target, RewritePatternSet &patterns, Chipset chipset,
- PatternBenefit benefit) {
- // gfx10 has no atomic adds.
- if (chipset.majorVersion == 10 || chipset < Chipset(9, 0, 8)) {
- target.addIllegalOp<RawBufferAtomicFaddOp>();
- }
- // gfx11 has no fp16 atomics
- if (chipset.majorVersion == 11) {
- target.addDynamicallyLegalOp<RawBufferAtomicFaddOp>(
- [](RawBufferAtomicFaddOp op) -> bool {
- Type elemType = getElementTypeOrSelf(op.getValue().getType());
- return !isa<Float16Type, BFloat16Type>(elemType);
+ ConversionTarget &target, RewritePatternSet &patterns,
+ const ROCDL::TargetInfo &targetInfo, PatternBenefit benefit) {
+ namespace AMDGPU = ::llvm::AMDGPU;
+
+ target.addDynamicallyLegalOp<RawBufferAtomicFaddOp>(
+ [targetInfo](RawBufferAtomicFaddOp op) -> bool {
+ return isFaddNativelySupported(targetInfo, op);
+ });
+
+ // Floating-point min and max are only emulated on gfx9; later generations
+ // have them for every type this op accepts. f64 is the one gfx9 case that
+ // may be native.
+ if (targetInfo.isGeneration(9)) {
+ target.addDynamicallyLegalOp<RawBufferAtomicFmaxOp>(
+ [targetInfo](RawBufferAtomicFmaxOp op) -> bool {
+ return op.getValue().getType().isF64() &&
+ targetInfo.has(AMDGPU::FEAT_ATOMIC_FMIN_FMAX_GLOBAL_F64);
});
}
- // gfx9 has no to a very limited support for floating-point min and max.
- if (chipset.majorVersion == 9) {
- if (chipset >= Chipset(9, 0, 0xa)) {
- // gfx90a supports f64 max (and min, but we don't have a min wrapper right
- // now) but all other types need to be emulated.
- target.addDynamicallyLegalOp<RawBufferAtomicFmaxOp>(
- [](RawBufferAtomicFmaxOp op) -> bool {
- return op.getValue().getType().isF64();
- });
- } else {
- target.addIllegalOp<RawBufferAtomicFmaxOp>();
- }
- // TODO(https://github.com/llvm/llvm-project/issues/129206): Refactor
- // this to avoid hardcoding ISA version: gfx950 has bf16 atomics.
- if (chipset < Chipset(9, 5, 0)) {
- target.addDynamicallyLegalOp<RawBufferAtomicFaddOp>(
- [](RawBufferAtomicFaddOp op) -> bool {
- Type elemType = getElementTypeOrSelf(op.getValue().getType());
- return !isa<BFloat16Type>(elemType);
- });
- }
- }
patterns.add<
RawBufferAtomicByCasPattern<RawBufferAtomicFaddOp, arith::AddFOp>,
RawBufferAtomicByCasPattern<RawBufferAtomicFmaxOp, arith::MaximumFOp>,
@@ -211,11 +217,10 @@ void mlir::amdgpu::populateAmdgpuEmulateAtomicsPatterns(
void AmdgpuEmulateAtomicsPass::runOnOperation() {
Operation *op = getOperation();
- FailureOr<Chipset> maybeChipset = Chipset::parse(chipset);
- if (failed(maybeChipset)) {
- emitError(op->getLoc(), "Invalid chipset name: " + chipset);
+ FailureOr<ROCDL::TargetInfo> targetInfo = ROCDL::TargetInfo::get(
+ triple, chip, features, [&] { return op->emitError(); });
+ if (failed(targetInfo))
return signalPassFailure();
- }
MLIRContext &ctx = getContext();
ConversionTarget target(ctx);
@@ -223,7 +228,7 @@ void AmdgpuEmulateAtomicsPass::runOnOperation() {
target.markUnknownOpDynamicallyLegal(
[](Operation *op) -> bool { return true; });
- populateAmdgpuEmulateAtomicsPatterns(target, patterns, *maybeChipset);
+ populateAmdgpuEmulateAtomicsPatterns(target, patterns, *targetInfo);
if (failed(applyPartialConversion(op, target, std::move(patterns))))
return signalPassFailure();
}
diff --git a/mlir/lib/Dialect/GPU/Pipelines/GPUToROCDLPipeline.cpp b/mlir/lib/Dialect/GPU/Pipelines/GPUToROCDLPipeline.cpp
index 1e5fd09a00a75..910e75924b128 100644
--- a/mlir/lib/Dialect/GPU/Pipelines/GPUToROCDLPipeline.cpp
+++ b/mlir/lib/Dialect/GPU/Pipelines/GPUToROCDLPipeline.cpp
@@ -40,6 +40,28 @@ namespace {
//===----------------------------------------------------------------------===//
// Common pipeline
//===----------------------------------------------------------------------===//
+/// The pipeline's `wave64` option and the wavefront-size target features say
+/// the same thing, so fold the former into the latter before handing them to
+/// the conversion passes. Only a GPU that runs at either size can honour the
+/// choice; on one that pins the size, naming the other is an error, and naming
+/// the same one is noise. A size the caller spelled out wins.
+static std::string
+resolveFeatures(const mlir::gpu::GPUToROCDLPipelineOptions &options) {
+ std::string features = options.features;
+ if (StringRef(features).contains("wavefrontsize"))
+ return features;
+
+ FailureOr<ROCDL::TargetInfo> target =
+ ROCDL::TargetInfo::get(options.triple, options.chip, options.features);
+ if (failed(target) || !target->supportsBothWavefrontSizes())
+ return features;
+
+ if (!features.empty())
+ features += ",";
+ features += options.wave64 ? "+wavefrontsize64" : "+wavefrontsize32";
+ return features;
+}
+
void buildCommonPassPipeline(
OpPassManager &pm, const mlir::gpu::GPUToROCDLPipelineOptions &options) {
// Lower AMDGPU dialect ops (e.g. amdgpu.lds_barrier, amdgpu.dpp,
@@ -47,7 +69,9 @@ void buildCommonPassPipeline(
// still live in unout-lined `gpu.launch` bodies. Mirrors the way NVVM's
// pipeline runs `convert-nvgpu-to-nvvm` before kernel outlining.
ConvertAMDGPUToROCDLPassOptions amdgpuToROCDLOpt;
- amdgpuToROCDLOpt.chipset = options.chip;
+ amdgpuToROCDLOpt.triple = options.triple;
+ amdgpuToROCDLOpt.chip = options.chip;
+ amdgpuToROCDLOpt.features = resolveFeatures(options);
pm.addPass(createConvertAMDGPUToROCDLPass(amdgpuToROCDLOpt));
pm.addPass(createGpuKernelOutliningPass());
@@ -80,7 +104,9 @@ void buildCommonPassPipeline(
void buildGpuPassPipeline(OpPassManager &pm,
const mlir::gpu::GPUToROCDLPipelineOptions &options) {
ConvertGpuOpsToROCDLOpsOptions opt;
- opt.chipset = options.chip;
+ opt.triple = options.triple;
+ opt.chip = options.chip;
+ opt.features = resolveFeatures(options);
opt.useBarePtrCallConv = options.kernelUseBarePtrCallConv;
opt.indexBitwidth = options.indexBitWidth;
// Always declare HIP as the runtime so that gpu.printf etc. lower to the
diff --git a/mlir/lib/Dialect/GPU/TransformOps/GPUTransformOps.cpp b/mlir/lib/Dialect/GPU/TransformOps/GPUTransformOps.cpp
index 9591b76a5330c..66528dfbe1414 100644
--- a/mlir/lib/Dialect/GPU/TransformOps/GPUTransformOps.cpp
+++ b/mlir/lib/Dialect/GPU/TransformOps/GPUTransformOps.cpp
@@ -14,7 +14,6 @@
#include "mlir/Conversion/GPUToROCDL/GPUToROCDLPass.h"
#include "mlir/Conversion/LLVMCommon/TypeConverter.h"
#include "mlir/Dialect/AMDGPU/IR/AMDGPUDialect.h"
-#include "mlir/Dialect/AMDGPU/Utils/Chipset.h"
#include "mlir/Dialect/Arith/IR/Arith.h"
#include "mlir/Dialect/GPU/IR/GPUDialect.h"
#include "mlir/Dialect/GPU/TransformOps/Utils.h"
@@ -108,21 +107,21 @@ void transform::ApplyGPUToROCDLConversionPatternsOp::populatePatterns(
TypeConverter &typeConverter, RewritePatternSet &patterns) {
auto &llvmTypeConverter = static_cast<LLVMTypeConverter &>(typeConverter);
amdgpu::populateCommonGPUTypeAndAttributeConversions(llvmTypeConverter);
- FailureOr<amdgpu::Chipset> maybeChipset =
- amdgpu::Chipset::parse(getChipset());
- assert(llvm::succeeded(maybeChipset) && "expected valid chipset");
+ // The verifier has already rejected anything unparseable.
+ FailureOr<ROCDL::TargetInfo> targetInfo = ROCDL::TargetInfo::get(
+ getTriple(), getChip().value_or(""), getFeatures().value_or(""));
+ assert(llvm::succeeded(targetInfo) && "verifier accepted this target");
populateGpuToROCDLConversionPatterns(
- llvmTypeConverter, patterns, mlir::gpu::amd::Runtime::HIP, *maybeChipset);
+ llvmTypeConverter, patterns, mlir::gpu::amd::Runtime::HIP, *targetInfo);
}
LogicalResult
transform::ApplyGPUToROCDLConversionPatternsOp::verifyTypeConverter(
transform::TypeConverterBuilderOpInterface builder) {
- FailureOr<amdgpu::Chipset> maybeChipset =
- amdgpu::Chipset::parse(getChipset());
- if (failed(maybeChipset)) {
- return emitOpError("Invalid chipset name: " + getChipset());
- }
+ if (failed(ROCDL::TargetInfo::get(getTriple(), getChip().value_or(""),
+ getFeatures().value_or(""),
+ [&] { return emitOpError(); })))
+ return failure();
if (builder.getTypeConverterType() != "LLVMTypeConverter")
return emitOpError("expected LLVMTypeConverter");
return success();
@@ -138,16 +137,34 @@ void ApplyGPURewritePatternsOp::populatePatterns(RewritePatternSet &patterns) {
void transform::ApplyGPUPromoteShuffleToAMDGPUPatternsOp::populatePatterns(
RewritePatternSet &patterns) {
- std::optional<StringRef> chipsetName = getChipset();
- std::optional<amdgpu::Chipset> maybeChipset;
- if (chipsetName) {
- FailureOr<amdgpu::Chipset> parsedChipset =
- amdgpu::Chipset::parse(*chipsetName);
- assert(llvm::succeeded(parsedChipset) && "expected valid chipset");
- maybeChipset = parsedChipset;
+ std::optional<StringRef> tripleName = getTriple();
+ std::optional<ROCDL::TargetInfo> targetInfo;
+ if (tripleName) {
+ // The verifier has already rejected anything unparseable.
+ FailureOr<ROCDL::TargetInfo> parsed = ROCDL::TargetInfo::get(
+ *tripleName, getChip().value_or(""), getFeatures().value_or(""));
+ assert(llvm::succeeded(parsed) && "verifier accepted this target");
+ targetInfo = *parsed;
+ }
+
+ populateGpuPromoteShuffleToAMDGPUPatterns(patterns, targetInfo);
+}
+
+LogicalResult transform::ApplyGPUPromoteShuffleToAMDGPUPatternsOp::verify() {
+ std::optional<StringRef> tripleName = getTriple();
+ if (!tripleName) {
+ // Without a target there is nothing for these to modify, and silently
+ // ignoring them would hide a typo'd or half-migrated script.
+ if (getChip() || getFeatures())
+ return emitOpError("'chip' and 'features' require a 'triple'");
+ return success();
}
- populateGpuPromoteShuffleToAMDGPUPatterns(patterns, maybeChipset);
+ if (failed(ROCDL::TargetInfo::get(*tripleName, getChip().value_or(""),
+ getFeatures().value_or(""),
+ [&] { return emitOpError(); })))
+ return failure();
+ return success();
}
//===----------------------------------------------------------------------===//
diff --git a/mlir/lib/Dialect/GPU/Transforms/PromoteShuffleToAMDGPU.cpp b/mlir/lib/Dialect/GPU/Transforms/PromoteShuffleToAMDGPU.cpp
index 01da26f88a84d..8fb62258c98ea 100644
--- a/mlir/lib/Dialect/GPU/Transforms/PromoteShuffleToAMDGPU.cpp
+++ b/mlir/lib/Dialect/GPU/Transforms/PromoteShuffleToAMDGPU.cpp
@@ -11,8 +11,8 @@
//
//===----------------------------------------------------------------------===//
-#include "mlir/Dialect/AMDGPU/Utils/Chipset.h"
#include "mlir/Dialect/GPU/Transforms/Passes.h"
+#include "mlir/Dialect/LLVMIR/ROCDLTargetInfo.h"
#include "mlir/Dialect/AMDGPU/IR/AMDGPUDialect.h"
#include "mlir/Dialect/Arith/IR/Arith.h"
@@ -24,8 +24,6 @@ using namespace mlir;
namespace {
-constexpr amdgpu::Chipset kGfx950 = amdgpu::Chipset(9, 5, 0);
-
/// Try to promote `gpu.shuffle` to `amdgpu.swizzle_bitmode`, width must be 64
/// and offset must be a constant integer in the range [0, 31].
struct PromoteShuffleToSwizzlePattern
@@ -100,10 +98,10 @@ struct PromoteShuffleToPermlanePattern
} // namespace
void mlir::populateGpuPromoteShuffleToAMDGPUPatterns(
- RewritePatternSet &patterns, std::optional<amdgpu::Chipset> maybeChipset) {
+ RewritePatternSet &patterns, std::optional<ROCDL::TargetInfo> target) {
patterns.add<PromoteShuffleToSwizzlePattern>(patterns.getContext(),
/*benefit*/ 1);
- if (maybeChipset && *maybeChipset >= kGfx950)
+ if (target && target->has(llvm::AMDGPU::FEAT_PERMLANE32_SWAP))
patterns.add<PromoteShuffleToPermlanePattern>(patterns.getContext(),
/*benefit*/ 2);
}
diff --git a/mlir/lib/Dialect/GPU/Transforms/SubgroupReduceLowering.cpp b/mlir/lib/Dialect/GPU/Transforms/SubgroupReduceLowering.cpp
index ec1571a56fe4a..23f1a16ca85aa 100644
--- a/mlir/lib/Dialect/GPU/Transforms/SubgroupReduceLowering.cpp
+++ b/mlir/lib/Dialect/GPU/Transforms/SubgroupReduceLowering.cpp
@@ -11,12 +11,12 @@
//===----------------------------------------------------------------------===//
#include "mlir/Dialect/AMDGPU/IR/AMDGPUDialect.h"
-#include "mlir/Dialect/AMDGPU/Utils/Chipset.h"
#include "mlir/Dialect/Arith/IR/Arith.h"
#include "mlir/Dialect/GPU/IR/GPUDialect.h"
#include "mlir/Dialect/GPU/Transforms/Passes.h"
#include "mlir/Dialect/GPU/Utils/GPUUtils.h"
#include "mlir/Dialect/LLVMIR/ROCDLDialect.h"
+#include "mlir/Dialect/LLVMIR/ROCDLTargetInfo.h"
#include "mlir/Dialect/Vector/IR/VectorOps.h"
#include "mlir/IR/BuiltinTypes.h"
#include "mlir/IR/Location.h"
@@ -370,7 +370,8 @@ struct VectorSubgroupReduceToShuffles final
static FailureOr<Value>
createSubgroupDPPReduction(PatternRewriter &rewriter, gpu::SubgroupReduceOp op,
Value input, gpu::AllReduceOperation mode,
- const ClusterInfo &ci, amdgpu::Chipset chipset) {
+ const ClusterInfo &ci,
+ const ROCDL::TargetInfo &target) {
Location loc = op.getLoc();
Value dpp;
Value res = input;
@@ -414,7 +415,7 @@ createSubgroupDPPReduction(PatternRewriter &rewriter, gpu::SubgroupReduceOp op,
gpu::convertReductionKind(mode), res, dpp);
}
if (ci.clusterSize >= 32) {
- if (chipset.majorVersion <= 9) {
+ if (!target.has(llvm::AMDGPU::FEAT_GFX10_INSTS)) {
// Broadcast last value from each row to next row.
// Use row mask to avoid polluting row 0 (and row 2 if wave-64).
dpp = amdgpu::DPPOp::create(rewriter, loc, res.getType(), res, res,
@@ -449,7 +450,7 @@ createSubgroupDPPReduction(PatternRewriter &rewriter, gpu::SubgroupReduceOp op,
/*or_mask=*/31,
/*xor_mask=*/0);
}
- } else if (chipset.majorVersion <= 12) {
+ } else if (!target.has(llvm::AMDGPU::FEAT_GFX13_INSTS)) {
// Use a permute lane to cross rows (row 1 <-> row 0, row 3 <-> row 2).
Value uint32Max = arith::ConstantOp::create(
rewriter, loc, rewriter.getI32Type(), rewriter.getI32IntegerAttr(-1));
@@ -472,7 +473,7 @@ createSubgroupDPPReduction(PatternRewriter &rewriter, gpu::SubgroupReduceOp op,
}
}
if (ci.clusterSize >= 64) {
- if (chipset.majorVersion <= 9) {
+ if (!target.has(llvm::AMDGPU::FEAT_GFX10_INSTS)) {
// Broadcast 31st lane value to rows 2 and 3.
dpp = amdgpu::DPPOp::create(rewriter, loc, res.getType(), res, res,
amdgpu::DPPPerm::row_bcast_31,
@@ -486,7 +487,7 @@ createSubgroupDPPReduction(PatternRewriter &rewriter, gpu::SubgroupReduceOp op,
res =
ROCDL::ReadlaneOp::create(rewriter, loc, res.getType(), res, lane63);
- } else if (chipset.majorVersion <= 12) {
+ } else if (!target.has(llvm::AMDGPU::FEAT_GFX13_INSTS)) {
// Assume reduction across 32 lanes has been done.
// Perform final reduction manually by summing values in lane 0 and
// lane 32.
@@ -516,10 +517,11 @@ createSubgroupDPPReduction(PatternRewriter &rewriter, gpu::SubgroupReduceOp op,
struct ScalarSubgroupReduceToDPP final
: OpRewritePattern<gpu::SubgroupReduceOp> {
ScalarSubgroupReduceToDPP(MLIRContext *ctx, unsigned subgroupSize,
- bool matchClustered, amdgpu::Chipset chipset,
+ bool matchClustered,
+ const ROCDL::TargetInfo &target,
PatternBenefit benefit)
: OpRewritePattern(ctx, benefit), subgroupSize(subgroupSize),
- matchClustered(matchClustered), chipset(chipset) {}
+ matchClustered(matchClustered), target(target) {}
LogicalResult matchAndRewrite(gpu::SubgroupReduceOp op,
PatternRewriter &rewriter) const override {
@@ -545,7 +547,7 @@ struct ScalarSubgroupReduceToDPP final
op, "Value type is not a compatible scalar.");
FailureOr<Value> dpp = createSubgroupDPPReduction(
- rewriter, op, op.getValue(), op.getOp(), *ci, chipset);
+ rewriter, op, op.getValue(), op.getOp(), *ci, target);
if (failed(dpp))
return failure();
@@ -556,7 +558,7 @@ struct ScalarSubgroupReduceToDPP final
private:
unsigned subgroupSize = 0;
bool matchClustered = false;
- amdgpu::Chipset chipset;
+ ROCDL::TargetInfo target;
};
} // namespace
@@ -569,18 +571,18 @@ void mlir::populateGpuBreakDownSubgroupReducePatterns(
}
void mlir::populateGpuLowerSubgroupReduceToDPPPatterns(
- RewritePatternSet &patterns, unsigned subgroupSize, amdgpu::Chipset chipset,
- PatternBenefit benefit) {
+ RewritePatternSet &patterns, unsigned subgroupSize,
+ const ROCDL::TargetInfo &target, PatternBenefit benefit) {
patterns.add<ScalarSubgroupReduceToDPP>(patterns.getContext(), subgroupSize,
- /*matchClustered=*/false, chipset,
+ /*matchClustered=*/false, target,
benefit);
}
void mlir::populateGpuLowerClusteredSubgroupReduceToDPPPatterns(
- RewritePatternSet &patterns, unsigned subgroupSize, amdgpu::Chipset chipset,
- PatternBenefit benefit) {
+ RewritePatternSet &patterns, unsigned subgroupSize,
+ const ROCDL::TargetInfo &target, PatternBenefit benefit) {
patterns.add<ScalarSubgroupReduceToDPP>(patterns.getContext(), subgroupSize,
- /*matchClustered=*/true, chipset,
+ /*matchClustered=*/true, target,
benefit);
}
diff --git a/mlir/lib/Dialect/LLVMIR/CMakeLists.txt b/mlir/lib/Dialect/LLVMIR/CMakeLists.txt
index 494a1e4c3ee97..f1bea9660e346 100644
--- a/mlir/lib/Dialect/LLVMIR/CMakeLists.txt
+++ b/mlir/lib/Dialect/LLVMIR/CMakeLists.txt
@@ -73,6 +73,7 @@ add_mlir_dialect_library(MLIRNVVMDialect
add_mlir_dialect_library(MLIRROCDLDialect
IR/ROCDLDialect.cpp
+ IR/ROCDLTargetInfo.cpp
ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/LLVMIR
@@ -86,6 +87,7 @@ add_mlir_dialect_library(MLIRROCDLDialect
LINK_COMPONENTS
AsmParser
Core
+ TargetParser
LINK_LIBS PUBLIC
MLIRIR
diff --git a/mlir/lib/Dialect/LLVMIR/IR/ROCDLTargetInfo.cpp b/mlir/lib/Dialect/LLVMIR/IR/ROCDLTargetInfo.cpp
new file mode 100644
index 0000000000000..651f0026b1d44
--- /dev/null
+++ b/mlir/lib/Dialect/LLVMIR/IR/ROCDLTargetInfo.cpp
@@ -0,0 +1,180 @@
+//===- ROCDLTargetInfo.cpp - AMDGPU target description --------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "mlir/Dialect/LLVMIR/ROCDLTargetInfo.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/Twine.h"
+
+using namespace mlir;
+using namespace mlir::ROCDL;
+
+namespace AMDGPU = ::llvm::AMDGPU;
+using ::llvm::Triple;
+
+namespace {
+/// Reports \p message through \p emitError if it is non-null, and returns
+/// failure.
+LogicalResult fail(function_ref<InFlightDiagnostic()> emitError,
+ const Twine &message) {
+ if (emitError)
+ emitError() << message;
+ return failure();
+}
+} // namespace
+
+/// Resolves the wavefront size in \p bits, mirroring the policy LLVM applies in
+/// fillAMDGCNFeatureMap: a target that only runs at one size rejects a request
+/// for the other, and a target that supports both defaults to wave32.
+static LogicalResult
+resolveWavefrontSize(AMDGPU::AMDGPUFeatureBitset &bits, bool targetWave32,
+ bool targetWave64,
+ function_ref<InFlightDiagnostic()> emitError) {
+ bool wave32 = bits.test(AMDGPU::FEAT_WAVEFRONTSIZE32);
+ bool wave64 = bits.test(AMDGPU::FEAT_WAVEFRONTSIZE64);
+
+ if (wave32 && wave64)
+ return fail(emitError,
+ "'+wavefrontsize32' and '+wavefrontsize64' are mutually "
+ "exclusive");
+ if (targetWave64 && !wave64)
+ return fail(emitError, "target only supports wavefrontsize64");
+ if (targetWave32 && !wave32)
+ return fail(emitError, "target only supports wavefrontsize32");
+
+ // A target that supports both sizes and was not asked for one runs wave32.
+ if (!wave32 && !wave64)
+ bits.set(AMDGPU::FEAT_WAVEFRONTSIZE32);
+ return success();
+}
+
+FailureOr<TargetInfo>
+TargetInfo::get(StringRef tripleOrChip, StringRef chip, StringRef features,
+ function_ref<InFlightDiagnostic()> emitError) {
+ if (tripleOrChip.empty())
+ return fail(emitError, "target triple cannot be empty");
+
+ TargetInfo info;
+
+ // A bare GPU name is accepted in place of a triple, so that "gfx942" keeps
+ // working where a chipset used to be given.
+ if (AMDGPU::GPUKind named = AMDGPU::parseArchAMDGCN(tripleOrChip)) {
+ if (!chip.empty() && chip != tripleOrChip)
+ return fail(emitError,
+ "conflicting GPUs '" + tripleOrChip + "' and '" + chip + "'");
+ info.kind = named;
+ info.subArch = AMDGPU::getSubArch(named);
+ } else {
+ Triple triple(Triple::normalize(tripleOrChip));
+ if (!triple.isAMDGCN())
+ return fail(emitError,
+ "'" + tripleOrChip + "' is not an AMDGCN triple or GPU name");
+
+ info.subArch = triple.getSubArch();
+ // Triple parsing maps any unrecognized "amdgpu..." arch to NoSubArch
+ // without complaining, so a typo would otherwise be silently accepted as a
+ // target with no features. Only the bare "amdgcn"/"amdgpu" spellings
+ // legitimately carry no subarch.
+ if (info.subArch == Triple::NoSubArch && triple.getArchName().size() != 6)
+ return fail(emitError, "unknown AMDGPU subarchitecture in triple '" +
+ tripleOrChip + "'");
+
+ if (!chip.empty()) {
+ if (!AMDGPU::isCPUValidForSubArch(info.subArch, chip))
+ return fail(emitError, "GPU '" + chip + "' is not valid for triple '" +
+ tripleOrChip + "'");
+ info.kind = AMDGPU::parseArchAMDGCN(chip);
+ // The chip pins down the exact GPU, which may be more specific than the
+ // triple's family subarch.
+ info.subArch = AMDGPU::getSubArch(info.kind);
+ } else {
+ info.kind = AMDGPU::getGPUKindFromSubArch(info.subArch);
+ }
+ }
+
+ info.featureBits = AMDGPU::getFeatureBitset(info.kind);
+
+ bool targetWave32 = info.featureBits.test(AMDGPU::FEAT_WAVEFRONTSIZE32);
+ bool targetWave64 = info.featureBits.test(AMDGPU::FEAT_WAVEFRONTSIZE64);
+ // Recorded before the modifiers and the default below pin a size.
+ info.dualWavefrontSize = !info.isUnknown() && !targetWave32 && !targetWave64;
+
+ if (std::optional<StringRef> bad =
+ AMDGPU::applyFeatureModifiers(features, info.featureBits))
+ return fail(emitError, "invalid target feature '" + *bad + "'");
+
+ if (!info.isUnknown() &&
+ failed(resolveWavefrontSize(info.featureBits, targetWave32, targetWave64,
+ emitError)))
+ return failure();
+
+ return info;
+}
+
+bool TargetInfo::isGeneration(unsigned major) const {
+ // The generation features are cumulative: a gfx12 target has every
+ // FEAT_GFX*_INSTS bit from gfx8 up to gfx12. So a target is *in* generation N
+ // when it has N's bit but not N+1's. This holds for generic targets too,
+ // unlike comparing ISA versions.
+ auto hasGen = [&](unsigned gen) {
+ switch (gen) {
+ case 7:
+ return has(AMDGPU::FEAT_CI_INSTS);
+ case 8:
+ return has(AMDGPU::FEAT_GFX8_INSTS);
+ case 9:
+ return has(AMDGPU::FEAT_GFX9_INSTS);
+ case 10:
+ return has(AMDGPU::FEAT_GFX10_INSTS);
+ case 11:
+ return has(AMDGPU::FEAT_GFX11_INSTS);
+ case 12:
+ return has(AMDGPU::FEAT_GFX12_INSTS);
+ case 13:
+ return has(AMDGPU::FEAT_GFX13_INSTS);
+ default:
+ return false;
+ }
+ };
+
+ if (isUnknown())
+ return false;
+ // gfx6 is the base: it has none of the generation features.
+ if (major == 6)
+ return !hasGen(7);
+ return hasGen(major) && !hasGen(major + 1);
+}
+
+std::optional<unsigned> TargetInfo::getBufferResourceNumRecordsWidth() const {
+ return AMDGPU::getBufferResourceNumRecordsWidth(kind);
+}
+
+std::optional<unsigned> TargetInfo::getMaxAddressableLocalMemorySize() const {
+ if (isUnknown())
+ return std::nullopt;
+ return AMDGPU::getMaxHWAddressableLocalMemorySize(kind);
+}
+
+std::optional<unsigned> TargetInfo::getWavefrontSize() const {
+ if (has(AMDGPU::FEAT_WAVEFRONTSIZE64))
+ return 64;
+ if (has(AMDGPU::FEAT_WAVEFRONTSIZE32))
+ return 32;
+ return std::nullopt;
+}
+
+AMDGPU::IsaVersion TargetInfo::getIsaVersion() const {
+ return AMDGPU::getIsaVersion(subArch);
+}
+
+StringRef TargetInfo::getArchName() const {
+ return AMDGPU::getArchNameAMDGCN(kind);
+}
+
+bool TargetInfo::isGeneric() const {
+ return !isUnknown() && AMDGPU::getMajorSubArch(subArch) == subArch;
+}
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/8-bit-floats-ocp-gfx1170.mlir b/mlir/test/Conversion/AMDGPUToROCDL/8-bit-floats-ocp-gfx1170.mlir
deleted file mode 100644
index 27692d540e5d4..0000000000000
--- a/mlir/test/Conversion/AMDGPUToROCDL/8-bit-floats-ocp-gfx1170.mlir
+++ /dev/null
@@ -1,28 +0,0 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx1170 --split-input-file --verify-diagnostics
-
-// gfx11.7 has FeatureOCPFP8ConversionInsts, so these conversions are available
-// on it. They are rejected today because the predicate deciding whether a
-// target uses the OCP fp8 formats is written as the version range "gfx9.5+ or
-// gfx12+", which skips over gfx11.7 entirely.
-
-func.func @ext_packed_fp8(%v: vector<4xf8E4M3FN>) -> f32 {
- // expected-error at below {{failed to legalize operation 'amdgpu.ext_packed_fp8'}}
- %ret = amdgpu.ext_packed_fp8 %v[0] : vector<4xf8E4M3FN> to f32
- func.return %ret : f32
-}
-
-// -----
-
-func.func @ext_packed_bf8(%v: vector<4xf8E5M2>) -> f32 {
- // expected-error at below {{failed to legalize operation 'amdgpu.ext_packed_fp8'}}
- %ret = amdgpu.ext_packed_fp8 %v[0] : vector<4xf8E5M2> to f32
- func.return %ret : f32
-}
-
-// -----
-
-func.func @packed_trunc_2xfp8(%v: f32) -> vector<4xf8E4M3FN> {
- // expected-error at below {{failed to legalize operation 'amdgpu.packed_trunc_2xfp8'}}
- %ret = amdgpu.packed_trunc_2xfp8 %v, undef into undef[word 0] : f32 to vector<4xf8E4M3FN>
- func.return %ret : vector<4xf8E4M3FN>
-}
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/8-bit-floats-ocp.mlir b/mlir/test/Conversion/AMDGPUToROCDL/8-bit-floats-ocp.mlir
index 464d47216c81b..83a86086a9f55 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/8-bit-floats-ocp.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/8-bit-floats-ocp.mlir
@@ -1,5 +1,7 @@
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx950 | FileCheck %s
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx1200 | FileCheck %s
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu9.50-amd-amdhsa | FileCheck %s
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu12.00-amd-amdhsa | FileCheck %s
+// gfx11.7 has the OCP fp8 conversions too.
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu11.70-amd-amdhsa | FileCheck %s
// CHECK-LABEL: func @ext_scalar
// CHECK: [[V:%.+]] = builtin.unrealized_conversion_cast %{{.+}} : f8E5M2 to i8
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/8-bit-floats.mlir b/mlir/test/Conversion/AMDGPUToROCDL/8-bit-floats.mlir
index 03fcb266a2e87..5951ff4de9a74 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/8-bit-floats.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/8-bit-floats.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx942 | FileCheck %s
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa | FileCheck %s
// CHECK-LABEL: func @ext_scalar
// CHECK: [[V:%.+]] = builtin.unrealized_conversion_cast %{{.+}} : f8E5M2FNUZ to i8
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/amdgpu-to-rocdl.mlir b/mlir/test/Conversion/AMDGPUToROCDL/amdgpu-to-rocdl.mlir
index 8086aa788c8ad..d70a7d7b4167c 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/amdgpu-to-rocdl.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/amdgpu-to-rocdl.mlir
@@ -1,10 +1,10 @@
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx908 | FileCheck %s --check-prefixes=CHECK,RECORDS32,GFX9,GFX908
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx90a | FileCheck %s --check-prefixes=CHECK,RECORDS32,GFX9,GFX90A
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx942 | FileCheck %s --check-prefixes=CHECK,RECORDS32,GFX9,GFX942
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx1030 | FileCheck %s --check-prefixes=CHECK,RECORDS32,GFX10,RDNA
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx1100 | FileCheck %s --check-prefixes=CHECK,RECORDS32,GFX11,RDNA
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx1201 | FileCheck %s --check-prefixes=CHECK,RECORDS32,GFX12,RDNA
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx1250 | FileCheck %s --check-prefixes=CHECK,RECORDS45,GFX1250
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu9.08-amd-amdhsa | FileCheck %s --check-prefixes=CHECK,RECORDS32,GFX9,GFX908
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu9.0a-amd-amdhsa | FileCheck %s --check-prefixes=CHECK,RECORDS32,GFX9,GFX90A
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa | FileCheck %s --check-prefixes=CHECK,RECORDS32,GFX9,GFX942
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu10.30-amd-amdhsa | FileCheck %s --check-prefixes=CHECK,RECORDS32,GFX10,RDNA
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu11.00-amd-amdhsa | FileCheck %s --check-prefixes=CHECK,RECORDS32,GFX11,RDNA
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu12.01-amd-amdhsa | FileCheck %s --check-prefixes=CHECK,RECORDS32,GFX12,RDNA
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu12.50-amd-amdhsa | FileCheck %s --check-prefixes=CHECK,RECORDS45,GFX1250
// CHECK: #[[$MMRA_TAG:.+]] = #llvm.mmra_tag<"amdgpu-synchronize-as":"local">
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/dot-gfx11.mlir b/mlir/test/Conversion/AMDGPUToROCDL/dot-gfx11.mlir
index a87227884dc2a..d6ec9229817d4 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/dot-gfx11.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/dot-gfx11.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx1100 | FileCheck %s
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu11.00-amd-amdhsa | FileCheck %s
// CHECK-LABEL: @dot_fdot2_f16_f16
func.func @dot_fdot2_f16_f16(%a: vector<2xf16>, %b: vector<2xf16>, %c: f16) -> f16 {
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/dot-gfx12.mlir b/mlir/test/Conversion/AMDGPUToROCDL/dot-gfx12.mlir
index 3213b5fa8f5c2..8363f27067b48 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/dot-gfx12.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/dot-gfx12.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx1200 | FileCheck %s
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu12.00-amd-amdhsa | FileCheck %s
// CHECK-LABEL: @dot_fp8_fp8
func.func @dot_fp8_fp8(%a: vector<4xf8E4M3FN>, %b: vector<4xf8E4M3FN>, %c: f32) -> f32 {
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/dot-gfx9.mlir b/mlir/test/Conversion/AMDGPUToROCDL/dot-gfx9.mlir
index e13a9976974dc..3eb512951e74f 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/dot-gfx9.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/dot-gfx9.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx906 | FileCheck %s
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu9.06-amd-amdhsa | FileCheck %s
// CHECK-LABEL: @dot_fdot2
func.func @dot_fdot2(%a: vector<2xf16>, %b: vector<2xf16>, %c: f32) -> f32 {
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/dot-invalid.mlir b/mlir/test/Conversion/AMDGPUToROCDL/dot-invalid.mlir
index dd26ab7040734..33ec036e563b4 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/dot-invalid.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/dot-invalid.mlir
@@ -1,5 +1,5 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx906 --split-input-file -verify-diagnostics
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx942 --split-input-file -verify-diagnostics
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu9.06-amd-amdhsa --split-input-file -verify-diagnostics
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa --split-input-file -verify-diagnostics
// fp8 dot4 is only available on gfx12+.
func.func @dot_fp8_requires_gfx12(%a: vector<4xf8E4M3FN>, %b: vector<4xf8E4M3FN>, %c: f32) -> f32 {
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/dpp.mlir b/mlir/test/Conversion/AMDGPUToROCDL/dpp.mlir
index a4c98111c2956..bc9f277e275a0 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/dpp.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/dpp.mlir
@@ -1,6 +1,6 @@
-// RUN: mlir-opt -convert-amdgpu-to-rocdl=chipset=gfx908 %s | FileCheck %s
-// RUN: mlir-opt -convert-amdgpu-to-rocdl=chipset=gfx90a %s | FileCheck %s
-// RUN: mlir-opt -convert-amdgpu-to-rocdl=chipset=gfx942 %s | FileCheck %s
+// RUN: mlir-opt -convert-amdgpu-to-rocdl=triple=amdgpu9.08-amd-amdhsa %s | FileCheck %s
+// RUN: mlir-opt -convert-amdgpu-to-rocdl=triple=amdgpu9.0a-amd-amdhsa %s | FileCheck %s
+// RUN: mlir-opt -convert-amdgpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa %s | FileCheck %s
func.func @test_dpp(%arg0: i32, %arg1: i32) -> i32 {
// CHECK-LABEL: func @test_dpp
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/gfx1250.mlir b/mlir/test/Conversion/AMDGPUToROCDL/gfx1250.mlir
index bc67513ff02a6..caa6b7fa8088b 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/gfx1250.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/gfx1250.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx1250 --split-input-file --verify-diagnostics \
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu12.50-amd-amdhsa --split-input-file --verify-diagnostics \
// RUN: | FileCheck %s
// CHECK-LABEL: @scaled_ext_packed_matrix_fp4
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/global-prefetch.mlir b/mlir/test/Conversion/AMDGPUToROCDL/global-prefetch.mlir
index f71de64cd071f..7c8d431e2090c 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/global-prefetch.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/global-prefetch.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx1250 --split-input-file --verify-diagnostics | FileCheck %s
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu12.50-amd-amdhsa --split-input-file --verify-diagnostics | FileCheck %s
// CHECK-LABEL: @glb_prefetch0
func.func @glb_prefetch0(%src : memref<64x64xf16, #gpu.address_space<global>>, %i : i64, %j : i64) {
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/global_transpose_load.mlir b/mlir/test/Conversion/AMDGPUToROCDL/global_transpose_load.mlir
index f378d7232d7b3..92f4e582f7f81 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/global_transpose_load.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/global_transpose_load.mlir
@@ -1,6 +1,6 @@
-// RUN: mlir-opt %s --split-input-file --verify-diagnostics -convert-amdgpu-to-rocdl=chipset=gfx1201 | FileCheck %s
-// RUN: mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=chipset=gfx1250 | FileCheck %s --check-prefixes=CHECK,CHECK-GFX1250
-// RUN: not mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=chipset=gfx942 2>&1 | FileCheck %s --check-prefix=CHECK-OLD
+// RUN: mlir-opt %s --split-input-file --verify-diagnostics -convert-amdgpu-to-rocdl=triple=amdgpu12.01-amd-amdhsa | FileCheck %s
+// RUN: mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=triple=amdgpu12.50-amd-amdhsa | FileCheck %s --check-prefixes=CHECK,CHECK-GFX1250
+// RUN: not mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa 2>&1 | FileCheck %s --check-prefix=CHECK-OLD
// CHECK-LABEL: func @global_transpose_load_8xf16
func.func @global_transpose_load_8xf16(%i : index, %j : index,
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/lds-barrier-gfx90c.mlir b/mlir/test/Conversion/AMDGPUToROCDL/lds-barrier-gfx90c.mlir
index d1c9919eb0d59..4309c4ed5e36b 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/lds-barrier-gfx90c.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/lds-barrier-gfx90c.mlir
@@ -1,14 +1,15 @@
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx90c | FileCheck %s
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu9.0c-amd-amdhsa | FileCheck %s
-// gfx90c sorts after gfx90a, so the version comparison guarding the inline asm
-// workaround treats it as having the hardware barrier back-off. It does not:
-// gfx90c is a Renoir-class APU and lacks FeatureBackOffBarrier, so a bare
-// s_barrier lets waits on global memory be introduced around the barrier.
+// gfx90c sorts after gfx90a, so the version comparison that used to guard the
+// inline asm workaround treated it as having the hardware barrier back-off. It
+// does not: gfx90c is a Renoir-class APU and lacks FeatureBackOffBarrier, so a
+// bare s_barrier lets waits on global memory be introduced around the barrier.
// CHECK-LABEL: func @lds_barrier
func.func @lds_barrier() {
// CHECK: llvm.fence syncscope("workgroup") release
- // CHECK-NEXT: rocdl.s.barrier
+ // CHECK-NEXT: llvm.inline_asm has_side_effects asm_dialect = att
+ // CHECK-SAME: ";;;WARNING: BREAKS DEBUG WATCHES\0As_barrier"
// CHECK-NEXT: llvm.fence syncscope("workgroup") acquire
amdgpu.lds_barrier
func.return
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/load_lds-gfx950.mlir b/mlir/test/Conversion/AMDGPUToROCDL/load_lds-gfx950.mlir
index ff8f19c33a437..4b0b3ec36e7c8 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/load_lds-gfx950.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/load_lds-gfx950.mlir
@@ -1,5 +1,5 @@
-// RUN: not mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=chipset=gfx942 2>&1 | FileCheck %s --check-prefix=GFX942
-// RUN: mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=chipset=gfx950 | FileCheck %s --check-prefix=GFX950
+// RUN: not mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa 2>&1 | FileCheck %s --check-prefix=GFX942
+// RUN: mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=triple=amdgpu9.50-amd-amdhsa | FileCheck %s --check-prefix=GFX950
// GFX950-LABEL: func @fat_buffer_load_to_rocdl_f96
// GFX950-SAME: (%[[ARG0:.*]]: memref<128x72xf32, #amdgpu.address_space<fat_raw_buffer>>)
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/load_lds.mlir b/mlir/test/Conversion/AMDGPUToROCDL/load_lds.mlir
index c2783c216d66d..e3069d14336b3 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/load_lds.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/load_lds.mlir
@@ -1,5 +1,5 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx942 | FileCheck %s
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx950 | FileCheck %s
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa | FileCheck %s
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu9.50-amd-amdhsa | FileCheck %s
// CHECK-LABEL: func @global_load_to_rocdl_f32
// CHECK-SAME: (%[[ARG0:.*]]: memref<128x72xf32, #gpu.address_space<global>>)
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/memory_counter_wait.mlir b/mlir/test/Conversion/AMDGPUToROCDL/memory_counter_wait.mlir
index 537ef59b503a6..e0d2a95f36273 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/memory_counter_wait.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/memory_counter_wait.mlir
@@ -1,7 +1,7 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx942 | FileCheck %s --check-prefixes=CHECK,GFX9
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx1030 | FileCheck %s --check-prefixes=CHECK,GFX10
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx1100 | FileCheck %s --check-prefixes=CHECK,GFX11
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx1201 | FileCheck %s --check-prefixes=CHECK,GFX12
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa | FileCheck %s --check-prefixes=CHECK,GFX9
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu10.30-amd-amdhsa | FileCheck %s --check-prefixes=CHECK,GFX10
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu11.00-amd-amdhsa | FileCheck %s --check-prefixes=CHECK,GFX11
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu12.01-amd-amdhsa | FileCheck %s --check-prefixes=CHECK,GFX12
// CHECK-LABEL: func @memory_counter_wait
func.func @memory_counter_wait() {
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/memory_counter_wait_tensor.mlir b/mlir/test/Conversion/AMDGPUToROCDL/memory_counter_wait_tensor.mlir
index 5b29e01abebdb..7b781ea501bf5 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/memory_counter_wait_tensor.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/memory_counter_wait_tensor.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx1250 | FileCheck %s
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu12.50-amd-amdhsa | FileCheck %s
// CHECK-LABEL: func @memory_counter_wait_tensor
func.func @memory_counter_wait_tensor() {
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/memory_counter_wait_unsupported.mlir b/mlir/test/Conversion/AMDGPUToROCDL/memory_counter_wait_unsupported.mlir
index 1d2f692bee488..3bb7653cf9a1c 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/memory_counter_wait_unsupported.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/memory_counter_wait_unsupported.mlir
@@ -1,6 +1,6 @@
-// RUN: mlir-opt %s --verify-diagnostics --convert-amdgpu-to-rocdl=chipset=gfx942
-// RUN: mlir-opt %s --verify-diagnostics --convert-amdgpu-to-rocdl=chipset=gfx1030
-// RUN: mlir-opt %s --verify-diagnostics --convert-amdgpu-to-rocdl=chipset=gfx1100
+// RUN: mlir-opt %s --verify-diagnostics --convert-amdgpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa
+// RUN: mlir-opt %s --verify-diagnostics --convert-amdgpu-to-rocdl=triple=amdgpu10.30-amd-amdhsa
+// RUN: mlir-opt %s --verify-diagnostics --convert-amdgpu-to-rocdl=triple=amdgpu11.00-amd-amdhsa
func.func @memory_counter_wait_tensor() {
// expected-error @below{{failed to legalize operation 'amdgpu.memory_counter_wait'}}
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/mfma-fp8-invalid.mlir b/mlir/test/Conversion/AMDGPUToROCDL/mfma-fp8-invalid.mlir
new file mode 100644
index 0000000000000..c8cd6ee8a9fcd
--- /dev/null
+++ b/mlir/test/Conversion/AMDGPUToROCDL/mfma-fp8-invalid.mlir
@@ -0,0 +1,21 @@
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu9.08-amd-amdhsa --split-input-file --verify-diagnostics
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu9.0a-amd-amdhsa --split-input-file --verify-diagnostics
+
+// gfx908 and gfx90a have FeatureMAIInsts but no fp8 conversions at all, so the
+// fp8 MFMAs -- which first appear on gfx942 -- must not be selected for them.
+
+func.func @mfma_bf8(%arg0 : vector<8xf8E5M2FNUZ>, %arg1 : vector<4xf32>) {
+ // expected-error at below {{op no intrinsic matching MFMA size on given chipset}}
+ // expected-error at below {{failed to legalize operation 'amdgpu.mfma'}}
+ amdgpu.mfma 16x16x32 %arg0 * %arg0 + %arg1 : vector<8xf8E5M2FNUZ>, vector<8xf8E5M2FNUZ>, vector<4xf32>
+ func.return
+}
+
+// -----
+
+func.func @mfma_fp8(%arg0 : vector<8xf8E4M3FNUZ>, %arg1 : vector<4xf32>) {
+ // expected-error at below {{op no intrinsic matching MFMA size on given chipset}}
+ // expected-error at below {{failed to legalize operation 'amdgpu.mfma'}}
+ amdgpu.mfma 16x16x32 %arg0 * %arg0 + %arg1 : vector<8xf8E4M3FNUZ>, vector<8xf8E4M3FNUZ>, vector<4xf32>
+ func.return
+}
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/mfma-gfx950.mlir b/mlir/test/Conversion/AMDGPUToROCDL/mfma-gfx950.mlir
index d124c33f19144..b4a6395169ae6 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/mfma-gfx950.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/mfma-gfx950.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx950 -cse | FileCheck %s
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu9.50-amd-amdhsa -cse | FileCheck %s
func.func @mfma_to_rocdl(%arg0 : vector<8xf16>, %arg1 : vector<16xf32>,
%arg2 : vector<4xf32>, %arg3 : vector<8xbf16>,
%arg4 : vector<16xi8>, %arg5 : vector<16xi32>,
@@ -96,17 +96,3 @@ func.func @scaled_mfma_to_rocdl(%arg0 : vector<16xf32>,
func.return
}
-
-// gfx950 does not have the xf32 MFMAs -- FeatureXF32Insts is set on gfx942
-// only -- but it compares greater than gfx942 by ISA version, so the
-// reduced-precision f32 MFMAs are currently selected for it.
-// CHECK-LABEL: func @mfma_reduce_precision_to_rocdl
-func.func @mfma_reduce_precision_to_rocdl(%arg0 : vector<2xf32>,
- %arg1 : vector<16xf32>,
- %arg2 : vector<4xf32>) {
- // CHECK: rocdl.mfma.f32.32x32x4.xf32
- amdgpu.mfma 32x32x4 %arg0 * %arg0 + %arg1 reducePrecision : vector<2xf32>, vector<2xf32>, vector<16xf32>
- // CHECK: rocdl.mfma.f32.16x16x8.xf32
- amdgpu.mfma 16x16x8 %arg0 * %arg0 + %arg2 reducePrecision : vector<2xf32>, vector<2xf32>, vector<4xf32>
- func.return
-}
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/mfma-reduce-precision-invalid.mlir b/mlir/test/Conversion/AMDGPUToROCDL/mfma-reduce-precision-invalid.mlir
new file mode 100644
index 0000000000000..f13b1820dad71
--- /dev/null
+++ b/mlir/test/Conversion/AMDGPUToROCDL/mfma-reduce-precision-invalid.mlir
@@ -0,0 +1,23 @@
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu9.50-amd-amdhsa --split-input-file --verify-diagnostics
+
+// The xf32 MFMAs come from FeatureXF32Insts, which only gfx942 has. gfx950
+// compares greater than gfx942 by ISA version, so a version-ordered check let
+// them through here.
+
+func.func @mfma_reduce_precision_32x32x4(%arg0 : vector<2xf32>,
+ %arg1 : vector<16xf32>) {
+ // expected-error at below {{op no intrinsic matching MFMA size on given chipset}}
+ // expected-error at below {{failed to legalize operation 'amdgpu.mfma'}}
+ amdgpu.mfma 32x32x4 %arg0 * %arg0 + %arg1 reducePrecision : vector<2xf32>, vector<2xf32>, vector<16xf32>
+ func.return
+}
+
+// -----
+
+func.func @mfma_reduce_precision_16x16x8(%arg0 : vector<2xf32>,
+ %arg1 : vector<4xf32>) {
+ // expected-error at below {{op no intrinsic matching MFMA size on given chipset}}
+ // expected-error at below {{failed to legalize operation 'amdgpu.mfma'}}
+ amdgpu.mfma 16x16x8 %arg0 * %arg0 + %arg1 reducePrecision : vector<2xf32>, vector<2xf32>, vector<4xf32>
+ func.return
+}
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/mfma.mlir b/mlir/test/Conversion/AMDGPUToROCDL/mfma.mlir
index 464d2d20048a2..996585ddf230a 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/mfma.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/mfma.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx942 -cse | FileCheck %s
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa -cse | FileCheck %s
func.func @mfma_to_rocdl(%arg0 : f32, %arg1 : vector<32xf32>,
%arg2 : vector<16xf32>, %arg3 : vector<4xf32>,
%arg4 : vector<4xf16>, %arg5 : vector<4xi8>,
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/packed-ext.mlir b/mlir/test/Conversion/AMDGPUToROCDL/packed-ext.mlir
index ad2e7684afc4a..98d2078294346 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/packed-ext.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/packed-ext.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx950 | FileCheck %s
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu9.50-amd-amdhsa | FileCheck %s
// CHECK-LABEL: func.func @scaled_ext_full_f8e4m3_f32
// CHECK-DAG: [[CAST:%.+]] = builtin.unrealized_conversion_cast %arg0 : vector<4xf8E4M3FN> to vector<4xi8>
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/packed-trunc-invalid.mlir b/mlir/test/Conversion/AMDGPUToROCDL/packed-trunc-invalid.mlir
index 93bfc60ce8f4b..e17d6ec7f3617 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/packed-trunc-invalid.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/packed-trunc-invalid.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx942 --split-input-file --verify-diagnostics
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa --split-input-file --verify-diagnostics
func.func @packed_trunc_ocp_type_requires_ocp_chipset(%arg0: f32) {
// expected-error at below {{'amdgpu.packed_trunc_2xfp8' op no truncation to result type available on given chipset}}
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/packed-trunc.mlir b/mlir/test/Conversion/AMDGPUToROCDL/packed-trunc.mlir
index e9764d34cefaf..e786abda2c2f3 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/packed-trunc.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/packed-trunc.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx950 | FileCheck %s
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu9.50-amd-amdhsa | FileCheck %s
// CHECK-LABEL: func.func @packed_scaled_trunc_f8e4m3_f32
// CHECK-DAG: [[ZERO:%.+]] = llvm.mlir.zero : vector<2xi16>
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/permlane-gfx1200-invalid.mlir b/mlir/test/Conversion/AMDGPUToROCDL/permlane-gfx1200-invalid.mlir
new file mode 100644
index 0000000000000..aba29f5883009
--- /dev/null
+++ b/mlir/test/Conversion/AMDGPUToROCDL/permlane-gfx1200-invalid.mlir
@@ -0,0 +1,21 @@
+// RUN: mlir-opt --convert-amdgpu-to-rocdl=triple=amdgpu12.00-amd-amdhsa --split-input-file --verify-diagnostics %s
+
+// gfx1200 has neither FeaturePermlane16Swap nor FeaturePermlane32Swap, but
+// compares greater than gfx950 by ISA version, so a version-ordered check
+// accepted both widths.
+
+func.func @permlane16(%arg0 : i32) -> i32 {
+ // expected-error at below {{op permlane_swap of row length 16 is not supported}}
+ // expected-error at below {{failed to legalize operation 'amdgpu.permlane_swap'}}
+ %0 = amdgpu.permlane_swap %arg0 16 : i32
+ return %0 : i32
+}
+
+// -----
+
+func.func @permlane32(%arg0 : i32) -> i32 {
+ // expected-error at below {{op permlane_swap of row length 32 is not supported}}
+ // expected-error at below {{failed to legalize operation 'amdgpu.permlane_swap'}}
+ %0 = amdgpu.permlane_swap %arg0 32 : i32
+ return %0 : i32
+}
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/permlane-gfx1250-invalid.mlir b/mlir/test/Conversion/AMDGPUToROCDL/permlane-gfx1250-invalid.mlir
new file mode 100644
index 0000000000000..0bec75f8a2fc6
--- /dev/null
+++ b/mlir/test/Conversion/AMDGPUToROCDL/permlane-gfx1250-invalid.mlir
@@ -0,0 +1,11 @@
+// RUN: mlir-opt --convert-amdgpu-to-rocdl=triple=amdgpu12.50-amd-amdhsa --split-input-file --verify-diagnostics %s
+
+// gfx1250 has FeaturePermlane16Swap but not FeaturePermlane32Swap; the 16-wide
+// form is covered as a positive case in permlane.mlir.
+
+func.func @permlane32(%arg0 : i32) -> i32 {
+ // expected-error at below {{op permlane_swap of row length 32 is not supported}}
+ // expected-error at below {{failed to legalize operation 'amdgpu.permlane_swap'}}
+ %0 = amdgpu.permlane_swap %arg0 32 : i32
+ return %0 : i32
+}
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/permlane-gfx1250.mlir b/mlir/test/Conversion/AMDGPUToROCDL/permlane-gfx1250.mlir
new file mode 100644
index 0000000000000..9b1be3810f21e
--- /dev/null
+++ b/mlir/test/Conversion/AMDGPUToROCDL/permlane-gfx1250.mlir
@@ -0,0 +1,12 @@
+// RUN: mlir-opt --convert-amdgpu-to-rocdl=triple=amdgpu12.50-amd-amdhsa --canonicalize %s | FileCheck %s
+
+// gfx1250 has FeaturePermlane16Swap. It does not have FeaturePermlane32Swap;
+// see permlane-gfx1250-invalid.mlir.
+
+// CHECK-LABEL: func @permlane16_i32
+// CHECK-SAME: (%[[ARG0:.*]]: i32)
+func.func @permlane16_i32(%arg0 : i32) -> i32 {
+// CHECK: %[[PERM:.*]] = rocdl.permlane16.swap %[[ARG0]], %[[ARG0]], false, false : (i32, i32) -> <(i32, i32)>
+ %0 = amdgpu.permlane_swap %arg0 16 : i32
+ return %0 : i32
+}
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/permlane-var.mlir b/mlir/test/Conversion/AMDGPUToROCDL/permlane-var.mlir
index e6e5a3061be5b..07dde24f5a129 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/permlane-var.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/permlane-var.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt --convert-amdgpu-to-rocdl=chipset=gfx1200 --canonicalize %s | FileCheck %s
+// RUN: mlir-opt --convert-amdgpu-to-rocdl=triple=amdgpu12.00-amd-amdhsa --canonicalize %s | FileCheck %s
// CHECK-LABEL: func @test_permlane_var_i32
// CHECK-SAME: (%[[SRC:.*]]: i32, %[[SEL:.*]]: i32)
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/permlane.mlir b/mlir/test/Conversion/AMDGPUToROCDL/permlane.mlir
index a8643604abb5f..03095164af58a 100755
--- a/mlir/test/Conversion/AMDGPUToROCDL/permlane.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/permlane.mlir
@@ -1,8 +1,4 @@
-// RUN: mlir-opt --convert-amdgpu-to-rocdl=chipset=gfx950 --canonicalize %s | FileCheck %s
-// The permlane swaps come from FeaturePermlane16Swap/FeaturePermlane32Swap,
-// which gfx1200 does not have -- but it compares greater than gfx950 by ISA
-// version, so `chipset < kGfx950` lets it through and it lowers identically.
-// RUN: mlir-opt --convert-amdgpu-to-rocdl=chipset=gfx1200 --canonicalize %s | FileCheck %s
+// RUN: mlir-opt --convert-amdgpu-to-rocdl=triple=amdgpu9.50-amd-amdhsa --canonicalize %s | FileCheck %s
// CHECK-LABEL: func @test_permlane16_i32
// CHECK-SAME: (%[[ARG0:.*]]: i32)
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/sparse-mfma-gfx950.mlir b/mlir/test/Conversion/AMDGPUToROCDL/sparse-mfma-gfx950.mlir
index abdfba9689c8f..aa17355a5dbfd 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/sparse-mfma-gfx950.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/sparse-mfma-gfx950.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx950 -cse | FileCheck %s
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu9.50-amd-amdhsa -cse | FileCheck %s
func.func @sparse_mfma_to_rocdl(%arg0 : vector<8xf16>, %arg1 : vector<16xf16>,
%arg2 : vector<4xf32>, %arg3 : vector<16xf32>,
%arg4 : vector<8xbf16>, %arg5 : vector<16xbf16>,
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/sparse-mfma.mlir b/mlir/test/Conversion/AMDGPUToROCDL/sparse-mfma.mlir
index 304f90351faf8..19217a51aef97 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/sparse-mfma.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/sparse-mfma.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=chipset=gfx942 -cse | FileCheck %s
+// RUN: mlir-opt %s -convert-amdgpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa -cse | FileCheck %s
func.func @sparse_mfma_to_rocdl(%arg0 : vector<4xf16>, %arg1 : vector<8xf16>,
%arg2 : vector<4xf32>, %arg3 : vector<16xf32>,
%arg4 : vector<4xbf16>, %arg5 : vector<8xbf16>,
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/swizzle.mlir b/mlir/test/Conversion/AMDGPUToROCDL/swizzle.mlir
index ef439efde1bd0..396f6f17c1875 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/swizzle.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/swizzle.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt -convert-amdgpu-to-rocdl --canonicalize %s | FileCheck %s
+// RUN: mlir-opt -convert-amdgpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa --canonicalize %s | FileCheck %s
// CHECK-LABEL: func @test_swizzle_i32
// CHECK-SAME: (%[[ARG0:.*]]: i32)
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/swmmac-gfx12.mlir b/mlir/test/Conversion/AMDGPUToROCDL/swmmac-gfx12.mlir
index 61d533b75907d..d918df07bc4a5 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/swmmac-gfx12.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/swmmac-gfx12.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx1200 --split-input-file --verify-diagnostics | FileCheck %s
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu12.00-amd-amdhsa --split-input-file --verify-diagnostics | FileCheck %s
// CHECK-LABEL: @rocdl.swmmac
func.func @rocdl.swmmac(
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/swmmac-gfx1250.mlir b/mlir/test/Conversion/AMDGPUToROCDL/swmmac-gfx1250.mlir
index 155e36c369a88..e831f86290054 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/swmmac-gfx1250.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/swmmac-gfx1250.mlir
@@ -1,5 +1,5 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx1250 --split-input-file --verify-diagnostics | FileCheck %s
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu12.50-amd-amdhsa --split-input-file --verify-diagnostics | FileCheck %s
// CHECK-LABEL: @rocdl.swmmac
func.func @rocdl.swmmac(
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/transpose_load.mlir b/mlir/test/Conversion/AMDGPUToROCDL/transpose_load.mlir
index dcc6624cdb37b..7229ff526ab0b 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/transpose_load.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/transpose_load.mlir
@@ -1,5 +1,5 @@
-// RUN: mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=chipset=gfx950 | FileCheck %s
-// RUN: not mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=chipset=gfx942 2>&1 | FileCheck %s --check-prefix=CHECK-OLD
+// RUN: mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=triple=amdgpu9.50-amd-amdhsa | FileCheck %s
+// RUN: not mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa 2>&1 | FileCheck %s --check-prefix=CHECK-OLD
// CHECK-LABEL: func @transpose_load_to_rocdl_4xf16
func.func @transpose_load_to_rocdl_4xf16(%idx1 : index, %idx2 : index, %wgmem : memref<128x72xf16, 3>) -> vector<4xf16> {
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_gfx1250.mlir b/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_gfx1250.mlir
index 98ce6b7ea3001..21f656b1f4850 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_gfx1250.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_gfx1250.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=chipset=gfx1250 | FileCheck %s
+// RUN: mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=triple=amdgpu12.50-amd-amdhsa | FileCheck %s
// CHECK-LABEL: func @transpose_load_to_rocdl_8xf16
func.func @transpose_load_to_rocdl_8xf16(%idx1 : index, %idx2 : index, %wgmem : memref<128x72xf16, 3>) -> vector<8xf16> {
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_gfx1250_invalid.mlir b/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_gfx1250_invalid.mlir
index 61acdfe245c7c..1f1357182784b 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_gfx1250_invalid.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_gfx1250_invalid.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s --split-input-file --verify-diagnostics -convert-amdgpu-to-rocdl=chipset=gfx1250
+// RUN: mlir-opt %s --split-input-file --verify-diagnostics -convert-amdgpu-to-rocdl=triple=amdgpu12.50-amd-amdhsa
func.func @transpose_load_to_rocdl_4xf16(%idx1 : index, %idx2 : index, %wgmem : memref<128x72xf16, 3>) -> vector<4xf16> {
// expected-error at +2 {{'amdgpu.transpose_load' op 16-bit transpose_load requires 8 elements on gfx1250+}}
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_gfx950_invalid.mlir b/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_gfx950_invalid.mlir
index 682f989ede83b..acfe1c2a056b3 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_gfx950_invalid.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_gfx950_invalid.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s --split-input-file --verify-diagnostics -convert-amdgpu-to-rocdl=chipset=gfx950
+// RUN: mlir-opt %s --split-input-file --verify-diagnostics -convert-amdgpu-to-rocdl=triple=amdgpu9.50-amd-amdhsa
func.func @transpose_load_to_rocdl_8xf16(%idx1 : index, %idx2 : index, %wgmem : memref<128x72xf16, 3>) -> vector<8xf16> {
// expected-error at +2 {{'amdgpu.transpose_load' op 16-bit transpose_load requires 4 elements on gfx950}}
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_reject.mlir b/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_reject.mlir
index a41051c904ed8..f05feedcef9c9 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_reject.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/transpose_load_reject.mlir
@@ -1,4 +1,4 @@
-// RUN: not mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=chipset=gfx950 2>&1 | FileCheck %s
+// RUN: not mlir-opt %s --split-input-file -convert-amdgpu-to-rocdl=triple=amdgpu9.50-amd-amdhsa 2>&1 | FileCheck %s
// -----
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/wmma-gfx11.mlir b/mlir/test/Conversion/AMDGPUToROCDL/wmma-gfx11.mlir
index 08fd68dfe158d..c4faad55222df 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/wmma-gfx11.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/wmma-gfx11.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx1100 | FileCheck %s
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu11.00-amd-amdhsa | FileCheck %s
// CHECK-LABEL: @wmma_to_rocdl
func.func @wmma_to_rocdl(%arg0 : vector<16xf16>, %arg1 : vector<8xf32>, %arg2 : vector<4xf32>,
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/wmma-gfx12.mlir b/mlir/test/Conversion/AMDGPUToROCDL/wmma-gfx12.mlir
index 1dac83946fc4c..7dcdc130a3b44 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/wmma-gfx12.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/wmma-gfx12.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx1200 \
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu12.00-amd-amdhsa \
// RUN: --split-input-file --verify-diagnostics | FileCheck %s
// CHECK-LABEL: @wmma_to_rocdl
diff --git a/mlir/test/Conversion/AMDGPUToROCDL/wmma-gfx1250.mlir b/mlir/test/Conversion/AMDGPUToROCDL/wmma-gfx1250.mlir
index 7f9605ad1a7eb..e8d8630d9c1eb 100644
--- a/mlir/test/Conversion/AMDGPUToROCDL/wmma-gfx1250.mlir
+++ b/mlir/test/Conversion/AMDGPUToROCDL/wmma-gfx1250.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=chipset=gfx1250 \
+// RUN: mlir-opt %s --convert-amdgpu-to-rocdl=triple=amdgpu12.50-amd-amdhsa \
// RUN: --split-input-file --verify-diagnostics | FileCheck %s
// CHECK-LABEL: @wmma_k4
diff --git a/mlir/test/Conversion/ArithToAMDGPU/16-bit-floats.mlir b/mlir/test/Conversion/ArithToAMDGPU/16-bit-floats.mlir
index 6077ef349408f..87f5099c91f9f 100644
--- a/mlir/test/Conversion/ArithToAMDGPU/16-bit-floats.mlir
+++ b/mlir/test/Conversion/ArithToAMDGPU/16-bit-floats.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="allow-packed-f16-round-to-zero=true" | FileCheck %s
+// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="triple=amdgpu9.42-amd-amdhsa allow-packed-f16-round-to-zero=true" | FileCheck %s
// CHECK-LABEL: @scalar_trunc
// CHECK-SAME: (%[[value:.*]]: f32)
diff --git a/mlir/test/Conversion/ArithToAMDGPU/8-bit-float-saturation-ocp.mlir b/mlir/test/Conversion/ArithToAMDGPU/8-bit-float-saturation-ocp.mlir
index e3c2ae9515939..0fb7a6b4bca7d 100644
--- a/mlir/test/Conversion/ArithToAMDGPU/8-bit-float-saturation-ocp.mlir
+++ b/mlir/test/Conversion/ArithToAMDGPU/8-bit-float-saturation-ocp.mlir
@@ -1,9 +1,9 @@
// RUN: mlir-opt --split-input-file %s \
-// RUN: --pass-pipeline='builtin.module(func.func(convert-arith-to-amdgpu{chipset=gfx950 saturate-fp8-truncf=true}))' \
+// RUN: --pass-pipeline='builtin.module(func.func(convert-arith-to-amdgpu{triple=amdgpu9.50-amd-amdhsa saturate-fp8-truncf=true}))' \
// RUN: | FileCheck %s
// RUN: mlir-opt --split-input-file %s \
-// RUN: --pass-pipeline='builtin.module(func.func(convert-arith-to-amdgpu{chipset=gfx1200 saturate-fp8-truncf=true}))' \
+// RUN: --pass-pipeline='builtin.module(func.func(convert-arith-to-amdgpu{triple=amdgpu12.00-amd-amdhsa saturate-fp8-truncf=true}))' \
// RUN: | FileCheck %s
// CHECK-LABEL: func.func @scalar_trunc
diff --git a/mlir/test/Conversion/ArithToAMDGPU/8-bit-float-saturation.mlir b/mlir/test/Conversion/ArithToAMDGPU/8-bit-float-saturation.mlir
index b6eabe391c0cd..21ec05bf3f73a 100644
--- a/mlir/test/Conversion/ArithToAMDGPU/8-bit-float-saturation.mlir
+++ b/mlir/test/Conversion/ArithToAMDGPU/8-bit-float-saturation.mlir
@@ -1,5 +1,5 @@
// RUN: mlir-opt --split-input-file %s \
-// RUN: --pass-pipeline='builtin.module(func.func(convert-arith-to-amdgpu{chipset=gfx942 saturate-fp8-truncf=true}))' \
+// RUN: --pass-pipeline='builtin.module(func.func(convert-arith-to-amdgpu{triple=amdgpu9.42-amd-amdhsa saturate-fp8-truncf=true}))' \
// RUN: | FileCheck %s
// CHECK-LABEL: func.func @scalar_trunc
diff --git a/mlir/test/Conversion/ArithToAMDGPU/8-bit-floats-ocp.mlir b/mlir/test/Conversion/ArithToAMDGPU/8-bit-floats-ocp.mlir
index 91a9a57898761..8e68192cdeb9c 100644
--- a/mlir/test/Conversion/ArithToAMDGPU/8-bit-floats-ocp.mlir
+++ b/mlir/test/Conversion/ArithToAMDGPU/8-bit-floats-ocp.mlir
@@ -1,5 +1,5 @@
-// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="chipset=gfx950" | FileCheck %s
-// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="chipset=gfx1200" | FileCheck %s
+// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="triple=amdgpu9.50-amd-amdhsa" | FileCheck %s
+// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="triple=amdgpu12.00-amd-amdhsa" | FileCheck %s
// CHECK-LABEL: func.func @scalar_ext
// CHECK-SAME: ([[V:%.+]]: f8E5M2)
diff --git a/mlir/test/Conversion/ArithToAMDGPU/8-bit-floats.mlir b/mlir/test/Conversion/ArithToAMDGPU/8-bit-floats.mlir
index cf3133cf09add..a75316e13b515 100644
--- a/mlir/test/Conversion/ArithToAMDGPU/8-bit-floats.mlir
+++ b/mlir/test/Conversion/ArithToAMDGPU/8-bit-floats.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="chipset=gfx942" | FileCheck %s
+// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="triple=amdgpu9.42-amd-amdhsa" | FileCheck %s
// CHECK-LABEL: func.func @scalar_ext
// CHECK-SAME: ([[V:%.+]]: f8E5M2FNUZ)
diff --git a/mlir/test/Conversion/ArithToAMDGPU/scaling-extf.mlir b/mlir/test/Conversion/ArithToAMDGPU/scaling-extf.mlir
index fe5b0520e37c1..f6968c905b051 100644
--- a/mlir/test/Conversion/ArithToAMDGPU/scaling-extf.mlir
+++ b/mlir/test/Conversion/ArithToAMDGPU/scaling-extf.mlir
@@ -1,5 +1,5 @@
-// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="chipset=gfx950" | FileCheck %s
-// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="chipset=gfx1100" | FileCheck %s --check-prefix=CHECK-GFX1100
+// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="triple=amdgpu9.50-amd-amdhsa" | FileCheck %s
+// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="triple=amdgpu11.00-amd-amdhsa" | FileCheck %s --check-prefix=CHECK-GFX1100
// CHECK-LABEL: @conversion_f8_f32_fallback
// CHECK: %[[CST:.+]] = arith.constant dense<0.000000e+00> : vector<2x2xf32>
diff --git a/mlir/test/Conversion/ArithToAMDGPU/scaling-truncf-tensor.mlir b/mlir/test/Conversion/ArithToAMDGPU/scaling-truncf-tensor.mlir
index d22f35a6d07f1..39f000c913a5e 100644
--- a/mlir/test/Conversion/ArithToAMDGPU/scaling-truncf-tensor.mlir
+++ b/mlir/test/Conversion/ArithToAMDGPU/scaling-truncf-tensor.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -convert-arith-to-amdgpu=chipset=gfx950 | FileCheck %s
+// RUN: mlir-opt %s -convert-arith-to-amdgpu=triple=amdgpu9.50-amd-amdhsa | FileCheck %s
// CHECK-LABEL: func.func @m0
// CHECK: arith.scaling_truncf
diff --git a/mlir/test/Conversion/ArithToAMDGPU/scaling-truncf.mlir b/mlir/test/Conversion/ArithToAMDGPU/scaling-truncf.mlir
index 4c70768037815..76aad7cad29e1 100644
--- a/mlir/test/Conversion/ArithToAMDGPU/scaling-truncf.mlir
+++ b/mlir/test/Conversion/ArithToAMDGPU/scaling-truncf.mlir
@@ -1,5 +1,5 @@
-// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="chipset=gfx950" | FileCheck %s
-// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="chipset=gfx1100" | FileCheck %s --check-prefix=CHECK-GFX1100
+// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="triple=amdgpu9.50-amd-amdhsa" | FileCheck %s
+// RUN: mlir-opt --split-input-file %s -convert-arith-to-amdgpu="triple=amdgpu11.00-amd-amdhsa" | FileCheck %s --check-prefix=CHECK-GFX1100
// CHECK-LABEL: @conversion_f8_fallback
// CHECK-DAG: %[[CST:.+]] = arith.constant dense<0.000000e+00> : vector<2x2xf8E5M2>
diff --git a/mlir/test/Conversion/GPUCommon/lower-global-id.mlir b/mlir/test/Conversion/GPUCommon/lower-global-id.mlir
index 94b9f90052769..dbde317565e03 100644
--- a/mlir/test/Conversion/GPUCommon/lower-global-id.mlir
+++ b/mlir/test/Conversion/GPUCommon/lower-global-id.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -split-input-file -convert-gpu-to-rocdl | FileCheck %s --check-prefixes=ROCDL
+// RUN: mlir-opt %s -split-input-file -convert-gpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa | FileCheck %s --check-prefixes=ROCDL
// RUN: mlir-opt %s -split-input-file -convert-gpu-to-nvvm | FileCheck %s --check-prefixes=NVVM
gpu.module @kernel {
diff --git a/mlir/test/Conversion/GPUCommon/lower-memory-space-attrs.mlir b/mlir/test/Conversion/GPUCommon/lower-memory-space-attrs.mlir
index 771f3185904bb..7964cf7cc4a1b 100644
--- a/mlir/test/Conversion/GPUCommon/lower-memory-space-attrs.mlir
+++ b/mlir/test/Conversion/GPUCommon/lower-memory-space-attrs.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -split-input-file -convert-gpu-to-rocdl | FileCheck %s --check-prefixes=CHECK,ROCDL
+// RUN: mlir-opt %s -split-input-file -convert-gpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa | FileCheck %s --check-prefixes=CHECK,ROCDL
// RUN: mlir-opt %s -split-input-file -convert-gpu-to-nvvm | FileCheck %s --check-prefixes=CHECK,NVVM
gpu.module @kernel {
diff --git a/mlir/test/Conversion/GPUCommon/memory-attrbution.mlir b/mlir/test/Conversion/GPUCommon/memory-attrbution.mlir
index 38e73ea9179ac..b6154a5d3bcfc 100644
--- a/mlir/test/Conversion/GPUCommon/memory-attrbution.mlir
+++ b/mlir/test/Conversion/GPUCommon/memory-attrbution.mlir
@@ -1,5 +1,5 @@
// RUN: mlir-opt -allow-unregistered-dialect --convert-gpu-to-nvvm --split-input-file %s | FileCheck --check-prefix=NVVM %s
-// RUN: mlir-opt -allow-unregistered-dialect --convert-gpu-to-rocdl --split-input-file %s | FileCheck --check-prefix=ROCDL %s
+// RUN: mlir-opt -allow-unregistered-dialect --convert-gpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa --split-input-file %s | FileCheck --check-prefix=ROCDL %s
gpu.module @kernel {
// NVVM-LABEL: llvm.func @private
diff --git a/mlir/test/Conversion/GPUCommon/memref-arg-attrs.mlir b/mlir/test/Conversion/GPUCommon/memref-arg-attrs.mlir
index e7c742067b4eb..c557fac7d5ebd 100644
--- a/mlir/test/Conversion/GPUCommon/memref-arg-attrs.mlir
+++ b/mlir/test/Conversion/GPUCommon/memref-arg-attrs.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -split-input-file -convert-gpu-to-rocdl='use-bare-ptr-memref-call-conv=0' | FileCheck %s --check-prefixes=CHECK,ROCDL
+// RUN: mlir-opt %s -split-input-file -convert-gpu-to-rocdl='triple=amdgpu9.42-amd-amdhsa use-bare-ptr-memref-call-conv=0' | FileCheck %s --check-prefixes=CHECK,ROCDL
// RUN: mlir-opt %s -split-input-file -convert-gpu-to-nvvm='use-bare-ptr-memref-call-conv=0' | FileCheck %s --check-prefixes=CHECK,NVVM
gpu.module @kernel {
diff --git a/mlir/test/Conversion/GPUCommon/memref-arg-noalias-attrs.mlir b/mlir/test/Conversion/GPUCommon/memref-arg-noalias-attrs.mlir
index 33cdc3348e513..c7eba391b5ff7 100644
--- a/mlir/test/Conversion/GPUCommon/memref-arg-noalias-attrs.mlir
+++ b/mlir/test/Conversion/GPUCommon/memref-arg-noalias-attrs.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -split-input-file -convert-gpu-to-rocdl='use-bare-ptr-memref-call-conv=1' | FileCheck %s --check-prefixes=CHECK,ROCDL
+// RUN: mlir-opt %s -split-input-file -convert-gpu-to-rocdl='triple=amdgpu9.42-amd-amdhsa use-bare-ptr-memref-call-conv=1' | FileCheck %s --check-prefixes=CHECK,ROCDL
// RUN: mlir-opt %s -split-input-file -convert-gpu-to-nvvm='use-bare-ptr-memref-call-conv=1' | FileCheck %s --check-prefixes=CHECK,NVVM
gpu.module @kernel {
diff --git a/mlir/test/Conversion/GPUCommon/memref-arg-noalias-warning.mlir b/mlir/test/Conversion/GPUCommon/memref-arg-noalias-warning.mlir
index 793df7380d78b..0a6d075b42156 100644
--- a/mlir/test/Conversion/GPUCommon/memref-arg-noalias-warning.mlir
+++ b/mlir/test/Conversion/GPUCommon/memref-arg-noalias-warning.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -split-input-file -convert-gpu-to-rocdl='use-bare-ptr-memref-call-conv=0' -verify-diagnostics
+// RUN: mlir-opt %s -split-input-file -convert-gpu-to-rocdl='triple=amdgpu9.42-amd-amdhsa use-bare-ptr-memref-call-conv=0' -verify-diagnostics
gpu.module @kernel {
// expected-warning @+1 {{Cannot copy noalias with non-bare pointers.}}
diff --git a/mlir/test/Conversion/GPUToROCDL/constant-address-space.mlir b/mlir/test/Conversion/GPUToROCDL/constant-address-space.mlir
index 738aece1769da..67881b2a7bd32 100644
--- a/mlir/test/Conversion/GPUToROCDL/constant-address-space.mlir
+++ b/mlir/test/Conversion/GPUToROCDL/constant-address-space.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt -convert-gpu-to-rocdl %s | FileCheck %s
+// RUN: mlir-opt -convert-gpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa %s | FileCheck %s
module attributes {gpu.container_module} {
gpu.module @kernel_module {
diff --git a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-barrier.mlir b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-barrier.mlir
index 618d1889b8478..95be675ddfca8 100644
--- a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-barrier.mlir
+++ b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-barrier.mlir
@@ -1,5 +1,5 @@
-// RUN: mlir-opt %s -convert-gpu-to-rocdl='chipset=gfx950' --mlir-print-local-scope | FileCheck %s --check-prefixes=CHECK,GFX9
-// RUN: mlir-opt %s -convert-gpu-to-rocdl='chipset=gfx1201' --mlir-print-local-scope | FileCheck %s --check-prefixes=CHECK,GFX12
+// RUN: mlir-opt %s -convert-gpu-to-rocdl='triple=amdgpu9.50-amd-amdhsa' --mlir-print-local-scope | FileCheck %s --check-prefixes=CHECK,GFX9
+// RUN: mlir-opt %s -convert-gpu-to-rocdl='triple=amdgpu12.01-amd-amdhsa' --mlir-print-local-scope | FileCheck %s --check-prefixes=CHECK,GFX12
gpu.module @test_module {
// CHECK-LABEL: func @barrier_default()
diff --git a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-barriers-gfx12.mlir b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-barriers-gfx12.mlir
index c6a9574ca43c1..879e40638a1dd 100644
--- a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-barriers-gfx12.mlir
+++ b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-barriers-gfx12.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -convert-gpu-to-rocdl='chipset=gfx1250' --mlir-print-local-scope | FileCheck %s
+// RUN: mlir-opt %s -convert-gpu-to-rocdl='triple=amdgpu12.50-amd-amdhsa' --mlir-print-local-scope | FileCheck %s
gpu.module @test_module {
diff --git a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-hip.mlir b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-hip.mlir
index 32da31202b688..7f41ebeb1e32e 100644
--- a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-hip.mlir
+++ b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-hip.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -convert-gpu-to-rocdl='runtime=HIP' -split-input-file | FileCheck %s
+// RUN: mlir-opt %s -convert-gpu-to-rocdl='triple=amdgpu9.42-amd-amdhsa runtime=HIP' -split-input-file | FileCheck %s
// CHECK-LABEL: gpu.module @test_module
gpu.module @test_module {
diff --git a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-invalid-ballot.mlir b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-invalid-ballot.mlir
index a94ab3b5bb780..d8a7cb588071b 100644
--- a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-invalid-ballot.mlir
+++ b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-invalid-ballot.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -convert-gpu-to-rocdl='chipset=gfx950' -split-input-file -verify-diagnostics
+// RUN: mlir-opt %s -convert-gpu-to-rocdl='triple=amdgpu9.50-amd-amdhsa' -split-input-file -verify-diagnostics
// -----
diff --git a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-invalid-dialect.mlir b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-invalid-dialect.mlir
index 117f7692669de..0d0ac1d09750c 100644
--- a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-invalid-dialect.mlir
+++ b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-invalid-dialect.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -convert-gpu-to-rocdl='allowed-dialects=test' -verify-diagnostics
+// RUN: mlir-opt %s -convert-gpu-to-rocdl='triple=amdgpu9.42-amd-amdhsa allowed-dialects=test' -verify-diagnostics
// expected-error @+1 {{dialect does not implement ConvertToLLVMPatternInterface: test}}
gpu.module @test_module_1 {
diff --git a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-invalid-named-barrier.mlir b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-invalid-named-barrier.mlir
index 3f39f4abcf396..907c834d72678 100644
--- a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-invalid-named-barrier.mlir
+++ b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-invalid-named-barrier.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -convert-gpu-to-rocdl='chipset=gfx1100' -split-input-file -verify-diagnostics
+// RUN: mlir-opt %s -convert-gpu-to-rocdl='triple=amdgpu11.00-amd-amdhsa' -split-input-file -verify-diagnostics
gpu.module @test_module {
func.func @initialize_named_barrier_pre_gfx12(%count : i32) {
diff --git a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-named-barrier-non-const.mlir b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-named-barrier-non-const.mlir
index c9ce2794f1422..749a8251ad398 100644
--- a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-named-barrier-non-const.mlir
+++ b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-named-barrier-non-const.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -convert-gpu-to-rocdl='chipset=gfx1250' -split-input-file -verify-diagnostics
+// RUN: mlir-opt %s -convert-gpu-to-rocdl='triple=amdgpu12.50-amd-amdhsa' -split-input-file -verify-diagnostics
gpu.module @test_module {
func.func @non_constant_member_count(%count : i32) {
diff --git a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-opencl.mlir b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-opencl.mlir
index 00d1d7d852680..312fa18b69a08 100644
--- a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-opencl.mlir
+++ b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-opencl.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -convert-gpu-to-rocdl='runtime=OpenCL' | FileCheck %s
+// RUN: mlir-opt %s -convert-gpu-to-rocdl='triple=amdgpu9.42-amd-amdhsa runtime=OpenCL' | FileCheck %s
gpu.module @test_module {
// CHECK: llvm.mlir.global internal constant @[[$PRINT_GLOBAL:[A-Za-z0-9_]+]]("Hello: %d\0A\00") {addr_space = 4 : i32}
diff --git a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-subgroup-id.mlir b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-subgroup-id.mlir
index 9cab3ff48f5bf..bda3426073c01 100644
--- a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-subgroup-id.mlir
+++ b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl-subgroup-id.mlir
@@ -1,5 +1,5 @@
-// RUN: mlir-opt %s -convert-gpu-to-rocdl='chipset=gfx942' | FileCheck %s --check-prefixes=CHECK,GFX9
-// RUN: mlir-opt %s -convert-gpu-to-rocdl='chipset=gfx1201' | FileCheck %s --check-prefixes=CHECK,GFX12
+// RUN: mlir-opt %s -convert-gpu-to-rocdl='triple=amdgpu9.42-amd-amdhsa' | FileCheck %s --check-prefixes=CHECK,GFX9
+// RUN: mlir-opt %s -convert-gpu-to-rocdl='triple=amdgpu12.01-amd-amdhsa' | FileCheck %s --check-prefixes=CHECK,GFX12
gpu.module @test_module {
// CHECK-LABEL: func @subgroup_id()
diff --git a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl.mlir b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl.mlir
index c01af31e9d4f1..fb628c9db1c59 100755
--- a/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl.mlir
+++ b/mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl.mlir
@@ -1,6 +1,6 @@
-// RUN: mlir-opt %s -convert-gpu-to-rocdl='chipset=gfx950' -split-input-file | FileCheck %s
-// RUN: mlir-opt %s -convert-gpu-to-rocdl='chipset=gfx950 allowed-dialects=func,arith,math' -split-input-file | FileCheck %s
-// RUN: mlir-opt %s -convert-gpu-to-rocdl='chipset=gfx950 index-bitwidth=32' -split-input-file | FileCheck --check-prefix=CHECK32 %s
+// RUN: mlir-opt %s -convert-gpu-to-rocdl='triple=amdgpu9.50-amd-amdhsa' -split-input-file | FileCheck %s
+// RUN: mlir-opt %s -convert-gpu-to-rocdl='triple=amdgpu9.50-amd-amdhsa allowed-dialects=func,arith,math' -split-input-file | FileCheck %s
+// RUN: mlir-opt %s -convert-gpu-to-rocdl='triple=amdgpu9.50-amd-amdhsa index-bitwidth=32' -split-input-file | FileCheck --check-prefix=CHECK32 %s
// CHECK-LABEL: @test_module
// CHECK-SAME: llvm.data_layout = "e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-p7:160:256:256:32-p8:128:128:128:48-p9:192:256:256:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7:8:9"
diff --git a/mlir/test/Conversion/GPUToROCDL/memref.mlir b/mlir/test/Conversion/GPUToROCDL/memref.mlir
index e645481c89230..c23a7494adadf 100644
--- a/mlir/test/Conversion/GPUToROCDL/memref.mlir
+++ b/mlir/test/Conversion/GPUToROCDL/memref.mlir
@@ -1,6 +1,6 @@
-// RUN: mlir-opt %s -convert-gpu-to-rocdl -split-input-file | FileCheck %s
+// RUN: mlir-opt %s -convert-gpu-to-rocdl=triple=amdgpu9.42-amd-amdhsa -split-input-file | FileCheck %s
// RUN: mlir-opt %s \
-// RUN: -convert-gpu-to-rocdl='use-bare-ptr-memref-call-conv=true' \
+// RUN: -convert-gpu-to-rocdl='triple=amdgpu9.42-amd-amdhsa use-bare-ptr-memref-call-conv=true' \
// RUN: -split-input-file \
// RUN: | FileCheck %s --check-prefix=BARE
diff --git a/mlir/test/Conversion/MathToROCDL/math-to-rocdl.mlir b/mlir/test/Conversion/MathToROCDL/math-to-rocdl.mlir
index 455f886839604..addcbaba54dd1 100644
--- a/mlir/test/Conversion/MathToROCDL/math-to-rocdl.mlir
+++ b/mlir/test/Conversion/MathToROCDL/math-to-rocdl.mlir
@@ -1,5 +1,5 @@
-// RUN: mlir-opt %s -allow-unregistered-dialect -split-input-file -pass-pipeline='builtin.module(convert-math-to-rocdl{chipset=gfx803})' | FileCheck %s --check-prefix=PRE9
-// RUN: mlir-opt %s -allow-unregistered-dialect -split-input-file -pass-pipeline='builtin.module(convert-math-to-rocdl{chipset=gfx942})' | FileCheck %s --check-prefix=POST9
+// RUN: mlir-opt %s -allow-unregistered-dialect -split-input-file -pass-pipeline='builtin.module(convert-math-to-rocdl{triple=amdgpu8.03-amd-amdhsa})' | FileCheck %s --check-prefix=PRE9
+// RUN: mlir-opt %s -allow-unregistered-dialect -split-input-file -pass-pipeline='builtin.module(convert-math-to-rocdl{triple=amdgpu9.42-amd-amdhsa})' | FileCheck %s --check-prefix=POST9
module @test_module {
// CHECK: llvm.func @__ocml_fmod_f16(f16, f16) -> f16
diff --git a/mlir/test/Dialect/AMDGPU/amdgpu-emulate-atomics.mlir b/mlir/test/Dialect/AMDGPU/amdgpu-emulate-atomics.mlir
index fa883bb96c1a0..35449a257af31 100644
--- a/mlir/test/Dialect/AMDGPU/amdgpu-emulate-atomics.mlir
+++ b/mlir/test/Dialect/AMDGPU/amdgpu-emulate-atomics.mlir
@@ -1,11 +1,11 @@
-// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=chipset=gfx908 %s | FileCheck %s --check-prefixes=CHECK,GFX9CAS,GFX908
-// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=chipset=gfx90a %s | FileCheck %s --check-prefixes=CHECK,GFX9CAS,GFX90A
-// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=chipset=gfx90c %s | FileCheck %s --check-prefixes=CHECK,GFX9CAS,GFX90C
-// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=chipset=gfx1030 %s | FileCheck %s --check-prefixes=CHECK,GFX10
-// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=chipset=gfx1100 %s | FileCheck %s --check-prefixes=CHECK,GFX11
-// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=chipset=gfx1200 %s | FileCheck %s --check-prefixes=CHECK,GFX12
-// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=chipset=gfx942 %s | FileCheck %s --check-prefixes=CHECK,GFX9CAS,GFX942
-// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=chipset=gfx950 %s | FileCheck %s --check-prefixes=CHECK,GFX9CAS,GFX950
+// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=triple=amdgpu9.08-amd-amdhsa %s | FileCheck %s --check-prefixes=CHECK,GFX9CAS,GFX9NOF64,GFX908
+// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=triple=amdgpu9.0a-amd-amdhsa %s | FileCheck %s --check-prefixes=CHECK,GFX9CAS,GFX90A
+// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=triple=amdgpu9.0c-amd-amdhsa %s | FileCheck %s --check-prefixes=CHECK,GFX9CAS,GFX9NOF64,GFX90C
+// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=triple=amdgpu10.30-amd-amdhsa %s | FileCheck %s --check-prefixes=CHECK,GFX10
+// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=triple=amdgpu11.00-amd-amdhsa %s | FileCheck %s --check-prefixes=CHECK,GFX11
+// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=triple=amdgpu12.00-amd-amdhsa %s | FileCheck %s --check-prefixes=CHECK,GFX12
+// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=triple=amdgpu9.42-amd-amdhsa %s | FileCheck %s --check-prefixes=CHECK,GFX9CAS,GFX942
+// RUN: mlir-opt -split-input-file -amdgpu-emulate-atomics=triple=amdgpu9.50-amd-amdhsa %s | FileCheck %s --check-prefixes=CHECK,GFX9CAS,GFX950
// -----
@@ -46,19 +46,17 @@ func.func @atomic_fmax_f64(%val: f64, %buffer: memref<?xf64>, %idx: i32) {
// GFX12: amdgpu.raw_buffer_atomic_fmax boundsCheck(true) [[val]] -> [[buffer]][[[idx]]]
// GFX942: amdgpu.raw_buffer_atomic_fmax boundsCheck(true) [[val]] -> [[buffer]][[[idx]]]
// GFX950: amdgpu.raw_buffer_atomic_fmax boundsCheck(true) [[val]] -> [[buffer]][[[idx]]]
-// gfx908 has no f64 buffer fmin/fmax, so it is emulated.
-// GFX908: [[ld:%.+]] = amdgpu.raw_buffer_load boundsCheck(true) [[buffer]][[[idx]]]
-// GFX908: cf.br [[loop:\^.+]]([[ld]] : f64)
-// GFX908: [[loop]]([[arg:%.+]]: f64):
-// GFX908: [[operated:%.+]] = arith.maximumf [[val]], [[arg]]
-// GFX908: [[atomicRes:%.+]] = amdgpu.raw_buffer_atomic_cmpswap boundsCheck(true) [[operated]], [[arg]] -> [[buffer]][[[idx]]]
-// GFX908: [[argCast:%.+]] = arith.bitcast [[arg]] : f64 to i64
-// GFX908: [[resCast:%.+]] = arith.bitcast [[atomicRes]] : f64 to i64
-// GFX908: [[test:%.+]] = arith.cmpi eq, [[resCast]], [[argCast]]
-// GFX908: cf.cond_br [[test]], [[post:\^.+]]([[arg]] : f64), [[loop]]([[atomicRes]] : f64)
-// GFX908: [[post]]([[old:%.+]]: f64):
-// gfx90c has none either, but sorts after gfx90a by ISA version.
-// GFX90C: amdgpu.raw_buffer_atomic_fmax boundsCheck(true) [[val]] -> [[buffer]][[[idx]]]
+// Neither gfx908 nor gfx90c has f64 buffer fmin/fmax.
+// GFX9NOF64: [[ld:%.+]] = amdgpu.raw_buffer_load boundsCheck(true) [[buffer]][[[idx]]]
+// GFX9NOF64: cf.br [[loop:\^.+]]([[ld]] : f64)
+// GFX9NOF64: [[loop]]([[arg:%.+]]: f64):
+// GFX9NOF64: [[operated:%.+]] = arith.maximumf [[val]], [[arg]]
+// GFX9NOF64: [[atomicRes:%.+]] = amdgpu.raw_buffer_atomic_cmpswap boundsCheck(true) [[operated]], [[arg]] -> [[buffer]][[[idx]]]
+// GFX9NOF64: [[argCast:%.+]] = arith.bitcast [[arg]] : f64 to i64
+// GFX9NOF64: [[resCast:%.+]] = arith.bitcast [[atomicRes]] : f64 to i64
+// GFX9NOF64: [[test:%.+]] = arith.cmpi eq, [[resCast]], [[argCast]]
+// GFX9NOF64: cf.cond_br [[test]], [[post:\^.+]]([[arg]] : f64), [[loop]]([[atomicRes]] : f64)
+// GFX9NOF64: [[post]]([[old:%.+]]: f64):
// CHECK-NEXT: gpu.printf "End\0A"
gpu.printf "Begin\n"
%old = amdgpu.raw_buffer_atomic_fmax boundsCheck(true) %val -> %buffer[%idx] : f64 -> memref<?xf64>, i32
@@ -77,8 +75,11 @@ func.func @atomic_fadd(%val: f32, %buffer: memref<?xf32>, %idx: i32) {
// GFX12: amdgpu.raw_buffer_atomic_fadd
// GFX942: amdgpu.raw_buffer_atomic_fadd
// GFX950: amdgpu.raw_buffer_atomic_fadd
+// gfx908 only has the no-return form, which suffices here as %old is unused.
// GFX908: amdgpu.raw_buffer_atomic_fadd
-// GFX90C: amdgpu.raw_buffer_atomic_fadd
+// gfx90c has no buffer fadd at all.
+// GFX90C: amdgpu.raw_buffer_load
+// GFX90C: amdgpu.raw_buffer_atomic_cmpswap
%old = amdgpu.raw_buffer_atomic_fadd boundsCheck(true) %val -> %buffer[%idx] : f32 -> memref<?xf32>, i32
func.return
}
@@ -100,8 +101,11 @@ func.func @atomic_fadd_v2f16(%val: vector<2xf16>, %buffer: memref<?xf16>, %idx:
// GFX942: amdgpu.raw_buffer_atomic_fadd
// GFX12: amdgpu.raw_buffer_atomic_fadd
// GFX950: amdgpu.raw_buffer_atomic_fadd
-// GFX908: amdgpu.raw_buffer_atomic_fadd
-// GFX90C: amdgpu.raw_buffer_atomic_fadd
+// Neither gfx908 nor gfx90c has the packed f16 buffer fadd.
+// GFX908: amdgpu.raw_buffer_load
+// GFX908: amdgpu.raw_buffer_atomic_cmpswap
+// GFX90C: amdgpu.raw_buffer_load
+// GFX90C: amdgpu.raw_buffer_atomic_cmpswap
%old = amdgpu.raw_buffer_atomic_fadd boundsCheck(true) %val -> %buffer[%idx] : vector<2xf16> -> memref<?xf16>, i32
func.return
}
@@ -125,3 +129,24 @@ func.func @atomic_fadd_v2bf16(%val: vector<2xbf16>, %buffer: memref<?xbf16>, %id
%old = amdgpu.raw_buffer_atomic_fadd boundsCheck(true) %val -> %buffer[%idx] : vector<2xbf16> -> memref<?xbf16>, i32
func.return
}
+
+// -----
+
+// gfx908 has only the no-return buffer fadd, so a *used* result has to be
+// emulated even though the discarded-result case above lowers natively.
+// CHECK: func @atomic_fadd_used_result
+func.func @atomic_fadd_used_result(%val: f32, %buffer: memref<?xf32>, %idx: i32) -> f32 {
+// GFX908: amdgpu.raw_buffer_load
+// GFX908: amdgpu.raw_buffer_atomic_cmpswap
+// GFX90A: amdgpu.raw_buffer_atomic_fadd
+// GFX942: amdgpu.raw_buffer_atomic_fadd
+// GFX950: amdgpu.raw_buffer_atomic_fadd
+// GFX90C: amdgpu.raw_buffer_load
+// GFX90C: amdgpu.raw_buffer_atomic_cmpswap
+// GFX10: amdgpu.raw_buffer_load
+// GFX10: amdgpu.raw_buffer_atomic_cmpswap
+// GFX11: amdgpu.raw_buffer_atomic_fadd
+// GFX12: amdgpu.raw_buffer_atomic_fadd
+ %old = amdgpu.raw_buffer_atomic_fadd boundsCheck(true) %val -> %buffer[%idx] : f32 -> memref<?xf32>, i32
+ func.return %old : f32
+}
diff --git a/mlir/test/Dialect/GPU/promote-shuffle-amdgpu-invalid.mlir b/mlir/test/Dialect/GPU/promote-shuffle-amdgpu-invalid.mlir
new file mode 100644
index 0000000000000..462305d36c02e
--- /dev/null
+++ b/mlir/test/Dialect/GPU/promote-shuffle-amdgpu-invalid.mlir
@@ -0,0 +1,38 @@
+// RUN: mlir-opt %s --transform-interpreter --split-input-file --verify-diagnostics
+
+module attributes {transform.with_named_sequence} {
+ transform.named_sequence @__transform_main(%module_op: !transform.any_op {transform.readonly}) {
+ %func = transform.structured.match ops{["func.func"]} in %module_op : (!transform.any_op) -> !transform.any_op
+ transform.apply_patterns to %func {
+ // expected-error at below {{'gfx999' is not an AMDGCN triple or GPU name}}
+ transform.apply_patterns.gpu.gpu_shuffle_to_amdgpu <triple = "gfx999">
+ } : !transform.any_op
+ transform.yield
+ }
+}
+
+// -----
+
+module attributes {transform.with_named_sequence} {
+ transform.named_sequence @__transform_main(%module_op: !transform.any_op {transform.readonly}) {
+ %func = transform.structured.match ops{["func.func"]} in %module_op : (!transform.any_op) -> !transform.any_op
+ transform.apply_patterns to %func {
+ // expected-error at below {{'chip' and 'features' require a 'triple'}}
+ transform.apply_patterns.gpu.gpu_shuffle_to_amdgpu <chip = "gfx950">
+ } : !transform.any_op
+ transform.yield
+ }
+}
+
+// -----
+
+module attributes {transform.with_named_sequence} {
+ transform.named_sequence @__transform_main(%module_op: !transform.any_op {transform.readonly}) {
+ %func = transform.structured.match ops{["func.func"]} in %module_op : (!transform.any_op) -> !transform.any_op
+ transform.apply_patterns to %func {
+ // expected-error at below {{invalid target feature '+not-a-feature'}}
+ transform.apply_patterns.gpu.gpu_shuffle_to_amdgpu <triple = "gfx950", features = "+not-a-feature">
+ } : !transform.any_op
+ transform.yield
+ }
+}
diff --git a/mlir/test/Dialect/GPU/promote-shuffle-amdgpu.mlir b/mlir/test/Dialect/GPU/promote-shuffle-amdgpu.mlir
index 747c997a3b441..16b8aab68acaa 100644
--- a/mlir/test/Dialect/GPU/promote-shuffle-amdgpu.mlir
+++ b/mlir/test/Dialect/GPU/promote-shuffle-amdgpu.mlir
@@ -4,7 +4,7 @@ module attributes {transform.with_named_sequence} {
transform.named_sequence @__transform_main(%module_op: !transform.any_op {transform.readonly}) {
%func = transform.structured.match ops{["func.func"]} in %module_op : (!transform.any_op) -> !transform.any_op
transform.apply_patterns to %func {
- transform.apply_patterns.gpu.gpu_shuffle_to_amdgpu chipset = "gfx950"
+ transform.apply_patterns.gpu.gpu_shuffle_to_amdgpu <triple = "amdgpu9.50-amd-amdhsa">
} : !transform.any_op
transform.yield
}
diff --git a/mlir/test/Integration/GPU/ROCM/gpu-to-hsaco.mlir b/mlir/test/Integration/GPU/ROCM/gpu-to-hsaco.mlir
index 5fa27eab3bba4..dd342b3cde044 100644
--- a/mlir/test/Integration/GPU/ROCM/gpu-to-hsaco.mlir
+++ b/mlir/test/Integration/GPU/ROCM/gpu-to-hsaco.mlir
@@ -1,6 +1,6 @@
// RUN: mlir-opt %s \
// RUN: | mlir-opt -gpu-kernel-outlining \
-// RUN: | mlir-opt -pass-pipeline='builtin.module(gpu.module(strip-debuginfo,convert-gpu-to-rocdl),rocdl-attach-target{chip=%chip})' \
+// RUN: | mlir-opt -pass-pipeline='builtin.module(gpu.module(strip-debuginfo,convert-gpu-to-rocdl{triple=%chip}),rocdl-attach-target{chip=%chip})' \
// RUN: | mlir-opt -gpu-to-llvm -reconcile-unrealized-casts -gpu-module-to-binary \
// RUN: | mlir-runner \
// RUN: --shared-libs=%mlir_rocm_runtime \
diff --git a/mlir/test/Integration/GPU/ROCM/printf.mlir b/mlir/test/Integration/GPU/ROCM/printf.mlir
index 8327ec428589d..232b145e14d3d 100644
--- a/mlir/test/Integration/GPU/ROCM/printf.mlir
+++ b/mlir/test/Integration/GPU/ROCM/printf.mlir
@@ -1,5 +1,5 @@
// RUN: mlir-opt %s \
-// RUN: | mlir-opt -pass-pipeline='builtin.module(gpu.module(strip-debuginfo,convert-gpu-to-rocdl{index-bitwidth=32 runtime=HIP}),rocdl-attach-target{chip=%chip})' \
+// RUN: | mlir-opt -pass-pipeline='builtin.module(gpu.module(strip-debuginfo,convert-gpu-to-rocdl{triple=%chip index-bitwidth=32 runtime=HIP}),rocdl-attach-target{chip=%chip})' \
// RUN: | mlir-opt -gpu-to-llvm -reconcile-unrealized-casts -gpu-module-to-binary \
// RUN: | mlir-runner \
// RUN: --shared-libs=%mlir_rocm_runtime \
diff --git a/mlir/test/Integration/GPU/ROCM/two-modules.mlir b/mlir/test/Integration/GPU/ROCM/two-modules.mlir
index f3062dbda86c8..3f08109837fac 100644
--- a/mlir/test/Integration/GPU/ROCM/two-modules.mlir
+++ b/mlir/test/Integration/GPU/ROCM/two-modules.mlir
@@ -1,6 +1,6 @@
// RUN: mlir-opt %s \
// RUN: | mlir-opt -gpu-kernel-outlining \
-// RUN: | mlir-opt -pass-pipeline='builtin.module(gpu.module(strip-debuginfo,convert-gpu-to-rocdl),rocdl-attach-target{chip=%chip})' \
+// RUN: | mlir-opt -pass-pipeline='builtin.module(gpu.module(strip-debuginfo,convert-gpu-to-rocdl{triple=%chip}),rocdl-attach-target{chip=%chip})' \
// RUN: | mlir-opt -gpu-to-llvm -reconcile-unrealized-casts -gpu-module-to-binary \
// RUN: | mlir-runner \
// RUN: --shared-libs=%mlir_rocm_runtime \
diff --git a/mlir/test/Integration/GPU/ROCM/vecadd.mlir b/mlir/test/Integration/GPU/ROCM/vecadd.mlir
index c3f8b982a131a..f4b9f3e731323 100644
--- a/mlir/test/Integration/GPU/ROCM/vecadd.mlir
+++ b/mlir/test/Integration/GPU/ROCM/vecadd.mlir
@@ -1,7 +1,7 @@
// RUN: mlir-opt %s \
// RUN: | mlir-opt -convert-scf-to-cf \
// RUN: | mlir-opt -gpu-kernel-outlining \
-// RUN: | mlir-opt -pass-pipeline='builtin.module(gpu.module(strip-debuginfo,convert-gpu-to-rocdl{use-bare-ptr-memref-call-conv=true}),rocdl-attach-target{chip=%chip})' \
+// RUN: | mlir-opt -pass-pipeline='builtin.module(gpu.module(strip-debuginfo,convert-gpu-to-rocdl{triple=%chip use-bare-ptr-memref-call-conv=true}),rocdl-attach-target{chip=%chip})' \
// RUN: | mlir-opt -gpu-to-llvm=use-bare-pointers-for-kernels=true -reconcile-unrealized-casts -gpu-module-to-binary \
// RUN: | mlir-runner \
// RUN: --shared-libs=%mlir_rocm_runtime \
diff --git a/mlir/test/Integration/GPU/ROCM/vector-transferops.mlir b/mlir/test/Integration/GPU/ROCM/vector-transferops.mlir
index a633edb8377af..54fb147e3feff 100644
--- a/mlir/test/Integration/GPU/ROCM/vector-transferops.mlir
+++ b/mlir/test/Integration/GPU/ROCM/vector-transferops.mlir
@@ -1,7 +1,7 @@
// RUN: mlir-opt %s \
// RUN: | mlir-opt -convert-scf-to-cf \
// RUN: | mlir-opt -gpu-kernel-outlining \
-// RUN: | mlir-opt -pass-pipeline='builtin.module(gpu.module(strip-debuginfo,convert-gpu-to-rocdl{chipset=%chip index-bitwidth=32}),rocdl-attach-target{chip=%chip})' \
+// RUN: | mlir-opt -pass-pipeline='builtin.module(gpu.module(strip-debuginfo,convert-gpu-to-rocdl{triple=%chip index-bitwidth=32}),rocdl-attach-target{chip=%chip})' \
// RUN: | mlir-opt -gpu-to-llvm -reconcile-unrealized-casts -gpu-module-to-binary \
// RUN: | mlir-runner \
// RUN: --shared-libs=%mlir_rocm_runtime \
diff --git a/mlir/test/lib/Dialect/GPU/TestGpuRewrite.cpp b/mlir/test/lib/Dialect/GPU/TestGpuRewrite.cpp
index 616f458e4824c..9ddf9c1a9658f 100644
--- a/mlir/test/lib/Dialect/GPU/TestGpuRewrite.cpp
+++ b/mlir/test/lib/Dialect/GPU/TestGpuRewrite.cpp
@@ -11,7 +11,6 @@
//===----------------------------------------------------------------------===//
#include "mlir/Dialect/AMDGPU/IR/AMDGPUDialect.h"
-#include "mlir/Dialect/AMDGPU/Utils/Chipset.h"
#include "mlir/Dialect/Arith/IR/Arith.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/GPU/Transforms/Passes.h"
@@ -91,12 +90,19 @@ struct TestGpuSubgroupReduceLoweringPass
/*maxShuffleBitwidth=*/32,
PatternBenefit(3));
if (expandToShuffles) {
- auto maybeChipset = amdgpu::Chipset::parse(target);
- if (succeeded(maybeChipset)) {
+ // No target means "shuffles only". A target that is given but not
+ // recognized used to be swallowed here, silently dropping the DPP
+ // patterns instead of reporting the typo.
+ if (!target.empty()) {
+ FailureOr<ROCDL::TargetInfo> targetInfo =
+ ROCDL::TargetInfo::get(target, /*chip=*/"", /*features=*/"",
+ [&] { return getOperation()->emitError(); });
+ if (failed(targetInfo))
+ return signalPassFailure();
populateGpuLowerSubgroupReduceToDPPPatterns(
- patterns, /*subgroupSize=*/64, *maybeChipset, PatternBenefit(2));
+ patterns, /*subgroupSize=*/64, *targetInfo, PatternBenefit(2));
populateGpuLowerClusteredSubgroupReduceToDPPPatterns(
- patterns, /*subgroupSize=*/64, *maybeChipset, PatternBenefit(2));
+ patterns, /*subgroupSize=*/64, *targetInfo, PatternBenefit(2));
}
populateGpuLowerSubgroupReduceToShufflePatterns(
patterns, /*subgroupSize=*/32, /*shuffleBitwidth=*/32);
diff --git a/mlir/unittests/Dialect/AMDGPU/AMDGPUUtilsTest.cpp b/mlir/unittests/Dialect/AMDGPU/AMDGPUUtilsTest.cpp
index 570d56f3c6ff1..33fbc79e77027 100644
--- a/mlir/unittests/Dialect/AMDGPU/AMDGPUUtilsTest.cpp
+++ b/mlir/unittests/Dialect/AMDGPU/AMDGPUUtilsTest.cpp
@@ -9,6 +9,14 @@
#include "mlir/Dialect/AMDGPU/Utils/Chipset.h"
#include "gtest/gtest.h"
+// Chipset is deprecated in favour of ROCDL::TargetInfo, but stays covered for
+// as long as it ships.
+#ifdef __clang__
+#pragma clang diagnostic ignored "-Wdeprecated-declarations"
+#elif defined(__GNUC__)
+#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+#endif
+
namespace mlir::amdgpu {
namespace {
diff --git a/mlir/unittests/Dialect/LLVMIR/CMakeLists.txt b/mlir/unittests/Dialect/LLVMIR/CMakeLists.txt
index 7cc130d02ad74..39205a53645a7 100644
--- a/mlir/unittests/Dialect/LLVMIR/CMakeLists.txt
+++ b/mlir/unittests/Dialect/LLVMIR/CMakeLists.txt
@@ -1,7 +1,9 @@
add_mlir_unittest(MLIRLLVMIRTests
LLVMTypeTest.cpp
+ ROCDLTargetInfoTest.cpp
)
mlir_target_link_libraries(MLIRLLVMIRTests
PRIVATE
MLIRLLVMDialect
+ MLIRROCDLDialect
)
diff --git a/mlir/unittests/Dialect/LLVMIR/ROCDLTargetInfoTest.cpp b/mlir/unittests/Dialect/LLVMIR/ROCDLTargetInfoTest.cpp
new file mode 100644
index 0000000000000..e050287e4c4df
--- /dev/null
+++ b/mlir/unittests/Dialect/LLVMIR/ROCDLTargetInfoTest.cpp
@@ -0,0 +1,246 @@
+//===- ROCDLTargetInfoTest.cpp - Unit tests for ROCDL::TargetInfo ---------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "mlir/Dialect/LLVMIR/ROCDLTargetInfo.h"
+#include "mlir/IR/Diagnostics.h"
+#include "mlir/IR/MLIRContext.h"
+#include "gtest/gtest.h"
+
+namespace mlir::ROCDL {
+namespace {
+
+/// Resolves a target, collecting anything reported through emitError.
+FailureOr<TargetInfo> resolve(StringRef tripleOrChip, StringRef chip,
+ StringRef features, std::string &error) {
+ MLIRContext ctx;
+ ScopedDiagnosticHandler handler(&ctx, [&](Diagnostic &diag) {
+ error = diag.str();
+ return success();
+ });
+ return TargetInfo::get(tripleOrChip, chip, features,
+ [&] { return emitError(UnknownLoc::get(&ctx)); });
+}
+
+/// Resolves a target, asserting that it succeeded.
+TargetInfo getTarget(StringRef tripleOrChip, StringRef chip = "",
+ StringRef features = "") {
+ std::string error;
+ FailureOr<TargetInfo> target = resolve(tripleOrChip, chip, features, error);
+ EXPECT_TRUE(succeeded(target)) << "'" << tripleOrChip << "': " << error;
+ return succeeded(target) ? *target : TargetInfo();
+}
+
+/// Returns the error message produced when resolving a target, or "" if it
+/// unexpectedly succeeded.
+std::string getTargetError(StringRef tripleOrChip, StringRef chip = "",
+ StringRef features = "") {
+ std::string error;
+ FailureOr<TargetInfo> target = resolve(tripleOrChip, chip, features, error);
+ EXPECT_TRUE(failed(target)) << "expected '" << tripleOrChip << "' to fail";
+ return error;
+}
+
+TEST(TargetInfoTest, ParseGPUName) {
+ TargetInfo gfx942 = getTarget("gfx942");
+ EXPECT_EQ(gfx942.getArchName(), "gfx942");
+ EXPECT_EQ(gfx942.getSubArch(), llvm::Triple::AMDGPUSubArch942);
+ EXPECT_FALSE(gfx942.isUnknown());
+ EXPECT_FALSE(gfx942.isGeneric());
+
+ llvm::AMDGPU::IsaVersion version = gfx942.getIsaVersion();
+ EXPECT_EQ(version.Major, 9u);
+ EXPECT_EQ(version.Minor, 4u);
+ EXPECT_EQ(version.Stepping, 2u);
+}
+
+TEST(TargetInfoTest, ParseTriple) {
+ // A subarch-bearing triple identifies the GPU on its own.
+ TargetInfo fromTriple = getTarget("amdgpu9.42-amd-amdhsa");
+ EXPECT_EQ(fromTriple.getArchName(), "gfx942");
+ EXPECT_EQ(fromTriple.getSubArch(), llvm::Triple::AMDGPUSubArch942);
+
+ // The legacy subarch-less triple names no GPU.
+ TargetInfo legacy = getTarget("amdgcn-amd-amdhsa");
+ EXPECT_TRUE(legacy.isUnknown());
+ EXPECT_EQ(legacy.getSubArch(), llvm::Triple::NoSubArch);
+ EXPECT_FALSE(legacy.has(llvm::AMDGPU::FEAT_GFX9_INSTS));
+
+ // A chip refines it, the way -mcpu does.
+ TargetInfo withChip = getTarget("amdgcn-amd-amdhsa", "gfx942");
+ EXPECT_EQ(withChip.getArchName(), "gfx942");
+ EXPECT_EQ(withChip.getSubArch(), llvm::Triple::AMDGPUSubArch942);
+
+ // A family triple plus a chip narrows to the exact GPU.
+ TargetInfo family = getTarget("amdgpu9.4-amd-amdhsa", "gfx950");
+ EXPECT_EQ(family.getArchName(), "gfx950");
+ EXPECT_EQ(family.getSubArch(), llvm::Triple::AMDGPUSubArch950);
+}
+
+TEST(TargetInfoTest, ParseGeneric) {
+ // Generic targets are representable, and carry only the features common to
+ // every GPU they cover.
+ TargetInfo generic = getTarget("gfx9-4-generic");
+ EXPECT_TRUE(generic.isGeneric());
+ EXPECT_EQ(generic.getArchName(), "gfx9-4-generic");
+
+ // gfx9-4-generic covers gfx942 and gfx950, so it must not claim anything
+ // exclusive to either.
+ EXPECT_TRUE(generic.has(llvm::AMDGPU::FEAT_GFX940_INSTS));
+ EXPECT_FALSE(generic.has(llvm::AMDGPU::FEAT_GFX950_INSTS));
+ EXPECT_FALSE(generic.has(llvm::AMDGPU::FEAT_XF32_INSTS));
+ EXPECT_TRUE(getTarget("gfx942").has(llvm::AMDGPU::FEAT_XF32_INSTS));
+}
+
+TEST(TargetInfoTest, ParseInvalid) {
+ // Unlike Chipset::parse, a well-formed but nonexistent GPU is rejected.
+ EXPECT_NE(getTargetError("gfx999"), "");
+ EXPECT_NE(getTargetError("gfx000"), "");
+ EXPECT_NE(getTargetError("navi33"), "");
+ EXPECT_NE(getTargetError("sm_80"), "");
+ EXPECT_NE(getTargetError("GFX942"), "");
+ EXPECT_NE(getTargetError(""), "");
+
+ // Triple parsing maps any unrecognized "amdgpu..." to NoSubArch rather than
+ // reporting an error, so a typo must not be mistaken for the legacy triple.
+ EXPECT_NE(getTargetError("amdgpu9.99-amd-amdhsa"), "");
+ EXPECT_NE(getTargetError("amdgputypo-amd-amdhsa"), "");
+
+ // A chip inconsistent with the triple's subarch is rejected.
+ EXPECT_NE(getTargetError("amdgpu9.42-amd-amdhsa", "gfx1030"), "");
+ EXPECT_NE(getTargetError("amdgcn-amd-amdhsa", "gfx999"), "");
+}
+
+TEST(TargetInfoTest, FeatureModifiers) {
+ TargetInfo target = getTarget("gfx942", /*chip=*/"", "-mai-insts,+dpp");
+ EXPECT_FALSE(target.has(llvm::AMDGPU::FEAT_MAI_INSTS));
+ EXPECT_TRUE(target.has(llvm::AMDGPU::FEAT_DPP));
+ EXPECT_TRUE(getTarget("gfx942").has(llvm::AMDGPU::FEAT_MAI_INSTS));
+
+ EXPECT_NE(getTargetError("gfx942", "", "+not-a-feature"), "");
+ EXPECT_NE(getTargetError("gfx942", "", "mai-insts"), "");
+}
+
+TEST(TargetInfoTest, WavefrontSize) {
+ // Single-mode targets report their only size.
+ EXPECT_EQ(getTarget("gfx90a").getWavefrontSize(), 64u);
+ EXPECT_EQ(getTarget("gfx942").getWavefrontSize(), 64u);
+ EXPECT_EQ(getTarget("gfx1250").getWavefrontSize(), 32u);
+
+ // Targets supporting both default to wave32, and honour an explicit request.
+ // This is the case a triple alone cannot express.
+ for (StringRef gpu : {"gfx1030", "gfx1100", "gfx1200"}) {
+ EXPECT_EQ(getTarget(gpu).getWavefrontSize(), 32u) << gpu;
+ EXPECT_EQ(getTarget(gpu, "", "+wavefrontsize64").getWavefrontSize(), 64u)
+ << gpu;
+ }
+
+ // Asking a single-mode target for the other size is an error, not a silent
+ // mis-lowering.
+ EXPECT_NE(getTargetError("gfx942", "", "+wavefrontsize32"), "");
+ EXPECT_NE(getTargetError("gfx1250", "", "+wavefrontsize64"), "");
+ EXPECT_NE(getTargetError("gfx1030", "", "+wavefrontsize32,+wavefrontsize64"),
+ "");
+
+ // An unknown target has no wavefront size.
+ EXPECT_EQ(getTarget("amdgcn-amd-amdhsa").getWavefrontSize(), std::nullopt);
+}
+
+TEST(TargetInfoTest, SupportsBothWavefrontSizes) {
+ // Only these leave the choice to the features; the rest pin a size, which is
+ // why the pipeline's wave64 option must not be forced onto them.
+ for (StringRef gpu : {"gfx1030", "gfx1100", "gfx1200"})
+ EXPECT_TRUE(getTarget(gpu).supportsBothWavefrontSizes()) << gpu;
+ for (StringRef gpu : {"gfx90a", "gfx942", "gfx1250"})
+ EXPECT_FALSE(getTarget(gpu).supportsBothWavefrontSizes()) << gpu;
+
+ // Naming a size does not change what the GPU is capable of.
+ EXPECT_TRUE(getTarget("gfx1030", "", "+wavefrontsize64")
+ .supportsBothWavefrontSizes());
+ EXPECT_FALSE(getTarget("amdgcn-amd-amdhsa").supportsBothWavefrontSizes());
+}
+
+TEST(TargetInfoTest, Fp8Formats) {
+ // hasFnuzFp8 is not the negation of hasOcpFp8: a target with no fp8
+ // conversions at all has neither.
+ EXPECT_TRUE(getTarget("gfx942").hasFnuzFp8());
+ EXPECT_FALSE(getTarget("gfx942").hasOcpFp8());
+ for (StringRef gpu : {"gfx950", "gfx1170", "gfx1200"}) {
+ EXPECT_FALSE(getTarget(gpu).hasFnuzFp8()) << gpu;
+ EXPECT_TRUE(getTarget(gpu).hasOcpFp8()) << gpu;
+ }
+ for (StringRef gpu : {"gfx908", "gfx90a", "gfx900"}) {
+ EXPECT_FALSE(getTarget(gpu).hasFnuzFp8()) << gpu;
+ EXPECT_FALSE(getTarget(gpu).hasOcpFp8()) << gpu;
+ }
+}
+
+TEST(TargetInfoTest, BufferResourceNumRecordsWidth) {
+ // A width rather than a capability bit, so that a third width would not need
+ // every caller to learn a new predicate.
+ for (StringRef gpu : {"gfx900", "gfx1030", "gfx1200", "gfx1201"})
+ EXPECT_EQ(getTarget(gpu).getBufferResourceNumRecordsWidth(), 32u) << gpu;
+ for (StringRef gpu : {"gfx1250", "gfx1251", "gfx1250-strict"})
+ EXPECT_EQ(getTarget(gpu).getBufferResourceNumRecordsWidth(), 45u) << gpu;
+
+ // Generic targets take the width of the family they cover.
+ EXPECT_EQ(getTarget("gfx12-generic").getBufferResourceNumRecordsWidth(), 32u);
+ EXPECT_EQ(getTarget("gfx12-5-generic").getBufferResourceNumRecordsWidth(),
+ 45u);
+
+ // An unknown target has no width, so a lowering that needs one must bail
+ // rather than assume the narrow case.
+ EXPECT_EQ(getTarget("amdgcn-amd-amdhsa").getBufferResourceNumRecordsWidth(),
+ std::nullopt);
+ EXPECT_EQ(TargetInfo().getBufferResourceNumRecordsWidth(), std::nullopt);
+}
+
+TEST(TargetInfoTest, MaxAddressableLocalMemorySize) {
+ EXPECT_EQ(getTarget("gfx900").getMaxAddressableLocalMemorySize(), 65536u);
+ EXPECT_EQ(getTarget("gfx1030").getMaxAddressableLocalMemorySize(), 65536u);
+ EXPECT_EQ(getTarget("gfx950").getMaxAddressableLocalMemorySize(), 163840u);
+ EXPECT_EQ(getTarget("gfx1250").getMaxAddressableLocalMemorySize(), 327680u);
+
+ EXPECT_EQ(getTarget("amdgcn-amd-amdhsa").getMaxAddressableLocalMemorySize(),
+ std::nullopt);
+}
+
+TEST(TargetInfoTest, Generation) {
+ EXPECT_TRUE(getTarget("gfx900").isGeneration(9));
+ EXPECT_TRUE(getTarget("gfx942").isGeneration(9));
+ EXPECT_TRUE(getTarget("gfx950").isGeneration(9));
+ EXPECT_FALSE(getTarget("gfx942").isGeneration(10));
+ EXPECT_FALSE(getTarget("gfx942").isGeneration(8));
+
+ EXPECT_TRUE(getTarget("gfx1010").isGeneration(10));
+ EXPECT_TRUE(getTarget("gfx1030").isGeneration(10));
+ EXPECT_TRUE(getTarget("gfx1100").isGeneration(11));
+ EXPECT_TRUE(getTarget("gfx1200").isGeneration(12));
+ EXPECT_TRUE(getTarget("gfx1250").isGeneration(12));
+ EXPECT_TRUE(getTarget("gfx803").isGeneration(8));
+ EXPECT_TRUE(getTarget("gfx700").isGeneration(7));
+ EXPECT_TRUE(getTarget("gfx600").isGeneration(6));
+
+ // Generic targets report the generation of the family they cover, which
+ // comparing ISA versions would get wrong.
+ EXPECT_TRUE(getTarget("gfx9-4-generic").isGeneration(9));
+ EXPECT_TRUE(getTarget("gfx11-generic").isGeneration(11));
+ EXPECT_TRUE(getTarget("gfx12-generic").isGeneration(12));
+
+ // An unknown target is in no generation.
+ EXPECT_FALSE(getTarget("amdgcn-amd-amdhsa").isGeneration(9));
+}
+
+TEST(TargetInfoTest, DefaultIsUnknown) {
+ TargetInfo target;
+ EXPECT_TRUE(target.isUnknown());
+ EXPECT_FALSE(target.has(llvm::AMDGPU::FEAT_GFX9_INSTS));
+ EXPECT_EQ(target.getArchName(), "");
+ EXPECT_EQ(target.getWavefrontSize(), std::nullopt);
+}
+} // namespace
+} // namespace mlir::ROCDL
More information about the llvm-branch-commits
mailing list