[llvm] [RFC][SPIR-V] Add llvm.arbitrary.fp.convert intrinsic (PR #164252)
Dmitry Sidorov via llvm-commits
llvm-commits at lists.llvm.org
Mon Oct 20 06:23:48 PDT 2025
https://github.com/MrSidims created https://github.com/llvm/llvm-project/pull/164252
The intrinsic performs conversions between values whose interpretation differs from their representation in LLVM IR. The intrinsic is overloaded on both its return type and first argument. Metadata operands describe how the raw bits should be interpreted before and after the conversion.
Current patch adds only lowering to SPIR-V.
Addresses https://discourse.llvm.org/t/rfc-spir-v-way-to-represent-float8-in-llvm-ir/87758/10
>From 9119408f6d8a90607f5b44e7bb78eac30932adf7 Mon Sep 17 00:00:00 2001
From: "Sidorov, Dmitry" <dmitry.sidorov at intel.com>
Date: Mon, 20 Oct 2025 06:14:47 -0700
Subject: [PATCH] [RFC][SPIR-V] Add llvm.arbitrary.fp.convert intrinsic
The intrinsic performs conversions between values whose interpretation differs
from their representation in LLVM IR. The intrinsic is overloaded on both its
return type and first argument. Metadata operands describe how the raw bits
should be interpreted before and after the conversion.
Current patch adds only lowering to SPIR-V.
Signed-off-by: Sidorov, Dmitry <dmitry.sidorov at intel.com>
---
llvm/docs/LangRef.rst | 63 +++
llvm/include/llvm/IR/Intrinsics.td | 8 +
llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp | 5 +-
llvm/lib/IR/Verifier.cpp | 47 ++
llvm/lib/Target/SPIRV/SPIRVCommandLine.cpp | 1 +
llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp | 37 ++
llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.h | 7 +
.../Target/SPIRV/SPIRVInstructionSelector.cpp | 440 ++++++++++++++++++
llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp | 2 +
llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp | 29 ++
llvm/lib/Target/SPIRV/SPIRVPreLegalizer.cpp | 91 ++++
.../lib/Target/SPIRV/SPIRVSymbolicOperands.td | 8 +
llvm/lib/Target/SPIRV/SPIRVUtils.cpp | 17 +
llvm/lib/Target/SPIRV/SPIRVUtils.h | 4 +
.../arbitrary-fp-convert-fp8-half.ll | 242 ++++++++++
llvm/test/Verifier/arbitrary-fp-convert.ll | 66 +++
16 files changed, 1066 insertions(+), 1 deletion(-)
create mode 100644 llvm/test/CodeGen/SPIRV/extensions/SPV_EXT_float8/arbitrary-fp-convert-fp8-half.ll
create mode 100644 llvm/test/Verifier/arbitrary-fp-convert.ll
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index 033910121a54f..fd21ecffa0aed 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -21406,6 +21406,69 @@ environment <floatenv>` *except* for the rounding mode.
This intrinsic is not supported on all targets. Some targets may not support
all rounding modes.
+'``llvm.arbitrary.fp.convert``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+ declare <type> @llvm.arbitrary.fp.convert(
+ <type> <value>, metadata <result interpretation>,
+ metadata <input interpretation>, metadata <rounding mode>,
+ i32 <saturation>)
+
+Overview:
+"""""""""
+
+The ``llvm.arbitrary.fp.convert`` intrinsic performs conversions
+between values whose interpretation differs from their representation
+in LLVM IR. The intrinsic is overloaded on both its return type and first
+argument. Metadata operands describe how the raw bits should be interpreted
+before and after the conversion.
+
+Arguments:
+""""""""""
+
+``value``
+ The value to convert. Its interpretation is described by ``input
+ interpretation``.
+
+``result interpretation``
+ A metadata string that describes the type of the result. The string
+ can be ``"none"`` (no conversion needed), ``"signed"`` or ``"unsigned"`` (for
+ integer types), or any target-specific string for floating-point formats.
+ For example ``"spv.E4M3EXT"`` and ``"spv.E5M2EXT"`` stand for FP8 SPIR-V formats.
+ Using ``"none"`` indicates the converted bits already have the desired LLVM IR type.
+
+``input interpretation``
+ Mirrors ``result interpretation`` but applies to the first argument. The
+ interpretation is target-specific and describes how to interpret the raw bits
+ of the input value.
+
+``rounding mode``
+ A metadata string. The permitted strings match those accepted by
+ :ref:`llvm.fptrunc.round <int_fptrunc_round>` (for example,
+ ``"round.tonearest"`` or ``"round.towardzero"``). The string ``"none"`` may be
+ used to indicate that the default rounding behaviour of the conversion should
+ be used.
+
+``saturation``
+ An integer constant (0 or 1) indicating whether saturation should be applied
+ to the conversion. When set to 1, values outside the representable range of
+ the result type are clamped to the minimum or maximum representable value
+ instead of wrapping. When set to 0, no saturation is applied.
+
+Semantics:
+""""""""""
+
+The intrinsic interprets the first argument according to ``input
+interpretation``, applies the requested rounding mode and saturation behavior,
+and produces a value whose type is described by ``result interpretation``.
+When saturation is enabled, values that exceed the representable range of the target
+format are clamped to the minimum or maximum representable value of that format.
+
Convergence Intrinsics
----------------------
diff --git a/llvm/include/llvm/IR/Intrinsics.td b/llvm/include/llvm/IR/Intrinsics.td
index 12d1c2528f977..b0c8ea1e47fc7 100644
--- a/llvm/include/llvm/IR/Intrinsics.td
+++ b/llvm/include/llvm/IR/Intrinsics.td
@@ -1091,6 +1091,14 @@ let IntrProperties = [IntrNoMem, IntrSpeculatable] in {
def int_fptrunc_round : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
[ llvm_anyfloat_ty, llvm_metadata_ty ]>;
+ // Convert between arbitrary interpreted floating-point and integer values.
+ def int_arbitrary_fp_convert
+ : DefaultAttrsIntrinsic<
+ [ llvm_any_ty ],
+ [ llvm_any_ty, llvm_metadata_ty, llvm_metadata_ty,
+ llvm_metadata_ty, llvm_i32_ty ],
+ [ IntrNoMem, IntrSpeculatable ]>;
+
def int_canonicalize : DefaultAttrsIntrinsic<[llvm_anyfloat_ty], [LLVMMatchType<0>],
[IntrNoMem]>;
// Arithmetic fence intrinsic.
diff --git a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
index 884c3f1692e94..f0a6c7082985e 100644
--- a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
@@ -2842,7 +2842,10 @@ bool IRTranslator::translateCall(const User &U, MachineIRBuilder &MIRBuilder) {
if (!MDN) {
if (auto *ConstMD = dyn_cast<ConstantAsMetadata>(MD))
MDN = MDNode::get(MF->getFunction().getContext(), ConstMD);
- else // This was probably an MDString.
+ else if (auto *MDS = dyn_cast<MDString>(MD)) {
+ Metadata *Ops[] = {MDS};
+ MDN = MDNode::get(MF->getFunction().getContext(), Ops);
+ } else
return false;
}
MIB.addMetadata(MDN);
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index 03da1547b652f..58b80191625c5 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -80,6 +80,7 @@
#include "llvm/IR/Dominators.h"
#include "llvm/IR/EHPersonalities.h"
#include "llvm/IR/Function.h"
+#include "llvm/IR/FPEnv.h"
#include "llvm/IR/GCStrategy.h"
#include "llvm/IR/GlobalAlias.h"
#include "llvm/IR/GlobalValue.h"
@@ -5848,6 +5849,52 @@ void Verifier::visitIntrinsicCall(Intrinsic::ID ID, CallBase &Call) {
"unsupported rounding mode argument", Call);
break;
}
+ case Intrinsic::arbitrary_fp_convert: {
+ auto *ResultMAV = dyn_cast<MetadataAsValue>(Call.getArgOperand(1));
+ Check(ResultMAV, "missing result interpretation metadata operand", Call);
+ auto *ResultStr = dyn_cast<MDString>(ResultMAV->getMetadata());
+ Check(ResultStr, "result interpretation metadata operand must be a string",
+ Call);
+ StringRef ResultInterp = ResultStr->getString();
+
+ auto *InputMAV = dyn_cast<MetadataAsValue>(Call.getArgOperand(2));
+ Check(InputMAV, "missing input interpretation metadata operand", Call);
+ auto *InputStr = dyn_cast<MDString>(InputMAV->getMetadata());
+ Check(InputStr, "input interpretation metadata operand must be a string",
+ Call);
+ StringRef InputInterp = InputStr->getString();
+
+ auto *RoundingMAV = dyn_cast<MetadataAsValue>(Call.getArgOperand(3));
+ Check(RoundingMAV, "missing rounding mode metadata operand", Call);
+ auto *RoundingStr = dyn_cast<MDString>(RoundingMAV->getMetadata());
+ Check(RoundingStr, "rounding mode metadata operand must be a string",
+ Call);
+ StringRef RoundingInterp = RoundingStr->getString();
+
+ // Check that interpretation strings are not empty. The actual interpretation
+ // values are target-specific and not validated here.
+ Check(!ResultInterp.empty(),
+ "result interpretation metadata string must not be empty", Call);
+ Check(!InputInterp.empty(),
+ "input interpretation metadata string must not be empty", Call);
+
+ if (RoundingInterp != "none") {
+ std::optional<RoundingMode> RM =
+ convertStrToRoundingMode(RoundingInterp);
+ Check(RM && *RM != RoundingMode::Dynamic,
+ "unsupported rounding mode argument", Call);
+ }
+
+ // Check saturation parameter (must be 0 or 1)
+ auto *SaturationOp = dyn_cast<ConstantInt>(Call.getArgOperand(4));
+ Check(SaturationOp, "saturation operand must be a constant integer", Call);
+ if (SaturationOp) {
+ uint64_t SatVal = SaturationOp->getZExtValue();
+ Check(SatVal == 0 || SatVal == 1,
+ "saturation operand must be 0 or 1", Call);
+ }
+ break;
+ }
#define BEGIN_REGISTER_VP_INTRINSIC(VPID, ...) case Intrinsic::VPID:
#include "llvm/IR/VPIntrinsics.def"
#undef BEGIN_REGISTER_VP_INTRINSIC
diff --git a/llvm/lib/Target/SPIRV/SPIRVCommandLine.cpp b/llvm/lib/Target/SPIRV/SPIRVCommandLine.cpp
index 96f5dee21bc2a..fe6c5783f61ed 100644
--- a/llvm/lib/Target/SPIRV/SPIRVCommandLine.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVCommandLine.cpp
@@ -149,6 +149,7 @@ static const std::map<std::string, SPIRV::Extension::Extension, std::less<>>
{"SPV_INTEL_tensor_float32_conversion",
SPIRV::Extension::Extension::SPV_INTEL_tensor_float32_conversion},
{"SPV_KHR_bfloat16", SPIRV::Extension::Extension::SPV_KHR_bfloat16},
+ {"SPV_EXT_float8", SPIRV::Extension::Extension::SPV_EXT_float8},
{"SPV_EXT_relaxed_printf_string_address_space",
SPIRV::Extension::Extension::
SPV_EXT_relaxed_printf_string_address_space},
diff --git a/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp b/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp
index 6fd1c7ed78c06..3d13e375c06e4 100644
--- a/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp
@@ -215,6 +215,43 @@ SPIRVGlobalRegistry::getOpTypeFloat(uint32_t Width,
});
}
+SPIRVType *SPIRVGlobalRegistry::getOrCreateOpTypeFloatWithEncoding(
+ uint32_t Width, MachineIRBuilder &MIRBuilder,
+ SPIRV::FPEncoding::FPEncoding FPEncode) {
+ auto Key = std::make_pair(Width, static_cast<unsigned>(FPEncode));
+ if (SPIRVType *Existing = FloatTypesWithEncoding.lookup(Key)) {
+ // Check if the existing type is from the current function
+ const MachineFunction *TypeMF = Existing->getParent()->getParent();
+ if (TypeMF == &MIRBuilder.getMF())
+ return Existing;
+ // Type is from a different function, need to create a new one for current function
+ }
+
+ SPIRVType *SpvType = getOpTypeFloat(Width, MIRBuilder, FPEncode);
+ LLVMContext &Ctx = MIRBuilder.getMF().getFunction().getContext();
+ Type *LLVMTy = nullptr;
+ switch (Width) {
+ case 8:
+ LLVMTy = Type::getInt8Ty(Ctx);
+ break;
+ case 16:
+ LLVMTy = Type::getHalfTy(Ctx);
+ break;
+ case 32:
+ LLVMTy = Type::getFloatTy(Ctx);
+ break;
+ case 64:
+ LLVMTy = Type::getDoubleTy(Ctx);
+ break;
+ default:
+ report_fatal_error("unsupported floating-point width for SPIR-V encoding");
+ }
+
+ SpvType = finishCreatingSPIRVType(LLVMTy, SpvType);
+ FloatTypesWithEncoding.try_emplace(Key, SpvType);
+ return SpvType;
+}
+
SPIRVType *SPIRVGlobalRegistry::getOpTypeVoid(MachineIRBuilder &MIRBuilder) {
return createOpType(MIRBuilder, [&](MachineIRBuilder &MIRBuilder) {
return MIRBuilder.buildInstr(SPIRV::OpTypeVoid)
diff --git a/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.h b/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.h
index a648defa0a888..47353fee10065 100644
--- a/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.h
+++ b/llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.h
@@ -40,6 +40,9 @@ class SPIRVGlobalRegistry : public SPIRVIRMapping {
DenseMap<SPIRVType *, const Type *> SPIRVToLLVMType;
+ DenseMap<std::pair<unsigned, unsigned>, SPIRVType *>
+ FloatTypesWithEncoding;
+
// map a Function to its definition (as a machine instruction operand)
DenseMap<const Function *, const MachineOperand *> FunctionToInstr;
DenseMap<const MachineInstr *, const Function *> FunctionToInstrRev;
@@ -413,6 +416,10 @@ class SPIRVGlobalRegistry : public SPIRVIRMapping {
// Return the number of bits SPIR-V pointers and size_t variables require.
unsigned getPointerSize() const { return PointerSize; }
+ SPIRVType *getOrCreateOpTypeFloatWithEncoding(
+ uint32_t Width, MachineIRBuilder &MIRBuilder,
+ SPIRV::FPEncoding::FPEncoding FPEncode);
+
// Returns true if two types are defined and are compatible in a sense of
// OpBitcast instruction
bool isBitcastCompatible(const SPIRVType *Type1,
diff --git a/llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp b/llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp
index a0cff4d82b500..3963d126d4f73 100644
--- a/llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp
@@ -22,6 +22,8 @@
#include "SPIRVUtils.h"
#include "llvm/ADT/APFloat.h"
#include "llvm/ADT/StringExtras.h"
+#include "llvm/ADT/StringSwitch.h"
+#include "llvm/IR/FPEnv.h"
#include "llvm/CodeGen/GlobalISel/GIMatchTableExecutorImpl.h"
#include "llvm/CodeGen/GlobalISel/GenericMachineInstrs.h"
#include "llvm/CodeGen/GlobalISel/InstructionSelector.h"
@@ -195,6 +197,9 @@ class SPIRVInstructionSelector : public InstructionSelector {
bool selectFloatDot(Register ResVReg, const SPIRVType *ResType,
MachineInstr &I) const;
+ bool selectArbitraryFPConvert(Register ResVReg, const SPIRVType *ResType,
+ MachineInstr &I) const;
+
bool selectOverflowArith(Register ResVReg, const SPIRVType *ResType,
MachineInstr &I, unsigned Opcode) const;
bool selectDebugTrap(Register ResVReg, const SPIRVType *ResType,
@@ -2101,6 +2106,439 @@ bool SPIRVInstructionSelector::selectFloatDot(Register ResVReg,
.constrainAllUses(TII, TRI, RBI);
}
+static std::optional<SPIRV::FPEncoding::FPEncoding>
+getFloat8EncodingFromString(StringRef Interpretation) {
+ return StringSwitch<std::optional<SPIRV::FPEncoding::FPEncoding>>(Interpretation)
+ .Case("spv.E4M3EXT", SPIRV::FPEncoding::Float8E4M3EXT)
+ .Case("spv.E5M2EXT", SPIRV::FPEncoding::Float8E5M2EXT)
+ .Default(std::nullopt);
+}
+
+// Enum to classify interpretation types
+enum class InterpretationType {
+ None,
+ Signed,
+ Unsigned,
+ Float8E4M3,
+ Float8E5M2,
+ Unknown
+};
+
+static InterpretationType classifyInterpretation(StringRef Interp) {
+ return StringSwitch<InterpretationType>(Interp)
+ .Case("none", InterpretationType::None)
+ .Case("signed", InterpretationType::Signed)
+ .Case("unsigned", InterpretationType::Unsigned)
+ .Case("spv.E4M3EXT", InterpretationType::Float8E4M3)
+ .Case("spv.E5M2EXT", InterpretationType::Float8E5M2)
+ .Default(InterpretationType::Unknown);
+}
+
+static std::optional<SPIRV::FPEncoding::FPEncoding>
+interpretationToFP8Encoding(InterpretationType Type) {
+ switch (Type) {
+ case InterpretationType::Float8E4M3:
+ return SPIRV::FPEncoding::Float8E4M3EXT;
+ case InterpretationType::Float8E5M2:
+ return SPIRV::FPEncoding::Float8E5M2EXT;
+ default:
+ return std::nullopt;
+ }
+}
+
+// Helper struct to hold parsed intrinsic parameters
+struct ArbitraryConvertParams {
+ Register SrcReg;
+ StringRef ResultInterp;
+ StringRef InputInterp;
+ StringRef RoundingInterp;
+ bool UseSaturation;
+
+ InterpretationType SrcType;
+ InterpretationType DstType;
+
+ static std::optional<ArbitraryConvertParams>
+ parse(const MachineInstr &I, const MachineRegisterInfo *MRI) {
+ unsigned IntrinsicIdx = I.getNumDefs();
+ if (IntrinsicIdx >= I.getNumOperands())
+ return std::nullopt;
+
+ unsigned ValueIdx = IntrinsicIdx + 1;
+ if (ValueIdx + 4 >= I.getNumOperands())
+ return std::nullopt;
+
+ const MachineOperand &ValueOp = I.getOperand(ValueIdx);
+ if (!ValueOp.isReg())
+ return std::nullopt;
+
+ auto GetStringFromMD = [&](unsigned OperandIdx) -> std::optional<StringRef> {
+ const MachineOperand &Op = I.getOperand(OperandIdx);
+ if (!Op.isMetadata())
+ return std::nullopt;
+ const MDNode *MD = Op.getMetadata();
+ if (!MD || MD->getNumOperands() != 1)
+ return std::nullopt;
+ if (auto *Str = dyn_cast<MDString>(MD->getOperand(0)))
+ return Str->getString();
+ return std::nullopt;
+ };
+
+ std::optional<StringRef> ResultInterp = GetStringFromMD(ValueIdx + 1);
+ std::optional<StringRef> InputInterp = GetStringFromMD(ValueIdx + 2);
+ std::optional<StringRef> RoundingInterp = GetStringFromMD(ValueIdx + 3);
+ if (!ResultInterp || !InputInterp || !RoundingInterp)
+ return std::nullopt;
+
+ // Get saturation parameter
+ const MachineOperand &SaturationOp = I.getOperand(ValueIdx + 4);
+ int64_t SaturationValue;
+ if (SaturationOp.isImm()) {
+ SaturationValue = SaturationOp.getImm();
+ } else if (SaturationOp.isReg()) {
+ SaturationValue = foldImm(SaturationOp, MRI);
+ } else {
+ return std::nullopt;
+ }
+
+ ArbitraryConvertParams Params;
+ Params.SrcReg = ValueOp.getReg();
+ Params.ResultInterp = *ResultInterp;
+ Params.InputInterp = *InputInterp;
+ Params.RoundingInterp = *RoundingInterp;
+ Params.UseSaturation = SaturationValue != 0;
+
+ Params.SrcType = classifyInterpretation(Params.InputInterp);
+ Params.DstType = classifyInterpretation(Params.ResultInterp);
+
+ return Params;
+ }
+
+ // Helper methods for type checking
+ bool isSrcFP8() const {
+ return SrcType == InterpretationType::Float8E4M3 ||
+ SrcType == InterpretationType::Float8E5M2;
+ }
+
+ bool isDstFP8() const {
+ return DstType == InterpretationType::Float8E4M3 ||
+ DstType == InterpretationType::Float8E5M2;
+ }
+
+ std::optional<SPIRV::FPEncoding::FPEncoding> getSrcFP8Encoding() const {
+ return interpretationToFP8Encoding(SrcType);
+ }
+
+ std::optional<SPIRV::FPEncoding::FPEncoding> getDstFP8Encoding() const {
+ return interpretationToFP8Encoding(DstType);
+ }
+};
+
+// Helper function to create Float8 type (scalar or vector)
+static SPIRVType *createFloat8Type(unsigned ComponentCount,
+ SPIRV::FPEncoding::FPEncoding Encoding,
+ MachineIRBuilder &MIRBuilder,
+ SPIRVGlobalRegistry &GR) {
+ SPIRVType *Float8ScalarType =
+ GR.getOrCreateOpTypeFloatWithEncoding(8, MIRBuilder, Encoding);
+ if (ComponentCount > 1)
+ return GR.getOrCreateSPIRVVectorType(Float8ScalarType, ComponentCount,
+ MIRBuilder, false);
+ return Float8ScalarType;
+}
+
+// Helper function to build bitcast if type conversion is needed
+static std::optional<Register>
+buildBitcastIfNeeded(Register SrcReg, SPIRVType *SrcType, SPIRVType *TargetType,
+ MachineInstr &I, const TargetInstrInfo &TII,
+ const TargetRegisterInfo &TRI,
+ const RegisterBankInfo &RBI, MachineRegisterInfo *MRI,
+ SPIRVGlobalRegistry &GR) {
+ if (SrcType == TargetType)
+ return SrcReg;
+
+ Register CastReg = MRI->createVirtualRegister(&SPIRV::iIDRegClass);
+ GR.assignSPIRVTypeToVReg(TargetType, CastReg, *I.getMF());
+ auto BitcastMIB =
+ BuildMI(*I.getParent(), I, I.getDebugLoc(), TII.get(SPIRV::OpBitcast))
+ .addDef(CastReg)
+ .addUse(GR.getSPIRVTypeID(TargetType))
+ .addUse(SrcReg);
+ if (!BitcastMIB.constrainAllUses(TII, TRI, RBI))
+ return std::nullopt;
+ return CastReg;
+}
+
+bool SPIRVInstructionSelector::selectArbitraryFPConvert(
+ Register ResVReg, const SPIRVType *ResType, MachineInstr &I) const {
+ // Parse intrinsic parameters
+ std::optional<ArbitraryConvertParams> MaybeParams =
+ ArbitraryConvertParams::parse(I, MRI);
+ if (!MaybeParams)
+ return false;
+
+ const ArbitraryConvertParams &Params = *MaybeParams;
+ Register SrcReg = Params.SrcReg;
+ SPIRVType *SrcType = GR.getSPIRVTypeForVReg(SrcReg);
+ LLT SrcLLT = MRI->getType(SrcReg);
+
+ // Parse and validate rounding mode
+ bool RoundingNone = Params.RoundingInterp == "none";
+ std::optional<RoundingMode> RM;
+ if (!RoundingNone) {
+ RM = convertStrToRoundingMode(Params.RoundingInterp);
+ if (!RM || *RM == RoundingMode::Dynamic ||
+ *RM == RoundingMode::NearestTiesToAway)
+ return false;
+ }
+
+ auto GetComponentInfo = [&](const SPIRVType *Type)
+ -> std::pair<const SPIRVType *, unsigned> {
+ if (!Type)
+ return {nullptr, 0};
+ return {GR.getScalarOrVectorComponentType(Type),
+ GR.getScalarOrVectorComponentCount(Type)};
+ };
+
+ MachineIRBuilder MIRBuilder(I);
+
+ // Conversion path 1: FP8 -> Float (e.g., spv.E4M3EXT -> none)
+ if (Params.DstType == InterpretationType::None && Params.isSrcFP8()) {
+ if (RM)
+ return false;
+
+ auto [ResScalarType, ComponentCount] = GetComponentInfo(ResType);
+ if (!ResScalarType || ResScalarType->getOpcode() != SPIRV::OpTypeFloat)
+ return false;
+
+ unsigned Width = ResScalarType->getOperand(1).getImm();
+ if (Width != 16 && Width != 32 && Width != 64)
+ return false;
+
+ unsigned SrcComponentCount = 0;
+ if (SrcType) {
+ SrcComponentCount = GR.getScalarOrVectorComponentCount(SrcType);
+ } else {
+ if (!SrcLLT.isValid())
+ return false;
+ SrcComponentCount = SrcLLT.isVector() ? SrcLLT.getNumElements() : 1;
+ }
+ if (SrcComponentCount != ComponentCount)
+ return false;
+
+ SPIRVType *Float8Type =
+ createFloat8Type(ComponentCount, *Params.getSrcFP8Encoding(), MIRBuilder, GR);
+
+ std::optional<Register> Float8Reg = buildBitcastIfNeeded(
+ SrcReg, SrcType, Float8Type, I, TII, TRI, RBI, MRI, GR);
+ if (!Float8Reg)
+ return false;
+
+ if (!RBI.constrainGenericRegister(ResVReg, SPIRV::iIDRegClass, *MRI))
+ return false;
+ auto MIB = BuildMI(*I.getParent(), I, I.getDebugLoc(),
+ TII.get(SPIRV::OpFConvert))
+ .addDef(ResVReg)
+ .addUse(GR.getSPIRVTypeID(ResType))
+ .addUse(*Float8Reg);
+ return MIB.constrainAllUses(TII, TRI, RBI);
+ }
+
+ // Conversion path 2: Float -> FP8 (e.g., none -> spv.E5M2EXT)
+ if (Params.SrcType == InterpretationType::None && Params.isDstFP8()) {
+ auto [SrcScalarType, ComponentCount] = GetComponentInfo(SrcType);
+ if (SrcType) {
+ if (!SrcScalarType || SrcScalarType->getOpcode() != SPIRV::OpTypeFloat)
+ return false;
+
+ unsigned Width = SrcScalarType->getOperand(1).getImm();
+ if (Width != 16 && Width != 32 && Width != 64)
+ return false;
+ } else {
+ if (!SrcLLT.isValid())
+ return false;
+ if (!SrcLLT.isScalar() && !SrcLLT.isVector())
+ return false;
+ unsigned Width = SrcLLT.getScalarSizeInBits();
+ if (Width != 16 && Width != 32 && Width != 64)
+ return false;
+ ComponentCount = SrcLLT.isVector() ? SrcLLT.getNumElements() : 1;
+ SrcScalarType = nullptr;
+ }
+
+ if (GR.getScalarOrVectorComponentCount(ResType) != ComponentCount)
+ return false;
+
+ SPIRVType *Float8Type =
+ createFloat8Type(ComponentCount, *Params.getDstFP8Encoding(), MIRBuilder, GR);
+
+ Register ConvertedReg =
+ MRI->createVirtualRegister(&SPIRV::iIDRegClass);
+ GR.assignSPIRVTypeToVReg(Float8Type, ConvertedReg, *I.getMF());
+
+ auto ConvertMIB = BuildMI(*I.getParent(), I, I.getDebugLoc(),
+ TII.get(SPIRV::OpFConvert))
+ .addDef(ConvertedReg)
+ .addUse(GR.getSPIRVTypeID(Float8Type))
+ .addUse(SrcReg);
+ if (!ConvertMIB.constrainAllUses(TII, TRI, RBI))
+ return false;
+
+ if (RM) {
+ auto MaybeRM = toSPIRVRoundingMode(*RM);
+ if (!MaybeRM)
+ return false;
+ buildOpDecorate(ConvertedReg, I, TII,
+ SPIRV::Decoration::FPRoundingMode,
+ {static_cast<uint32_t>(*MaybeRM)});
+ } else if (!RoundingNone) {
+ return false;
+ }
+
+ // Add saturation decoration if requested
+ if (Params.UseSaturation) {
+ buildOpDecorate(ConvertedReg, I, TII,
+ SPIRV::Decoration::SaturatedToLargestFloat8NormalConversionEXT,
+ {});
+ }
+
+ if (!RBI.constrainGenericRegister(ResVReg, SPIRV::iIDRegClass, *MRI))
+ return false;
+ auto BitcastMIB = BuildMI(*I.getParent(), I, I.getDebugLoc(),
+ TII.get(SPIRV::OpBitcast))
+ .addDef(ResVReg)
+ .addUse(GR.getSPIRVTypeID(ResType))
+ .addUse(ConvertedReg);
+ return BitcastMIB.constrainAllUses(TII, TRI, RBI);
+ }
+
+ // Conversion path 3: FP8 -> Int (e.g., spv.E4M3EXT -> signed/unsigned)
+ if ((Params.DstType == InterpretationType::Signed ||
+ Params.DstType == InterpretationType::Unsigned) && Params.isSrcFP8()) {
+ if (RM)
+ return false;
+
+ auto [ResScalarType, ComponentCount] = GetComponentInfo(ResType);
+ if (!ResScalarType || ResScalarType->getOpcode() != SPIRV::OpTypeInt)
+ return false;
+
+ unsigned ResultWidth = ResScalarType->getOperand(1).getImm();
+ if (ResultWidth != 8 && ResultWidth != 16 && ResultWidth != 32 &&
+ ResultWidth != 64)
+ return false;
+
+ unsigned SrcComponentCount = 0;
+ if (SrcType) {
+ auto [SrcScalarType, Count] = GetComponentInfo(SrcType);
+ if (!SrcScalarType || SrcScalarType->getOpcode() != SPIRV::OpTypeInt ||
+ SrcScalarType->getOperand(1).getImm() != 8)
+ return false;
+ SrcComponentCount = Count;
+ } else {
+ if (!SrcLLT.isValid())
+ return false;
+ if (!SrcLLT.isScalar() && !SrcLLT.isVector())
+ return false;
+ if (SrcLLT.getScalarSizeInBits() != 8)
+ return false;
+ SrcComponentCount = SrcLLT.isVector() ? SrcLLT.getNumElements() : 1;
+ }
+
+ if (SrcComponentCount != ComponentCount)
+ return false;
+
+ SPIRVType *Float8Type =
+ createFloat8Type(ComponentCount, *Params.getSrcFP8Encoding(), MIRBuilder, GR);
+
+ std::optional<Register> Float8Reg = buildBitcastIfNeeded(
+ SrcReg, SrcType, Float8Type, I, TII, TRI, RBI, MRI, GR);
+ if (!Float8Reg)
+ return false;
+
+ if (!RBI.constrainGenericRegister(ResVReg, SPIRV::iIDRegClass, *MRI))
+ return false;
+
+ unsigned Opcode = Params.DstType == InterpretationType::Signed
+ ? SPIRV::OpConvertFToS
+ : SPIRV::OpConvertFToU;
+ auto MIB = BuildMI(*I.getParent(), I, I.getDebugLoc(), TII.get(Opcode))
+ .addDef(ResVReg)
+ .addUse(GR.getSPIRVTypeID(ResType))
+ .addUse(*Float8Reg);
+ return MIB.constrainAllUses(TII, TRI, RBI);
+ }
+
+ // Conversion path 4: Int -> FP8 (e.g., signed/unsigned -> spv.E5M2EXT)
+ if ((Params.SrcType == InterpretationType::Signed ||
+ Params.SrcType == InterpretationType::Unsigned) && Params.isDstFP8()) {
+ if (RM)
+ return false;
+
+ unsigned ComponentCount = 0;
+ unsigned SrcWidth = 0;
+ if (SrcType) {
+ auto [SrcScalarType, Count] = GetComponentInfo(SrcType);
+ if (!SrcScalarType || SrcScalarType->getOpcode() != SPIRV::OpTypeInt)
+ return false;
+ SrcWidth = SrcScalarType->getOperand(1).getImm();
+ ComponentCount = Count;
+ } else {
+ if (!SrcLLT.isValid())
+ return false;
+ if (!SrcLLT.isScalar() && !SrcLLT.isVector())
+ return false;
+ SrcWidth = SrcLLT.getScalarSizeInBits();
+ ComponentCount = SrcLLT.isVector() ? SrcLLT.getNumElements() : 1;
+ }
+
+ if (SrcWidth != 8 && SrcWidth != 16 && SrcWidth != 32 && SrcWidth != 64)
+ return false;
+
+ auto [ResScalarType, ResComponentCount] = GetComponentInfo(ResType);
+ if (!ResScalarType || ResScalarType->getOpcode() != SPIRV::OpTypeInt ||
+ ResScalarType->getOperand(1).getImm() != 8)
+ return false;
+
+ if (ResComponentCount != ComponentCount)
+ return false;
+
+ SPIRVType *Float8Type =
+ createFloat8Type(ComponentCount, *Params.getDstFP8Encoding(), MIRBuilder, GR);
+
+ Register ConvertedReg =
+ MRI->createVirtualRegister(&SPIRV::iIDRegClass);
+ GR.assignSPIRVTypeToVReg(Float8Type, ConvertedReg, *I.getMF());
+
+ unsigned Opcode = Params.SrcType == InterpretationType::Signed
+ ? SPIRV::OpConvertSToF
+ : SPIRV::OpConvertUToF;
+ auto ConvertMIB = BuildMI(*I.getParent(), I, I.getDebugLoc(),
+ TII.get(Opcode))
+ .addDef(ConvertedReg)
+ .addUse(GR.getSPIRVTypeID(Float8Type))
+ .addUse(SrcReg);
+ if (!ConvertMIB.constrainAllUses(TII, TRI, RBI))
+ return false;
+
+ // Add saturation decoration if requested
+ if (Params.UseSaturation) {
+ buildOpDecorate(ConvertedReg, I, TII,
+ SPIRV::Decoration::SaturatedToLargestFloat8NormalConversionEXT,
+ {});
+ }
+
+ if (!RBI.constrainGenericRegister(ResVReg, SPIRV::iIDRegClass, *MRI))
+ return false;
+ auto BitcastMIB = BuildMI(*I.getParent(), I, I.getDebugLoc(),
+ TII.get(SPIRV::OpBitcast))
+ .addDef(ResVReg)
+ .addUse(GR.getSPIRVTypeID(ResType))
+ .addUse(ConvertedReg);
+ return BitcastMIB.constrainAllUses(TII, TRI, RBI);
+ }
+
+ return false;
+}
+
bool SPIRVInstructionSelector::selectIntegerDot(Register ResVReg,
const SPIRVType *ResType,
MachineInstr &I,
@@ -3440,6 +3878,8 @@ bool SPIRVInstructionSelector::selectIntrinsic(Register ResVReg,
return selectExtInst(ResVReg, ResType, I, CL::step, GL::Step);
case Intrinsic::spv_radians:
return selectExtInst(ResVReg, ResType, I, CL::radians, GL::Radians);
+ case Intrinsic::arbitrary_fp_convert:
+ return selectArbitraryFPConvert(ResVReg, ResType, I);
// Discard intrinsics which we do not expect to actually represent code after
// lowering or intrinsics which are not implemented but should not crash when
// found in a customer's LLVM IR input.
diff --git a/llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp b/llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp
index 53074ea3b2597..21f5cda6136a0 100644
--- a/llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp
@@ -14,11 +14,13 @@
#include "SPIRV.h"
#include "SPIRVGlobalRegistry.h"
#include "SPIRVSubtarget.h"
+#include "llvm/CodeGen/GlobalISel/GenericMachineInstrs.h"
#include "llvm/CodeGen/GlobalISel/LegalizerHelper.h"
#include "llvm/CodeGen/GlobalISel/MachineIRBuilder.h"
#include "llvm/CodeGen/MachineInstr.h"
#include "llvm/CodeGen/MachineRegisterInfo.h"
#include "llvm/CodeGen/TargetOpcodes.h"
+#include "llvm/IR/Intrinsics.h"
using namespace llvm;
using namespace llvm::LegalizeActions;
diff --git a/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp b/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp
index 61a0bbef90891..26688e40227c5 100644
--- a/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp
@@ -1336,6 +1336,15 @@ static bool isBFloat16Type(const SPIRVType *TypeDef) {
TypeDef->getOperand(2).getImm() == SPIRV::FPEncoding::BFloat16KHR;
}
+static bool isFloat8Type(const SPIRVType *TypeDef) {
+ if (!TypeDef || TypeDef->getOpcode() != SPIRV::OpTypeFloat ||
+ TypeDef->getNumOperands() != 3 || TypeDef->getOperand(1).getImm() != 8)
+ return false;
+ uint32_t Encoding = TypeDef->getOperand(2).getImm();
+ return Encoding == SPIRV::FPEncoding::Float8E4M3EXT ||
+ Encoding == SPIRV::FPEncoding::Float8E5M2EXT;
+}
+
void addInstrRequirements(const MachineInstr &MI,
SPIRV::ModuleAnalysisInfo &MAI,
const SPIRVSubtarget &ST) {
@@ -1398,6 +1407,17 @@ void addInstrRequirements(const MachineInstr &MI,
} else {
Reqs.addCapability(SPIRV::Capability::Float16);
}
+ } else if (BitWidth == 8) {
+ if (!isFloat8Type(&MI))
+ report_fatal_error("OpTypeFloat type with width 8 requires a Float8 "
+ "encoding",
+ false);
+ if (!ST.canUseExtension(SPIRV::Extension::SPV_EXT_float8))
+ report_fatal_error("OpTypeFloat type with Float8 encoding requires the "
+ "following SPIR-V extension: SPV_EXT_float8",
+ false);
+ Reqs.addExtension(SPIRV::Extension::SPV_EXT_float8);
+ Reqs.addCapability(SPIRV::Capability::Float8EXT);
}
break;
}
@@ -1744,6 +1764,15 @@ void addInstrRequirements(const MachineInstr &MI,
SPIRVType *TypeDef = MRI.getVRegDef(MI.getOperand(1).getReg());
if (isBFloat16Type(TypeDef))
Reqs.addCapability(SPIRV::Capability::BFloat16CooperativeMatrixKHR);
+ if (isFloat8Type(TypeDef)) {
+ if (!ST.canUseExtension(SPIRV::Extension::SPV_EXT_float8))
+ report_fatal_error(
+ "OpTypeCooperativeMatrixKHR with Float8 component requires the "
+ "following SPIR-V extension: SPV_EXT_float8",
+ false);
+ Reqs.addExtension(SPIRV::Extension::SPV_EXT_float8);
+ Reqs.addCapability(SPIRV::Capability::Float8CooperativeMatrixEXT);
+ }
break;
}
case SPIRV::OpArithmeticFenceEXT:
diff --git a/llvm/lib/Target/SPIRV/SPIRVPreLegalizer.cpp b/llvm/lib/Target/SPIRV/SPIRVPreLegalizer.cpp
index db6f2d61e8f29..31f15f6bc3008 100644
--- a/llvm/lib/Target/SPIRV/SPIRVPreLegalizer.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVPreLegalizer.cpp
@@ -18,14 +18,22 @@
#include "llvm/ADT/PostOrderIterator.h"
#include "llvm/CodeGen/GlobalISel/CSEInfo.h"
#include "llvm/CodeGen/GlobalISel/GISelValueTracking.h"
+#include "llvm/CodeGen/GlobalISel/GenericMachineInstrs.h"
#include "llvm/IR/Attributes.h"
#include "llvm/IR/Constants.h"
+#include "llvm/IR/DerivedTypes.h"
#include "llvm/IR/IntrinsicsSPIRV.h"
#define DEBUG_TYPE "spirv-prelegalizer"
using namespace llvm;
+namespace llvm {
+void insertAssignInstr(Register Reg, Type *Ty, SPIRVType *SpvType,
+ SPIRVGlobalRegistry *GR, MachineIRBuilder &MIB,
+ MachineRegisterInfo &MRI);
+}
+
namespace {
class SPIRVPreLegalizer : public MachineFunctionPass {
public:
@@ -351,6 +359,89 @@ static SPIRVType *propagateSPIRVType(MachineInstr *MI, SPIRVGlobalRegistry *GR,
SpvType = propagateSPIRVType(Def, GR, MRI, MIB);
break;
}
+ case TargetOpcode::G_INTRINSIC: {
+ auto *GI = dyn_cast<GIntrinsic>(MI);
+ if (!GI)
+ break;
+ if (GI->getIntrinsicID() != Intrinsic::arbitrary_fp_convert)
+ break;
+
+ unsigned IntrinsicIdx = MI->getNumExplicitDefs();
+ if (IntrinsicIdx >= MI->getNumOperands())
+ break;
+ unsigned ValueIdx = IntrinsicIdx + 1;
+ if (ValueIdx + 3 >= MI->getNumOperands())
+ break;
+
+ auto GetStringFromMD = [&](unsigned OperandIdx)
+ -> std::optional<StringRef> {
+ const MachineOperand &OpMO = MI->getOperand(OperandIdx);
+ if (!OpMO.isMetadata())
+ return std::nullopt;
+ const MDNode *MD = OpMO.getMetadata();
+ if (!MD || MD->getNumOperands() != 1)
+ return std::nullopt;
+ if (auto *Str = dyn_cast<MDString>(MD->getOperand(0)))
+ return Str->getString();
+ return std::nullopt;
+ };
+
+ std::optional<StringRef> ResultInterp = GetStringFromMD(ValueIdx + 1);
+ if (!ResultInterp)
+ break;
+
+ LLT ResLLT = MRI.getType(Reg);
+ if (!ResLLT.isValid())
+ break;
+
+ unsigned NumElements = ResLLT.isVector() ? ResLLT.getNumElements() : 1;
+ unsigned Width = ResLLT.getScalarSizeInBits();
+ LLVMContext &Ctx = MIB.getMF().getFunction().getContext();
+
+ Type *ScalarTy = nullptr;
+ if (*ResultInterp == "none") {
+ switch (Width) {
+ case 16:
+ ScalarTy = Type::getHalfTy(Ctx);
+ break;
+ case 32:
+ ScalarTy = Type::getFloatTy(Ctx);
+ break;
+ case 64:
+ ScalarTy = Type::getDoubleTy(Ctx);
+ break;
+ default:
+ ScalarTy = Type::getIntNTy(Ctx, Width);
+ break;
+ }
+ } else if (*ResultInterp == "signed" || *ResultInterp == "unsigned") {
+ ScalarTy = Type::getIntNTy(Ctx, Width);
+ } else {
+ // For target-specific interpretations (e.g., "spv.E4M3EXT"), use the
+ // width from the result LLT to determine the appropriate LLVM type.
+ // FP8 types (width 8) are represented as i8 in LLVM.
+ if (Width == 8)
+ ScalarTy = Type::getInt8Ty(Ctx);
+ else
+ ScalarTy = Type::getIntNTy(Ctx, Width);
+ }
+
+ if (!ScalarTy)
+ break;
+
+ Type *ResTy = ScalarTy;
+ if (NumElements > 1)
+ ResTy = FixedVectorType::get(ScalarTy, NumElements);
+
+ SPIRVType *AssignedType = GR->getOrCreateSPIRVType(
+ ResTy, MIB, SPIRV::AccessQualifier::ReadWrite, true);
+ GR->assignSPIRVTypeToVReg(AssignedType, Reg, MIB.getMF());
+ if (!MRI.getType(Reg).isValid())
+ MRI.setType(Reg, ResLLT);
+ MRI.setRegClass(Reg, &SPIRV::iIDRegClass);
+ SpvType = AssignedType;
+ break;
+ }
default:
break;
}
diff --git a/llvm/lib/Target/SPIRV/SPIRVSymbolicOperands.td b/llvm/lib/Target/SPIRV/SPIRVSymbolicOperands.td
index 7d08b29a51a6e..3351c432cd88e 100644
--- a/llvm/lib/Target/SPIRV/SPIRVSymbolicOperands.td
+++ b/llvm/lib/Target/SPIRV/SPIRVSymbolicOperands.td
@@ -387,6 +387,7 @@ defm SPV_INTEL_tensor_float32_conversion : ExtensionOperand<125, [EnvOpenCL]>;
defm SPV_KHR_bfloat16 : ExtensionOperand<126, [EnvVulkan, EnvOpenCL]>;
defm SPV_INTEL_predicated_io : ExtensionOperand<127, [EnvOpenCL]>;
defm SPV_KHR_maximal_reconvergence : ExtensionOperand<128, [EnvVulkan]>;
+defm SPV_EXT_float8 : ExtensionOperand<127, [EnvVulkan, EnvOpenCL]>;
//===----------------------------------------------------------------------===//
// Multiclass used to define Capabilities enum values and at the same time
@@ -603,6 +604,9 @@ defm TensorFloat32RoundingINTEL : CapabilityOperand<6425, 0, 0, [SPV_INTEL_tenso
defm BFloat16TypeKHR : CapabilityOperand<5116, 0, 0, [SPV_KHR_bfloat16], []>;
defm BFloat16DotProductKHR : CapabilityOperand<5117, 0, 0, [SPV_KHR_bfloat16], [BFloat16TypeKHR]>;
defm BFloat16CooperativeMatrixKHR : CapabilityOperand<5118, 0, 0, [SPV_KHR_bfloat16], [BFloat16TypeKHR, CooperativeMatrixKHR]>;
+defm Float8EXT : CapabilityOperand<4212, 0, 0, [SPV_EXT_float8], []>;
+defm Float8CooperativeMatrixEXT : CapabilityOperand<4213, 0, 0, [SPV_EXT_float8],
+ [Float8EXT, CooperativeMatrixKHR]>;
//===----------------------------------------------------------------------===//
// Multiclass used to define SourceLanguage enum values and at the same time
@@ -1352,6 +1356,8 @@ defm AlignmentId : DecorationOperand<46, 0, 0, [], [Kernel]>;
defm MaxByteOffsetId : DecorationOperand<47, 0, 0, [], [Addresses]>;
defm NoSignedWrap : DecorationOperand<4469, 0x10400, 0, [SPV_KHR_no_integer_wrap_decoration], []>;
defm NoUnsignedWrap : DecorationOperand<4470, 0x10400, 0, [SPV_KHR_no_integer_wrap_decoration], []>;
+defm SaturatedToLargestFloat8NormalConversionEXT
+ : DecorationOperand<4216, 0, 0, [SPV_EXT_float8], [Float8EXT]>;
defm ExplicitInterpAMD : DecorationOperand<4999, 0, 0, [], []>;
defm OverrideCoverageNV : DecorationOperand<5248, 0, 0, [], [SampleMaskOverrideCoverageNV]>;
defm PassthroughNV : DecorationOperand<5250, 0, 0, [], [GeometryShaderPassthroughNV]>;
@@ -2035,6 +2041,8 @@ multiclass FPEncodingOperand<bits<32> value, list<Extension> reqExtensions>{
}
defm BFloat16KHR : FPEncodingOperand<0, [SPV_KHR_bfloat16]>;
+defm Float8E4M3EXT : FPEncodingOperand<4214, [SPV_EXT_float8]>;
+defm Float8E5M2EXT : FPEncodingOperand<4215, [SPV_EXT_float8]>;
def PackedVectorFormats : GenericEnum, Operand<i32> {
let FilterClass = "PackedVectorFormats";
diff --git a/llvm/lib/Target/SPIRV/SPIRVUtils.cpp b/llvm/lib/Target/SPIRV/SPIRVUtils.cpp
index 1d47c892d03b1..afdaa96c0b6db 100644
--- a/llvm/lib/Target/SPIRV/SPIRVUtils.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVUtils.cpp
@@ -356,6 +356,23 @@ SPIRV::Scope::Scope getMemScope(LLVMContext &Ctx, SyncScope::ID Id) {
return SPIRV::Scope::CrossDevice;
}
+std::optional<SPIRV::FPRoundingMode::FPRoundingMode>
+toSPIRVRoundingMode(RoundingMode Mode) {
+ using namespace SPIRV::FPRoundingMode;
+ switch (Mode) {
+ case RoundingMode::NearestTiesToEven:
+ return FPRoundingMode::RTE;
+ case RoundingMode::TowardNegative:
+ return FPRoundingMode::RTN;
+ case RoundingMode::TowardPositive:
+ return FPRoundingMode::RTP;
+ case RoundingMode::TowardZero:
+ return FPRoundingMode::RTZ;
+ default:
+ return std::nullopt;
+ }
+}
+
MachineInstr *getDefInstrMaybeConstant(Register &ConstReg,
const MachineRegisterInfo *MRI) {
MachineInstr *MI = MRI->getVRegDef(ConstReg);
diff --git a/llvm/lib/Target/SPIRV/SPIRVUtils.h b/llvm/lib/Target/SPIRV/SPIRVUtils.h
index 5777a24faabed..a1b91715363c3 100644
--- a/llvm/lib/Target/SPIRV/SPIRVUtils.h
+++ b/llvm/lib/Target/SPIRV/SPIRVUtils.h
@@ -280,6 +280,10 @@ SPIRV::MemorySemantics::MemorySemantics getMemSemantics(AtomicOrdering Ord);
SPIRV::Scope::Scope getMemScope(LLVMContext &Ctx, SyncScope::ID Id);
+// Convert RoundingMode to SPIR-V FPRoundingMode
+std::optional<SPIRV::FPRoundingMode::FPRoundingMode>
+toSPIRVRoundingMode(RoundingMode Mode);
+
// Find def instruction for the given ConstReg, walking through
// spv_track_constant and ASSIGN_TYPE instructions. Updates ConstReg by def
// of OpConstant instruction.
diff --git a/llvm/test/CodeGen/SPIRV/extensions/SPV_EXT_float8/arbitrary-fp-convert-fp8-half.ll b/llvm/test/CodeGen/SPIRV/extensions/SPV_EXT_float8/arbitrary-fp-convert-fp8-half.ll
new file mode 100644
index 0000000000000..de904d409bf96
--- /dev/null
+++ b/llvm/test/CodeGen/SPIRV/extensions/SPV_EXT_float8/arbitrary-fp-convert-fp8-half.ll
@@ -0,0 +1,242 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 6
+; RUN: llc -verify-machineinstrs -mtriple=spirv32-unknown-unknown --spirv-ext=+SPV_EXT_float8 < %s | FileCheck %s
+
+; CHECK-DAG: OpExtension "SPV_EXT_float8"
+; CHECK-DAG: OpCapability Float8EXT
+; CHECK-DAG: OpDecorate %[[HALF_TO_FP8:[0-9]+]] FPRoundingMode RTZ
+; CHECK-DAG: OpDecorate %[[HALF_TO_FP8_SAT:[0-9]+]] SaturatedToLargestFloat8NormalConversionEXT
+; CHECK-DAG: %[[HALF:[0-9]+]] = OpTypeFloat 16
+; CHECK-DAG: %[[FLOAT:[0-9]+]] = OpTypeFloat 32
+; CHECK-DAG: %[[DOUBLE:[0-9]+]] = OpTypeFloat 64
+; CHECK-DAG: %[[FP8E4:[0-9]+]] = OpTypeFloat 8 4214
+; CHECK-DAG: %[[FP8E5:[0-9]+]] = OpTypeFloat 8 4215
+; CHECK-DAG: %[[INT8:[0-9]+]] = OpTypeInt 8 {{[01]}}
+; CHECK-DAG: %[[INT16:[0-9]+]] = OpTypeInt 16 {{[01]}}
+; CHECK-DAG: %[[INT32:[0-9]+]] = OpTypeInt 32 {{[01]}}
+; CHECK-DAG: %[[INT64:[0-9]+]] = OpTypeInt 64 {{[01]}}
+; CHECK-DAG: %[[VHALF:[0-9]+]] = OpTypeVector %[[HALF]] 4
+; CHECK-DAG: %[[VFP8:[0-9]+]] = OpTypeVector %[[FP8E4]] 4
+; CHECK-DAG: %[[VINT8:[0-9]+]] = OpTypeVector %[[INT8]] 4
+
+; CHECK: %[[FROM_HALF:]] = OpFunction %[[HALF]] None
+; CHECK: %[[ARG:]] = OpFunctionParameter %[[INT8]]
+; CHECK: %[[FP8VAL:]] = OpBitcast %[[FP8E4]] %[[ARG]]
+; CHECK: %[[HALFRES:]] = OpFConvert %[[HALF]] %[[FP8VAL]]
+; CHECK: OpReturnValue %[[HALFRES]]
+
+; CHECK: %[[TO_HALF:]] = OpFunction %[[INT8]] None
+; CHECK: %[[HALFARG:]] = OpFunctionParameter %[[HALF]]
+; CHECK: %[[HALF_TO_FP8]] = OpFConvert %[[FP8E4]] %[[HALFARG]]
+; CHECK: %[[BITCAST:]] = OpBitcast %[[INT8]] %[[HALF_TO_FP8]]
+; CHECK: OpReturnValue %[[BITCAST]]
+
+; CHECK: %[[TO_HALF_SAT:]] = OpFunction %[[INT8]] None
+; CHECK: %[[HALFARG_SAT:]] = OpFunctionParameter %[[HALF]]
+; CHECK: %[[HALF_TO_FP8_SAT]] = OpFConvert %[[FP8E4]] %[[HALFARG_SAT]]
+; CHECK: %[[BITCAST_SAT:]] = OpBitcast %[[INT8]] %[[HALF_TO_FP8_SAT]]
+; CHECK: OpReturnValue %[[BITCAST_SAT]]
+
+; CHECK: %[[FROMVEC:]] = OpFunction %[[VHALF]] None
+; CHECK: %[[VARG:]] = OpFunctionParameter %[[VINT8]]
+; CHECK: %[[VFP8VAL:]] = OpBitcast %[[VFP8]] %[[VARG]]
+; CHECK: %[[VHALFRES:]] = OpFConvert %[[VHALF]] %[[VFP8VAL]]
+; CHECK: OpReturnValue %[[VHALFRES]]
+
+; CHECK: %[[TOVEC:]] = OpFunction %[[VINT8]] None
+; CHECK: %[[VHALFARG:]] = OpFunctionParameter %[[VHALF]]
+; CHECK: %[[VCONV:]] = OpFConvert %[[VFP8]] %[[VHALFARG]]
+; CHECK: %[[VBITCAST:]] = OpBitcast %[[VINT8]] %[[VCONV]]
+; CHECK: OpReturnValue %[[VBITCAST]]
+
+; CHECK: %[[FROM_FLOAT:]] = OpFunction %[[FLOAT]] None
+; CHECK: %[[FARG:]] = OpFunctionParameter %[[INT8]]
+; CHECK: %[[FFP8VAL:]] = OpBitcast %[[FP8E5]] %[[FARG]]
+; CHECK: %[[FRES:]] = OpFConvert %[[FLOAT]] %[[FFP8VAL]]
+; CHECK: OpReturnValue %[[FRES]]
+
+; CHECK: %[[TO_FLOAT:]] = OpFunction %[[INT8]] None
+; CHECK: %[[FLARG:]] = OpFunctionParameter %[[FLOAT]]
+; CHECK: %[[FLTCONV:]] = OpFConvert %[[FP8E5]] %[[FLARG]]
+; CHECK: %[[FLBIT:]] = OpBitcast %[[INT8]] %[[FLTCONV]]
+; CHECK: OpReturnValue %[[FLBIT]]
+
+; CHECK: %[[FROM_DOUBLE:]] = OpFunction %[[DOUBLE]] None
+; CHECK: %[[DARG:]] = OpFunctionParameter %[[INT8]]
+; CHECK: %[[DFP8VAL:]] = OpBitcast %[[FP8E5]] %[[DARG]]
+; CHECK: %[[DRES:]] = OpFConvert %[[DOUBLE]] %[[DFP8VAL]]
+; CHECK: OpReturnValue %[[DRES]]
+
+; CHECK: %[[TO_DOUBLE:]] = OpFunction %[[INT8]] None
+; CHECK: %[[DLARG:]] = OpFunctionParameter %[[DOUBLE]]
+; CHECK: %[[DLCONV:]] = OpFConvert %[[FP8E5]] %[[DLARG]]
+; CHECK: %[[DLBIT:]] = OpBitcast %[[INT8]] %[[DLCONV]]
+; CHECK: OpReturnValue %[[DLBIT]]
+
+; CHECK: %[[FP8_TO_I32:]] = OpFunction %[[INT32]] None
+; CHECK: %[[I32SRC:]] = OpFunctionParameter %[[INT8]]
+; CHECK: %[[I32FP8:]] = OpBitcast %[[FP8E4]] %[[I32SRC]]
+; CHECK: %[[I32RES:]] = OpConvertFToS %[[INT32]] %[[I32FP8]]
+; CHECK: OpReturnValue %[[I32RES]]
+
+; CHECK: %[[FP8_TO_U16:]] = OpFunction %[[INT16]] None
+; CHECK: %[[U16SRC:]] = OpFunctionParameter %[[INT8]]
+; CHECK: %[[U16FP8:]] = OpBitcast %[[FP8E5]] %[[U16SRC]]
+; CHECK: %[[U16RES:]] = OpConvertFToU %[[INT16]] %[[U16FP8]]
+; CHECK: OpReturnValue %[[U16RES]]
+
+; CHECK: %[[FP8_TO_I64:]] = OpFunction %[[INT64]] None
+; CHECK: %[[I64SRC:]] = OpFunctionParameter %[[INT8]]
+; CHECK: %[[I64FP8:]] = OpBitcast %[[FP8E5]] %[[I64SRC]]
+; CHECK: %[[I64RES:]] = OpConvertFToS %[[INT64]] %[[I64FP8]]
+; CHECK: OpReturnValue %[[I64RES]]
+
+; CHECK: %[[FP8_TO_I8:]] = OpFunction %[[INT8]] None
+; CHECK: %[[I8SRC:]] = OpFunctionParameter %[[INT8]]
+; CHECK: %[[I8FP8:]] = OpBitcast %[[FP8E4]] %[[I8SRC]]
+; CHECK: %[[I8RES:]] = OpConvertFToS %[[INT8]] %[[I8FP8]]
+; CHECK: OpReturnValue %[[I8RES]]
+
+; CHECK: %[[I32_TO_FP8_FN:]] = OpFunction %[[INT8]] None
+; CHECK: %[[I32ARG:]] = OpFunctionParameter %[[INT32]]
+; CHECK: %[[I32_TO_FP8]] = OpConvertSToF %[[FP8E4]] %[[I32ARG]]
+; CHECK: %[[I32BIT:]] = OpBitcast %[[INT8]] %[[I32_TO_FP8]]
+; CHECK: OpReturnValue %[[I32BIT]]
+
+; CHECK: %[[U16_TO_FP8_FN:]] = OpFunction %[[INT8]] None
+; CHECK: %[[U16ARG:]] = OpFunctionParameter %[[INT16]]
+; CHECK: %[[U16CONV:]] = OpConvertUToF %[[FP8E5]] %[[U16ARG]]
+; CHECK: %[[U16BIT:]] = OpBitcast %[[INT8]] %[[U16CONV]]
+; CHECK: OpReturnValue %[[U16BIT]]
+
+; CHECK: %[[I64_TO_FP8_FN:]] = OpFunction %[[INT8]] None
+; CHECK: %[[I64ARG:]] = OpFunctionParameter %[[INT64]]
+; CHECK: %[[I64CONV:]] = OpConvertSToF %[[FP8E5]] %[[I64ARG]]
+; CHECK: %[[I64BIT:]] = OpBitcast %[[INT8]] %[[I64CONV]]
+; CHECK: OpReturnValue %[[I64BIT]]
+
+; CHECK: %[[I8_TO_FP8_FN:]] = OpFunction %[[INT8]] None
+; CHECK: %[[I8ARG:]] = OpFunctionParameter %[[INT8]]
+; CHECK: %[[I8CONV:]] = OpConvertSToF %[[FP8E4]] %[[I8ARG]]
+; CHECK: %[[I8BIT:]] = OpBitcast %[[INT8]] %[[I8CONV]]
+; CHECK: OpReturnValue %[[I8BIT]]
+
+declare half @llvm.arbitrary.fp.convert.half.i8(i8, metadata, metadata, metadata, i32)
+declare i8 @llvm.arbitrary.fp.convert.i8.half(half, metadata, metadata, metadata, i32)
+declare <4 x half> @llvm.arbitrary.fp.convert.v4f16.v4i8(<4 x i8>, metadata, metadata, metadata, i32)
+declare <4 x i8> @llvm.arbitrary.fp.convert.v4i8.v4f16(<4 x half>, metadata, metadata, metadata, i32)
+declare float @llvm.arbitrary.fp.convert.f32.i8(i8, metadata, metadata, metadata, i32)
+declare i8 @llvm.arbitrary.fp.convert.i8.f32(float, metadata, metadata, metadata, i32)
+declare double @llvm.arbitrary.fp.convert.f64.i8(i8, metadata, metadata, metadata, i32)
+declare i8 @llvm.arbitrary.fp.convert.i8.f64(double, metadata, metadata, metadata, i32)
+declare i32 @llvm.arbitrary.fp.convert.i32.i8(i8, metadata, metadata, metadata, i32)
+declare i16 @llvm.arbitrary.fp.convert.i16.i8(i8, metadata, metadata, metadata, i32)
+declare i64 @llvm.arbitrary.fp.convert.i64.i8(i8, metadata, metadata, metadata, i32)
+declare i8 @llvm.arbitrary.fp.convert.i8.i32(i32, metadata, metadata, metadata, i32)
+declare i8 @llvm.arbitrary.fp.convert.i8.i16(i16, metadata, metadata, metadata, i32)
+declare i8 @llvm.arbitrary.fp.convert.i8.i64(i64, metadata, metadata, metadata, i32)
+declare i8 @llvm.arbitrary.fp.convert.i8.i8(i8, metadata, metadata, metadata, i32)
+
+define half @from_fp8(i8 %v) {
+ %r = call half @llvm.arbitrary.fp.convert.half.i8(
+ i8 %v, metadata !"none", metadata !"spv.E4M3EXT", metadata !"none", i32 0)
+ ret half %r
+}
+
+define i8 @to_fp8(half %v) {
+ %r = call i8 @llvm.arbitrary.fp.convert.i8.half(
+ half %v, metadata !"spv.E4M3EXT", metadata !"none", metadata !"round.towardzero", i32 0)
+ ret i8 %r
+}
+
+define i8 @to_fp8_saturated(half %v) {
+ %r = call i8 @llvm.arbitrary.fp.convert.i8.half(
+ half %v, metadata !"spv.E4M3EXT", metadata !"none", metadata !"round.towardzero", i32 1)
+ ret i8 %r
+}
+
+define <4 x half> @from_fp8_vec(<4 x i8> %v) {
+ %r = call <4 x half> @llvm.arbitrary.fp.convert.v4f16.v4i8(
+ <4 x i8> %v, metadata !"none", metadata !"spv.E4M3EXT", metadata !"none", i32 0)
+ ret <4 x half> %r
+}
+
+define <4 x i8> @to_fp8_vec(<4 x half> %v) {
+ %r = call <4 x i8> @llvm.arbitrary.fp.convert.v4i8.v4f16(
+ <4 x half> %v, metadata !"spv.E4M3EXT", metadata !"none", metadata !"round.towardzero", i32 0)
+ ret <4 x i8> %r
+}
+
+define float @from_fp8_to_float(i8 %v) {
+ %r = call float @llvm.arbitrary.fp.convert.f32.i8(
+ i8 %v, metadata !"none", metadata !"spv.E5M2EXT", metadata !"none", i32 0)
+ ret float %r
+}
+
+define i8 @to_fp8_from_float(float %v) {
+ %r = call i8 @llvm.arbitrary.fp.convert.i8.f32(
+ float %v, metadata !"spv.E5M2EXT", metadata !"none", metadata !"round.towardzero", i32 0)
+ ret i8 %r
+}
+
+define double @from_fp8_to_double(i8 %v) {
+ %r = call double @llvm.arbitrary.fp.convert.f64.i8(
+ i8 %v, metadata !"none", metadata !"spv.E5M2EXT", metadata !"none", i32 0)
+ ret double %r
+}
+
+define i8 @to_fp8_from_double(double %v) {
+ %r = call i8 @llvm.arbitrary.fp.convert.i8.f64(
+ double %v, metadata !"spv.E5M2EXT", metadata !"none", metadata !"round.towardzero", i32 0)
+ ret i8 %r
+}
+
+define i32 @from_fp8_to_i32(i8 %v) {
+ %r = call i32 @llvm.arbitrary.fp.convert.i32.i8(
+ i8 %v, metadata !"signed", metadata !"spv.E4M3EXT", metadata !"none", i32 0)
+ ret i32 %r
+}
+
+define i16 @from_fp8_to_u16(i8 %v) {
+ %r = call i16 @llvm.arbitrary.fp.convert.i16.i8(
+ i8 %v, metadata !"unsigned", metadata !"spv.E5M2EXT", metadata !"none", i32 0)
+ ret i16 %r
+}
+
+define i8 @to_fp8_from_i32(i32 %v) {
+ %r = call i8 @llvm.arbitrary.fp.convert.i8.i32(
+ i32 %v, metadata !"spv.E4M3EXT", metadata !"signed", metadata !"none", i32 0)
+ ret i8 %r
+}
+
+define i8 @to_fp8_from_u16(i16 %v) {
+ %r = call i8 @llvm.arbitrary.fp.convert.i8.i16(
+ i16 %v, metadata !"spv.E5M2EXT", metadata !"unsigned", metadata !"none", i32 0)
+ ret i8 %r
+}
+
+define i64 @from_fp8_to_i64(i8 %v) {
+ %r = call i64 @llvm.arbitrary.fp.convert.i64.i8(
+ i8 %v, metadata !"signed", metadata !"spv.E5M2EXT", metadata !"none", i32 0)
+ ret i64 %r
+}
+
+define i8 @from_fp8_to_i8(i8 %v) {
+ %r = call i8 @llvm.arbitrary.fp.convert.i8.i8(
+ i8 %v, metadata !"signed", metadata !"spv.E4M3EXT", metadata !"none", i32 0)
+ ret i8 %r
+}
+
+define i8 @to_fp8_from_i64(i64 %v) {
+ %r = call i8 @llvm.arbitrary.fp.convert.i8.i64(
+ i64 %v, metadata !"spv.E5M2EXT", metadata !"signed", metadata !"none", i32 0)
+ ret i8 %r
+}
+
+define i8 @to_fp8_from_i8(i8 %v) {
+ %r = call i8 @llvm.arbitrary.fp.convert.i8.i8(
+ i8 %v, metadata !"spv.E4M3EXT", metadata !"signed", metadata !"none", i32 0)
+ ret i8 %r
+}
+
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; CHECK: {{.*}}
diff --git a/llvm/test/Verifier/arbitrary-fp-convert.ll b/llvm/test/Verifier/arbitrary-fp-convert.ll
new file mode 100644
index 0000000000000..68cab2cb05dc1
--- /dev/null
+++ b/llvm/test/Verifier/arbitrary-fp-convert.ll
@@ -0,0 +1,66 @@
+; RUN: split-file %s %t
+; RUN: not opt -S -passes=verify %t/bad-result.ll 2>&1 | FileCheck %s --check-prefix=BADRESULT
+; RUN: not opt -S -passes=verify %t/bad-rounding.ll 2>&1 | FileCheck %s --check-prefix=BADROUND
+; RUN: not opt -S -passes=verify %t/bad-saturation.ll 2>&1 | FileCheck %s --check-prefix=BADSAT
+; RUN: opt -S -passes=verify %t/good.ll
+
+;--- bad-result.ll
+; BADRESULT: result interpretation metadata string must not be empty
+declare half @llvm.arbitrary.fp.convert.half.i8(i8, metadata, metadata, metadata, i32)
+
+define half @bad_result(i8 %v) {
+ %r = call half @llvm.arbitrary.fp.convert.half.i8(
+ i8 %v, metadata !"", metadata !"spv.E5M2EXT", metadata !"none", i32 0)
+ ret half %r
+}
+
+;--- bad-rounding.ll
+; BADROUND: unsupported rounding mode argument
+declare i8 @llvm.arbitrary.fp.convert.i8.half(half, metadata, metadata, metadata, i32)
+
+define i8 @bad_rounding(half %v) {
+ %r = call i8 @llvm.arbitrary.fp.convert.i8.half(
+ half %v, metadata !"spv.E4M3EXT", metadata !"none", metadata !"round.dynamic", i32 0)
+ ret i8 %r
+}
+
+;--- bad-saturation.ll
+; BADSAT: saturation operand must be 0 or 1
+declare i8 @llvm.arbitrary.fp.convert.i8.half.sat(half, metadata, metadata, metadata, i32)
+
+define i8 @bad_saturation(half %v) {
+ %r = call i8 @llvm.arbitrary.fp.convert.i8.half.sat(
+ half %v, metadata !"spv.E4M3EXT", metadata !"none", metadata !"round.towardzero", i32 2)
+ ret i8 %r
+}
+
+;--- good.ll
+declare half @llvm.arbitrary.fp.convert.half.i8(i8, metadata, metadata, metadata, i32)
+declare i8 @llvm.arbitrary.fp.convert.i8.half(half, metadata, metadata, metadata, i32)
+declare i32 @llvm.arbitrary.fp.convert.i32.i8(i8, metadata, metadata, metadata, i32)
+declare i8 @llvm.arbitrary.fp.convert.i8.i32(i32, metadata, metadata, metadata, i32)
+
+define half @good_from(i8 %v) {
+ %r = call half @llvm.arbitrary.fp.convert.half.i8(
+ i8 %v, metadata !"none", metadata !"spv.E4M3EXT", metadata !"none", i32 0)
+ ret half %r
+}
+
+define i8 @good_to(half %v) {
+ %r = call i8 @llvm.arbitrary.fp.convert.i8.half(
+ half %v, metadata !"spv.E4M3EXT", metadata !"none", metadata !"round.towardzero", i32 0)
+ ret i8 %r
+}
+
+; Test integer conversions with rounding modes - these are now allowed
+define i32 @good_int_rounding(i8 %v) {
+ %r = call i32 @llvm.arbitrary.fp.convert.i32.i8(
+ i8 %v, metadata !"signed", metadata !"spv.E4M3EXT", metadata !"none", i32 0)
+ ret i32 %r
+}
+
+define i8 @good_input_rounding(i32 %v) {
+ %r = call i8 @llvm.arbitrary.fp.convert.i8.i32(
+ i32 %v, metadata !"spv.E4M3EXT", metadata !"signed", metadata !"none", i32 0)
+ ret i8 %r
+}
More information about the llvm-commits
mailing list