[clang] [llvm] [Hexagon] Add XQFloat code generation and post-RA QFP handling (PR #198902)
via cfe-commits
cfe-commits at lists.llvm.org
Tue Jun 9 08:27:04 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-clang-driver
Author: Fateme Hosseini (fhossein-quic)
<details>
<summary>Changes</summary>
Introduce two new passes for the Hexagon HVX floating-point pipeline, targeting v79+ where QFloat (qf16/qf32) is the native HVX FP format.
HexagonXQFloatGenerator lowers IEEE-754 HVX floating-point sequences (sf/hf) to native QFloat (qf16/qf32) operations. QFloat instructions are faster and more power-efficient than their IEEE counterparts, with optional accuracy trade-offs. The pass exposes four modes:
* Strict IEEE-754 compliant
* IEEE-754 compliant (extended dynamic range and subnormal precision, no IEEE-754 overflow handling)
* Lossy subnormals
* Legacy
HexagonPostRAHandleQFP runs after register allocation and corrects the spill/refill paths. QFloat operands carry four extra precision bits that are silently dropped if the value passes through a spill slot or a non-HVX instruction. The pass uses the Register DataFlow Graph (RDF) to walk use-def chains in non-SSA form, inserts qf->IEEE conversions before spills and non-HVX uses, and rewrites saturating instruction opcodes that can absorb IEEE operands directly.
Co-authored-by: Sumanth Gundapaneni <sgundapa@<!-- -->quicinc.com>
Co-authored-by: Santanu Das <santdas@<!-- -->qti.qualcomm.com>
---
Patch is 438.55 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/198902.diff
52 Files Affected:
- (modified) clang/include/clang/Options/Options.td (+3)
- (modified) clang/lib/Driver/ToolChains/Clang.cpp (+112)
- (modified) clang/lib/Driver/ToolChains/Hexagon.cpp (+23-6)
- (added) clang/test/Driver/hexagon-hvx-ieee-qfloat.c (+25)
- (added) clang/test/Driver/hexagon-hvx-qfloat-backend.c (+43)
- (modified) llvm/include/llvm/CodeGen/RDFGraph.h (+2)
- (modified) llvm/lib/Target/Hexagon/CMakeLists.txt (+2)
- (modified) llvm/lib/Target/Hexagon/Hexagon.h (+4-1)
- (modified) llvm/lib/Target/Hexagon/HexagonInstrInfo.cpp (+24)
- (modified) llvm/lib/Target/Hexagon/HexagonInstrInfo.h (+3)
- (added) llvm/lib/Target/Hexagon/HexagonPostRAHandleQFP.cpp (+1854)
- (modified) llvm/lib/Target/Hexagon/HexagonTargetMachine.cpp (+31-1)
- (modified) llvm/lib/Target/Hexagon/HexagonTargetMachine.h (+3)
- (added) llvm/lib/Target/Hexagon/HexagonXQFloatGenerator.cpp (+2177)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-add-qf.ll (+157)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-assertion1.ll (+84)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-check-free-reg.ll (+110)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-check-qf-instrs.ll (+73)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-compliant-ieee-mul-qf16.ll (+86)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-compliant-ieee-mul-qf32.ll (+136)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-convert-elim.ll (+77)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-corner-case1.ll (+147)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-fix-invalid-opcode.ll (+72)
- (removed) llvm/test/CodeGen/Hexagon/autohvx/xqf-fixup-qfp1.ll (-372)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-handle-conv.ll (+180)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-input-rt.ll (+63)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-lossy-mul-qf16.ll (+74)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-lossy-mul-qf32.ll (+109)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-mode-flags.ll (+76)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-multi-conv.ll (+133)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-multidef.ll (+49)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-normalization-assert.ll (+459)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-conv-double.mir (+120)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-conv-double2.ll (+28)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-copy3.ll (+20)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-fakereg.ll (+130)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-handle-crash.ll (+23)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-handle-crash2.mir (+86)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-handle-qf32-mul.ll (+69)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-legacy-mode.ll (+30)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-subreg2.ll (+99)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-subreg3.ll (+45)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-warnings.ll (+60)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-strict-ieee-mul-qf16.ll (+91)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-strictieee-mul-qf32.ll (+123)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-unary-crash.ll (+25)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-v81/xqf-v81-compliant-ieee-mul-qf32.ll (+109)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-v81/xqf-v81-lossy-mul-qf32.ll (+98)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-v81/xqf-v81-strict-mul-qf32.ll (+119)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-v81/xqf-v81-vsub.ll (+164)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-vsub.ll (+130)
- (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-warnings.ll (+143)
``````````diff
diff --git a/clang/include/clang/Options/Options.td b/clang/include/clang/Options/Options.td
index 4fd892e58df86..c82c00c98d3ef 100644
--- a/clang/include/clang/Options/Options.td
+++ b/clang/include/clang/Options/Options.td
@@ -6986,6 +6986,9 @@ def mhexagon_hvx_length_EQ : Joined<["-"], "mhvx-length=">,
def mhexagon_hvx_qfloat : Flag<["-"], "mhvx-qfloat">,
Group<m_hexagon_Features_HVX_Group>,
HelpText<"Enable Hexagon HVX QFloat instructions">;
+def mhexagon_hvx_qfloat_EQ : Joined<["-"], "mhvx-qfloat=">,
+ Group<m_hexagon_Features_HVX_Group>,
+ HelpText<"Enable Hexagon HVX QFloat instructions with mode: strict-ieee, ieee, lossy, legacy (v79+)">;
def mno_hexagon_hvx_qfloat : Flag<["-"], "mno-hvx-qfloat">,
Group<m_hexagon_Features_HVX_Group>,
HelpText<"Disable Hexagon HVX QFloat instructions">;
diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp
index 8a0efd70e6c0d..b942e74d8933f 100644
--- a/clang/lib/Driver/ToolChains/Clang.cpp
+++ b/clang/lib/Driver/ToolChains/Clang.cpp
@@ -61,6 +61,7 @@
#include "llvm/TargetParser/RISCVISAInfo.h"
#include "llvm/TargetParser/RISCVTargetParser.h"
#include <cctype>
+#include <iterator>
using namespace clang::driver;
using namespace clang::driver::tools;
@@ -2235,6 +2236,115 @@ void Clang::AddX86TargetArgs(const ArgList &Args,
}
}
+static StringRef getOptionName(StringRef Option, const char Delimiter = '=') {
+ size_t Index = Option.find(Delimiter);
+ if (Index != StringRef::npos)
+ Option = Option.substr(0, Index);
+ return Option;
+}
+
+static void checkAndRemoveLLVMArg(ArgStringList &CmdArgs, StringRef Opt) {
+ Opt = getOptionName(Opt);
+ if (CmdArgs.size() < 2)
+ return;
+
+ for (auto It = std::next(CmdArgs.begin()); It != CmdArgs.end(); ++It) {
+ StringRef Option = *It;
+ if (!Option.starts_with(Opt))
+ continue;
+ Option = getOptionName(Option);
+ if (Option != Opt)
+ continue;
+ if (StringRef(*(It - 1)) != "-mllvm")
+ continue;
+
+ It = CmdArgs.erase(It);
+ CmdArgs.erase(It - 1);
+ return;
+ }
+}
+
+static void pushBackLLVMArg(ArgStringList &CmdArgs, const char *A) {
+ checkAndRemoveLLVMArg(CmdArgs, A);
+ CmdArgs.push_back("-mllvm");
+ CmdArgs.push_back(A);
+}
+
+static void addQFloatLossyFastMathArgs(ArgStringList &CmdArgs) {
+ for (auto It = CmdArgs.begin(), Ie = CmdArgs.end(); It != Ie;) {
+ StringRef Option = *It;
+ if (Option == "-fmath-errno" || Option == "-ffp-contract=on") {
+ It = CmdArgs.erase(It);
+ Ie = CmdArgs.end();
+ } else {
+ ++It;
+ }
+ }
+
+ CmdArgs.push_back("-menable-no-infs");
+ CmdArgs.push_back("-menable-no-nans");
+ CmdArgs.push_back("-fapprox-func");
+ CmdArgs.push_back("-funsafe-math-optimizations");
+ CmdArgs.push_back("-fno-signed-zeros");
+ CmdArgs.push_back("-mreassociate");
+ CmdArgs.push_back("-freciprocal-math");
+ CmdArgs.push_back("-ffp-contract=fast");
+ CmdArgs.push_back("-ffast-math");
+ CmdArgs.push_back("-ffinite-math-only");
+ CmdArgs.push_back("-D__FAST_MATH__");
+ pushBackLLVMArg(CmdArgs, "-fast-math=true");
+}
+
+static void addQFloatBackendArg(const Driver &D, const ArgList &Args,
+ ArgStringList &CmdArgs) {
+ auto HvxVerOpt = toolchains::HexagonToolChain::GetHVXVersion(Args);
+ bool HasHVX = HvxVerOpt.has_value();
+ std::string HvxVer = HasHVX ? *HvxVerOpt : std::string();
+ if (Args.hasArg(options::OPT_mhexagon_hvx, options::OPT_mhexagon_hvx_EQ,
+ options::OPT_mhexagon_hvx_ieee_fp) &&
+ HasHVX) {
+ unsigned HvxVerNum = 0;
+ if (StringRef(HvxVer).drop_front(1).getAsInteger(10, HvxVerNum))
+ HvxVerNum = 0;
+
+ if (Arg *A = Args.getLastArg(options::OPT_mhexagon_hvx_qfloat,
+ options::OPT_mhexagon_hvx_qfloat_EQ,
+ options::OPT_mhexagon_hvx_ieee_fp)) {
+ if (HvxVerNum >= 79) {
+ if (A->getOption().matches(options::OPT_mhexagon_hvx_qfloat_EQ)) {
+ const char *Mode =
+ llvm::StringSwitch<const char *>(StringRef(A->getValue()).lower())
+ .Case("strict-ieee", "-hexagon-qfloat-mode=strict-ieee")
+ .Case("ieee", "-hexagon-qfloat-mode=ieee")
+ .Case("lossy", "-hexagon-qfloat-mode=lossy")
+ .Case("legacy", "-hexagon-qfloat-mode=legacy")
+ .Default("-hexagon-qfloat-mode=lossy");
+ pushBackLLVMArg(CmdArgs, Mode);
+ if (strcmp(Mode, "-hexagon-qfloat-mode=lossy") == 0)
+ addQFloatLossyFastMathArgs(CmdArgs);
+ } else if (A->getOption().matches(options::OPT_mhexagon_hvx_qfloat)) {
+ pushBackLLVMArg(CmdArgs, "-hexagon-qfloat-mode=lossy");
+ addQFloatLossyFastMathArgs(CmdArgs);
+ } else {
+ pushBackLLVMArg(CmdArgs, "-hexagon-qfloat-mode=ieee");
+ }
+ } else {
+ if (Arg *QFloatArg =
+ Args.getLastArg(options::OPT_mhexagon_hvx_qfloat,
+ options::OPT_mhexagon_hvx_qfloat_EQ,
+ options::OPT_mno_hexagon_hvx_qfloat);
+ QFloatArg && QFloatArg->getOption().matches(
+ options::OPT_mhexagon_hvx_qfloat_EQ)) {
+ D.Diag(diag::warn_drv_unsupported_option_part_for_target)
+ << QFloatArg->getValue() << QFloatArg->getAsString(Args)
+ << (std::string("HVX ") + HvxVer +
+ "; falling back to legacy qfloat mode");
+ }
+ }
+ }
+ }
+}
+
void Clang::AddHexagonTargetArgs(const ArgList &Args,
ArgStringList &CmdArgs) const {
CmdArgs.push_back("-mqdsp6-compat");
@@ -2254,6 +2364,8 @@ void Clang::AddHexagonTargetArgs(const ArgList &Args,
}
CmdArgs.push_back("-mllvm");
CmdArgs.push_back("-machine-sink-split=0");
+
+ addQFloatBackendArg(getToolChain().getDriver(), Args, CmdArgs);
}
void Clang::AddLanaiTargetArgs(const ArgList &Args,
diff --git a/clang/lib/Driver/ToolChains/Hexagon.cpp b/clang/lib/Driver/ToolChains/Hexagon.cpp
index 0e7055797a1f0..d60b0f201f1e0 100644
--- a/clang/lib/Driver/ToolChains/Hexagon.cpp
+++ b/clang/lib/Driver/ToolChains/Hexagon.cpp
@@ -106,14 +106,14 @@ static void handleHVXTargetFeatures(const Driver &D, const ArgList &Args,
// Handle HVX floating point flags.
auto checkFlagHvxVersion =
- [&](auto FlagOn, auto FlagOff,
+ [&](auto FlagOn, auto FlagOnWithModes, auto FlagOff, bool CheckMode,
unsigned MinVerNum) -> std::optional<StringRef> {
// Return an std::optional<StringRef>:
// - std::nullopt indicates a verification failure, or that the flag was not
// present in Args.
// - Otherwise the returned value is that name of the feature to add
// to Features.
- Arg *A = Args.getLastArg(FlagOn, FlagOff);
+ Arg *A = Args.getLastArg(FlagOn, FlagOnWithModes, FlagOff);
if (!A)
return std::nullopt;
@@ -130,17 +130,34 @@ static void handleHVXTargetFeatures(const Driver &D, const ArgList &Args,
<< withMinus(OptName) << ("v" + std::to_string(HvxVerNum));
return std::nullopt;
}
+
+ if (CheckMode && A->getOption().matches(FlagOnWithModes)) {
+ bool ValidMode =
+ llvm::StringSwitch<bool>(StringRef(A->getValue()).lower())
+ .Cases({"strict-ieee", "ieee", "lossy", "legacy"}, true)
+ .Default(false);
+ if (!ValidMode)
+ D.Diag(diag::err_drv_invalid_value)
+ << A->getAsString(Args) << A->getValue();
+ }
return makeFeature(OptName, true);
};
- if (auto F = checkFlagHvxVersion(options::OPT_mhexagon_hvx_qfloat,
- options::OPT_mno_hexagon_hvx_qfloat, 68)) {
+ if (auto F = checkFlagHvxVersion(
+ options::OPT_mhexagon_hvx_qfloat, options::OPT_mhexagon_hvx_qfloat_EQ,
+ options::OPT_mno_hexagon_hvx_qfloat, /*CheckMode=*/true, 68)) {
Features.push_back(*F);
}
- if (auto F = checkFlagHvxVersion(options::OPT_mhexagon_hvx_ieee_fp,
- options::OPT_mno_hexagon_hvx_ieee_fp, 68)) {
+ if (auto F = checkFlagHvxVersion(
+ options::OPT_mhexagon_hvx_ieee_fp, options::OPT_mhexagon_hvx_ieee_fp,
+ options::OPT_mno_hexagon_hvx_ieee_fp, /*CheckMode=*/false, 68)) {
Features.push_back(*F);
}
+
+ // On v79 and above, there is no IEEE hardware. Treat -mhvx-ieee-fp
+ // as "qfloat mode ieee".
+ if (HvxVerNum >= 79 && Args.getLastArg(options::OPT_mhexagon_hvx_ieee_fp))
+ Features.push_back("+hvx-qfloat");
}
// Hexagon target features.
diff --git a/clang/test/Driver/hexagon-hvx-ieee-qfloat.c b/clang/test/Driver/hexagon-hvx-ieee-qfloat.c
new file mode 100644
index 0000000000000..ee8dc9de25751
--- /dev/null
+++ b/clang/test/Driver/hexagon-hvx-ieee-qfloat.c
@@ -0,0 +1,25 @@
+// ---------------------------------------------------------------------------
+// Tests for the hvx qfloat target feature and backend flag if -mhvx-ieee-fp is
+// passed on v79 and above.
+// ---------------------------------------------------------------------------
+
+// Test for v79, the correct backend flag is passed for -mhvx-ieee-fp.
+// CHECK-IEEE: "-mllvm" "-hexagon-qfloat-mode=ieee"
+// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \
+// RUN: -mhvx-ieee-fp 2>&1 | FileCheck -check-prefix=CHECK-IEEE %s
+
+// Test for arches lower than v79 does not pass any backend flag.
+// CHECK-MODE-NOT: "-mllvm" "-hexagon-qfloat-mode="
+// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv75 -mhvx \
+// RUN: -mhvx-ieee-fp 2>&1 | FileCheck -check-prefix=CHECK-MODE %s
+
+// Test for v79, the correct qfloat target feature is set when -mhvx-ieee-fp is
+// passed.
+// CHECK-HVX-QFLOAT-ON: "-target-feature" "+hvx-qfloat"
+// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \
+// RUN: -mhvx-ieee-fp 2>&1 | FileCheck -check-prefix=CHECK-HVX-QFLOAT-ON %s
+
+// Test for arches lower than v79 does not set the qfloat target feature.
+// CHECK-FEATURE-NOT: "-target-feature" "+hvx-qfloat"
+// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv75 -mhvx \
+// RUN: -mhvx-ieee-fp 2>&1 | FileCheck -check-prefix=CHECK-FEATURE %s
diff --git a/clang/test/Driver/hexagon-hvx-qfloat-backend.c b/clang/test/Driver/hexagon-hvx-qfloat-backend.c
new file mode 100644
index 0000000000000..ae6866777acff
--- /dev/null
+++ b/clang/test/Driver/hexagon-hvx-qfloat-backend.c
@@ -0,0 +1,43 @@
+// ---------------------------------------------------------------------------
+// Tests for the hvx qfloat modes backend flag.
+// ---------------------------------------------------------------------------
+
+// Test for correct backend flag with case-insensitive values.
+// CHECK-STRICT-IEEE: "-mllvm" "-hexagon-qfloat-mode=strict-ieee"
+// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \
+// RUN: -mhvx-qfloat=strict-ieee 2>&1 | FileCheck -check-prefix=CHECK-STRICT-IEEE %s
+// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \
+// RUN: -mhvx-qfloat=sTriCt-Ieee 2>&1 | FileCheck -check-prefix=CHECK-STRICT-IEEE %s
+
+// CHECK-IEEE: "-mllvm" "-hexagon-qfloat-mode=ieee"
+// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \
+// RUN: -mhvx-qfloat=ieee 2>&1 | FileCheck -check-prefix=CHECK-IEEE %s
+// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \
+// RUN: -mhvx-qfloat=IEEE 2>&1 | FileCheck -check-prefix=CHECK-IEEE %s
+
+// CHECK-LOSSY: "-mllvm" "-hexagon-qfloat-mode=lossy"
+// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \
+// RUN: -mhvx-qfloat=lossy 2>&1 | FileCheck -check-prefix=CHECK-LOSSY %s
+// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \
+// RUN: -mhvx-qfloat=lOSSy 2>&1 | FileCheck -check-prefix=CHECK-LOSSY %s
+
+// CHECK-LEGACY: "-mllvm" "-hexagon-qfloat-mode=legacy"
+// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \
+// RUN: -mhvx-qfloat=legacy 2>&1 | FileCheck -check-prefix=CHECK-LEGACY %s
+// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \
+// RUN: -mhvx-qfloat=LEGacy 2>&1 | FileCheck -check-prefix=CHECK-LEGACY %s
+
+// Test for default mode, if no mode is specified on v79.
+// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \
+// RUN: -mhvx-qfloat 2>&1 | FileCheck -check-prefix=CHECK-LOSSY %s
+
+// Test for arches lower than v79 does not pass any backend flag.
+// CHECK-MODE-NOT: "-mllvm" "-hexagon-qfloat-mode="
+// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv75 -mhvx \
+// RUN: -mhvx-qfloat 2>&1 | FileCheck -check-prefix=CHECK-MODE %s
+
+// Test for arches lower than v79 warns that qfloat mode is ignored.
+// CHECK-MODE-WARN: warning: ignoring 'ieee' in '-mhvx-qfloat=ieee' option as it is not currently supported for target 'HVX v75'
+// CHECK-MODE-WARN-NOT: "-mllvm" "-hexagon-qfloat-mode="
+// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv75 -mhvx \
+// RUN: -mhvx-qfloat=ieee 2>&1 | FileCheck -check-prefix=CHECK-MODE-WARN %s
diff --git a/llvm/include/llvm/CodeGen/RDFGraph.h b/llvm/include/llvm/CodeGen/RDFGraph.h
index c1ec2ddff14a3..8d96efd8e2f0f 100644
--- a/llvm/include/llvm/CodeGen/RDFGraph.h
+++ b/llvm/include/llvm/CodeGen/RDFGraph.h
@@ -358,6 +358,8 @@ template <typename T> struct NodeAddr {
return !operator==(NA);
}
+ bool operator<(const NodeAddr<T> &NA) const { return Id < NA.Id; }
+
T Addr = nullptr;
NodeId Id = 0;
};
diff --git a/llvm/lib/Target/Hexagon/CMakeLists.txt b/llvm/lib/Target/Hexagon/CMakeLists.txt
index 38dcc09282330..f98b519fe4974 100644
--- a/llvm/lib/Target/Hexagon/CMakeLists.txt
+++ b/llvm/lib/Target/Hexagon/CMakeLists.txt
@@ -58,9 +58,11 @@ add_llvm_target(HexagonCodeGen
HexagonOptAddrMode.cpp
HexagonOptimizeSZextends.cpp
HexagonPeephole.cpp
+ HexagonPostRAHandleQFP.cpp
HexagonQFPOptimizer.cpp
HexagonRDFOpt.cpp
HexagonRegisterInfo.cpp
+ HexagonXQFloatGenerator.cpp
HexagonSelectionDAGInfo.cpp
HexagonSplitConst32AndConst64.cpp
HexagonSplitDouble.cpp
diff --git a/llvm/lib/Target/Hexagon/Hexagon.h b/llvm/lib/Target/Hexagon/Hexagon.h
index 1db2326b274dc..e9ee7b7c48e3e 100644
--- a/llvm/lib/Target/Hexagon/Hexagon.h
+++ b/llvm/lib/Target/Hexagon/Hexagon.h
@@ -69,8 +69,9 @@ void initializeHexagonOptimizeSZextendsPass(PassRegistry &);
void initializeHexagonPeepholePass(PassRegistry &);
void initializeHexagonSplitConst32AndConst64Pass(PassRegistry &);
void initializeHexagonVectorPrintPass(PassRegistry &);
-
void initializeHexagonQFPOptimizerPass(PassRegistry &);
+void initializeHexagonPostRAHandleQFPPass(PassRegistry &);
+void initializeHexagonXQFloatGeneratorPass(PassRegistry &);
Pass *createHexagonLoopIdiomPass();
Pass *createHexagonVectorLoopCarriedReuseLegacyPass();
@@ -119,6 +120,8 @@ FunctionPass *createHexagonVectorPrint();
FunctionPass *createHexagonVExtract();
FunctionPass *createHexagonExpandCondsets();
FunctionPass *createHexagonQFPOptimizer();
+FunctionPass *createHexagonPostRAHandleQFP();
+FunctionPass *createHexagonXQFloatGenerator();
} // end namespace llvm;
diff --git a/llvm/lib/Target/Hexagon/HexagonInstrInfo.cpp b/llvm/lib/Target/Hexagon/HexagonInstrInfo.cpp
index 6c95d54bf111b..5e8578a5d407d 100644
--- a/llvm/lib/Target/Hexagon/HexagonInstrInfo.cpp
+++ b/llvm/lib/Target/Hexagon/HexagonInstrInfo.cpp
@@ -4950,6 +4950,30 @@ bool HexagonInstrInfo::isQFPInstr(MachineInstr *MI) const {
return isQFP32Instr(MI) || isQFP16Instr(MI);
}
+// Return true if the function contains any qf-generating instructions.
+bool HexagonInstrInfo::hasQFPInstrs(const MachineFunction &MF) const {
+ for (const MachineBasicBlock &MBB : MF)
+ for (const MachineInstr &MI : MBB)
+ if (isQFPInstr(const_cast<MachineInstr *>(&MI)))
+ return true;
+ return false;
+}
+
+// Returns true if A appears before B within the same basic block.
+bool HexagonInstrInfo::isMIBefore(const MachineInstr *A,
+ const MachineInstr *B) const {
+ if (!A || !B || A->getParent() != B->getParent())
+ return false;
+
+ for (const MachineInstr &MI : *A->getParent()) {
+ if (&MI == A)
+ return true;
+ if (&MI == B)
+ return false;
+ }
+ return false;
+}
+
// Addressing mode relations.
short HexagonInstrInfo::changeAddrMode_abs_io(short Opc) const {
return Opc >= 0 ? Hexagon::changeAddrMode_abs_io(Opc) : Opc;
diff --git a/llvm/lib/Target/Hexagon/HexagonInstrInfo.h b/llvm/lib/Target/Hexagon/HexagonInstrInfo.h
index 230f5d2228457..1901b260926d2 100644
--- a/llvm/lib/Target/Hexagon/HexagonInstrInfo.h
+++ b/llvm/lib/Target/Hexagon/HexagonInstrInfo.h
@@ -52,6 +52,9 @@ class HexagonInstrInfo : public HexagonGenInstrInfo {
const HexagonRegisterInfo &getRegisterInfo() const { return RegInfo; }
+ bool isMIBefore(const MachineInstr *A, const MachineInstr *B) const;
+ bool hasQFPInstrs(const MachineFunction &MF) const;
+
/// TargetInstrInfo overrides.
/// If the specified machine instruction is a direct
diff --git a/llvm/lib/Target/Hexagon/HexagonPostRAHandleQFP.cpp b/llvm/lib/Target/Hexagon/HexagonPostRAHandleQFP.cpp
new file mode 100644
index 0000000000000..31a97918d3c1f
--- /dev/null
+++ b/llvm/lib/Target/Hexagon/HexagonPostRAHandleQFP.cpp
@@ -0,0 +1,1854 @@
+//===--------------------- HexagonPostRAHandleQFP.cpp --------------------------
+//===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===---------------------------------------------------------------------===//
+// For v79 and above, we generate qf operations for HVX which includes vadd,
+// vsub and vmpy instructions. These qf operations with qf operands are fast,
+// maintain similar accuracy as IEEE and saves power.
+//
+// However, these qf operands should always be converted back to IEEE format
+// when used in non-HVX instructions. This is because of how the qf values
+// are stored in memory. qf operands have 4 extra bits. If used in non-HVX
+// operations, these bits get dropped resulting in incorrect value being
+// used. So, before use in any non-HVX operation we need to convert these
+// qf values to IEEE format.
+//
+// During register allocation, when no more physical registers are available
+// the qf operands may be spilled to memory. This instantly causes loss of
+// accuracy. This pass prevents that by:
+// 1. Inserting qf type to IEEE type conversion instructions before the spill.
+// 2. Iterating over the uses of qf def (created before the spill) and
+// changing their opcodes to handle IEEE type operands for saturating
+// instructions. This is because, the refills will use IEEE type operands, but
+// the instructions will still assume qf operands. For non-saturating
+// instructions which uses qf, we incorporate a conversion to IEEE before that.
+// 3. Iterating over the uses of qf def created by the spill and replacing
+// them with appropiate opcode (which uses IEEE operands) for saturating
+// instructions. For non-saturating instructions which uses qf,
+// we incorporate a conversion to IEEE before that.
+// 4. Iterating over the copy instructions and checking their uses,
+// inserting conversions from qf to IEEE whenever required. The conversions
+// are inserted after their reaching def since there can be multiple defs
+// for use in non-SSA form.
+//
+// To get the use-def chains, we make use of Register DataFlow Graph (RDF),
+// since after register allocation SSA form is lost. This can be done during
+// spills and fills during Frame Lowering for register allocation. However,
+// that was abandoned due to the intermediate state of the code.
+// Liveness is preserved in this pass.
+//
+// NOTE:
+// Saturating instructions: Instructions for which transformation involves
+// only changing the opcode. Eg. vmpy(qf32, sf) saturates to vmpy(sf, sf) when
+// we see that the first operand is now a sf type.
+// Non-Saturating instructions: Instructions for which conversion(s) have
+// to be inserted. Eg. Vd.f8=Vu.qf16. If the use operand is now hf type,
+// we have to insert a conversion qf16 = hf before this instruction.
+//
+// FIXME tags have been added for potential errors, along with the underlying
+// assumption.
+// FIXME Implement v81 specific optimizations as below. At the moment, we add
+// converts.
+// Vd.qf16=Vu....
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/198902
More information about the cfe-commits
mailing list