[llvm] Intrinsic: introduce minimumnum and maximumnum (PR #96299)
YunQiang Su via llvm-commits
llvm-commits at lists.llvm.org
Fri Jun 21 04:34:40 PDT 2024
https://github.com/wzssyqa created https://github.com/llvm/llvm-project/pull/96299
Currently, on different platform, the behaivor of llvm.minnum is different if one operand is sNaN:
When we compare sNaN vs NUM:
ARM/AArch64/PowerPC: follow the IEEE754-2008's minNUM: return qNaN. RISC-V/Hexagon follow the IEEE754-2019's minimumNumber: return NUM. X86: Returns NUM but not same with IEEE754-2019's minimumNumber as
+0.0 is not always greater than -0.0.
MIPS/LoongArch/Generic: return NUM.
LIBCALL: returns qNaN.
So, let's introduce llvm.minmumnum/llvm.maximumnum, which always follow IEEE754-2019's minimumNumber/maximumNumber.
Half-fix: #93033
>From 08f8e1608e70ae605c9dadbf2cb4eaf854b08574 Mon Sep 17 00:00:00 2001
From: YunQiang Su <syq at debian.org>
Date: Fri, 21 Jun 2024 11:53:08 +0800
Subject: [PATCH] Intrinsic: introduce minimumnum and maximumnum
Currently, on different platform, the behaivor of llvm.minnum is
different if one operand is sNaN:
When we compare sNaN vs NUM:
ARM/AArch64/PowerPC: follow the IEEE754-2008's minNUM: return qNaN.
RISC-V/Hexagon follow the IEEE754-2019's minimumNumber: return NUM. X86:
Returns NUM but not same with IEEE754-2019's minimumNumber as
+0.0 is not always greater than -0.0.
MIPS/LoongArch/Generic: return NUM.
LIBCALL: returns qNaN.
So, let's introduce llvm.minmumnum/llvm.maximumnum, which always follow
IEEE754-2019's minimumNumber/maximumNumber.
Half-fix: #93033
---
llvm/docs/LangRef.rst | 550 ++++++++++++++++++
llvm/include/llvm/ADT/APFloat.h | 26 +
.../llvm/Analysis/TargetLibraryInfo.def | 33 ++
llvm/include/llvm/CodeGen/BasicTTIImpl.h | 10 +
.../llvm/CodeGen/GlobalISel/CombinerHelper.h | 2 +
.../CodeGen/GlobalISel/GenericMachineInstrs.h | 12 +
.../llvm/CodeGen/GlobalISel/LegalizerHelper.h | 1 +
.../CodeGen/GlobalISel/MachineIRBuilder.h | 24 +
llvm/include/llvm/CodeGen/GlobalISel/Utils.h | 4 +
llvm/include/llvm/CodeGen/ISDOpcodes.h | 11 +
llvm/include/llvm/CodeGen/TargetLowering.h | 7 +
llvm/include/llvm/IR/ConstrainedOps.def | 2 +
llvm/include/llvm/IR/IRBuilder.h | 12 +
llvm/include/llvm/IR/IntrinsicInst.h | 2 +
llvm/include/llvm/IR/Intrinsics.td | 40 ++
llvm/include/llvm/IR/RuntimeLibcalls.def | 10 +
llvm/include/llvm/IR/VPIntrinsics.def | 22 +
llvm/include/llvm/Support/TargetOpcodes.def | 6 +
llvm/include/llvm/Target/GenericOpcodes.td | 19 +
.../include/llvm/Target/GlobalISel/Combine.td | 6 +-
.../Target/GlobalISel/SelectionDAGCompat.td | 4 +
.../include/llvm/Target/TargetSelectionDAG.td | 18 +
llvm/lib/CodeGen/ExpandVectorPredication.cpp | 18 +-
llvm/lib/CodeGen/GlobalISel/CSEMIRBuilder.cpp | 2 +
.../lib/CodeGen/GlobalISel/CombinerHelper.cpp | 2 +
llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp | 8 +
.../CodeGen/GlobalISel/LegalizerHelper.cpp | 53 +-
llvm/lib/CodeGen/GlobalISel/Utils.cpp | 10 +-
llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp | 12 +-
llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp | 32 +
.../SelectionDAG/LegalizeFloatTypes.cpp | 62 +-
llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h | 4 +
.../SelectionDAG/LegalizeVectorOps.cpp | 10 +
.../SelectionDAG/LegalizeVectorTypes.cpp | 14 +
.../lib/CodeGen/SelectionDAG/SelectionDAG.cpp | 14 +-
.../SelectionDAG/SelectionDAGBuilder.cpp | 30 +
.../SelectionDAG/SelectionDAGDumper.cpp | 8 +
.../CodeGen/SelectionDAG/TargetLowering.cpp | 106 +++-
llvm/lib/CodeGen/TargetLoweringBase.cpp | 1 +
.../Target/AArch64/AArch64ISelLowering.cpp | 3 +
llvm/lib/Target/AArch64/AArch64InstrInfo.td | 12 +
.../Target/Hexagon/HexagonISelLowering.cpp | 5 +
.../Target/Hexagon/HexagonISelLoweringHVX.cpp | 6 +
llvm/lib/Target/Hexagon/HexagonPatterns.td | 4 +
llvm/lib/Target/Hexagon/HexagonPatternsHVX.td | 8 +
.../RISCV/GISel/RISCVRegisterBankInfo.cpp | 4 +-
llvm/lib/Target/RISCV/RISCVISelLowering.cpp | 32 +-
llvm/lib/Target/RISCV/RISCVInstrInfoD.td | 2 +
llvm/lib/Target/RISCV/RISCVInstrInfoF.td | 2 +
.../Target/RISCV/RISCVInstrInfoVSDPatterns.td | 2 +
llvm/lib/Target/X86/X86ISelLowering.cpp | 4 +
.../lib/Target/X86/X86TargetTransformInfo.cpp | 13 +-
.../GlobalISel/legalizer-info-validation.mir | 12 +
.../AArch64/combine_andor_with_cmps.ll | 11 +-
.../AArch64/fp-maximumnum-minimumnum.ll | 51 ++
llvm/test/CodeGen/Hexagon/fminmax-v67.ll | 40 ++
.../LoongArch/fp-maximumnum-minimumnum.ll | 154 +++++
.../CodeGen/Mips/fp-maximumnum-minimumnum.ll | 100 ++++
llvm/test/CodeGen/RISCV/double-intrinsics.ll | 178 ++++--
llvm/test/CodeGen/RISCV/float-intrinsics.ll | 184 ++++--
llvm/test/TableGen/GlobalISelEmitter.td | 2 +-
.../tools/llvm-tli-checker/ps4-tli-check.yaml | 14 +-
llvm/unittests/ADT/APFloatTest.cpp | 48 ++
.../Analysis/TargetLibraryInfoTest.cpp | 8 +-
llvm/unittests/IR/VPIntrinsicTest.cpp | 11 +-
65 files changed, 1973 insertions(+), 144 deletions(-)
create mode 100644 llvm/test/CodeGen/AArch64/fp-maximumnum-minimumnum.ll
create mode 100644 llvm/test/CodeGen/LoongArch/fp-maximumnum-minimumnum.ll
create mode 100644 llvm/test/CodeGen/Mips/fp-maximumnum-minimumnum.ll
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index 9a0f73e4d6d86..f902e5604e6ae 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -16080,6 +16080,100 @@ The returned value is completely identical to the input except for the sign bit;
in particular, if the input is a NaN, then the quiet/signaling bit and payload
are perfectly preserved.
+.. _i_fminmax_family:
+
+'``llvm.min.*``' Intrinsics Comparation
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Standard:
+"""""""""
+
+IEEE754 and ISO C define some min/max operations, and they have some differences
+on working with qNaN/sNaN and +0.0/-0.0. Here is the list:
+
+.. list-table::
+ :header-rows: 2
+
+ * - ``ISO C``
+ - fmin/fmax
+ - fmininum/fmaximum
+ - fminimum_num/fmaximum_num
+
+ * - ``IEEE754``
+ - minNum/maxNum (2008)
+ - minimum/maximum (2019)
+ - minimumNumber/maximumNumber (2019)
+
+ * - ``+0.0 vs -0.0``
+ - either one
+ - +0.0 > -0.0
+ - +0.0 > -0.0
+
+ * - ``NUM vs sNaN``
+ - qNaN, invalid exception
+ - qNaN, invalid exception
+ - NUM, invalid exception
+
+ * - ``qNaN vs sNaN``
+ - qNaN, invalid exception
+ - qNaN, invalid exception
+ - qNaN, invalid exception
+
+ * - ``NUM vs qNaN``
+ - NUM, no exception
+ - qNaN, no exception
+ - NUM, no exception
+
+LLVM Implementation:
+""""""""""""""""""""
+
+LLVM implements all ISO C flavors as listed in this table.
+Only basic intrinsics list here. The constrained version
+ones may have different behaivor on exception.
+
+Since some architectures implement minNum/maxNum with +0.0>-0.0,
+so we define internal ISD::MINNUM_IEEE and ISD::MAXNUM_IEEE.
+They will be helpful to implement minimumnum/maximumnum.
+
+.. list-table::
+ :header-rows: 1
+ :widths: 16 28 28 28
+
+ * - Operation
+ - minnum/maxnum
+ - minimum/maximum
+ - minimumnum/maximumnum
+
+ * - ``NUM vs qNaN``
+ - NUM, no exception
+ - qNaN, no exception
+ - NUM, no exception
+
+ * - ``NUM vs sNaN``
+ - qNaN, invalid exception
+ - qNaN, invalid exception
+ - NUM, invalid exception
+
+ * - ``qNaN vs sNaN``
+ - qNaN, invalid exception
+ - qNaN, invalid exception
+ - qNaN, invalid exception
+
+ * - ``sNaN vs sNaN``
+ - qNaN, invalid exception
+ - qNaN, invalid exception
+ - qNaN, invalid exception
+
+ * - ``+0.0 vs -0.0``
+ - either one
+ - +0.0(max)/-0.0(min)
+ - +0.0(max)/-0.0(min)
+
+ * - ``NUM vs NUM``
+ - larger(max)/smaller(min)
+ - larger(max)/smaller(min)
+ - larger(max)/smaller(min)
+
.. _i_minnum:
'``llvm.minnum.*``' Intrinsic
@@ -16261,6 +16355,98 @@ of the two arguments. -0.0 is considered to be less than +0.0 for this
intrinsic. Note that these are the semantics specified in the draft of
IEEE 754-2019.
+.. _i_minimumnum:
+
+'``llvm.minimumnum.*``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+This is an overloaded intrinsic. You can use ``llvm.minimumnum`` on any
+floating-point or vector of floating-point type. Not all targets support
+all types however.
+
+::
+
+ declare float @llvm.minimumnum.f32(float %Val0, float %Val1)
+ declare double @llvm.minimumnum.f64(double %Val0, double %Val1)
+ declare x86_fp80 @llvm.minimumnum.f80(x86_fp80 %Val0, x86_fp80 %Val1)
+ declare fp128 @llvm.minimumnum.f128(fp128 %Val0, fp128 %Val1)
+ declare ppc_fp128 @llvm.minimumnum.ppcf128(ppc_fp128 %Val0, ppc_fp128 %Val1)
+
+Overview:
+"""""""""
+
+The '``llvm.minimumnum.*``' intrinsics return the minimum of the two
+arguments, not propagating NaNs and treating -0.0 as less than +0.0.
+
+
+Arguments:
+""""""""""
+
+The arguments and return value are floating-point numbers of the same
+type.
+
+Semantics:
+""""""""""
+If both operands are NaNs (including sNaN), returns qNaN. If one operand
+is NaN (including sNaN) and another operand is a number, return the number.
+Otherwise returns the lesser of the two arguments. -0.0 is considered to
+be less than +0.0 for this intrinsic.
+
+Note that these are the semantics of minimumNumber specified in IEEE 754-2019.
+
+It has some different with '``llvm.minnum.*``':
+1)'``llvm.minnum.*``' will return qNaN if either operand is sNaN.
+2)'``llvm.minnum*``' may return either one if we compare +0.0 vs -0.0.
+
+.. _i_maximumnum:
+
+'``llvm.maximumnum.*``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+This is an overloaded intrinsic. You can use ``llvm.maximumnum`` on any
+floating-point or vector of floating-point type. Not all targets support
+all types however.
+
+::
+
+ declare float @llvm.maximumnum.f32(float %Val0, float %Val1)
+ declare double @llvm.maximumnum.f64(double %Val0, double %Val1)
+ declare x86_fp80 @llvm.maximumnum.f80(x86_fp80 %Val0, x86_fp80 %Val1)
+ declare fp128 @llvm.maximumnum.f128(fp128 %Val0, fp128 %Val1)
+ declare ppc_fp128 @llvm.maximumnum.ppcf128(ppc_fp128 %Val0, ppc_fp128 %Val1)
+
+Overview:
+"""""""""
+
+The '``llvm.maximumnum.*``' intrinsics return the maximum of the two
+arguments, not propagating NaNs and treating -0.0 as less than +0.0.
+
+
+Arguments:
+""""""""""
+
+The arguments and return value are floating-point numbers of the same
+type.
+
+Semantics:
+""""""""""
+If both operands are NaNs (including sNaN), returns qNaN. If one operand
+is NaN (including sNaN) and another operand is a number, return the number.
+Otherwise returns the greater of the two arguments. -0.0 is considered to
+be less than +0.0 for this intrinsic.
+
+Note that these are the semantics of maximumNumber specified in IEEE 754-2019.
+
+It has some different with '``llvm.maxnum.*``':
+1)'``llvm.maxnum.*``' will return qNaN if either operand is sNaN.
+2)'``llvm.maxnum*``' may return either one if we compare +0.0 vs -0.0.
+
.. _int_copysign:
'``llvm.copysign.*``' Intrinsic
@@ -19067,6 +19253,65 @@ Arguments:
""""""""""
The argument to this intrinsic must be a vector of floating-point values.
+.. _int_vector_reduce_fmaximumnum:
+
+'``llvm.vector.reduce.fmaximumnum.*``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+This is an overloaded intrinsic.
+
+::
+
+ declare float @llvm.vector.reduce.fmaximumnum.v4f32(<4 x float> %a)
+ declare double @llvm.vector.reduce.fmaximumnum.v2f64(<2 x double> %a)
+
+Overview:
+"""""""""
+
+The '``llvm.vector.reduce.fmaximumnum.*``' intrinsics do a floating-point
+``MAX`` reduction of a vector, returning the result as a scalar. The return type
+matches the element-type of the vector input.
+
+This instruction has the same comparison semantics as the '``llvm.maximumnum.*``'
+intrinsic. That is, this intrinsic not propagates NaNs (even sNaNs) and +0.0 is
+considered greater than -0.0. If all elements of the vector are NaNs, the result is NaN.
+
+Arguments:
+""""""""""
+The argument to this intrinsic must be a vector of floating-point values.
+
+.. _int_vector_reduce_fminimumnum:
+
+'``llvm.vector.reduce.fminimumnum.*``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+This is an overloaded intrinsic.
+
+::
+
+ declare float @llvm.vector.reduce.fminimumnum.v4f32(<4 x float> %a)
+ declare double @llvm.vector.reduce.fminimumnum.v2f64(<2 x double> %a)
+
+Overview:
+"""""""""
+
+The '``llvm.vector.reduce.fminimumnum.*``' intrinsics do a floating-point
+``MIN`` reduction of a vector, returning the result as a scalar. The return type
+matches the element-type of the vector input.
+
+This instruction has the same comparison semantics as the '``llvm.minimumnum.*``'
+intrinsic. That is, this intrinsic not propagates NaNs (even sNaNs) and -0.0 is
+considered less than +0.0. If all elements of the vector are NaNs, the result is NaN.
+
+Arguments:
+""""""""""
+The argument to this intrinsic must be a vector of floating-point values.
+
+
'``llvm.vector.insert``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -21253,6 +21498,107 @@ Examples:
%also.r = select <4 x i1> %mask, <4 x float> %t, <4 x float> poison
+.. _int_vp_minimumnum:
+
+'``llvm.vp.minimumnum.*``' Intrinsics
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+This is an overloaded intrinsic.
+
+::
+
+ declare <16 x float> @llvm.vp.minimumnum.v16f32 (<16 x float> <left_op>, <16 x float> <right_op>, <16 x i1> <mask>, i32 <vector_length>)
+ declare <vscale x 4 x float> @llvm.vp.minimumnum.nxv4f32 (<vscale x 4 x float> <left_op>, <vscale x 4 x float> <right_op>, <vscale x 4 x i1> <mask>, i32 <vector_length>)
+ declare <256 x double> @llvm.vp.minimumnum.v256f64 (<256 x double> <left_op>, <256 x double> <right_op>, <256 x i1> <mask>, i32 <vector_length>)
+
+Overview:
+"""""""""
+
+Predicated floating-point minimum of two vectors of floating-point values,
+not propagating NaNs and treating -0.0 as less than +0.0.
+
+Arguments:
+""""""""""
+
+The first two operands and the result have the same vector of floating-point type. The
+third operand is the vector mask and has the same number of elements as the
+result vector type. The fourth operand is the explicit vector length of the
+operation.
+
+Semantics:
+""""""""""
+
+The '``llvm.vp.minimumnum``' intrinsic performs floating-point minimumNumber (:ref:`minimumnum <i_minimumnum>`)
+of the first and second vector operand on each enabled lane, the result being
+NaN if both operands are NaN. Even sNaN vs Num results NUM. -0.0 is considered
+to be less than +0.0 for this intrinsic. The result on disabled lanes is a :ref:`poison value <poisonvalues>`.
+The operation is performed in the default floating-point environment.
+
+Examples:
+"""""""""
+
+.. code-block:: llvm
+
+ %r = call <4 x float> @llvm.vp.minimumnum.v4f32(<4 x float> %a, <4 x float> %b, <4 x i1> %mask, i32 %evl)
+ ;; For all lanes below %evl, %r is lane-wise equivalent to %also.r
+
+ %t = call <4 x float> @llvm.minimumnum.v4f32(<4 x float> %a, <4 x float> %b)
+ %also.r = select <4 x i1> %mask, <4 x float> %t, <4 x float> poison
+
+
+.. _int_vp_maximumnum:
+
+'``llvm.vp.maximumnum.*``' Intrinsics
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+This is an overloaded intrinsic.
+
+::
+
+ declare <16 x float> @llvm.vp.maximumnum.v16f32 (<16 x float> <left_op>, <16 x float> <right_op>, <16 x i1> <mask>, i32 <vector_length>)
+ declare <vscale x 4 x float> @llvm.vp.maximumnum.nxv4f32 (<vscale x 4 x float> <left_op>, <vscale x 4 x float> <right_op>, <vscale x 4 x i1> <mask>, i32 <vector_length>)
+ declare <256 x double> @llvm.vp.maximumnum.v256f64 (<256 x double> <left_op>, <256 x double> <right_op>, <256 x i1> <mask>, i32 <vector_length>)
+
+Overview:
+"""""""""
+
+Predicated floating-point maximum of two vectors of floating-point values,
+not propagating NaNs and treating -0.0 as less than +0.0.
+
+Arguments:
+""""""""""
+
+The first two operands and the result have the same vector of floating-point type. The
+third operand is the vector mask and has the same number of elements as the
+result vector type. The fourth operand is the explicit vector length of the
+operation.
+
+Semantics:
+""""""""""
+
+The '``llvm.vp.maximumnum``' intrinsic performs floating-point maximumNumber (:ref:`maximumnum <i_maximumnum>`)
+of the first and second vector operand on each enabled lane, the result being
+NaN if both operands are NaN. sNaN vs Num results NUM. -0.0 is considered
+to be less than +0.0 for this intrinsic. The result on disabled lanes is a :ref:`poison value <poisonvalues>`.
+The operation is performed in the default floating-point environment.
+
+Examples:
+"""""""""
+
+.. code-block:: llvm
+
+ %r = call <4 x float> @llvm.vp.maximumnum.v4f32(<4 x float> %a, <4 x float> %b, <4 x i1> %mask, i32 %evl)
+ ;; For all lanes below %evl, %r is lane-wise equivalent to %also.r
+
+ %t = call <4 x float> @llvm.maximumnum.v4f32(<4 x float> %a, <4 x float> %b, <4 x i1> %mask, i32 %evl)
+ %also.r = select <4 x i1> %mask, <4 x float> %t, <4 x float> poison
+
+
+
.. _int_vp_fadd:
'``llvm.vp.fadd.*``' Intrinsics
@@ -22652,6 +22998,146 @@ Examples:
%also.r = call float @llvm.minimum.f32(float %reduction, float %start)
+.. _int_vp_reduce_fmaximumnum:
+
+'``llvm.vp.reduce.fmaximumnum.*``' Intrinsics
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+This is an overloaded intrinsic.
+
+::
+
+ declare float @llvm.vp.reduce.fmaximumnum.v4f32(float <start_value>, <4 x float> <val>, <4 x i1> <mask>, float <vector_length>)
+ declare double @llvm.vp.reduce.fmaximumnum.nxv8f64(double <start_value>, <vscale x 8 x double> <val>, <vscale x 8 x i1> <mask>, i32 <vector_length>)
+
+Overview:
+"""""""""
+
+Predicated floating-point ``MAX`` reduction of a vector and a scalar starting
+value, returning the result as a scalar.
+
+
+Arguments:
+""""""""""
+
+The first operand is the start value of the reduction, which must be a scalar
+floating-point type equal to the result type. The second operand is the vector
+on which the reduction is performed and must be a vector of floating-point
+values whose element type is the result/start type. The third operand is the
+vector mask and is a vector of boolean values with the same number of elements
+as the vector operand. The fourth operand is the explicit vector length of the
+operation.
+
+Semantics:
+""""""""""
+
+The '``llvm.vp.reduce.fmaximumnum``' intrinsic performs the floating-point ``MAX``
+reduction (:ref:`llvm.vector.reduce.fmaximumnum <int_vector_reduce_fmaximumnum>`) of
+the vector operand ``val`` on each enabled lane, taking the maximum of that and
+the scalar ``start_value``. Disabled lanes are treated as containing the
+neutral value (i.e. having no effect on the reduction operation). If the vector
+length is zero, the result is the start value.
+
+The neutral value is dependent on the :ref:`fast-math flags <fastmath>`. If no
+flags are set or only the ``nnan`` is set, the neutral value is ``-Infinity``.
+If ``ninf`` is set, then the neutral value is the smallest floating-point value
+for the result type.
+
+This instruction has the same comparison semantics as the
+:ref:`llvm.vector.reduce.fmaximumnum <int_vector_reduce_fmaximumnum>` intrinsic (and
+thus the '``llvm.maximumnum.*``' intrinsic). That is, the result will always be a
+number unless all of the elements in the vector and the starting value is
+``NaN``. Namely, this intrinsic not propagates ``NaN``, even for ``sNaN``.
+Also, -0.0 is considered less than +0.0.
+
+To ignore the start value, the neutral value can be used.
+
+Examples:
+"""""""""
+
+.. code-block:: llvm
+
+ %r = call float @llvm.vp.reduce.fmaximumnum.v4f32(float %float, <4 x float> %a, <4 x i1> %mask, i32 %evl)
+ ; %r is equivalent to %also.r, where lanes greater than or equal to %evl
+ ; are treated as though %mask were false for those lanes.
+
+ %masked.a = select <4 x i1> %mask, <4 x float> %a, <4 x float> <float -infinity, float -infinity, float -infinity, float -infinity>
+ %reduction = call float @llvm.vector.reduce.fmaximumnum.v4f32(<4 x float> %masked.a)
+ %also.r = call float @llvm.maximumnum.f32(float %reduction, float %start)
+
+
+.. _int_vp_reduce_fminimumnum:
+
+'``llvm.vp.reduce.fminimumnum.*``' Intrinsics
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+This is an overloaded intrinsic.
+
+::
+
+ declare float @llvm.vp.reduce.fminimumnum.v4f32(float <start_value>, <4 x float> <val>, <4 x i1> <mask>, float <vector_length>)
+ declare double @llvm.vp.reduce.fminimumnum.nxv8f64(double <start_value>, <vscale x 8 x double> <val>, <vscale x 8 x i1> <mask>, i32 <vector_length>)
+
+Overview:
+"""""""""
+
+Predicated floating-point ``MIN`` reduction of a vector and a scalar starting
+value, returning the result as a scalar.
+
+
+Arguments:
+""""""""""
+
+The first operand is the start value of the reduction, which must be a scalar
+floating-point type equal to the result type. The second operand is the vector
+on which the reduction is performed and must be a vector of floating-point
+values whose element type is the result/start type. The third operand is the
+vector mask and is a vector of boolean values with the same number of elements
+as the vector operand. The fourth operand is the explicit vector length of the
+operation.
+
+Semantics:
+""""""""""
+
+The '``llvm.vp.reduce.fminimumnum``' intrinsic performs the floating-point ``MIN``
+reduction (:ref:`llvm.vector.reduce.fminimumnum <int_vector_reduce_fminimumnum>`) of
+the vector operand ``val`` on each enabled lane, taking the minimum of that and
+the scalar ``start_value``. Disabled lanes are treated as containing the
+neutral value (i.e. having no effect on the reduction operation). If the vector
+length is zero, the result is the start value.
+
+The neutral value is dependent on the :ref:`fast-math flags <fastmath>`. If no
+flags are set or only the ``nnan`` is set, the neutral value is ``+Infinity``.
+If ``ninf`` is set, then the neutral value is the largest floating-point value
+for the result type.
+
+This instruction has the same comparison semantics as the
+:ref:`llvm.vector.reduce.fminimumnum <int_vector_reduce_fminimumnum>` intrinsic (and
+thus the '``llvm.minimumnum.*``' intrinsic). That is, the result will always be a
+number unless all of the elements in the vector and the starting value is
+``NaN``. Namely, this intrinsic not propagates ``NaN``, even for ``sNaN``.
+Also, -0.0 is considered less than +0.0.
+
+To ignore the start value, the neutral value can be used.
+
+Examples:
+"""""""""
+
+.. code-block:: llvm
+
+ %r = call float @llvm.vp.reduce.fmaximumnum.v4f32(float %float, <4 x float> %a, <4 x i1> %mask, i32 %evl)
+ ; %r is equivalent to %also.r, where lanes greater than or equal to %evl
+ ; are treated as though %mask were false for those lanes.
+
+ %masked.a = select <4 x i1> %mask, <4 x float> %a, <4 x float> <float -infinity, float -infinity, float -infinity, float -infinity>
+ %reduction = call float @llvm.vector.reduce.fmaximumnum.v4f32(<4 x float> %masked.a)
+ %also.r = call float @llvm.maximumnum.f32(float %reduction, float %start)
+
+
.. _int_get_active_lane_mask:
'``llvm.get.active.lane.mask.*``' Intrinsics
@@ -26957,6 +27443,70 @@ Semantics:
This function follows semantics specified in the draft of IEEE 754-2019.
+'``llvm.experimental.constrained.maximumnum``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+ declare <type>
+ @llvm.experimental.constrained.maximumnum(<type> <op1>, <type> <op2>
+ metadata <exception behavior>)
+
+Overview:
+"""""""""
+
+The '``llvm.experimental.constrained.maximumnum``' intrinsic returns the maximum
+of the two arguments, not propagating NaNs and treating -0.0 as less than +0.0.
+
+Arguments:
+""""""""""
+
+The first two arguments and the return value are floating-point numbers
+of the same type.
+
+The third argument specifies the exception behavior as described above.
+
+Semantics:
+""""""""""
+
+This function follows semantics specified in IEEE 754-2019.
+
+
+'``llvm.experimental.constrained.minimumnum``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+ declare <type>
+ @llvm.experimental.constrained.minimumnum(<type> <op1>, <type> <op2>
+ metadata <exception behavior>)
+
+Overview:
+"""""""""
+
+The '``llvm.experimental.constrained.minimumnum``' intrinsic returns the minimum
+of the two arguments, not propagating NaNs and treating -0.0 as less than +0.0.
+
+Arguments:
+""""""""""
+
+The first two arguments and the return value are floating-point numbers
+of the same type.
+
+The third argument specifies the exception behavior as described above.
+
+Semantics:
+""""""""""
+
+This function follows semantics specified in IEEE 754-2019.
+
+
'``llvm.experimental.constrained.ceil``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/llvm/include/llvm/ADT/APFloat.h b/llvm/include/llvm/ADT/APFloat.h
index c24eae8da3797..db2fa480655c6 100644
--- a/llvm/include/llvm/ADT/APFloat.h
+++ b/llvm/include/llvm/ADT/APFloat.h
@@ -1483,6 +1483,19 @@ inline APFloat minimum(const APFloat &A, const APFloat &B) {
return B < A ? B : A;
}
+/// Implements IEEE 754-2019 minimumNumber semantics. Returns the smaller
+/// of 2 arguments, not propagating NaNs and treating -0 as less than +0.
+LLVM_READONLY
+inline APFloat minimumnum(const APFloat &A, const APFloat &B) {
+ if (A.isNaN())
+ return B.isNaN() ? B.makeQuiet() : B;
+ if (B.isNaN())
+ return A;
+ if (A.isZero() && B.isZero() && (A.isNegative() != B.isNegative()))
+ return A.isNegative() ? A : B;
+ return B < A ? B : A;
+}
+
/// Implements IEEE 754-2019 maximum semantics. Returns the larger of 2
/// arguments, propagating NaNs and treating -0 as less than +0.
LLVM_READONLY
@@ -1496,6 +1509,19 @@ inline APFloat maximum(const APFloat &A, const APFloat &B) {
return A < B ? B : A;
}
+/// Implements IEEE 754-2019 maximumNumber semantics. Returns the larger
+/// of 2 arguments, not propagating NaNs and treating -0 as less than +0.
+LLVM_READONLY
+inline APFloat maximumnum(const APFloat &A, const APFloat &B) {
+ if (A.isNaN())
+ return B.isNaN() ? B.makeQuiet() : B;
+ if (B.isNaN())
+ return A;
+ if (A.isZero() && B.isZero() && (A.isNegative() != B.isNegative()))
+ return A.isNegative() ? B : A;
+ return A < B ? B : A;
+}
+
// We want the following functions to be available in the header for inlining.
// We cannot define them inline in the class definition of `DoubleAPFloat`
// because doing so would instantiate `std::unique_ptr<APFloat[]>` before
diff --git a/llvm/include/llvm/Analysis/TargetLibraryInfo.def b/llvm/include/llvm/Analysis/TargetLibraryInfo.def
index 717693a7cf63c..8b938f97a679d 100644
--- a/llvm/include/llvm/Analysis/TargetLibraryInfo.def
+++ b/llvm/include/llvm/Analysis/TargetLibraryInfo.def
@@ -1347,6 +1347,39 @@ TLI_DEFINE_ENUM_INTERNAL(fminl)
TLI_DEFINE_STRING_INTERNAL("fminl")
TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same)
+// Calls to fmaximum_num and fminimum_num library functions expand to the llvm.maximumnum and
+// llvm.minimumnum intrinsics with the correct parameter types for the arguments
+// (all types must match).
+/// double fmaximum_num(double x, double y);
+TLI_DEFINE_ENUM_INTERNAL(fmaximum_num)
+TLI_DEFINE_STRING_INTERNAL("fmaximum_num")
+TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same)
+
+/// float fmaximum_numf(float x, float y);
+TLI_DEFINE_ENUM_INTERNAL(fmaximum_numf)
+TLI_DEFINE_STRING_INTERNAL("fmaximum_numf")
+TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same)
+
+/// long double fmaximum_numl(long double x, long double y);
+TLI_DEFINE_ENUM_INTERNAL(fmaximum_numl)
+TLI_DEFINE_STRING_INTERNAL("fmaximum_numl")
+TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same)
+
+/// double fminimum_num(double x, double y);
+TLI_DEFINE_ENUM_INTERNAL(fminimum_num)
+TLI_DEFINE_STRING_INTERNAL("fminimum_num")
+TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same)
+
+/// float fminimum_numf(float x, float y);
+TLI_DEFINE_ENUM_INTERNAL(fminimum_numf)
+TLI_DEFINE_STRING_INTERNAL("fminimum_numf")
+TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same)
+
+/// long double fminimum_numl(long double x, long double y);
+TLI_DEFINE_ENUM_INTERNAL(fminimum_numl)
+TLI_DEFINE_STRING_INTERNAL("fminimum_numl")
+TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same)
+
/// double fmod(double x, double y);
TLI_DEFINE_ENUM_INTERNAL(fmod)
TLI_DEFINE_STRING_INTERNAL("fmod")
diff --git a/llvm/include/llvm/CodeGen/BasicTTIImpl.h b/llvm/include/llvm/CodeGen/BasicTTIImpl.h
index 9f8d3ded9b3c1..4385fdb0d1e59 100644
--- a/llvm/include/llvm/CodeGen/BasicTTIImpl.h
+++ b/llvm/include/llvm/CodeGen/BasicTTIImpl.h
@@ -1690,6 +1690,8 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> {
case Intrinsic::vector_reduce_fmin:
case Intrinsic::vector_reduce_fmaximum:
case Intrinsic::vector_reduce_fminimum:
+ case Intrinsic::vector_reduce_fmaximumnum:
+ case Intrinsic::vector_reduce_fminimumnum:
case Intrinsic::vector_reduce_umax:
case Intrinsic::vector_reduce_umin: {
IntrinsicCostAttributes Attrs(IID, RetTy, Args[0]->getType(), FMF, I, 1);
@@ -2016,6 +2018,12 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> {
case Intrinsic::maximum:
ISD = ISD::FMAXIMUM;
break;
+ case Intrinsic::minimumnum:
+ ISD = ISD::FMINIMUMNUM;
+ break;
+ case Intrinsic::maximumnum:
+ ISD = ISD::FMAXIMUMNUM;
+ break;
case Intrinsic::copysign:
ISD = ISD::FCOPYSIGN;
break;
@@ -2097,6 +2105,8 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> {
case Intrinsic::vector_reduce_fmin:
case Intrinsic::vector_reduce_fmaximum:
case Intrinsic::vector_reduce_fminimum:
+ case Intrinsic::vector_reduce_fmaximumnum:
+ case Intrinsic::vector_reduce_fminimumnum:
return thisT()->getMinMaxReductionCost(getMinMaxReductionIntrinsicOp(IID),
VecOpTy, ICA.getFlags(), CostKind);
case Intrinsic::abs: {
diff --git a/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h b/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h
index 43659564d5ace..89ae4c65b29cb 100644
--- a/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h
+++ b/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h
@@ -954,8 +954,10 @@ class CombinerHelper {
///
/// * G_FMAXNUM
/// * G_FMAXIMUM
+ /// * G_FMAXIMUMNUM
/// * G_FMINNUM
/// * G_FMINIMUM
+ /// * G_FMINIMUMNUM
///
/// Helper function for matchFPSelectToMinMax.
unsigned getFPMinMaxOpcForSelect(CmpInst::Predicate Pred, LLT DstTy,
diff --git a/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h b/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h
index 995031f7c00be..240a12b3f8441 100644
--- a/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h
+++ b/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h
@@ -553,6 +553,8 @@ class GVecReduce : public GenericMachineInstr {
case TargetOpcode::G_VECREDUCE_FMIN:
case TargetOpcode::G_VECREDUCE_FMAXIMUM:
case TargetOpcode::G_VECREDUCE_FMINIMUM:
+ case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM:
+ case TargetOpcode::G_VECREDUCE_FMINIMUMNUM:
case TargetOpcode::G_VECREDUCE_ADD:
case TargetOpcode::G_VECREDUCE_MUL:
case TargetOpcode::G_VECREDUCE_AND:
@@ -591,6 +593,12 @@ class GVecReduce : public GenericMachineInstr {
case TargetOpcode::G_VECREDUCE_FMINIMUM:
ScalarOpc = TargetOpcode::G_FMINIMUM;
break;
+ case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM:
+ ScalarOpc = TargetOpcode::G_FMAXIMUMNUM;
+ break;
+ case TargetOpcode::G_VECREDUCE_FMINIMUMNUM:
+ ScalarOpc = TargetOpcode::G_FMINIMUMNUM;
+ break;
case TargetOpcode::G_VECREDUCE_ADD:
ScalarOpc = TargetOpcode::G_ADD;
break;
@@ -671,6 +679,8 @@ class GBinOp : public GenericMachineInstr {
case TargetOpcode::G_FMAXNUM_IEEE:
case TargetOpcode::G_FMINIMUM:
case TargetOpcode::G_FMAXIMUM:
+ case TargetOpcode::G_FMINIMUMNUM:
+ case TargetOpcode::G_FMAXIMUMNUM:
case TargetOpcode::G_FADD:
case TargetOpcode::G_FSUB:
case TargetOpcode::G_FMUL:
@@ -721,6 +731,8 @@ class GFBinOp : public GBinOp {
case TargetOpcode::G_FMAXNUM_IEEE:
case TargetOpcode::G_FMINIMUM:
case TargetOpcode::G_FMAXIMUM:
+ case TargetOpcode::G_FMINIMUMNUM:
+ case TargetOpcode::G_FMAXIMUMNUM:
case TargetOpcode::G_FADD:
case TargetOpcode::G_FSUB:
case TargetOpcode::G_FMUL:
diff --git a/llvm/include/llvm/CodeGen/GlobalISel/LegalizerHelper.h b/llvm/include/llvm/CodeGen/GlobalISel/LegalizerHelper.h
index 284f434fbb9b0..8ce6b44c7e1c6 100644
--- a/llvm/include/llvm/CodeGen/GlobalISel/LegalizerHelper.h
+++ b/llvm/include/llvm/CodeGen/GlobalISel/LegalizerHelper.h
@@ -403,6 +403,7 @@ class LegalizerHelper {
LegalizeResult lowerMinMax(MachineInstr &MI);
LegalizeResult lowerFCopySign(MachineInstr &MI);
LegalizeResult lowerFMinNumMaxNum(MachineInstr &MI);
+ LegalizeResult lowerFMinimumNumMaximumNum(MachineInstr &MI);
LegalizeResult lowerFMad(MachineInstr &MI);
LegalizeResult lowerIntrinsicRound(MachineInstr &MI);
LegalizeResult lowerFFloor(MachineInstr &MI);
diff --git a/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h b/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h
index 92e05ee858a75..d322e32c9317d 100644
--- a/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h
+++ b/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h
@@ -1723,6 +1723,18 @@ class MachineIRBuilder {
return buildInstr(TargetOpcode::G_FMAXNUM_IEEE, {Dst}, {Src0, Src1}, Flags);
}
+ MachineInstrBuilder
+ buildFMinimumnum(const DstOp &Dst, const SrcOp &Src0, const SrcOp &Src1,
+ std::optional<unsigned> Flags = std::nullopt) {
+ return buildInstr(TargetOpcode::G_FMINIMUMNUM, {Dst}, {Src0, Src1}, Flags);
+ }
+
+ MachineInstrBuilder
+ buildFMaximumnum(const DstOp &Dst, const SrcOp &Src0, const SrcOp &Src1,
+ std::optional<unsigned> Flags = std::nullopt) {
+ return buildInstr(TargetOpcode::G_FMAXIMUMNUM, {Dst}, {Src0, Src1}, Flags);
+ }
+
MachineInstrBuilder buildShl(const DstOp &Dst, const SrcOp &Src0,
const SrcOp &Src1,
std::optional<unsigned> Flags = std::nullopt) {
@@ -2074,6 +2086,18 @@ class MachineIRBuilder {
return buildInstr(TargetOpcode::G_VECREDUCE_FMINIMUM, {Dst}, {Src});
}
+ /// Build and insert \p Res = G_VECREDUCE_FMAXIMUMNUM \p Src
+ MachineInstrBuilder buildVecReduceFMaximumnum(const DstOp &Dst,
+ const SrcOp &Src) {
+ return buildInstr(TargetOpcode::G_VECREDUCE_FMAXIMUMNUM, {Dst}, {Src});
+ }
+
+ /// Build and insert \p Res = G_VECREDUCE_FMINIMUMNUM \p Src
+ MachineInstrBuilder buildVecReduceFMinimumnum(const DstOp &Dst,
+ const SrcOp &Src) {
+ return buildInstr(TargetOpcode::G_VECREDUCE_FMINIMUMNUM, {Dst}, {Src});
+ }
+
/// Build and insert \p Res = G_VECREDUCE_ADD \p Src
MachineInstrBuilder buildVecReduceAdd(const DstOp &Dst, const SrcOp &Src) {
return buildInstr(TargetOpcode::G_VECREDUCE_ADD, {Dst}, {Src});
diff --git a/llvm/include/llvm/CodeGen/GlobalISel/Utils.h b/llvm/include/llvm/CodeGen/GlobalISel/Utils.h
index 08736acebca8a..882e8e221229b 100644
--- a/llvm/include/llvm/CodeGen/GlobalISel/Utils.h
+++ b/llvm/include/llvm/CodeGen/GlobalISel/Utils.h
@@ -60,6 +60,8 @@ class APFloat;
case TargetOpcode::G_VECREDUCE_FMIN: \
case TargetOpcode::G_VECREDUCE_FMAXIMUM: \
case TargetOpcode::G_VECREDUCE_FMINIMUM: \
+ case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM: \
+ case TargetOpcode::G_VECREDUCE_FMINIMUMNUM: \
case TargetOpcode::G_VECREDUCE_ADD: \
case TargetOpcode::G_VECREDUCE_MUL: \
case TargetOpcode::G_VECREDUCE_AND: \
@@ -77,6 +79,8 @@ class APFloat;
case TargetOpcode::G_VECREDUCE_FMIN: \
case TargetOpcode::G_VECREDUCE_FMAXIMUM: \
case TargetOpcode::G_VECREDUCE_FMINIMUM: \
+ case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM: \
+ case TargetOpcode::G_VECREDUCE_FMINIMUMNUM: \
case TargetOpcode::G_VECREDUCE_ADD: \
case TargetOpcode::G_VECREDUCE_MUL: \
case TargetOpcode::G_VECREDUCE_AND: \
diff --git a/llvm/include/llvm/CodeGen/ISDOpcodes.h b/llvm/include/llvm/CodeGen/ISDOpcodes.h
index 6bb89fb58a296..ca868686483a3 100644
--- a/llvm/include/llvm/CodeGen/ISDOpcodes.h
+++ b/llvm/include/llvm/CodeGen/ISDOpcodes.h
@@ -436,6 +436,8 @@ enum NodeType {
STRICT_LLRINT,
STRICT_FMAXIMUM,
STRICT_FMINIMUM,
+ STRICT_FMAXIMUMNUM,
+ STRICT_FMINIMUMNUM,
/// STRICT_FP_TO_[US]INT - Convert a floating point value to a signed or
/// unsigned integer. These have the same semantics as fptosi and fptoui
@@ -999,6 +1001,11 @@ enum NodeType {
FMINIMUM,
FMAXIMUM,
+ /// FMINIMUMNUM/FMAXIMUMNUM - minimumnum/maximumnum that is same with
+ /// FMINNUM_IEEE and FMAXNUM_IEEE besides if either operand is sNaN.
+ FMINIMUMNUM,
+ FMAXIMUMNUM,
+
/// FSINCOS - Compute both fsin and fcos as a single operation.
FSINCOS,
@@ -1373,6 +1380,10 @@ enum NodeType {
/// llvm.minimum and llvm.maximum semantics.
VECREDUCE_FMAXIMUM,
VECREDUCE_FMINIMUM,
+ /// FMINIMUMNUM/FMAXIMUMNUM nodes don't propatate NaNs and signed zeroes using
+ /// the llvm.minimumnum and llvm.maximumnum semantics.
+ VECREDUCE_FMAXIMUMNUM,
+ VECREDUCE_FMINIMUMNUM,
/// Integer reductions may have a result type larger than the vector element
/// type. However, the reduction is performed using the vector element type
/// and the value in the top bits is unspecified.
diff --git a/llvm/include/llvm/CodeGen/TargetLowering.h b/llvm/include/llvm/CodeGen/TargetLowering.h
index 06f7ee2a589c8..a992a2024281d 100644
--- a/llvm/include/llvm/CodeGen/TargetLowering.h
+++ b/llvm/include/llvm/CodeGen/TargetLowering.h
@@ -2910,6 +2910,8 @@ class TargetLoweringBase {
case ISD::FMAXNUM_IEEE:
case ISD::FMINIMUM:
case ISD::FMAXIMUM:
+ case ISD::FMINIMUMNUM:
+ case ISD::FMAXIMUMNUM:
case ISD::AVGFLOORS:
case ISD::AVGFLOORU:
case ISD::AVGCEILS:
@@ -5130,6 +5132,8 @@ class TargetLowering : public TargetLoweringBase {
/// through to the default expansion/soften to libcall, we might introduce a
/// link-time dependency on libm into a file that originally did not have one.
SDValue createSelectForFMINNUM_FMAXNUM(SDNode *Node, SelectionDAG &DAG) const;
+ SDValue createSelectForFMINIMUMNUM_FMAXIMUMNUM(SDNode *Node,
+ SelectionDAG &DAG) const;
/// Return a reciprocal estimate value for the input operand.
/// \p Enabled is a ReciprocalEstimate enum with value either 'Unspecified' or
@@ -5261,6 +5265,9 @@ class TargetLowering : public TargetLoweringBase {
/// Expand fminimum/fmaximum into multiple comparison with selects.
SDValue expandFMINIMUM_FMAXIMUM(SDNode *N, SelectionDAG &DAG) const;
+ /// Expand fminimumnum/fmaximumnum into multiple comparison with selects.
+ SDValue expandFMINIMUMNUM_FMAXIMUMNUM(SDNode *N, SelectionDAG &DAG) const;
+
/// Expand FP_TO_[US]INT_SAT into FP_TO_[US]INT and selects or min/max.
/// \param N Node to expand
/// \returns The expansion result
diff --git a/llvm/include/llvm/IR/ConstrainedOps.def b/llvm/include/llvm/IR/ConstrainedOps.def
index a7b37c5cb204d..cb40fcad4fd51 100644
--- a/llvm/include/llvm/IR/ConstrainedOps.def
+++ b/llvm/include/llvm/IR/ConstrainedOps.def
@@ -86,6 +86,8 @@ DAG_FUNCTION(maxnum, 2, 0, experimental_constrained_maxnum, FMAXNUM
DAG_FUNCTION(minnum, 2, 0, experimental_constrained_minnum, FMINNUM)
DAG_FUNCTION(maximum, 2, 0, experimental_constrained_maximum, FMAXIMUM)
DAG_FUNCTION(minimum, 2, 0, experimental_constrained_minimum, FMINIMUM)
+DAG_FUNCTION(maximumnum, 2, 0, experimental_constrained_maximumnum, FMAXIMUMNUM)
+DAG_FUNCTION(minimumnum, 2, 0, experimental_constrained_minimumnum, FMINIMUMNUM)
DAG_FUNCTION(nearbyint, 1, 1, experimental_constrained_nearbyint, FNEARBYINT)
DAG_FUNCTION(pow, 2, 1, experimental_constrained_pow, FPOW)
DAG_FUNCTION(powi, 2, 1, experimental_constrained_powi, FPOWI)
diff --git a/llvm/include/llvm/IR/IRBuilder.h b/llvm/include/llvm/IR/IRBuilder.h
index c10ea33a4ee13..f4149c0041a68 100644
--- a/llvm/include/llvm/IR/IRBuilder.h
+++ b/llvm/include/llvm/IR/IRBuilder.h
@@ -1016,6 +1016,18 @@ class IRBuilderBase {
return CreateBinaryIntrinsic(Intrinsic::maximum, LHS, RHS, nullptr, Name);
}
+ /// Create call to the minimumnum intrinsic.
+ Value *CreateMinimumNum(Value *LHS, Value *RHS, const Twine &Name = "") {
+ return CreateBinaryIntrinsic(Intrinsic::minimumnum, LHS, RHS, nullptr,
+ Name);
+ }
+
+ /// Create call to the maximum intrinsic.
+ Value *CreateMaximumNum(Value *LHS, Value *RHS, const Twine &Name = "") {
+ return CreateBinaryIntrinsic(Intrinsic::maximumnum, LHS, RHS, nullptr,
+ Name);
+ }
+
/// Create call to the copysign intrinsic.
Value *CreateCopySign(Value *LHS, Value *RHS,
Instruction *FMFSource = nullptr,
diff --git a/llvm/include/llvm/IR/IntrinsicInst.h b/llvm/include/llvm/IR/IntrinsicInst.h
index 3963a5c8ab8f9..dce0d0229b257 100644
--- a/llvm/include/llvm/IR/IntrinsicInst.h
+++ b/llvm/include/llvm/IR/IntrinsicInst.h
@@ -76,6 +76,8 @@ class IntrinsicInst : public CallInst {
case Intrinsic::minnum:
case Intrinsic::maximum:
case Intrinsic::minimum:
+ case Intrinsic::maximumnum:
+ case Intrinsic::minimumnum:
case Intrinsic::smax:
case Intrinsic::smin:
case Intrinsic::umax:
diff --git a/llvm/include/llvm/IR/Intrinsics.td b/llvm/include/llvm/IR/Intrinsics.td
index c7d383a5d0c0c..52de62c63b1bf 100644
--- a/llvm/include/llvm/IR/Intrinsics.td
+++ b/llvm/include/llvm/IR/Intrinsics.td
@@ -1086,6 +1086,14 @@ def int_maximum : DefaultAttrsIntrinsic<[llvm_anyfloat_ty],
[LLVMMatchType<0>, LLVMMatchType<0>],
[IntrNoMem, IntrSpeculatable, IntrWillReturn, Commutative]
>;
+def int_minimumnum : DefaultAttrsIntrinsic<[llvm_anyfloat_ty],
+ [LLVMMatchType<0>, LLVMMatchType<0>],
+ [IntrNoMem, IntrSpeculatable, IntrWillReturn, Commutative]
+>;
+def int_maximumnum : DefaultAttrsIntrinsic<[llvm_anyfloat_ty],
+ [LLVMMatchType<0>, LLVMMatchType<0>],
+ [IntrNoMem, IntrSpeculatable, IntrWillReturn, Commutative]
+>;
// Internal interface for object size checking
def int_objectsize : DefaultAttrsIntrinsic<[llvm_anyint_ty],
@@ -1280,6 +1288,14 @@ let IntrProperties = [IntrInaccessibleMemOnly, IntrWillReturn, IntrStrictFP] in
[ LLVMMatchType<0>,
LLVMMatchType<0>,
llvm_metadata_ty ]>;
+ def int_experimental_constrained_maximumnum : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+ [ LLVMMatchType<0>,
+ LLVMMatchType<0>,
+ llvm_metadata_ty ]>;
+ def int_experimental_constrained_minimumnum : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+ [ LLVMMatchType<0>,
+ LLVMMatchType<0>,
+ llvm_metadata_ty ]>;
def int_experimental_constrained_ceil : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
[ LLVMMatchType<0>,
llvm_metadata_ty ]>;
@@ -2077,6 +2093,16 @@ let IntrProperties = [IntrNoMem, IntrNoSync, IntrWillReturn] in {
LLVMMatchType<0>,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
llvm_i32_ty]>;
+ def int_vp_minimumnum : DefaultAttrsIntrinsic<[ llvm_anyvector_ty ],
+ [ LLVMMatchType<0>,
+ LLVMMatchType<0>,
+ LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
+ llvm_i32_ty]>;
+ def int_vp_maximumnum : DefaultAttrsIntrinsic<[ llvm_anyvector_ty ],
+ [ LLVMMatchType<0>,
+ LLVMMatchType<0>,
+ LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
+ llvm_i32_ty]>;
def int_vp_copysign : DefaultAttrsIntrinsic<[ llvm_anyvector_ty ],
[ LLVMMatchType<0>,
LLVMMatchType<0>,
@@ -2266,6 +2292,16 @@ let IntrProperties = [IntrNoMem, IntrNoSync, IntrWillReturn] in {
llvm_anyvector_ty,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
llvm_i32_ty]>;
+ def int_vp_reduce_fmaximumnum : DefaultAttrsIntrinsic<[LLVMVectorElementType<0>],
+ [ LLVMVectorElementType<0>,
+ llvm_anyvector_ty,
+ LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
+ llvm_i32_ty]>;
+ def int_vp_reduce_fminimumnum : DefaultAttrsIntrinsic<[LLVMVectorElementType<0>],
+ [ LLVMVectorElementType<0>,
+ llvm_anyvector_ty,
+ LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
+ llvm_i32_ty]>;
}
let IntrProperties = [IntrNoMem, IntrNoSync, IntrWillReturn, ImmArg<ArgIndex<1>>] in {
@@ -2504,6 +2540,10 @@ let IntrProperties = [IntrNoMem, IntrSpeculatable] in {
[llvm_anyvector_ty]>;
def int_vector_reduce_fmaximum: DefaultAttrsIntrinsic<[LLVMVectorElementType<0>],
[llvm_anyvector_ty]>;
+ def int_vector_reduce_fminimumnum: DefaultAttrsIntrinsic<[LLVMVectorElementType<0>],
+ [llvm_anyvector_ty]>;
+ def int_vector_reduce_fmaximumnum: DefaultAttrsIntrinsic<[LLVMVectorElementType<0>],
+ [llvm_anyvector_ty]>;
}
//===----- Matrix intrinsics ---------------------------------------------===//
diff --git a/llvm/include/llvm/IR/RuntimeLibcalls.def b/llvm/include/llvm/IR/RuntimeLibcalls.def
index d8eab80656c06..e6764dfa15830 100644
--- a/llvm/include/llvm/IR/RuntimeLibcalls.def
+++ b/llvm/include/llvm/IR/RuntimeLibcalls.def
@@ -269,6 +269,16 @@ HANDLE_LIBCALL(FMAX_F64, "fmax")
HANDLE_LIBCALL(FMAX_F80, "fmaxl")
HANDLE_LIBCALL(FMAX_F128, "fmaxl")
HANDLE_LIBCALL(FMAX_PPCF128, "fmaxl")
+HANDLE_LIBCALL(FMINIMUMNUM_F32, "fminimum_numf")
+HANDLE_LIBCALL(FMINIMUMNUM_F64, "fminimum_num")
+HANDLE_LIBCALL(FMINIMUMNUM_F80, "fminimum_numl")
+HANDLE_LIBCALL(FMINIMUMNUM_F128, "fminmum_numl")
+HANDLE_LIBCALL(FMINIMUMNUM_PPCF128, "fminimum_numl")
+HANDLE_LIBCALL(FMAXIMUMNUM_F32, "fmaximum_numf")
+HANDLE_LIBCALL(FMAXIMUMNUM_F64, "fmaximum_num")
+HANDLE_LIBCALL(FMAXIMUMNUM_F80, "fmaximum_numl")
+HANDLE_LIBCALL(FMAXIMUMNUM_F128, "fmaxmum_numl")
+HANDLE_LIBCALL(FMAXIMUMNUM_PPCF128, "fmaximum_numl")
HANDLE_LIBCALL(LROUND_F32, "lroundf")
HANDLE_LIBCALL(LROUND_F64, "lround")
HANDLE_LIBCALL(LROUND_F80, "lroundl")
diff --git a/llvm/include/llvm/IR/VPIntrinsics.def b/llvm/include/llvm/IR/VPIntrinsics.def
index 8eced073501e8..94187fe11c593 100644
--- a/llvm/include/llvm/IR/VPIntrinsics.def
+++ b/llvm/include/llvm/IR/VPIntrinsics.def
@@ -428,6 +428,20 @@ VP_PROPERTY_FUNCTIONAL_SDOPC(FMAXIMUM)
VP_PROPERTY_FUNCTIONAL_INTRINSIC(maximum)
END_REGISTER_VP(vp_maximum, VP_FMAXIMUM)
+// llvm.vp.minimumnum(x,y,mask,vlen)
+BEGIN_REGISTER_VP(vp_minimumnum, 2, 3, VP_FMINIMUMNUM, -1)
+VP_PROPERTY_BINARYOP
+VP_PROPERTY_FUNCTIONAL_SDOPC(FMINIMUMNUM)
+VP_PROPERTY_FUNCTIONAL_INTRINSIC(minimumnum)
+END_REGISTER_VP(vp_minimumnum, VP_FMINIMUMNUM)
+
+// llvm.vp.maximumnum(x,y,mask,vlen)
+BEGIN_REGISTER_VP(vp_maximumnum, 2, 3, VP_FMAXIMUMNUM, -1)
+VP_PROPERTY_BINARYOP
+VP_PROPERTY_FUNCTIONAL_SDOPC(FMAXIMUMNUM)
+VP_PROPERTY_FUNCTIONAL_INTRINSIC(maximumnum)
+END_REGISTER_VP(vp_maximumnum, VP_FMAXIMUMNUM)
+
// llvm.vp.ceil(x,mask,vlen)
BEGIN_REGISTER_VP(vp_ceil, 1, 2, VP_FCEIL, -1)
VP_PROPERTY_FUNCTIONAL_INTRINSIC(ceil)
@@ -709,6 +723,14 @@ HELPER_REGISTER_REDUCTION_VP(vp_reduce_fmaximum, VP_REDUCE_FMAXIMUM,
HELPER_REGISTER_REDUCTION_VP(vp_reduce_fminimum, VP_REDUCE_FMINIMUM,
vector_reduce_fminimum)
+// llvm.vp.reduce.fmaximumnum(start,x,mask,vlen)
+HELPER_REGISTER_REDUCTION_VP(vp_reduce_fmaximumnum, VP_REDUCE_FMAXIMUMNUM,
+ vector_reduce_fmaximumnum)
+
+// llvm.vp.reduce.fminimumnum(start,x,mask,vlen)
+HELPER_REGISTER_REDUCTION_VP(vp_reduce_fminimumnum, VP_REDUCE_FMINIMUMNUM,
+ vector_reduce_fminimumnum)
+
#undef HELPER_REGISTER_REDUCTION_VP
// Specialized helper macro for VP reductions as above but with two forms:
diff --git a/llvm/include/llvm/Support/TargetOpcodes.def b/llvm/include/llvm/Support/TargetOpcodes.def
index df4b264af72a8..d4da86fdef5d8 100644
--- a/llvm/include/llvm/Support/TargetOpcodes.def
+++ b/llvm/include/llvm/Support/TargetOpcodes.def
@@ -692,6 +692,10 @@ HANDLE_TARGET_OPCODE(G_FMAXNUM_IEEE)
HANDLE_TARGET_OPCODE(G_FMINIMUM)
HANDLE_TARGET_OPCODE(G_FMAXIMUM)
+/// FP min_num/max_num matching IEEE-754 2019 semantics.
+HANDLE_TARGET_OPCODE(G_FMINIMUMNUM)
+HANDLE_TARGET_OPCODE(G_FMAXIMUMNUM)
+
/// Access to FP environment.
HANDLE_TARGET_OPCODE(G_GET_FPENV)
HANDLE_TARGET_OPCODE(G_SET_FPENV)
@@ -875,6 +879,8 @@ HANDLE_TARGET_OPCODE(G_VECREDUCE_FMAX)
HANDLE_TARGET_OPCODE(G_VECREDUCE_FMIN)
HANDLE_TARGET_OPCODE(G_VECREDUCE_FMAXIMUM)
HANDLE_TARGET_OPCODE(G_VECREDUCE_FMINIMUM)
+HANDLE_TARGET_OPCODE(G_VECREDUCE_FMAXIMUMNUM)
+HANDLE_TARGET_OPCODE(G_VECREDUCE_FMINIMUMNUM)
HANDLE_TARGET_OPCODE(G_VECREDUCE_ADD)
HANDLE_TARGET_OPCODE(G_VECREDUCE_MUL)
HANDLE_TARGET_OPCODE(G_VECREDUCE_AND)
diff --git a/llvm/include/llvm/Target/GenericOpcodes.td b/llvm/include/llvm/Target/GenericOpcodes.td
index 4abffe6476c85..ec6fc0cc10809 100644
--- a/llvm/include/llvm/Target/GenericOpcodes.td
+++ b/llvm/include/llvm/Target/GenericOpcodes.td
@@ -837,6 +837,23 @@ def G_FMAXIMUM : GenericInstruction {
let isCommutable = true;
}
+// FMINIMUMNUM/FMAXIMUMNUM - minimumnum/maximumnum that also treat -0.0
+// as less than 0.0. While FMINNUM_IEEE/FMAXNUM_IEEE follow IEEE 754-2008
+// semantics, FMINIMUM/FMAXIMUM follow IEEE 754-2019 semantics: if one operand
+// is SNaN, 2008 returns QNaN, while 2019 returns another non-NaN operand.
+def G_FMINIMUMNUM : GenericInstruction {
+ let OutOperandList = (outs type0:$dst);
+ let InOperandList = (ins type0:$src1, type0:$src2);
+ let hasSideEffects = false;
+ let isCommutable = true;
+}
+
+def G_FMAXIMUMNUM : GenericInstruction {
+ let OutOperandList = (outs type0:$dst);
+ let InOperandList = (ins type0:$src1, type0:$src2);
+ let hasSideEffects = false;
+ let isCommutable = true;
+}
//------------------------------------------------------------------------------
// Floating Point Binary ops.
//------------------------------------------------------------------------------
@@ -1571,6 +1588,8 @@ def G_VECREDUCE_FMAX : VectorReduction;
def G_VECREDUCE_FMIN : VectorReduction;
def G_VECREDUCE_FMAXIMUM : VectorReduction;
def G_VECREDUCE_FMINIMUM : VectorReduction;
+def G_VECREDUCE_FMAXIMUMNUM : VectorReduction;
+def G_VECREDUCE_FMINIMUMNUM : VectorReduction;
def G_VECREDUCE_ADD : VectorReduction;
def G_VECREDUCE_MUL : VectorReduction;
diff --git a/llvm/include/llvm/Target/GlobalISel/Combine.td b/llvm/include/llvm/Target/GlobalISel/Combine.td
index bd43b95899030..c2a8bb867c5cf 100644
--- a/llvm/include/llvm/Target/GlobalISel/Combine.td
+++ b/llvm/include/llvm/Target/GlobalISel/Combine.td
@@ -486,7 +486,7 @@ def commute_fp_constant_to_rhs : GICombineRule<
(defs root:$root),
(match (wip_match_opcode G_FADD, G_FMUL, G_FMINNUM, G_FMAXNUM,
G_FMINNUM_IEEE, G_FMAXNUM_IEEE,
- G_FMINIMUM, G_FMAXIMUM):$root,
+ G_FMINIMUM, G_FMAXIMUM, G_FMINIMUMNUM, G_FMAXIMUMNUM):$root,
[{ return Helper.matchCommuteFPConstantToRHS(*${root}); }]),
(apply [{ Helper.applyCommuteBinOpOperands(*${root}); }])
>;
@@ -583,7 +583,7 @@ def fold_binop_into_select : GICombineRule<
G_SDIV, G_SREM, G_UDIV, G_UREM, G_LSHR, G_ASHR, G_SHL,
G_SMIN, G_SMAX, G_UMIN, G_UMAX,
G_FMUL, G_FADD, G_FSUB, G_FDIV, G_FREM,
- G_FMINNUM, G_FMAXNUM, G_FMINIMUM, G_FMAXIMUM):$root,
+ G_FMINNUM, G_FMAXNUM, G_FMINIMUM, G_FMAXIMUM, G_FMINIMUMNUM, G_FMAXIMUMNUM):$root,
[{ return Helper.matchFoldBinOpIntoSelect(*${root}, ${select_op_no}); }]),
(apply [{ Helper.applyFoldBinOpIntoSelect(*${root}, ${select_op_no}); }])
>;
@@ -1249,7 +1249,7 @@ def combine_fsub_fpext_fneg_fmul_to_fmad_or_fma: GICombineRule<
def combine_minmax_nan: GICombineRule<
(defs root:$root, unsigned_matchinfo:$info),
- (match (wip_match_opcode G_FMINNUM, G_FMAXNUM, G_FMINIMUM, G_FMAXIMUM):$root,
+ (match (wip_match_opcode G_FMINNUM, G_FMAXNUM, G_FMINIMUM, G_FMAXIMUM, G_FMINIMUMNUM, G_FMAXIMUMNUM):$root,
[{ return Helper.matchCombineFMinMaxNaN(*${root}, ${info}); }]),
(apply [{ Helper.replaceSingleDefInstWithOperand(*${root}, ${info}); }])>;
diff --git a/llvm/include/llvm/Target/GlobalISel/SelectionDAGCompat.td b/llvm/include/llvm/Target/GlobalISel/SelectionDAGCompat.td
index 560d3b434d07d..27a16555ab6a3 100644
--- a/llvm/include/llvm/Target/GlobalISel/SelectionDAGCompat.td
+++ b/llvm/include/llvm/Target/GlobalISel/SelectionDAGCompat.td
@@ -171,6 +171,8 @@ def : GINodeEquiv<G_FMINNUM_IEEE, fminnum_ieee>;
def : GINodeEquiv<G_FMAXNUM_IEEE, fmaxnum_ieee>;
def : GINodeEquiv<G_FMAXIMUM, fmaximum>;
def : GINodeEquiv<G_FMINIMUM, fminimum>;
+def : GINodeEquiv<G_FMAXIMUMNUM, fmaximumnum>;
+def : GINodeEquiv<G_FMINIMUMNUM, fminimumnum>;
def : GINodeEquiv<G_READCYCLECOUNTER, readcyclecounter>;
def : GINodeEquiv<G_READSTEADYCOUNTER, readsteadycounter>;
def : GINodeEquiv<G_ROTR, rotr>;
@@ -182,6 +184,8 @@ def : GINodeEquiv<G_VECREDUCE_FMAX, vecreduce_fmax>;
def : GINodeEquiv<G_VECREDUCE_FMIN, vecreduce_fmin>;
def : GINodeEquiv<G_VECREDUCE_FMAXIMUM, vecreduce_fmaximum>;
def : GINodeEquiv<G_VECREDUCE_FMINIMUM, vecreduce_fminimum>;
+def : GINodeEquiv<G_VECREDUCE_FMAXIMUMNUM, vecreduce_fmaximumnum>;
+def : GINodeEquiv<G_VECREDUCE_FMINIMUMNUM, vecreduce_fminimumnum>;
def : GINodeEquiv<G_VECREDUCE_UMIN, vecreduce_umin>;
def : GINodeEquiv<G_VECREDUCE_UMAX, vecreduce_umax>;
def : GINodeEquiv<G_VECREDUCE_SMIN, vecreduce_smin>;
diff --git a/llvm/include/llvm/Target/TargetSelectionDAG.td b/llvm/include/llvm/Target/TargetSelectionDAG.td
index 8cbf98cd58ca9..d2179f775ab79 100644
--- a/llvm/include/llvm/Target/TargetSelectionDAG.td
+++ b/llvm/include/llvm/Target/TargetSelectionDAG.td
@@ -487,6 +487,8 @@ def vecreduce_fmin : SDNode<"ISD::VECREDUCE_FMIN", SDTFPVecReduce>;
def vecreduce_fmax : SDNode<"ISD::VECREDUCE_FMAX", SDTFPVecReduce>;
def vecreduce_fminimum : SDNode<"ISD::VECREDUCE_FMINIMUM", SDTFPVecReduce>;
def vecreduce_fmaximum : SDNode<"ISD::VECREDUCE_FMAXIMUM", SDTFPVecReduce>;
+def vecreduce_fminimumnum : SDNode<"ISD::VECREDUCE_FMINIMUMNUM", SDTFPVecReduce>;
+def vecreduce_fmaximumnum : SDNode<"ISD::VECREDUCE_FMAXIMUMNUM", SDTFPVecReduce>;
def fadd : SDNode<"ISD::FADD" , SDTFPBinOp, [SDNPCommutative]>;
def fsub : SDNode<"ISD::FSUB" , SDTFPBinOp>;
@@ -508,6 +510,10 @@ def fminimum : SDNode<"ISD::FMINIMUM" , SDTFPBinOp,
[SDNPCommutative, SDNPAssociative]>;
def fmaximum : SDNode<"ISD::FMAXIMUM" , SDTFPBinOp,
[SDNPCommutative, SDNPAssociative]>;
+def fminimumnum : SDNode<"ISD::FMINIMUMNUM" , SDTFPBinOp,
+ [SDNPCommutative, SDNPAssociative]>;
+def fmaximumnum : SDNode<"ISD::FMAXIMUMNUM" , SDTFPBinOp,
+ [SDNPCommutative, SDNPAssociative]>;
def fgetsign : SDNode<"ISD::FGETSIGN" , SDTFPToIntOp>;
def fcanonicalize : SDNode<"ISD::FCANONICALIZE", SDTFPUnaryOp>;
def fneg : SDNode<"ISD::FNEG" , SDTFPUnaryOp>;
@@ -612,6 +618,12 @@ def strict_fminimum : SDNode<"ISD::STRICT_FMINIMUM",
def strict_fmaximum : SDNode<"ISD::STRICT_FMAXIMUM",
SDTFPBinOp, [SDNPHasChain,
SDNPCommutative, SDNPAssociative]>;
+def strict_fminimumnum : SDNode<"ISD::STRICT_FMINIMUMNUM",
+ SDTFPBinOp, [SDNPHasChain,
+ SDNPCommutative, SDNPAssociative]>;
+def strict_fmaximumnum : SDNode<"ISD::STRICT_FMAXIMUMNUM",
+ SDTFPBinOp, [SDNPHasChain,
+ SDNPCommutative, SDNPAssociative]>;
def strict_fpround : SDNode<"ISD::STRICT_FP_ROUND",
SDTFPRoundOp, [SDNPHasChain]>;
def strict_fpextend : SDNode<"ISD::STRICT_FP_EXTEND",
@@ -1585,6 +1597,12 @@ def any_fmaximum : PatFrags<(ops node:$lhs, node:$rhs),
def any_fminimum : PatFrags<(ops node:$lhs, node:$rhs),
[(strict_fminimum node:$lhs, node:$rhs),
(fminimum node:$lhs, node:$rhs)]>;
+def any_fmaximumnum : PatFrags<(ops node:$lhs, node:$rhs),
+ [(strict_fmaximumnum node:$lhs, node:$rhs),
+ (fmaximumnum node:$lhs, node:$rhs)]>;
+def any_fminimumnum : PatFrags<(ops node:$lhs, node:$rhs),
+ [(strict_fminimumnum node:$lhs, node:$rhs),
+ (fminimumnum node:$lhs, node:$rhs)]>;
def any_fpround : PatFrags<(ops node:$src),
[(strict_fpround node:$src),
(fpround node:$src)]>;
diff --git a/llvm/lib/CodeGen/ExpandVectorPredication.cpp b/llvm/lib/CodeGen/ExpandVectorPredication.cpp
index dc35f33a3a059..c8a9f9074d95e 100644
--- a/llvm/lib/CodeGen/ExpandVectorPredication.cpp
+++ b/llvm/lib/CodeGen/ExpandVectorPredication.cpp
@@ -331,7 +331,9 @@ Value *CachingVPExpander::expandPredicationToFPCall(
return NewOp;
}
case Intrinsic::maxnum:
- case Intrinsic::minnum: {
+ case Intrinsic::minnum:
+ case Intrinsic::maximumnum:
+ case Intrinsic::minimumnum: {
Value *Op0 = VPI.getOperand(0);
Value *Op1 = VPI.getOperand(1);
Function *Fn = Intrinsic::getDeclaration(
@@ -498,6 +500,18 @@ CachingVPExpander::expandPredicationInReduction(IRBuilder<> &Builder,
Reduction =
Builder.CreateBinaryIntrinsic(Intrinsic::minimum, Reduction, Start);
break;
+ case Intrinsic::vp_reduce_fmaximumnum:
+ Reduction = Builder.CreateFPMaximumReduce(RedOp);
+ transferDecorations(*Reduction, VPI);
+ Reduction =
+ Builder.CreateBinaryIntrinsic(Intrinsic::maximumnum, Reduction, Start);
+ break;
+ case Intrinsic::vp_reduce_fminimumnum:
+ Reduction = Builder.CreateFPMinimumReduce(RedOp);
+ transferDecorations(*Reduction, VPI);
+ Reduction =
+ Builder.CreateBinaryIntrinsic(Intrinsic::minimumnum, Reduction, Start);
+ break;
case Intrinsic::vp_reduce_fadd:
Reduction = Builder.CreateFAddReduce(Start, RedOp);
break;
@@ -755,6 +769,8 @@ Value *CachingVPExpander::expandPredication(VPIntrinsic &VPI) {
case Intrinsic::vp_minnum:
case Intrinsic::vp_maximum:
case Intrinsic::vp_minimum:
+ case Intrinsic::vp_maximumnum:
+ case Intrinsic::vp_minimumnum:
case Intrinsic::vp_fma:
case Intrinsic::vp_fmuladd:
return expandPredicationToFPCall(Builder, VPI,
diff --git a/llvm/lib/CodeGen/GlobalISel/CSEMIRBuilder.cpp b/llvm/lib/CodeGen/GlobalISel/CSEMIRBuilder.cpp
index 547529bbe699a..ad2ef19fc67eb 100644
--- a/llvm/lib/CodeGen/GlobalISel/CSEMIRBuilder.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/CSEMIRBuilder.cpp
@@ -245,6 +245,8 @@ MachineInstrBuilder CSEMIRBuilder::buildInstr(unsigned Opc,
case TargetOpcode::G_FMAXNUM_IEEE:
case TargetOpcode::G_FMINIMUM:
case TargetOpcode::G_FMAXIMUM:
+ case TargetOpcode::G_FMINIMUMNUM:
+ case TargetOpcode::G_FMAXIMUMNUM:
case TargetOpcode::G_FCOPYSIGN: {
// Try to constant fold these.
assert(SrcOps.size() == 2 && "Invalid sources");
diff --git a/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp b/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
index 31030accd43f7..ec93cbe5bc281 100644
--- a/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
@@ -6175,6 +6175,8 @@ bool CombinerHelper::matchCombineFMinMaxNaN(MachineInstr &MI,
return false;
case TargetOpcode::G_FMINNUM:
case TargetOpcode::G_FMAXNUM:
+ case TargetOpcode::G_FMINIMUMNUM:
+ case TargetOpcode::G_FMAXIMUMNUM:
PropagateNaN = false;
break;
case TargetOpcode::G_FMINIMUM:
diff --git a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
index c06b35a98e434..4b118a982047f 100644
--- a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
@@ -1919,6 +1919,10 @@ unsigned IRTranslator::getSimpleIntrinsicOpcode(Intrinsic::ID ID) {
return TargetOpcode::G_FMINIMUM;
case Intrinsic::maximum:
return TargetOpcode::G_FMAXIMUM;
+ case Intrinsic::minimumnum:
+ return TargetOpcode::G_FMINIMUMNUM;
+ case Intrinsic::maximumnum:
+ return TargetOpcode::G_FMAXIMUMNUM;
case Intrinsic::canonicalize:
return TargetOpcode::G_FCANONICALIZE;
case Intrinsic::floor:
@@ -1976,6 +1980,10 @@ unsigned IRTranslator::getSimpleIntrinsicOpcode(Intrinsic::ID ID) {
return TargetOpcode::G_VECREDUCE_FMINIMUM;
case Intrinsic::vector_reduce_fmaximum:
return TargetOpcode::G_VECREDUCE_FMAXIMUM;
+ case Intrinsic::vector_reduce_fminimumnum:
+ return TargetOpcode::G_VECREDUCE_FMINIMUMNUM;
+ case Intrinsic::vector_reduce_fmaximumnum:
+ return TargetOpcode::G_VECREDUCE_FMAXIMUMNUM;
case Intrinsic::vector_reduce_add:
return TargetOpcode::G_VECREDUCE_ADD;
case Intrinsic::vector_reduce_mul:
diff --git a/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp b/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
index 430fcae731689..638cd5ec7162f 100644
--- a/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
@@ -467,6 +467,10 @@ static RTLIB::Libcall getRTLibDesc(unsigned Opcode, unsigned Size) {
RTLIBCASE(FMIN_F);
case TargetOpcode::G_FMAXNUM:
RTLIBCASE(FMAX_F);
+ case TargetOpcode::G_FMINIMUMNUM:
+ RTLIBCASE(FMINIMUMNUM_F);
+ case TargetOpcode::G_FMAXIMUMNUM:
+ RTLIBCASE(FMAXIMUMNUM_F);
case TargetOpcode::G_FSQRT:
RTLIBCASE(SQRT_F);
case TargetOpcode::G_FRINT:
@@ -1051,6 +1055,8 @@ LegalizerHelper::libcall(MachineInstr &MI, LostDebugLocObserver &LocObserver) {
case TargetOpcode::G_FFLOOR:
case TargetOpcode::G_FMINNUM:
case TargetOpcode::G_FMAXNUM:
+ case TargetOpcode::G_FMINIMUMNUM:
+ case TargetOpcode::G_FMAXIMUMNUM:
case TargetOpcode::G_FSQRT:
case TargetOpcode::G_FRINT:
case TargetOpcode::G_FNEARBYINT:
@@ -2888,6 +2894,8 @@ LegalizerHelper::widenScalar(MachineInstr &MI, unsigned TypeIdx, LLT WideTy) {
case TargetOpcode::G_FMAXNUM_IEEE:
case TargetOpcode::G_FMINIMUM:
case TargetOpcode::G_FMAXIMUM:
+ case TargetOpcode::G_FMINIMUMNUM:
+ case TargetOpcode::G_FMAXIMUMNUM:
case TargetOpcode::G_FDIV:
case TargetOpcode::G_FREM:
case TargetOpcode::G_FCEIL:
@@ -3011,7 +3019,9 @@ LegalizerHelper::widenScalar(MachineInstr &MI, unsigned TypeIdx, LLT WideTy) {
case TargetOpcode::G_VECREDUCE_FMIN:
case TargetOpcode::G_VECREDUCE_FMAX:
case TargetOpcode::G_VECREDUCE_FMINIMUM:
- case TargetOpcode::G_VECREDUCE_FMAXIMUM: {
+ case TargetOpcode::G_VECREDUCE_FMAXIMUM:
+ case TargetOpcode::G_VECREDUCE_FMINIMUMNUM:
+ case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM: {
if (TypeIdx != 0)
return UnableToLegalize;
Observer.changingInstr(MI);
@@ -3928,6 +3938,9 @@ LegalizerHelper::lower(MachineInstr &MI, unsigned TypeIdx, LLT LowerHintTy) {
case G_FMINNUM:
case G_FMAXNUM:
return lowerFMinNumMaxNum(MI);
+ case G_FMINIMUMNUM:
+ case G_FMAXIMUMNUM:
+ return lowerFMinimumNumMaximumNum(MI);
case G_MERGE_VALUES:
return lowerMergeValues(MI);
case G_UNMERGE_VALUES:
@@ -4682,6 +4695,8 @@ LegalizerHelper::fewerElementsVector(MachineInstr &MI, unsigned TypeIdx,
case G_FMAXNUM_IEEE:
case G_FMINIMUM:
case G_FMAXIMUM:
+ case G_FMINIMUMNUM:
+ case G_FMAXIMUMNUM:
case G_FSHL:
case G_FSHR:
case G_ROTL:
@@ -5340,8 +5355,10 @@ MachineInstrBuilder LegalizerHelper::getNeutralElementForVecReduce(
return MIRBuilder.buildFConstant(Ty, 1.0);
case TargetOpcode::G_VECREDUCE_FMINIMUM:
case TargetOpcode::G_VECREDUCE_FMAXIMUM:
+ case TargetOpcode::G_VECREDUCE_FMINIMUMNUM:
+ case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM:
assert(false && "getNeutralElementForVecReduce unimplemented for "
- "G_VECREDUCE_FMINIMUM and G_VECREDUCE_FMAXIMUM!");
+ "G_VECREDUCE_FMINIMUM and G_VECREDUCE_FMAXIMUM(_NUM)!");
}
llvm_unreachable("switch expected to return!");
}
@@ -5392,6 +5409,8 @@ LegalizerHelper::moreElementsVector(MachineInstr &MI, unsigned TypeIdx,
case TargetOpcode::G_FMAXNUM_IEEE:
case TargetOpcode::G_FMINIMUM:
case TargetOpcode::G_FMAXIMUM:
+ case TargetOpcode::G_FMINIMUMNUM:
+ case TargetOpcode::G_FMAXIMUMNUM:
case TargetOpcode::G_STRICT_FADD:
case TargetOpcode::G_STRICT_FSUB:
case TargetOpcode::G_STRICT_FMUL:
@@ -7247,6 +7266,36 @@ LegalizerHelper::lowerFMinNumMaxNum(MachineInstr &MI) {
return Legalized;
}
+LegalizerHelper::LegalizeResult
+LegalizerHelper::lowerFMinimumNumMaximumNum(MachineInstr &MI) {
+ unsigned NewOp = MI.getOpcode() == TargetOpcode::G_FMINIMUMNUM
+ ? TargetOpcode::G_FMINNUM_IEEE
+ : TargetOpcode::G_FMAXNUM_IEEE;
+
+ auto [Dst, Src0, Src1] = MI.getFirst3Regs();
+ LLT Ty = MRI.getType(Dst);
+
+ if (!MI.getFlag(MachineInstr::FmNoNans)) {
+ // Insert canonicalizes if it's possible we need to quiet to get correct
+ // sNaN behavior.
+
+ // Note this must be done here, and not as an optimization combine in the
+ // absence of a dedicate quiet-snan instruction as we're using an
+ // omni-purpose G_FCANONICALIZE.
+ if (!isKnownNeverSNaN(Src0, MRI))
+ Src0 = MIRBuilder.buildFCanonicalize(Ty, Src0, MI.getFlags()).getReg(0);
+
+ if (!isKnownNeverSNaN(Src1, MRI))
+ Src1 = MIRBuilder.buildFCanonicalize(Ty, Src1, MI.getFlags()).getReg(0);
+ }
+
+ // If there are no nans, it's safe to simply replace this with the non-IEEE
+ // version.
+ MIRBuilder.buildInstr(NewOp, {Dst}, {Src0, Src1}, MI.getFlags());
+ MI.eraseFromParent();
+ return Legalized;
+}
+
LegalizerHelper::LegalizeResult LegalizerHelper::lowerFMad(MachineInstr &MI) {
// Expand G_FMAD a, b, c -> G_FADD (G_FMUL a, b), c
Register DstReg = MI.getOperand(0).getReg();
diff --git a/llvm/lib/CodeGen/GlobalISel/Utils.cpp b/llvm/lib/CodeGen/GlobalISel/Utils.cpp
index 328a146580452..fe74fec2472b1 100644
--- a/llvm/lib/CodeGen/GlobalISel/Utils.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/Utils.cpp
@@ -763,6 +763,10 @@ llvm::ConstantFoldFPBinOp(unsigned Opcode, const Register Op1,
return minimum(C1, C2);
case TargetOpcode::G_FMAXIMUM:
return maximum(C1, C2);
+ case TargetOpcode::G_FMINIMUMNUM:
+ return minimumnum(C1, C2);
+ case TargetOpcode::G_FMAXIMUMNUM:
+ return maximumnum(C1, C2);
case TargetOpcode::G_FMINNUM_IEEE:
case TargetOpcode::G_FMAXNUM_IEEE:
// FIXME: These operations were unfortunately named. fminnum/fmaxnum do not
@@ -853,7 +857,9 @@ bool llvm::isKnownNeverNaN(Register Val, const MachineRegisterInfo &MRI,
isKnownNeverNaN(DefMI->getOperand(2).getReg(), MRI));
}
case TargetOpcode::G_FMINNUM:
- case TargetOpcode::G_FMAXNUM: {
+ case TargetOpcode::G_FMAXNUM:
+ case TargetOpcode::G_FMINIMUMNUM:
+ case TargetOpcode::G_FMAXIMUMNUM: {
// Only one needs to be known not-nan, since it will be returned if the
// other ends up being one.
return isKnownNeverNaN(DefMI->getOperand(1).getReg(), MRI, SNaN) ||
@@ -1700,9 +1706,11 @@ bool llvm::isPreISelGenericFloatingPointOpcode(unsigned Opc) {
case TargetOpcode::G_FMA:
case TargetOpcode::G_FMAD:
case TargetOpcode::G_FMAXIMUM:
+ case TargetOpcode::G_FMAXIMUMNUM:
case TargetOpcode::G_FMAXNUM:
case TargetOpcode::G_FMAXNUM_IEEE:
case TargetOpcode::G_FMINIMUM:
+ case TargetOpcode::G_FMINIMUMNUM:
case TargetOpcode::G_FMINNUM:
case TargetOpcode::G_FMINNUM_IEEE:
case TargetOpcode::G_FMUL:
diff --git a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
index 254d63abdf805..a9eddc9e64af7 100644
--- a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
@@ -1936,7 +1936,9 @@ SDValue DAGCombiner::visit(SDNode *N) {
case ISD::FMINNUM:
case ISD::FMAXNUM:
case ISD::FMINIMUM:
- case ISD::FMAXIMUM: return visitFMinMax(N);
+ case ISD::FMAXIMUM:
+ case ISD::FMINIMUMNUM:
+ case ISD::FMAXIMUMNUM: return visitFMinMax(N);
case ISD::FCEIL: return visitFCEIL(N);
case ISD::FTRUNC: return visitFTRUNC(N);
case ISD::FFREXP: return visitFFREXP(N);
@@ -1979,7 +1981,9 @@ SDValue DAGCombiner::visit(SDNode *N) {
case ISD::VECREDUCE_FMAX:
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAXIMUM:
- case ISD::VECREDUCE_FMINIMUM: return visitVECREDUCE(N);
+ case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM: return visitVECREDUCE(N);
#define BEGIN_REGISTER_VP_SDNODE(SDOPC, ...) case ISD::SDOPC:
#include "llvm/IR/VPIntrinsics.def"
return visitVPOp(N);
@@ -6113,6 +6117,7 @@ static bool arebothOperandsNotNan(SDValue Operand1, SDValue Operand2,
return DAG.isKnownNeverNaN(Operand2) && DAG.isKnownNeverNaN(Operand1);
}
+// FIXME: use FMINIMUMNUM if possible, such as for RISC-V.
static unsigned getMinMaxOpcodeForFP(SDValue Operand1, SDValue Operand2,
ISD::CondCode CC, unsigned OrAndOpcode,
SelectionDAG &DAG,
@@ -18067,7 +18072,8 @@ SDValue DAGCombiner::visitFMinMax(SDNode *N) {
const SDNodeFlags Flags = N->getFlags();
unsigned Opc = N->getOpcode();
bool PropagatesNaN = Opc == ISD::FMINIMUM || Opc == ISD::FMAXIMUM;
- bool IsMin = Opc == ISD::FMINNUM || Opc == ISD::FMINIMUM;
+ bool IsMin =
+ Opc == ISD::FMINNUM || Opc == ISD::FMINIMUM || Opc == ISD::FMINIMUMNUM;
SelectionDAG::FlagInserter FlagsInserter(DAG, N);
// Constant fold.
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
index dfc24f01eb112..954218531a631 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
@@ -1210,6 +1210,8 @@ void SelectionDAGLegalize::LegalizeOp(SDNode *Node) {
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
case ISD::IS_FPCLASS:
Action = TLI.getOperationAction(
Node->getOpcode(), Node->getOperand(0).getValueType());
@@ -1231,6 +1233,8 @@ void SelectionDAGLegalize::LegalizeOp(SDNode *Node) {
case ISD::VP_REDUCE_FMIN:
case ISD::VP_REDUCE_FMAXIMUM:
case ISD::VP_REDUCE_FMINIMUM:
+ case ISD::VP_REDUCE_FMAXIMUMNUM:
+ case ISD::VP_REDUCE_FMINIMUMNUM:
case ISD::VP_REDUCE_SEQ_FADD:
case ISD::VP_REDUCE_SEQ_FMUL:
Action = TLI.getOperationAction(
@@ -3635,6 +3639,12 @@ bool SelectionDAGLegalize::ExpandNode(SDNode *Node) {
Results.push_back(Expanded);
break;
}
+ case ISD::FMINIMUMNUM:
+ case ISD::FMAXIMUMNUM: {
+ if (SDValue Expanded = TLI.expandFMINIMUMNUM_FMAXIMUMNUM(Node, DAG))
+ Results.push_back(Expanded);
+ break;
+ }
case ISD::FSIN:
case ISD::FCOS: {
EVT VT = Node->getValueType(0);
@@ -4310,6 +4320,8 @@ bool SelectionDAGLegalize::ExpandNode(SDNode *Node) {
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
Results.push_back(TLI.expandVecReduce(Node, DAG));
break;
case ISD::VP_CTTZ_ELTS:
@@ -4504,6 +4516,18 @@ void SelectionDAGLegalize::ConvertNodeToLibcall(SDNode *Node) {
RTLIB::FMAX_F80, RTLIB::FMAX_F128,
RTLIB::FMAX_PPCF128, Results);
break;
+ case ISD::FMINIMUMNUM:
+ case ISD::STRICT_FMINIMUMNUM:
+ ExpandFPLibCall(Node, RTLIB::FMINIMUMNUM_F32, RTLIB::FMINIMUMNUM_F64,
+ RTLIB::FMINIMUMNUM_F80, RTLIB::FMINIMUMNUM_F128,
+ RTLIB::FMINIMUMNUM_PPCF128, Results);
+ break;
+ case ISD::FMAXIMUMNUM:
+ case ISD::STRICT_FMAXIMUMNUM:
+ ExpandFPLibCall(Node, RTLIB::FMAXIMUMNUM_F32, RTLIB::FMAXIMUMNUM_F64,
+ RTLIB::FMAXIMUMNUM_F80, RTLIB::FMAXIMUMNUM_F128,
+ RTLIB::FMAXIMUMNUM_PPCF128, Results);
+ break;
case ISD::FSQRT:
case ISD::STRICT_FSQRT:
ExpandFPLibCall(Node, RTLIB::SQRT_F32, RTLIB::SQRT_F64,
@@ -5066,6 +5090,8 @@ void SelectionDAGLegalize::PromoteNode(SDNode *Node) {
Node->getOpcode() == ISD::VP_REDUCE_FMIN ||
Node->getOpcode() == ISD::VP_REDUCE_FMAXIMUM ||
Node->getOpcode() == ISD::VP_REDUCE_FMINIMUM ||
+ Node->getOpcode() == ISD::VP_REDUCE_FMAXIMUMNUM ||
+ Node->getOpcode() == ISD::VP_REDUCE_FMINIMUMNUM ||
Node->getOpcode() == ISD::VP_REDUCE_SEQ_FADD)
OVT = Node->getOperand(1).getSimpleValueType();
if (Node->getOpcode() == ISD::BR_CC ||
@@ -5381,6 +5407,8 @@ void SelectionDAGLegalize::PromoteNode(SDNode *Node) {
case ISD::FMAXNUM:
case ISD::FMINIMUM:
case ISD::FMAXIMUM:
+ case ISD::FMINIMUMNUM:
+ case ISD::FMAXIMUMNUM:
case ISD::FPOW:
Tmp1 = DAG.getNode(ISD::FP_EXTEND, dl, NVT, Node->getOperand(0));
Tmp2 = DAG.getNode(ISD::FP_EXTEND, dl, NVT, Node->getOperand(1));
@@ -5396,6 +5424,8 @@ void SelectionDAGLegalize::PromoteNode(SDNode *Node) {
case ISD::STRICT_FDIV:
case ISD::STRICT_FMINNUM:
case ISD::STRICT_FMAXNUM:
+ case ISD::STRICT_FMINIMUMNUM:
+ case ISD::STRICT_FMAXIMUMNUM:
case ISD::STRICT_FREM:
case ISD::STRICT_FPOW:
Tmp1 = DAG.getNode(ISD::STRICT_FP_EXTEND, dl, {NVT, MVT::Other},
@@ -5744,6 +5774,8 @@ void SelectionDAGLegalize::PromoteNode(SDNode *Node) {
case ISD::VP_REDUCE_FMIN:
case ISD::VP_REDUCE_FMAXIMUM:
case ISD::VP_REDUCE_FMINIMUM:
+ case ISD::VP_REDUCE_FMAXIMUMNUM:
+ case ISD::VP_REDUCE_FMINIMUMNUM:
case ISD::VP_REDUCE_SEQ_FADD:
Results.push_back(PromoteReduction(Node));
break;
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp
index aa116c9de5d8c..53b2c9abce95c 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp
@@ -74,6 +74,10 @@ void DAGTypeLegalizer::SoftenFloatResult(SDNode *N, unsigned ResNo) {
case ISD::FMINNUM: R = SoftenFloatRes_FMINNUM(N); break;
case ISD::STRICT_FMAXNUM:
case ISD::FMAXNUM: R = SoftenFloatRes_FMAXNUM(N); break;
+ case ISD::STRICT_FMINIMUMNUM:
+ case ISD::FMINIMUMNUM: R = SoftenFloatRes_FMINIMUMNUM(N); break;
+ case ISD::STRICT_FMAXIMUMNUM:
+ case ISD::FMAXIMUMNUM: R = SoftenFloatRes_FMAXIMUMNUM(N); break;
case ISD::STRICT_FADD:
case ISD::FADD: R = SoftenFloatRes_FADD(N); break;
case ISD::FCBRT: R = SoftenFloatRes_FCBRT(N); break;
@@ -152,7 +156,9 @@ void DAGTypeLegalizer::SoftenFloatResult(SDNode *N, unsigned ResNo) {
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAX:
case ISD::VECREDUCE_FMAXIMUM:
- case ISD::VECREDUCE_FMINIMUM: R = SoftenFloatRes_VECREDUCE(N); break;
+ case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM: R = SoftenFloatRes_VECREDUCE(N); break;
case ISD::VECREDUCE_SEQ_FADD:
case ISD::VECREDUCE_SEQ_FMUL: R = SoftenFloatRes_VECREDUCE_SEQ(N); break;
// clang-format on
@@ -311,6 +317,24 @@ SDValue DAGTypeLegalizer::SoftenFloatRes_FMAXNUM(SDNode *N) {
RTLIB::FMAX_PPCF128));
}
+SDValue DAGTypeLegalizer::SoftenFloatRes_FMINIMUMNUM(SDNode *N) {
+ if (SDValue SelCC = TLI.createSelectForFMINIMUMNUM_FMAXIMUMNUM(N, DAG))
+ return SoftenFloatRes_SELECT_CC(SelCC.getNode());
+ return SoftenFloatRes_Binary(
+ N, GetFPLibCall(N->getValueType(0), RTLIB::FMINIMUMNUM_F32,
+ RTLIB::FMINIMUMNUM_F64, RTLIB::FMINIMUMNUM_F80,
+ RTLIB::FMINIMUMNUM_F128, RTLIB::FMINIMUMNUM_PPCF128));
+}
+
+SDValue DAGTypeLegalizer::SoftenFloatRes_FMAXIMUMNUM(SDNode *N) {
+ if (SDValue SelCC = TLI.createSelectForFMINIMUMNUM_FMAXIMUMNUM(N, DAG))
+ return SoftenFloatRes_SELECT_CC(SelCC.getNode());
+ return SoftenFloatRes_Binary(
+ N, GetFPLibCall(N->getValueType(0), RTLIB::FMAXIMUMNUM_F32,
+ RTLIB::FMAXIMUMNUM_F64, RTLIB::FMAXIMUMNUM_F80,
+ RTLIB::FMAXIMUMNUM_F128, RTLIB::FMAXIMUMNUM_PPCF128));
+}
+
SDValue DAGTypeLegalizer::SoftenFloatRes_FADD(SDNode *N) {
return SoftenFloatRes_Binary(N, GetFPLibCall(N->getValueType(0),
RTLIB::ADD_F32,
@@ -1356,6 +1380,14 @@ void DAGTypeLegalizer::ExpandFloatResult(SDNode *N, unsigned ResNo) {
case ISD::FMINNUM: ExpandFloatRes_FMINNUM(N, Lo, Hi); break;
case ISD::STRICT_FMAXNUM:
case ISD::FMAXNUM: ExpandFloatRes_FMAXNUM(N, Lo, Hi); break;
+ case ISD::STRICT_FMINIMUMNUM:
+ case ISD::FMINIMUMNUM:
+ ExpandFloatRes_FMINIMUMNUM(N, Lo, Hi);
+ break;
+ case ISD::STRICT_FMAXIMUMNUM:
+ case ISD::FMAXIMUMNUM:
+ ExpandFloatRes_FMAXIMUMNUM(N, Lo, Hi);
+ break;
case ISD::STRICT_FADD:
case ISD::FADD: ExpandFloatRes_FADD(N, Lo, Hi); break;
case ISD::FCBRT: ExpandFloatRes_FCBRT(N, Lo, Hi); break;
@@ -1501,6 +1533,26 @@ void DAGTypeLegalizer::ExpandFloatRes_FMAXNUM(SDNode *N, SDValue &Lo,
RTLIB::FMAX_PPCF128), Lo, Hi);
}
+void DAGTypeLegalizer::ExpandFloatRes_FMINIMUMNUM(SDNode *N, SDValue &Lo,
+ SDValue &Hi) {
+ ExpandFloatRes_Binary(
+ N,
+ GetFPLibCall(N->getValueType(0), RTLIB::FMINIMUMNUM_F32,
+ RTLIB::FMINIMUMNUM_F64, RTLIB::FMINIMUMNUM_F80,
+ RTLIB::FMINIMUMNUM_F128, RTLIB::FMINIMUMNUM_PPCF128),
+ Lo, Hi);
+}
+
+void DAGTypeLegalizer::ExpandFloatRes_FMAXIMUMNUM(SDNode *N, SDValue &Lo,
+ SDValue &Hi) {
+ ExpandFloatRes_Binary(
+ N,
+ GetFPLibCall(N->getValueType(0), RTLIB::FMAXIMUMNUM_F32,
+ RTLIB::FMAXIMUMNUM_F64, RTLIB::FMAXIMUMNUM_F80,
+ RTLIB::FMAXIMUMNUM_F128, RTLIB::FMAXIMUMNUM_PPCF128),
+ Lo, Hi);
+}
+
void DAGTypeLegalizer::ExpandFloatRes_FADD(SDNode *N, SDValue &Lo,
SDValue &Hi) {
ExpandFloatRes_Binary(N, GetFPLibCall(N->getValueType(0),
@@ -2507,6 +2559,8 @@ void DAGTypeLegalizer::PromoteFloatResult(SDNode *N, unsigned ResNo) {
case ISD::FDIV:
case ISD::FMAXIMUM:
case ISD::FMINIMUM:
+ case ISD::FMAXIMUMNUM:
+ case ISD::FMINIMUMNUM:
case ISD::FMAXNUM:
case ISD::FMINNUM:
case ISD::FMAXNUM_IEEE:
@@ -2544,6 +2598,8 @@ void DAGTypeLegalizer::PromoteFloatResult(SDNode *N, unsigned ResNo) {
case ISD::VECREDUCE_FMAX:
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
R = PromoteFloatRes_VECREDUCE(N);
break;
case ISD::VECREDUCE_SEQ_FADD:
@@ -2943,6 +2999,8 @@ void DAGTypeLegalizer::SoftPromoteHalfResult(SDNode *N, unsigned ResNo) {
case ISD::FDIV:
case ISD::FMAXIMUM:
case ISD::FMINIMUM:
+ case ISD::FMAXIMUMNUM:
+ case ISD::FMINIMUMNUM:
case ISD::FMAXNUM:
case ISD::FMINNUM:
case ISD::FMUL:
@@ -2974,6 +3032,8 @@ void DAGTypeLegalizer::SoftPromoteHalfResult(SDNode *N, unsigned ResNo) {
case ISD::VECREDUCE_FMAX:
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
R = SoftPromoteHalfRes_VECREDUCE(N);
break;
case ISD::VECREDUCE_SEQ_FADD:
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h b/llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h
index 85f947efe2c75..a73e8ecf85e60 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h
@@ -560,6 +560,8 @@ class LLVM_LIBRARY_VISIBILITY DAGTypeLegalizer {
SDValue SoftenFloatRes_FABS(SDNode *N);
SDValue SoftenFloatRes_FMINNUM(SDNode *N);
SDValue SoftenFloatRes_FMAXNUM(SDNode *N);
+ SDValue SoftenFloatRes_FMINIMUMNUM(SDNode *N);
+ SDValue SoftenFloatRes_FMAXIMUMNUM(SDNode *N);
SDValue SoftenFloatRes_FADD(SDNode *N);
SDValue SoftenFloatRes_FCBRT(SDNode *N);
SDValue SoftenFloatRes_FCEIL(SDNode *N);
@@ -646,6 +648,8 @@ class LLVM_LIBRARY_VISIBILITY DAGTypeLegalizer {
void ExpandFloatRes_FABS (SDNode *N, SDValue &Lo, SDValue &Hi);
void ExpandFloatRes_FMINNUM (SDNode *N, SDValue &Lo, SDValue &Hi);
void ExpandFloatRes_FMAXNUM (SDNode *N, SDValue &Lo, SDValue &Hi);
+ void ExpandFloatRes_FMINIMUMNUM(SDNode *N, SDValue &Lo, SDValue &Hi);
+ void ExpandFloatRes_FMAXIMUMNUM(SDNode *N, SDValue &Lo, SDValue &Hi);
void ExpandFloatRes_FADD (SDNode *N, SDValue &Lo, SDValue &Hi);
void ExpandFloatRes_FCBRT (SDNode *N, SDValue &Lo, SDValue &Hi);
void ExpandFloatRes_FCEIL (SDNode *N, SDValue &Lo, SDValue &Hi);
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp
index 14b147cc5b01b..764e7091da01f 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp
@@ -397,6 +397,8 @@ SDValue VectorLegalizer::LegalizeOp(SDValue Op) {
case ISD::FMAXNUM_IEEE:
case ISD::FMINIMUM:
case ISD::FMAXIMUM:
+ case ISD::FMINIMUMNUM:
+ case ISD::FMAXIMUMNUM:
case ISD::FCOPYSIGN:
case ISD::FSQRT:
case ISD::FSIN:
@@ -485,6 +487,8 @@ SDValue VectorLegalizer::LegalizeOp(SDValue Op) {
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
Action = TLI.getOperationAction(Node->getOpcode(),
Node->getOperand(0).getValueType());
break;
@@ -1009,6 +1013,10 @@ void VectorLegalizer::Expand(SDNode *Node, SmallVectorImpl<SDValue> &Results) {
case ISD::FMAXIMUM:
Results.push_back(TLI.expandFMINIMUM_FMAXIMUM(Node, DAG));
return;
+ case ISD::FMINIMUMNUM:
+ case ISD::FMAXIMUMNUM:
+ Results.push_back(TLI.expandFMINIMUMNUM_FMAXIMUMNUM(Node, DAG));
+ return;
case ISD::SMIN:
case ISD::SMAX:
case ISD::UMIN:
@@ -1097,6 +1105,8 @@ void VectorLegalizer::Expand(SDNode *Node, SmallVectorImpl<SDValue> &Results) {
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
Results.push_back(TLI.expandVecReduce(Node, DAG));
return;
case ISD::VECREDUCE_SEQ_FADD:
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp
index 532c6306fb3d1..863747e11b30c 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp
@@ -140,6 +140,8 @@ void DAGTypeLegalizer::ScalarizeVectorResult(SDNode *N, unsigned ResNo) {
case ISD::FMAXNUM_IEEE:
case ISD::FMINIMUM:
case ISD::FMAXIMUM:
+ case ISD::FMINIMUMNUM:
+ case ISD::FMAXIMUMNUM:
case ISD::FLDEXP:
case ISD::SMIN:
case ISD::SMAX:
@@ -1231,6 +1233,10 @@ void DAGTypeLegalizer::SplitVectorResult(SDNode *N, unsigned ResNo) {
case ISD::VP_FMINIMUM:
case ISD::FMAXIMUM:
case ISD::VP_FMAXIMUM:
+ case ISD::FMINIMUMNUM:
+ case ISD::VP_FMINIMUMNUM:
+ case ISD::FMAXIMUMNUM:
+ case ISD::VP_FMAXIMUMNUM:
case ISD::SDIV: case ISD::VP_SDIV:
case ISD::UDIV: case ISD::VP_UDIV:
case ISD::FDIV: case ISD::VP_FDIV:
@@ -3191,6 +3197,8 @@ bool DAGTypeLegalizer::SplitVectorOperand(SDNode *N, unsigned OpNo) {
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
Res = SplitVecOp_VECREDUCE(N, OpNo);
break;
case ISD::VECREDUCE_SEQ_FADD:
@@ -3214,6 +3222,8 @@ bool DAGTypeLegalizer::SplitVectorOperand(SDNode *N, unsigned OpNo) {
case ISD::VP_REDUCE_FMIN:
case ISD::VP_REDUCE_FMAXIMUM:
case ISD::VP_REDUCE_FMINIMUM:
+ case ISD::VP_REDUCE_FMAXIMUMNUM:
+ case ISD::VP_REDUCE_FMINIMUMNUM:
Res = SplitVecOp_VP_REDUCE(N, OpNo);
break;
case ISD::VP_CTTZ_ELTS:
@@ -4340,6 +4350,10 @@ void DAGTypeLegalizer::WidenVectorResult(SDNode *N, unsigned ResNo) {
case ISD::VP_FMINIMUM:
case ISD::FMAXIMUM:
case ISD::VP_FMAXIMUM:
+ case ISD::FMINIMUMNUM:
+ case ISD::VP_FMINIMUMNUM:
+ case ISD::FMAXIMUMNUM:
+ case ISD::VP_FMAXIMUMNUM:
case ISD::SMIN: case ISD::VP_SMIN:
case ISD::SMAX: case ISD::VP_SMAX:
case ISD::UMIN: case ISD::VP_UMIN:
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
index 8463e94d7f933..22f69f2a72f3a 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
@@ -474,6 +474,12 @@ ISD::NodeType ISD::getVecReduceBaseOpcode(unsigned VecReduceOpcode) {
case ISD::VECREDUCE_FMINIMUM:
case ISD::VP_REDUCE_FMINIMUM:
return ISD::FMINIMUM;
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VP_REDUCE_FMAXIMUMNUM:
+ return ISD::FMAXIMUMNUM;
+ case ISD::VECREDUCE_FMINIMUMNUM:
+ case ISD::VP_REDUCE_FMINIMUMNUM:
+ return ISD::FMINIMUMNUM;
}
}
@@ -5429,7 +5435,9 @@ bool SelectionDAG::isKnownNeverNaN(SDValue Op, bool SNaN, unsigned Depth) const
return false;
}
case ISD::FMINNUM:
- case ISD::FMAXNUM: {
+ case ISD::FMAXNUM:
+ case ISD::FMINIMUMNUM:
+ case ISD::FMAXIMUMNUM: {
// Only one needs to be known not-nan, since it will be returned if the
// other ends up being one.
return isKnownNeverNaN(Op.getOperand(0), SNaN, Depth + 1) ||
@@ -6760,6 +6768,10 @@ SDValue SelectionDAG::foldConstantFPMath(unsigned Opcode, const SDLoc &DL,
return getConstantFP(minimum(C1, C2), DL, VT);
case ISD::FMAXIMUM:
return getConstantFP(maximum(C1, C2), DL, VT);
+ case ISD::FMINIMUMNUM:
+ return getConstantFP(minimumnum(C1, C2), DL, VT);
+ case ISD::FMAXIMUMNUM:
+ return getConstantFP(maximumnum(C1, C2), DL, VT);
default: break;
}
}
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index 296b06187ec0f..6e771c4ab220a 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -6841,6 +6841,18 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
getValue(I.getArgOperand(0)),
getValue(I.getArgOperand(1)), Flags));
return;
+ case Intrinsic::minimumnum:
+ setValue(&I, DAG.getNode(ISD::FMINIMUMNUM, sdl,
+ getValue(I.getArgOperand(0)).getValueType(),
+ getValue(I.getArgOperand(0)),
+ getValue(I.getArgOperand(1)), Flags));
+ return;
+ case Intrinsic::maximumnum:
+ setValue(&I, DAG.getNode(ISD::FMAXIMUMNUM, sdl,
+ getValue(I.getArgOperand(0)).getValueType(),
+ getValue(I.getArgOperand(0)),
+ getValue(I.getArgOperand(1)), Flags));
+ return;
case Intrinsic::copysign:
setValue(&I, DAG.getNode(ISD::FCOPYSIGN, sdl,
getValue(I.getArgOperand(0)).getValueType(),
@@ -9182,6 +9194,18 @@ void SelectionDAGBuilder::visitCall(const CallInst &I) {
if (visitBinaryFloatCall(I, ISD::FMAXNUM))
return;
break;
+ case LibFunc_fminimum_num:
+ case LibFunc_fminimum_numf:
+ case LibFunc_fminimum_numl:
+ if (visitBinaryFloatCall(I, ISD::FMINIMUMNUM))
+ return;
+ break;
+ case LibFunc_fmaximum_num:
+ case LibFunc_fmaximum_numf:
+ case LibFunc_fmaximum_numl:
+ if (visitBinaryFloatCall(I, ISD::FMAXIMUMNUM))
+ return;
+ break;
case LibFunc_sin:
case LibFunc_sinf:
case LibFunc_sinl:
@@ -10644,6 +10668,12 @@ void SelectionDAGBuilder::visitVectorReduce(const CallInst &I,
case Intrinsic::vector_reduce_fminimum:
Res = DAG.getNode(ISD::VECREDUCE_FMINIMUM, dl, VT, Op1, SDFlags);
break;
+ case Intrinsic::vector_reduce_fmaximumnum:
+ Res = DAG.getNode(ISD::VECREDUCE_FMAXIMUMNUM, dl, VT, Op1, SDFlags);
+ break;
+ case Intrinsic::vector_reduce_fminimumnum:
+ Res = DAG.getNode(ISD::VECREDUCE_FMINIMUMNUM, dl, VT, Op1, SDFlags);
+ break;
default:
llvm_unreachable("Unhandled vector reduce intrinsic");
}
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp
index a7555d6d31f26..7b25abb5f1ed5 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp
@@ -203,6 +203,10 @@ std::string SDNode::getOperationName(const SelectionDAG *G) const {
case ISD::STRICT_FMINIMUM: return "strict_fminimum";
case ISD::FMAXIMUM: return "fmaximum";
case ISD::STRICT_FMAXIMUM: return "strict_fmaximum";
+ case ISD::FMINIMUMNUM: return "fminimumnum";
+ case ISD::STRICT_FMINIMUMNUM: return "strict_fminimumnum";
+ case ISD::FMAXIMUMNUM: return "fmaximumnum";
+ case ISD::STRICT_FMAXIMUMNUM: return "strict_fmaximumnum";
case ISD::FNEG: return "fneg";
case ISD::FSQRT: return "fsqrt";
case ISD::STRICT_FSQRT: return "strict_fsqrt";
@@ -530,6 +534,10 @@ std::string SDNode::getOperationName(const SelectionDAG *G) const {
case ISD::VECREDUCE_FMIN: return "vecreduce_fmin";
case ISD::VECREDUCE_FMAXIMUM: return "vecreduce_fmaximum";
case ISD::VECREDUCE_FMINIMUM: return "vecreduce_fminimum";
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ return "vecreduce_fmaximumnum";
+ case ISD::VECREDUCE_FMINIMUMNUM:
+ return "vecreduce_fminimumnum";
case ISD::STACKMAP:
return "stackmap";
case ISD::PATCHPOINT:
diff --git a/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp b/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp
index ad957aaa8f141..0165a656509b6 100644
--- a/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp
@@ -8469,6 +8469,83 @@ SDValue TargetLowering::expandFMINIMUM_FMAXIMUM(SDNode *N,
return MinMax;
}
+SDValue TargetLowering::createSelectForFMINIMUMNUM_FMAXIMUMNUM(
+ SDNode *Node, SelectionDAG &DAG) const {
+ unsigned Opcode = Node->getOpcode();
+ assert((Opcode == ISD::FMINIMUMNUM || Opcode == ISD::FMAXIMUMNUM ||
+ Opcode == ISD::STRICT_FMINIMUMNUM ||
+ Opcode == ISD::STRICT_FMAXIMUMNUM) &&
+ "Wrong opcode");
+
+ if (Node->getFlags().hasNoNaNs()) {
+ ISD::CondCode Pred = Opcode == ISD::FMINIMUMNUM ? ISD::SETLT : ISD::SETGT;
+ SDValue Op1 = Node->getOperand(0);
+ SDValue Op2 = Node->getOperand(1);
+ SDValue SelCC = DAG.getSelectCC(SDLoc(Node), Op1, Op2, Op1, Op2, Pred);
+ // Copy FMF flags, but always set the no-signed-zeros flag
+ // as this is implied by the FMINNUM/FMAXNUM semantics.
+ SDNodeFlags Flags = Node->getFlags();
+ Flags.setNoSignedZeros(true);
+ SelCC->setFlags(Flags);
+ return SelCC;
+ }
+
+ return SDValue();
+}
+
+SDValue TargetLowering::expandFMINIMUMNUM_FMAXIMUMNUM(SDNode *Node,
+ SelectionDAG &DAG) const {
+ SDLoc dl(Node);
+ unsigned NewOp = Node->getOpcode() == ISD::FMINIMUMNUM ? ISD::FMINNUM_IEEE
+ : ISD::FMAXNUM_IEEE;
+ EVT VT = Node->getValueType(0);
+
+ if (VT.isScalableVector())
+ report_fatal_error(
+ "Expanding fminimumnum/fmaximumnum for scalable vectors is undefined.");
+
+ if (isOperationLegalOrCustom(NewOp, VT)) {
+ SDValue Quiet0 = Node->getOperand(0);
+ SDValue Quiet1 = Node->getOperand(1);
+
+ if (!Node->getFlags().hasNoNaNs()) {
+ // Insert canonicalizes if it's possible we need to quiet to get correct
+ // sNaN behavior.
+ if (!DAG.isKnownNeverSNaN(Quiet0)) {
+ Quiet0 =
+ DAG.getNode(ISD::FCANONICALIZE, dl, VT, Quiet0, Node->getFlags());
+ }
+ if (!DAG.isKnownNeverSNaN(Quiet1)) {
+ Quiet1 =
+ DAG.getNode(ISD::FCANONICALIZE, dl, VT, Quiet1, Node->getFlags());
+ }
+ }
+
+ return DAG.getNode(NewOp, dl, VT, Quiet0, Quiet1, Node->getFlags());
+ }
+
+ // If the target has FMINIMUM/FMAXIMUM but not FMINNUM/FMAXNUM use that
+ // instead if there are no NaNs and there can't be an incompatible zero
+ // compare: at least one operand isn't +/-0, or there are no signed-zeros.
+ if ((Node->getFlags().hasNoNaNs() ||
+ (DAG.isKnownNeverNaN(Node->getOperand(0)) &&
+ DAG.isKnownNeverNaN(Node->getOperand(1)))) &&
+ (Node->getFlags().hasNoSignedZeros() ||
+ DAG.isKnownNeverZeroFloat(Node->getOperand(0)) ||
+ DAG.isKnownNeverZeroFloat(Node->getOperand(1)))) {
+ unsigned IEEE2018Op =
+ Node->getOpcode() == ISD::FMINIMUMNUM ? ISD::FMINIMUM : ISD::FMAXIMUM;
+ if (isOperationLegalOrCustom(IEEE2018Op, VT))
+ return DAG.getNode(IEEE2018Op, dl, VT, Node->getOperand(0),
+ Node->getOperand(1), Node->getFlags());
+ }
+
+ if (SDValue SelCC = createSelectForFMINIMUMNUM_FMAXIMUMNUM(Node, DAG))
+ return SelCC;
+
+ return SDValue();
+}
+
/// Returns a true value if if this FPClassTest can be performed with an ordered
/// fcmp to 0, and a false value if it's an unordered fcmp to 0. Returns
/// std::nullopt if it cannot be performed as a compare with 0.
@@ -11083,13 +11160,32 @@ SDValue TargetLowering::expandFP_TO_INT_SAT(SDNode *Node,
// of comparisons and selects.
bool MinMaxLegal = isOperationLegal(ISD::FMINNUM, SrcVT) &&
isOperationLegal(ISD::FMAXNUM, SrcVT);
- if (AreExactFloatBounds && MinMaxLegal) {
+ bool MinMaxIEEELegal = isOperationLegal(ISD::FMINNUM_IEEE, SrcVT) &&
+ isOperationLegal(ISD::FMAXNUM_IEEE, SrcVT);
+ bool MinMaxNumLegal = isOperationLegal(ISD::FMINIMUMNUM, SrcVT) &&
+ isOperationLegal(ISD::FMAXIMUMNUM, SrcVT);
+ bool CanonLegal = isOperationLegal(ISD::FCANONICALIZE, SrcVT);
+ if (AreExactFloatBounds &&
+ (MinMaxNumLegal || ((MinMaxIEEELegal || MinMaxLegal) && CanonLegal))) {
SDValue Clamped = Src;
- // Clamp Src by MinFloat from below. If Src is NaN the result is MinFloat.
- Clamped = DAG.getNode(ISD::FMAXNUM, dl, SrcVT, Clamped, MinFloatNode);
- // Clamp by MaxFloat from above. NaN cannot occur.
- Clamped = DAG.getNode(ISD::FMINNUM, dl, SrcVT, Clamped, MaxFloatNode);
+ if (MinMaxNumLegal) {
+ Clamped = DAG.getNode(ISD::FMAXIMUMNUM, dl, SrcVT, Clamped, MinFloatNode);
+ Clamped = DAG.getNode(ISD::FMINIMUMNUM, dl, SrcVT, Clamped, MaxFloatNode);
+ } else if (MinMaxIEEELegal && CanonLegal) {
+ Clamped = DAG.getNode(ISD::FCANONICALIZE, dl, SrcVT, Clamped);
+ Clamped =
+ DAG.getNode(ISD::FMAXNUM_IEEE, dl, SrcVT, Clamped, MinFloatNode);
+ Clamped =
+ DAG.getNode(ISD::FMINNUM_IEEE, dl, SrcVT, Clamped, MaxFloatNode);
+ } else if (MinMaxLegal && CanonLegal) {
+ // Clamp Src by MinFloat from below. If Src is NaN the result is MinFloat.
+ Clamped = DAG.getNode(ISD::FMAXNUM, dl, SrcVT, Clamped, MinFloatNode);
+ // Clamp by MaxFloat from above. NaN cannot occur.
+ Clamped = DAG.getNode(ISD::FMINNUM, dl, SrcVT, Clamped, MaxFloatNode);
+ } else {
+ llvm_unreachable("No usable fmax/fmin instructions!");
+ }
// Convert clamped value to integer.
SDValue FpToInt = DAG.getNode(IsSigned ? ISD::FP_TO_SINT : ISD::FP_TO_UINT,
dl, DstVT, Clamped);
diff --git a/llvm/lib/CodeGen/TargetLoweringBase.cpp b/llvm/lib/CodeGen/TargetLoweringBase.cpp
index ff684c7cb6bba..be28f72a79326 100644
--- a/llvm/lib/CodeGen/TargetLoweringBase.cpp
+++ b/llvm/lib/CodeGen/TargetLoweringBase.cpp
@@ -909,6 +909,7 @@ void TargetLoweringBase::initActions() {
ISD::FMINNUM, ISD::FMAXNUM,
ISD::FMINNUM_IEEE, ISD::FMAXNUM_IEEE,
ISD::FMINIMUM, ISD::FMAXIMUM,
+ ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM,
ISD::FMAD, ISD::SMIN,
ISD::SMAX, ISD::UMIN,
ISD::UMAX, ISD::ABS,
diff --git a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
index 7f821715c910d..983fee397e441 100644
--- a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
+++ b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
@@ -856,6 +856,7 @@ AArch64TargetLowering::AArch64TargetLowering(const TargetMachine &TM,
{ISD::FFLOOR, ISD::FNEARBYINT, ISD::FCEIL,
ISD::FRINT, ISD::FTRUNC, ISD::FROUND,
ISD::FROUNDEVEN, ISD::FMINNUM, ISD::FMAXNUM,
+ ISD::FMINNUM_IEEE, ISD::FMAXNUM_IEEE,
ISD::FMINIMUM, ISD::FMAXIMUM, ISD::LROUND,
ISD::LLROUND, ISD::LRINT, ISD::LLRINT,
ISD::STRICT_FFLOOR, ISD::STRICT_FCEIL, ISD::STRICT_FNEARBYINT,
@@ -1185,6 +1186,7 @@ AArch64TargetLowering::AArch64TargetLowering(const TargetMachine &TM,
ISD::FEXP10, ISD::FRINT, ISD::FROUND,
ISD::FROUNDEVEN, ISD::FTRUNC, ISD::FMINNUM,
ISD::FMAXNUM, ISD::FMINIMUM, ISD::FMAXIMUM,
+ ISD::FMINNUM_IEEE, ISD::FMAXNUM_IEEE,
ISD::STRICT_FADD, ISD::STRICT_FSUB, ISD::STRICT_FMUL,
ISD::STRICT_FDIV, ISD::STRICT_FMA, ISD::STRICT_FCEIL,
ISD::STRICT_FFLOOR, ISD::STRICT_FSQRT, ISD::STRICT_FRINT,
@@ -1876,6 +1878,7 @@ void AArch64TargetLowering::addTypeForNEON(MVT VT) {
(VT.getVectorElementType() != MVT::f16 || Subtarget->hasFullFP16()))
for (unsigned Opcode :
{ISD::FMINIMUM, ISD::FMAXIMUM, ISD::FMINNUM, ISD::FMAXNUM,
+ ISD::FMINNUM_IEEE, ISD::FMAXNUM_IEEE,
ISD::STRICT_FMINIMUM, ISD::STRICT_FMAXIMUM, ISD::STRICT_FMINNUM,
ISD::STRICT_FMAXNUM, ISD::STRICT_FADD, ISD::STRICT_FSUB,
ISD::STRICT_FMUL, ISD::STRICT_FDIV, ISD::STRICT_FMA,
diff --git a/llvm/lib/Target/AArch64/AArch64InstrInfo.td b/llvm/lib/Target/AArch64/AArch64InstrInfo.td
index 6afee9bd388a6..9e5f2a75ba522 100644
--- a/llvm/lib/Target/AArch64/AArch64InstrInfo.td
+++ b/llvm/lib/Target/AArch64/AArch64InstrInfo.td
@@ -4916,6 +4916,18 @@ def : Pat<(v1f64 (fmaxnum (v1f64 FPR64:$Rn), (v1f64 FPR64:$Rm))),
(FMAXNMDrr FPR64:$Rn, FPR64:$Rm)>;
def : Pat<(v1f64 (fminnum (v1f64 FPR64:$Rn), (v1f64 FPR64:$Rm))),
(FMINNMDrr FPR64:$Rn, FPR64:$Rm)>;
+def : Pat<(fminnum_ieee (f64 FPR64:$a), (f64 FPR64:$b)),
+ (FMINNMDrr FPR64:$a, FPR64:$b)>;
+def : Pat<(fminnum_ieee (f32 FPR32:$a), (f32 FPR32:$b)),
+ (FMINNMSrr FPR32:$a, FPR32:$b)>;
+def : Pat<(fmaxnum_ieee (f64 FPR64:$a), (f64 FPR64:$b)),
+ (FMAXNMDrr FPR64:$a, FPR64:$b)>;
+def : Pat<(fmaxnum_ieee (f32 FPR32:$a), (f32 FPR32:$b)),
+ (FMAXNMSrr FPR32:$a, FPR32:$b)>;
+def : Pat<(f32 (fcanonicalize f32:$a)),
+ (FMINSrr f32:$a, f32:$a)>;
+def : Pat<(f64 (fcanonicalize f64:$a)),
+ (FMINDrr f64:$a, f64:$a)>;
//===----------------------------------------------------------------------===//
// Floating point three operand instructions.
diff --git a/llvm/lib/Target/Hexagon/HexagonISelLowering.cpp b/llvm/lib/Target/Hexagon/HexagonISelLowering.cpp
index 22a88734afd44..3810d71fd40c9 100644
--- a/llvm/lib/Target/Hexagon/HexagonISelLowering.cpp
+++ b/llvm/lib/Target/Hexagon/HexagonISelLowering.cpp
@@ -1656,6 +1656,7 @@ HexagonTargetLowering::HexagonTargetLowering(const TargetMachine &TM,
ISD::FLOG10, ISD::FEXP, ISD::FEXP2, ISD::FCEIL, ISD::FTRUNC,
ISD::FRINT, ISD::FNEARBYINT, ISD::FROUND, ISD::FFLOOR,
ISD::FMINNUM, ISD::FMAXNUM, ISD::FSINCOS, ISD::FLDEXP,
+ ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM,
// Misc:
ISD::BR_CC, ISD::SELECT_CC, ISD::ConstantPool,
// Vector:
@@ -1789,6 +1790,8 @@ HexagonTargetLowering::HexagonTargetLowering(const TargetMachine &TM,
setOperationAction(ISD::FMINNUM, MVT::f32, Legal);
setOperationAction(ISD::FMAXNUM, MVT::f32, Legal);
+ setOperationAction(ISD::FMINIMUMNUM, MVT::f32, Legal);
+ setOperationAction(ISD::FMAXIMUMNUM, MVT::f32, Legal);
setOperationAction(ISD::FP_TO_UINT, MVT::i1, Promote);
setOperationAction(ISD::FP_TO_UINT, MVT::i8, Promote);
@@ -1838,6 +1841,8 @@ HexagonTargetLowering::HexagonTargetLowering(const TargetMachine &TM,
if (Subtarget.hasV67Ops()) {
setOperationAction(ISD::FMINNUM, MVT::f64, Legal);
setOperationAction(ISD::FMAXNUM, MVT::f64, Legal);
+ setOperationAction(ISD::FMINIMUMNUM, MVT::f64, Legal);
+ setOperationAction(ISD::FMAXIMUMNUM, MVT::f64, Legal);
setOperationAction(ISD::FMUL, MVT::f64, Legal);
}
diff --git a/llvm/lib/Target/Hexagon/HexagonISelLoweringHVX.cpp b/llvm/lib/Target/Hexagon/HexagonISelLoweringHVX.cpp
index 81035849491bc..939974b9e905e 100644
--- a/llvm/lib/Target/Hexagon/HexagonISelLoweringHVX.cpp
+++ b/llvm/lib/Target/Hexagon/HexagonISelLoweringHVX.cpp
@@ -129,6 +129,8 @@ HexagonTargetLowering::initializeHVXLowering() {
setOperationAction(ISD::FMUL, T, Legal);
setOperationAction(ISD::FMINNUM, T, Legal);
setOperationAction(ISD::FMAXNUM, T, Legal);
+ setOperationAction(ISD::FMINIMUMNUM, T, Legal);
+ setOperationAction(ISD::FMAXIMUMNUM, T, Legal);
setOperationAction(ISD::INSERT_SUBVECTOR, T, Custom);
setOperationAction(ISD::EXTRACT_SUBVECTOR, T, Custom);
@@ -166,6 +168,8 @@ HexagonTargetLowering::initializeHVXLowering() {
setOperationAction(ISD::FMUL, P, Custom);
setOperationAction(ISD::FMINNUM, P, Custom);
setOperationAction(ISD::FMAXNUM, P, Custom);
+ setOperationAction(ISD::FMINIMUMNUM, P, Custom);
+ setOperationAction(ISD::FMAXIMUMNUM, P, Custom);
setOperationAction(ISD::SETCC, P, Custom);
setOperationAction(ISD::VSELECT, P, Custom);
@@ -3168,6 +3172,8 @@ HexagonTargetLowering::LowerHvxOperation(SDValue Op, SelectionDAG &DAG) const {
case ISD::FMUL:
case ISD::FMINNUM:
case ISD::FMAXNUM:
+ case ISD::FMINIMUMNUM:
+ case ISD::FMAXIMUMNUM:
case ISD::MULHS:
case ISD::MULHU:
case ISD::AND:
diff --git a/llvm/lib/Target/Hexagon/HexagonPatterns.td b/llvm/lib/Target/Hexagon/HexagonPatterns.td
index baa552fcd220d..08f6e93b5c1f0 100644
--- a/llvm/lib/Target/Hexagon/HexagonPatterns.td
+++ b/llvm/lib/Target/Hexagon/HexagonPatterns.td
@@ -1563,6 +1563,8 @@ def: OpR_RR_pat<F2_sfsub, pf2<fsub>, f32, F32>;
def: OpR_RR_pat<F2_sfmpy, pf2<fmul>, f32, F32>;
def: OpR_RR_pat<F2_sfmin, pf2<fminnum>, f32, F32>;
def: OpR_RR_pat<F2_sfmax, pf2<fmaxnum>, f32, F32>;
+def: OpR_RR_pat<F2_sfmin, pf2<fminimumnum>, f32, F32>;
+def: OpR_RR_pat<F2_sfmax, pf2<fmaximumnum>, f32, F32>;
let Predicates = [HasV66] in {
def: OpR_RR_pat<F2_dfadd, pf2<fadd>, f64, F64>;
@@ -1584,6 +1586,8 @@ let Predicates = [HasV67,UseUnsafeMath], AddedComplexity = 50 in {
let Predicates = [HasV67] in {
def: OpR_RR_pat<F2_dfmin, pf2<fminnum>, f64, F64>;
def: OpR_RR_pat<F2_dfmax, pf2<fmaxnum>, f64, F64>;
+ def: OpR_RR_pat<F2_dfmin, pf2<fminimumnum>, f64, F64>;
+ def: OpR_RR_pat<F2_dfmax, pf2<fmaximumnum>, f64, F64>;
def: Pat<(fmul F64:$Rs, F64:$Rt), (DfMpy (F2_dfmpyfix $Rs, $Rt),
(F2_dfmpyfix $Rt, $Rs))>;
diff --git a/llvm/lib/Target/Hexagon/HexagonPatternsHVX.td b/llvm/lib/Target/Hexagon/HexagonPatternsHVX.td
index 5b8386416a5f0..32e7c65bf4db4 100644
--- a/llvm/lib/Target/Hexagon/HexagonPatternsHVX.td
+++ b/llvm/lib/Target/Hexagon/HexagonPatternsHVX.td
@@ -512,6 +512,10 @@ let Predicates = [UseHVXV68, UseHVX128B, UseHVXQFloat] in {
def: OpR_RR_pat<V6_vmax_hf, pf2<fmaxnum>, VecF16, HVF16>;
def: OpR_RR_pat<V6_vmin_sf, pf2<fminnum>, VecF32, HVF32>;
def: OpR_RR_pat<V6_vmax_sf, pf2<fmaxnum>, VecF32, HVF32>;
+ def: OpR_RR_pat<V6_vmin_hf, pf2<fminimumnum>, VecF16, HVF16>;
+ def: OpR_RR_pat<V6_vmax_hf, pf2<fmaximumnum>, VecF16, HVF16>;
+ def: OpR_RR_pat<V6_vmin_sf, pf2<fminimumnum>, VecF32, HVF32>;
+ def: OpR_RR_pat<V6_vmax_sf, pf2<fmaximumnum>, VecF32, HVF32>;
}
let Predicates = [UseHVXV68, UseHVX128B, UseHVXIEEEFP] in {
@@ -525,6 +529,10 @@ let Predicates = [UseHVXV68, UseHVX128B, UseHVXIEEEFP] in {
def: OpR_RR_pat<V6_vfmax_hf, pf2<fmaxnum>, VecF16, HVF16>;
def: OpR_RR_pat<V6_vfmin_sf, pf2<fminnum>, VecF32, HVF32>;
def: OpR_RR_pat<V6_vfmax_sf, pf2<fmaxnum>, VecF32, HVF32>;
+ def: OpR_RR_pat<V6_vfmin_hf, pf2<fminimumnum>, VecF16, HVF16>;
+ def: OpR_RR_pat<V6_vfmax_hf, pf2<fmaximumnum>, VecF16, HVF16>;
+ def: OpR_RR_pat<V6_vfmin_sf, pf2<fminimumnum>, VecF32, HVF32>;
+ def: OpR_RR_pat<V6_vfmax_sf, pf2<fmaximumnum>, VecF32, HVF32>;
}
let Predicates = [UseHVX] in {
diff --git a/llvm/lib/Target/RISCV/GISel/RISCVRegisterBankInfo.cpp b/llvm/lib/Target/RISCV/GISel/RISCVRegisterBankInfo.cpp
index 686c8d89a7321..4ad5808fb075f 100644
--- a/llvm/lib/Target/RISCV/GISel/RISCVRegisterBankInfo.cpp
+++ b/llvm/lib/Target/RISCV/GISel/RISCVRegisterBankInfo.cpp
@@ -281,7 +281,9 @@ RISCVRegisterBankInfo::getInstrMapping(const MachineInstr &MI) const {
case TargetOpcode::G_FNEG:
case TargetOpcode::G_FSQRT:
case TargetOpcode::G_FMAXNUM:
- case TargetOpcode::G_FMINNUM: {
+ case TargetOpcode::G_FMINNUM:
+ case TargetOpcode::G_FMAXIMUMNUM:
+ case TargetOpcode::G_FMINIMUMNUM: {
LLT Ty = MRI.getType(MI.getOperand(0).getReg());
TypeSize Size = Ty.getSizeInBits();
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
index a02fd5bd1b65e..92cdc22c1a10c 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
@@ -436,7 +436,8 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
ISD::STRICT_LRINT, ISD::STRICT_LLRINT, ISD::STRICT_LROUND,
ISD::STRICT_LLROUND, ISD::STRICT_FMA, ISD::STRICT_FADD,
ISD::STRICT_FSUB, ISD::STRICT_FMUL, ISD::STRICT_FDIV,
- ISD::STRICT_FSQRT, ISD::STRICT_FSETCC, ISD::STRICT_FSETCCS};
+ ISD::STRICT_FSQRT, ISD::STRICT_FSETCC, ISD::STRICT_FSETCCS,
+ ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM};
static const ISD::CondCode FPCCToExpand[] = {
ISD::SETOGT, ISD::SETOGE, ISD::SETONE, ISD::SETUEQ, ISD::SETUGT,
@@ -463,7 +464,8 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
ISD::STRICT_FSQRT, ISD::STRICT_FSETCC, ISD::STRICT_FSETCCS,
ISD::SETCC, ISD::FCEIL, ISD::FFLOOR,
ISD::FTRUNC, ISD::FRINT, ISD::FROUND,
- ISD::FROUNDEVEN, ISD::SELECT};
+ ISD::FROUNDEVEN, ISD::SELECT, ISD::FMINIMUMNUM,
+ ISD::FMAXIMUMNUM};
if (Subtarget.hasStdExtZfbfmin()) {
setOperationAction(ISD::BITCAST, MVT::i16, Custom);
@@ -717,6 +719,7 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
ISD::VP_FROUNDEVEN, ISD::VP_FCOPYSIGN, ISD::VP_FROUNDTOZERO,
ISD::VP_FRINT, ISD::VP_FNEARBYINT, ISD::VP_IS_FPCLASS,
ISD::VP_FMINIMUM, ISD::VP_FMAXIMUM, ISD::VP_LRINT,
+ ISD::VP_FMINIMUMNUM, ISD::VP_FMAXIMUMNUM,
ISD::VP_LLRINT, ISD::EXPERIMENTAL_VP_REVERSE,
ISD::EXPERIMENTAL_VP_SPLICE, ISD::VP_REDUCE_FMINIMUM,
ISD::VP_REDUCE_FMAXIMUM};
@@ -952,7 +955,8 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
ISD::FABS, ISD::FNEG, ISD::FCOPYSIGN, ISD::FCEIL,
ISD::FFLOOR, ISD::FROUND, ISD::FROUNDEVEN, ISD::FRINT,
ISD::FNEARBYINT, ISD::IS_FPCLASS, ISD::SETCC, ISD::FMAXIMUM,
- ISD::FMINIMUM, ISD::STRICT_FADD, ISD::STRICT_FSUB, ISD::STRICT_FMUL,
+ ISD::FMINIMUM, ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM,
+ ISD::STRICT_FADD, ISD::STRICT_FSUB, ISD::STRICT_FMUL,
ISD::STRICT_FDIV, ISD::STRICT_FSQRT, ISD::STRICT_FMA};
// TODO: support more vp ops.
@@ -965,7 +969,8 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
ISD::VP_FFLOOR, ISD::VP_FROUND, ISD::VP_FROUNDEVEN,
ISD::VP_FCOPYSIGN, ISD::VP_FROUNDTOZERO, ISD::VP_FRINT,
ISD::VP_FNEARBYINT, ISD::VP_SETCC, ISD::VP_FMINIMUM,
- ISD::VP_FMAXIMUM, ISD::VP_REDUCE_FMINIMUM, ISD::VP_REDUCE_FMAXIMUM};
+ ISD::VP_FMAXIMUM, ISD::VP_REDUCE_FMINIMUM, ISD::VP_REDUCE_FMAXIMUM,
+ ISD::VP_FMINIMUMNUM, ISD::VP_FMAXIMUMNUM};
// Sets common operation actions on RVV floating-point vector types.
const auto SetCommonVFPActions = [&](MVT VT) {
@@ -983,6 +988,7 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
setCondCodeAction(VFPCCToExpand, VT, Expand);
setOperationAction({ISD::FMINNUM, ISD::FMAXNUM}, VT, Legal);
+ setOperationAction({ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM}, VT, Legal);
setOperationAction({ISD::FMAXIMUM, ISD::FMINIMUM}, VT, Custom);
setOperationAction({ISD::FTRUNC, ISD::FCEIL, ISD::FFLOOR, ISD::FROUND,
@@ -1372,7 +1378,8 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
setOperationAction({ISD::FADD, ISD::FSUB, ISD::FMUL, ISD::FDIV,
ISD::FNEG, ISD::FABS, ISD::FCOPYSIGN, ISD::FSQRT,
ISD::FMA, ISD::FMINNUM, ISD::FMAXNUM,
- ISD::IS_FPCLASS, ISD::FMAXIMUM, ISD::FMINIMUM},
+ ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM, ISD::IS_FPCLASS,
+ ISD::FMAXIMUM, ISD::FMINIMUM},
VT, Custom);
setOperationAction({ISD::FP_ROUND, ISD::FP_EXTEND}, VT, Custom);
@@ -1472,7 +1479,8 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
setTargetDAGCombine(ISD::SRA);
if (Subtarget.hasStdExtFOrZfinx())
- setTargetDAGCombine({ISD::FADD, ISD::FMAXNUM, ISD::FMINNUM});
+ setTargetDAGCombine({ISD::FADD, ISD::FMAXNUM, ISD::FMINNUM,
+ ISD::FMAXIMUMNUM, ISD::FMINIMUMNUM});
if (Subtarget.hasStdExtZbb())
setTargetDAGCombine({ISD::UMAX, ISD::UMIN, ISD::SMAX, ISD::SMIN});
@@ -5964,9 +5972,11 @@ static unsigned getRISCVVLOp(SDValue Op) {
case ISD::VP_FP_TO_UINT:
return RISCVISD::VFCVT_RTZ_XU_F_VL;
case ISD::FMINNUM:
+ case ISD::FMINIMUMNUM:
case ISD::VP_FMINNUM:
return RISCVISD::VFMIN_VL;
case ISD::FMAXNUM:
+ case ISD::FMAXIMUMNUM:
case ISD::VP_FMAXNUM:
return RISCVISD::VFMAX_VL;
case ISD::LRINT:
@@ -6911,6 +6921,8 @@ SDValue RISCVTargetLowering::LowerOperation(SDValue Op,
case ISD::FMA:
case ISD::FMINNUM:
case ISD::FMAXNUM:
+ case ISD::FMINIMUMNUM:
+ case ISD::FMAXIMUMNUM:
if (Op.getValueType() == MVT::nxv32f16 &&
(Subtarget.hasVInstructionsF16Minimal() &&
!Subtarget.hasVInstructionsF16()))
@@ -7034,6 +7046,8 @@ SDValue RISCVTargetLowering::LowerOperation(SDValue Op,
case ISD::VP_FMA:
case ISD::VP_FMINNUM:
case ISD::VP_FMAXNUM:
+ case ISD::VP_FMINIMUMNUM:
+ case ISD::VP_FMAXIMUMNUM:
case ISD::VP_FCOPYSIGN:
if (Op.getValueType() == MVT::nxv32f16 &&
(Subtarget.hasVInstructionsF16Minimal() &&
@@ -12924,8 +12938,10 @@ static SDValue combineBinOpToReduce(SDNode *N, SelectionDAG &DAG,
case ISD::FADD:
return RISCVISD::VECREDUCE_FADD_VL;
case ISD::FMAXNUM:
+ case ISD::FMAXIMUMNUM:
return RISCVISD::VECREDUCE_FMAX_VL;
case ISD::FMINNUM:
+ case ISD::FMINIMUMNUM:
return RISCVISD::VECREDUCE_FMIN_VL;
}
};
@@ -16637,7 +16653,9 @@ SDValue RISCVTargetLowering::PerformDAGCombine(SDNode *N,
case ISD::SMAX:
case ISD::SMIN:
case ISD::FMAXNUM:
- case ISD::FMINNUM: {
+ case ISD::FMINNUM:
+ case ISD::FMAXIMUMNUM:
+ case ISD::FMINIMUMNUM: {
if (SDValue V = combineBinOpToReduce(N, DAG, Subtarget))
return V;
if (SDValue V = combineBinOpOfExtractToReduceTree(N, DAG, Subtarget))
diff --git a/llvm/lib/Target/RISCV/RISCVInstrInfoD.td b/llvm/lib/Target/RISCV/RISCVInstrInfoD.td
index 8efefee383a6a..00e61889d8ad9 100644
--- a/llvm/lib/Target/RISCV/RISCVInstrInfoD.td
+++ b/llvm/lib/Target/RISCV/RISCVInstrInfoD.td
@@ -389,6 +389,8 @@ def : Pat<(fneg (any_fma_nsz FPR64IN32X:$rs1, FPR64IN32X:$rs2, FPR64IN32X:$rs3))
foreach Ext = DExts in {
defm : PatFprFpr_m<fminnum, FMIN_D, Ext>;
defm : PatFprFpr_m<fmaxnum, FMAX_D, Ext>;
+ defm : PatFprFpr_m<fminimumnum, FMIN_D, Ext>;
+ defm : PatFprFpr_m<fmaximumnum, FMAX_D, Ext>;
defm : PatFprFpr_m<riscv_fmin, FMIN_D, Ext>;
defm : PatFprFpr_m<riscv_fmax, FMAX_D, Ext>;
}
diff --git a/llvm/lib/Target/RISCV/RISCVInstrInfoF.td b/llvm/lib/Target/RISCV/RISCVInstrInfoF.td
index 7d89608de1223..edc5d2467b2a7 100644
--- a/llvm/lib/Target/RISCV/RISCVInstrInfoF.td
+++ b/llvm/lib/Target/RISCV/RISCVInstrInfoF.td
@@ -596,6 +596,8 @@ def : Pat<(fneg (any_fma_nsz FPR32INX:$rs1, FPR32INX:$rs2, FPR32INX:$rs3)),
foreach Ext = FExts in {
defm : PatFprFpr_m<fminnum, FMIN_S, Ext>;
defm : PatFprFpr_m<fmaxnum, FMAX_S, Ext>;
+ defm : PatFprFpr_m<fminimumnum, FMIN_S, Ext>;
+ defm : PatFprFpr_m<fmaximumnum, FMAX_S, Ext>;
defm : PatFprFpr_m<riscv_fmin, FMIN_S, Ext>;
defm : PatFprFpr_m<riscv_fmax, FMAX_S, Ext>;
}
diff --git a/llvm/lib/Target/RISCV/RISCVInstrInfoVSDPatterns.td b/llvm/lib/Target/RISCV/RISCVInstrInfoVSDPatterns.td
index e82625f085bec..34c9669489107 100644
--- a/llvm/lib/Target/RISCV/RISCVInstrInfoVSDPatterns.td
+++ b/llvm/lib/Target/RISCV/RISCVInstrInfoVSDPatterns.td
@@ -1377,6 +1377,8 @@ foreach vti = AllFloatVectors in {
// 13.11. Vector Floating-Point MIN/MAX Instructions
defm : VPatBinaryFPSDNode_VV_VF<fminnum, "PseudoVFMIN", isSEWAware=1>;
defm : VPatBinaryFPSDNode_VV_VF<fmaxnum, "PseudoVFMAX", isSEWAware=1>;
+defm : VPatBinaryFPSDNode_VV_VF<fminimumnum, "PseudoVFMIN", isSEWAware=1>;
+defm : VPatBinaryFPSDNode_VV_VF<fmaximumnum, "PseudoVFMAX", isSEWAware=1>;
// 13.13. Vector Floating-Point Compare Instructions
defm : VPatFPSetCCSDNode_VV_VF_FV<SETEQ, "PseudoVMFEQ", "PseudoVMFEQ">;
diff --git a/llvm/lib/Target/X86/X86ISelLowering.cpp b/llvm/lib/Target/X86/X86ISelLowering.cpp
index 2d5066dd8854c..abaa249563a03 100644
--- a/llvm/lib/Target/X86/X86ISelLowering.cpp
+++ b/llvm/lib/Target/X86/X86ISelLowering.cpp
@@ -947,6 +947,8 @@ X86TargetLowering::X86TargetLowering(const X86TargetMachine &TM,
setOperationAction(ISD::FEXP10, MVT::f80, Expand);
setOperationAction(ISD::FMINNUM, MVT::f80, Expand);
setOperationAction(ISD::FMAXNUM, MVT::f80, Expand);
+ setOperationAction(ISD::FMINIMUMNUM, MVT::f80, Expand);
+ setOperationAction(ISD::FMAXIMUMNUM, MVT::f80, Expand);
// Some FP actions are always expanded for vector types.
for (auto VT : { MVT::v8f16, MVT::v16f16, MVT::v32f16,
@@ -44813,6 +44815,8 @@ static SDValue scalarizeExtEltFP(SDNode *ExtElt, SelectionDAG &DAG,
case ISD::FMAXNUM_IEEE:
case ISD::FMAXIMUM:
case ISD::FMINIMUM:
+ case ISD::FMAXIMUMNUM:
+ case ISD::FMINIMUMNUM:
case X86ISD::FMAX:
case X86ISD::FMIN:
case ISD::FABS: // Begin 1 operand
diff --git a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
index de0144331dba3..7b27754f6d183 100644
--- a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
+++ b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
@@ -4214,6 +4214,10 @@ X86TTIImpl::getIntrinsicInstrCost(const IntrinsicCostAttributes &ICA,
// FMINNUM has same costs so don't duplicate.
ISD = ISD::FMAXNUM;
break;
+ case Intrinsic::maximumnum:
+ case Intrinsic::minimumnum:
+ ISD = ISD::FMAXIMUMNUM;
+ break;
case Intrinsic::sadd_sat:
ISD = ISD::SADDSAT;
break;
@@ -5407,9 +5411,12 @@ X86TTIImpl::getMinMaxReductionCost(Intrinsic::ID IID, VectorType *ValTy,
} else {
assert(ValTy->isFPOrFPVectorTy() &&
"Expected float point or integer vector type.");
- ISD = (IID == Intrinsic::minnum || IID == Intrinsic::maxnum)
- ? ISD::FMINNUM
- : ISD::FMINIMUM;
+ if (IID == Intrinsic::minimumnum || IID == Intrinsic::maximumnum)
+ ISD = ISD::FMINIMUMNUM;
+ else
+ ISD = (IID == Intrinsic::minnum || IID == Intrinsic::maxnum)
+ ? ISD::FMINNUM
+ : ISD::FMINIMUM;
}
// We use the Intel Architecture Code Analyzer(IACA) to measure the throughput
diff --git a/llvm/test/CodeGen/AArch64/GlobalISel/legalizer-info-validation.mir b/llvm/test/CodeGen/AArch64/GlobalISel/legalizer-info-validation.mir
index f03491924f7f4..6e8c6cce66da8 100644
--- a/llvm/test/CodeGen/AArch64/GlobalISel/legalizer-info-validation.mir
+++ b/llvm/test/CodeGen/AArch64/GlobalISel/legalizer-info-validation.mir
@@ -561,6 +561,12 @@
# DEBUG-NEXT: .. opcode {{[0-9]+}} is aliased to {{[0-9]+}}
# DEBUG-NEXT: .. type index coverage check SKIPPED: user-defined predicate detected
# DEBUG-NEXT: .. imm index coverage check SKIPPED: user-defined predicate detected
+# DEBUG-NEXT: G_FMINIMUMNUM (opcode {{[0-9]+}}): 1 type index
+# DEBUG-NEXT: .. type index coverage check SKIPPED: no rules defined
+# DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined
+# DEBUG-NEXT: G_FMAXIMUMNUM (opcode {{[0-9]+}}): 1 type index
+# DEBUG-NEXT: .. type index coverage check SKIPPED: no rules defined
+# DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined
# DEBUG-NEXT: G_GET_FPENV (opcode {{[0-9]+}}): 1 type index, 0 imm indices
# DEBUG-NEXT: .. type index coverage check SKIPPED: user-defined predicate detected
# DEBUG-NEXT: .. imm index coverage check SKIPPED: user-defined predicate detected
@@ -816,6 +822,12 @@
# DEBUG-NEXT: .. opcode {{[0-9]+}} is aliased to {{[0-9]+}}
# DEBUG-NEXT: .. type index coverage check SKIPPED: user-defined predicate detected
# DEBUG-NEXT: .. imm index coverage check SKIPPED: user-defined predicate detected
+# DEBUG-NEXT: G_VECREDUCE_FMAXIMUMNUM (opcode {{[0-9]+}}): 2 type indices, 0 imm indices
+# DEBUG-NEXT: .. type index coverage check SKIPPED: no rules defined
+# DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined
+# DEBUG-NEXT: G_VECREDUCE_FMINIMUMNUM (opcode {{[0-9]+}}): 2 type indices, 0 imm indices
+# DEBUG-NEXT: .. type index coverage check SKIPPED: no rules defined
+# DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined
# DEBUG-NEXT: G_VECREDUCE_ADD (opcode {{[0-9]+}}): 2 type indices, 0 imm indices
# DEBUG-NEXT: .. type index coverage check SKIPPED: user-defined predicate detected
# DEBUG-NEXT: .. imm index coverage check SKIPPED: user-defined predicate detected
diff --git a/llvm/test/CodeGen/AArch64/combine_andor_with_cmps.ll b/llvm/test/CodeGen/AArch64/combine_andor_with_cmps.ll
index 783683cf7e844..7cb5f7583b2ef 100644
--- a/llvm/test/CodeGen/AArch64/combine_andor_with_cmps.ll
+++ b/llvm/test/CodeGen/AArch64/combine_andor_with_cmps.ll
@@ -31,9 +31,6 @@ define i1 @test2(double %arg1, double %arg2, double %arg3) #0 {
ret i1 %or1
}
-; It is illegal to apply the optimization in the following two test cases
-; because FMINNUM_IEEE and FMAXNUM_IEEE are not supported.
-
define i1 @test3(float %arg1, float %arg2, float %arg3) {
; CHECK-LABEL: test3:
; CHECK: // %bb.0:
@@ -41,8 +38,8 @@ define i1 @test3(float %arg1, float %arg2, float %arg3) {
; CHECK-NEXT: fadd s0, s0, s3
; CHECK-NEXT: fmov s3, #2.00000000
; CHECK-NEXT: fadd s1, s1, s3
-; CHECK-NEXT: fcmp s1, s2
-; CHECK-NEXT: fccmp s0, s2, #0, lt
+; CHECK-NEXT: fmaxnm s0, s0, s1
+; CHECK-NEXT: fcmp s0, s2
; CHECK-NEXT: cset w0, lt
; CHECK-NEXT: ret
%add1 = fadd nnan float %arg1, 1.0
@@ -60,8 +57,8 @@ define i1 @test4(float %arg1, float %arg2, float %arg3) {
; CHECK-NEXT: fadd s0, s0, s3
; CHECK-NEXT: fmov s3, #2.00000000
; CHECK-NEXT: fadd s1, s1, s3
-; CHECK-NEXT: fcmp s1, s2
-; CHECK-NEXT: fccmp s0, s2, #4, gt
+; CHECK-NEXT: fminnm s0, s0, s1
+; CHECK-NEXT: fcmp s0, s2
; CHECK-NEXT: cset w0, gt
; CHECK-NEXT: ret
%add1 = fadd nnan float %arg1, 1.0
diff --git a/llvm/test/CodeGen/AArch64/fp-maximumnum-minimumnum.ll b/llvm/test/CodeGen/AArch64/fp-maximumnum-minimumnum.ll
new file mode 100644
index 0000000000000..6a1ffa75a7566
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/fp-maximumnum-minimumnum.ll
@@ -0,0 +1,51 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc --mtriple=aarch64-linux-gnu < %s | FileCheck %s --check-prefix=AARCH64
+
+declare float @llvm.maximumnum.f32(float, float)
+declare double @llvm.maximumnum.f64(double, double)
+declare float @llvm.minimumnum.f32(float, float)
+declare double @llvm.minimumnum.f64(double, double)
+
+define float @maximumnum_float(float %x, float %y) {
+; AARCH64-LABEL: maximumnum_float:
+; AARCH64: // %bb.0:
+; AARCH64-NEXT: fmin s1, s1, s1
+; AARCH64-NEXT: fmin s0, s0, s0
+; AARCH64-NEXT: fmaxnm s0, s0, s1
+; AARCH64-NEXT: ret
+ %z = tail call float @llvm.maximumnum.f32(float %x, float %y)
+ ret float %z
+}
+
+define double @maximumnum_double(double %x, double %y) {
+; AARCH64-LABEL: maximumnum_double:
+; AARCH64: // %bb.0:
+; AARCH64-NEXT: fmin d1, d1, d1
+; AARCH64-NEXT: fmin d0, d0, d0
+; AARCH64-NEXT: fmaxnm d0, d0, d1
+; AARCH64-NEXT: ret
+ %z = call double @llvm.maximumnum.f64(double %x, double %y)
+ ret double %z
+}
+
+define float @minimumnum_float(float %x, float %y) {
+; AARCH64-LABEL: minimumnum_float:
+; AARCH64: // %bb.0:
+; AARCH64-NEXT: fmin s1, s1, s1
+; AARCH64-NEXT: fmin s0, s0, s0
+; AARCH64-NEXT: fminnm s0, s0, s1
+; AARCH64-NEXT: ret
+ %z = call float @llvm.minimumnum.f32(float %x, float %y)
+ ret float %z
+}
+
+define double @minimumnum_double(double %x, double %y) {
+; AARCH64-LABEL: minimumnum_double:
+; AARCH64: // %bb.0:
+; AARCH64-NEXT: fmin d1, d1, d1
+; AARCH64-NEXT: fmin d0, d0, d0
+; AARCH64-NEXT: fminnm d0, d0, d1
+; AARCH64-NEXT: ret
+ %z = call double @llvm.minimumnum.f64(double %x, double %y)
+ ret double %z
+}
diff --git a/llvm/test/CodeGen/Hexagon/fminmax-v67.ll b/llvm/test/CodeGen/Hexagon/fminmax-v67.ll
index d5b803c6c8926..c40f1da22eb6e 100644
--- a/llvm/test/CodeGen/Hexagon/fminmax-v67.ll
+++ b/llvm/test/CodeGen/Hexagon/fminmax-v67.ll
@@ -37,9 +37,49 @@ entry:
ret float %0
}
+; CHECK-LABEL: t5
+; CHECK: dfmax
+
+define dso_local double @t5(double %a, double %b) local_unnamed_addr {
+entry:
+ %0 = tail call double @llvm.maximumnum.f64(double %a, double %b)
+ ret double %0
+}
+
+; CHECK-LABEL: t6
+; CHECK: dfmin
+
+define dso_local double @t6(double %a, double %b) local_unnamed_addr {
+entry:
+ %0 = tail call double @llvm.minimumnum.f64(double %a, double %b)
+ ret double %0
+}
+
+; CHECK-LABEL: t7
+; CHECK: sfmax
+
+define dso_local float @t7(float %a, float %b) local_unnamed_addr {
+entry:
+ %0 = tail call float @llvm.maximumnum.f32(float %a, float %b)
+ ret float %0
+}
+
+; CHECK-LABEL: t8
+; CHECK: sfmin
+
+define dso_local float @t8(float %a, float %b) local_unnamed_addr {
+entry:
+ %0 = tail call float @llvm.minimumnum.f32(float %a, float %b)
+ ret float %0
+}
+
declare double @llvm.minnum.f64(double, double) #1
declare double @llvm.maxnum.f64(double, double) #1
declare float @llvm.maxnum.f32(float, float) #1
declare float @llvm.minnum.f32(float, float) #1
+declare double @llvm.minimumnum.f64(double, double) #1
+declare double @llvm.maximumnum.f64(double, double) #1
+declare float @llvm.maximumnum.f32(float, float) #1
+declare float @llvm.minimumnum.f32(float, float) #1
attributes #1 = { nounwind readnone speculatable }
diff --git a/llvm/test/CodeGen/LoongArch/fp-maximumnum-minimumnum.ll b/llvm/test/CodeGen/LoongArch/fp-maximumnum-minimumnum.ll
new file mode 100644
index 0000000000000..5f86efaa45e99
--- /dev/null
+++ b/llvm/test/CodeGen/LoongArch/fp-maximumnum-minimumnum.ll
@@ -0,0 +1,154 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc --mtriple=loongarch32 --mattr=+f,-d < %s | FileCheck %s --check-prefix=LA32F
+; RUN: llc --mtriple=loongarch32 --mattr=+d < %s | FileCheck %s --check-prefix=LA32D
+; RUN: llc --mtriple=loongarch64 --mattr=+f,-d < %s | FileCheck %s --check-prefix=LA64F
+; RUN: llc --mtriple=loongarch64 --mattr=+d < %s | FileCheck %s --check-prefix=LA64D
+
+declare float @llvm.maximumnum.f32(float, float)
+declare double @llvm.maximumnum.f64(double, double)
+declare float @llvm.minimumnum.f32(float, float)
+declare double @llvm.minimumnum.f64(double, double)
+
+define float @maximumnum_float(float %x, float %y) {
+; LA32F-LABEL: maximumnum_float:
+; LA32F: # %bb.0:
+; LA32F-NEXT: fmax.s $fa1, $fa1, $fa1
+; LA32F-NEXT: fmax.s $fa0, $fa0, $fa0
+; LA32F-NEXT: fmax.s $fa0, $fa0, $fa1
+; LA32F-NEXT: ret
+;
+; LA32D-LABEL: maximumnum_float:
+; LA32D: # %bb.0:
+; LA32D-NEXT: fmax.s $fa1, $fa1, $fa1
+; LA32D-NEXT: fmax.s $fa0, $fa0, $fa0
+; LA32D-NEXT: fmax.s $fa0, $fa0, $fa1
+; LA32D-NEXT: ret
+;
+; LA64F-LABEL: maximumnum_float:
+; LA64F: # %bb.0:
+; LA64F-NEXT: fmax.s $fa1, $fa1, $fa1
+; LA64F-NEXT: fmax.s $fa0, $fa0, $fa0
+; LA64F-NEXT: fmax.s $fa0, $fa0, $fa1
+; LA64F-NEXT: ret
+;
+; LA64D-LABEL: maximumnum_float:
+; LA64D: # %bb.0:
+; LA64D-NEXT: fmax.s $fa1, $fa1, $fa1
+; LA64D-NEXT: fmax.s $fa0, $fa0, $fa0
+; LA64D-NEXT: fmax.s $fa0, $fa0, $fa1
+; LA64D-NEXT: ret
+ %z = call float @llvm.maximumnum.f32(float %x, float %y)
+ ret float %z
+}
+
+define double @maximumnum_double(double %x, double %y) {
+; LA32F-LABEL: maximumnum_double:
+; LA32F: # %bb.0:
+; LA32F-NEXT: addi.w $sp, $sp, -16
+; LA32F-NEXT: .cfi_def_cfa_offset 16
+; LA32F-NEXT: st.w $ra, $sp, 12 # 4-byte Folded Spill
+; LA32F-NEXT: .cfi_offset 1, -4
+; LA32F-NEXT: bl %plt(fmaximum_num)
+; LA32F-NEXT: ld.w $ra, $sp, 12 # 4-byte Folded Reload
+; LA32F-NEXT: addi.w $sp, $sp, 16
+; LA32F-NEXT: ret
+;
+; LA32D-LABEL: maximumnum_double:
+; LA32D: # %bb.0:
+; LA32D-NEXT: fmax.d $fa1, $fa1, $fa1
+; LA32D-NEXT: fmax.d $fa0, $fa0, $fa0
+; LA32D-NEXT: fmax.d $fa0, $fa0, $fa1
+; LA32D-NEXT: ret
+;
+; LA64F-LABEL: maximumnum_double:
+; LA64F: # %bb.0:
+; LA64F-NEXT: addi.d $sp, $sp, -16
+; LA64F-NEXT: .cfi_def_cfa_offset 16
+; LA64F-NEXT: st.d $ra, $sp, 8 # 8-byte Folded Spill
+; LA64F-NEXT: .cfi_offset 1, -8
+; LA64F-NEXT: bl %plt(fmaximum_num)
+; LA64F-NEXT: ld.d $ra, $sp, 8 # 8-byte Folded Reload
+; LA64F-NEXT: addi.d $sp, $sp, 16
+; LA64F-NEXT: ret
+;
+; LA64D-LABEL: maximumnum_double:
+; LA64D: # %bb.0:
+; LA64D-NEXT: fmax.d $fa1, $fa1, $fa1
+; LA64D-NEXT: fmax.d $fa0, $fa0, $fa0
+; LA64D-NEXT: fmax.d $fa0, $fa0, $fa1
+; LA64D-NEXT: ret
+ %z = call double @llvm.maximumnum.f64(double %x, double %y)
+ ret double %z
+}
+
+define float @minimumnum_float(float %x, float %y) {
+; LA32F-LABEL: minimumnum_float:
+; LA32F: # %bb.0:
+; LA32F-NEXT: fmax.s $fa1, $fa1, $fa1
+; LA32F-NEXT: fmax.s $fa0, $fa0, $fa0
+; LA32F-NEXT: fmin.s $fa0, $fa0, $fa1
+; LA32F-NEXT: ret
+;
+; LA32D-LABEL: minimumnum_float:
+; LA32D: # %bb.0:
+; LA32D-NEXT: fmax.s $fa1, $fa1, $fa1
+; LA32D-NEXT: fmax.s $fa0, $fa0, $fa0
+; LA32D-NEXT: fmin.s $fa0, $fa0, $fa1
+; LA32D-NEXT: ret
+;
+; LA64F-LABEL: minimumnum_float:
+; LA64F: # %bb.0:
+; LA64F-NEXT: fmax.s $fa1, $fa1, $fa1
+; LA64F-NEXT: fmax.s $fa0, $fa0, $fa0
+; LA64F-NEXT: fmin.s $fa0, $fa0, $fa1
+; LA64F-NEXT: ret
+;
+; LA64D-LABEL: minimumnum_float:
+; LA64D: # %bb.0:
+; LA64D-NEXT: fmax.s $fa1, $fa1, $fa1
+; LA64D-NEXT: fmax.s $fa0, $fa0, $fa0
+; LA64D-NEXT: fmin.s $fa0, $fa0, $fa1
+; LA64D-NEXT: ret
+ %z = call float @llvm.minimumnum.f32(float %x, float %y)
+ ret float %z
+}
+
+define double @minimumnum_double(double %x, double %y) {
+; LA32F-LABEL: minimumnum_double:
+; LA32F: # %bb.0:
+; LA32F-NEXT: addi.w $sp, $sp, -16
+; LA32F-NEXT: .cfi_def_cfa_offset 16
+; LA32F-NEXT: st.w $ra, $sp, 12 # 4-byte Folded Spill
+; LA32F-NEXT: .cfi_offset 1, -4
+; LA32F-NEXT: bl %plt(fminimum_num)
+; LA32F-NEXT: ld.w $ra, $sp, 12 # 4-byte Folded Reload
+; LA32F-NEXT: addi.w $sp, $sp, 16
+; LA32F-NEXT: ret
+;
+; LA32D-LABEL: minimumnum_double:
+; LA32D: # %bb.0:
+; LA32D-NEXT: fmax.d $fa1, $fa1, $fa1
+; LA32D-NEXT: fmax.d $fa0, $fa0, $fa0
+; LA32D-NEXT: fmin.d $fa0, $fa0, $fa1
+; LA32D-NEXT: ret
+;
+; LA64F-LABEL: minimumnum_double:
+; LA64F: # %bb.0:
+; LA64F-NEXT: addi.d $sp, $sp, -16
+; LA64F-NEXT: .cfi_def_cfa_offset 16
+; LA64F-NEXT: st.d $ra, $sp, 8 # 8-byte Folded Spill
+; LA64F-NEXT: .cfi_offset 1, -8
+; LA64F-NEXT: bl %plt(fminimum_num)
+; LA64F-NEXT: ld.d $ra, $sp, 8 # 8-byte Folded Reload
+; LA64F-NEXT: addi.d $sp, $sp, 16
+; LA64F-NEXT: ret
+;
+; LA64D-LABEL: minimumnum_double:
+; LA64D: # %bb.0:
+; LA64D-NEXT: fmax.d $fa1, $fa1, $fa1
+; LA64D-NEXT: fmax.d $fa0, $fa0, $fa0
+; LA64D-NEXT: fmin.d $fa0, $fa0, $fa1
+; LA64D-NEXT: ret
+ %z = call double @llvm.minimumnum.f64(double %x, double %y)
+ ret double %z
+}
diff --git a/llvm/test/CodeGen/Mips/fp-maximumnum-minimumnum.ll b/llvm/test/CodeGen/Mips/fp-maximumnum-minimumnum.ll
new file mode 100644
index 0000000000000..8ee4070e53434
--- /dev/null
+++ b/llvm/test/CodeGen/Mips/fp-maximumnum-minimumnum.ll
@@ -0,0 +1,100 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc --mtriple=mipsisa32r6 < %s | FileCheck %s --check-prefix=MIPS32R6
+; RUN: llc --mtriple=mips < %s | FileCheck %s --check-prefix=MIPS32
+
+declare float @llvm.maximumnum.f32(float, float)
+declare double @llvm.maximumnum.f64(double, double)
+declare float @llvm.minimumnum.f32(float, float)
+declare double @llvm.minimumnum.f64(double, double)
+
+define float @maximumnum_float(float %x, float %y) {
+; MIPS32R6-LABEL: maximumnum_float:
+; MIPS32R6: # %bb.0:
+; MIPS32R6-NEXT: min.s $f0, $f14, $f14
+; MIPS32R6-NEXT: min.s $f1, $f12, $f12
+; MIPS32R6-NEXT: jr $ra
+; MIPS32R6-NEXT: max.s $f0, $f1, $f0
+;
+; MIPS32-LABEL: maximumnum_float:
+; MIPS32: # %bb.0:
+; MIPS32-NEXT: addiu $sp, $sp, -24
+; MIPS32-NEXT: .cfi_def_cfa_offset 24
+; MIPS32-NEXT: sw $ra, 20($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: .cfi_offset 31, -4
+; MIPS32-NEXT: jal fmaximum_numf
+; MIPS32-NEXT: nop
+; MIPS32-NEXT: lw $ra, 20($sp) # 4-byte Folded Reload
+; MIPS32-NEXT: jr $ra
+; MIPS32-NEXT: addiu $sp, $sp, 24
+ %z = call float @llvm.maximumnum.f32(float %x, float %y)
+ ret float %z
+}
+
+define double @maximumnum_double(double %x, double %y) {
+; MIPS32R6-LABEL: maximumnum_double:
+; MIPS32R6: # %bb.0:
+; MIPS32R6-NEXT: min.d $f0, $f14, $f14
+; MIPS32R6-NEXT: min.d $f1, $f12, $f12
+; MIPS32R6-NEXT: jr $ra
+; MIPS32R6-NEXT: max.d $f0, $f1, $f0
+;
+; MIPS32-LABEL: maximumnum_double:
+; MIPS32: # %bb.0:
+; MIPS32-NEXT: addiu $sp, $sp, -24
+; MIPS32-NEXT: .cfi_def_cfa_offset 24
+; MIPS32-NEXT: sw $ra, 20($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: .cfi_offset 31, -4
+; MIPS32-NEXT: jal fmaximum_num
+; MIPS32-NEXT: nop
+; MIPS32-NEXT: lw $ra, 20($sp) # 4-byte Folded Reload
+; MIPS32-NEXT: jr $ra
+; MIPS32-NEXT: addiu $sp, $sp, 24
+ %z = call double @llvm.maximumnum.f64(double %x, double %y)
+ ret double %z
+}
+
+define float @minimumnum_float(float %x, float %y) {
+; MIPS32R6-LABEL: minimumnum_float:
+; MIPS32R6: # %bb.0:
+; MIPS32R6-NEXT: min.s $f0, $f14, $f14
+; MIPS32R6-NEXT: min.s $f1, $f12, $f12
+; MIPS32R6-NEXT: jr $ra
+; MIPS32R6-NEXT: min.s $f0, $f1, $f0
+;
+; MIPS32-LABEL: minimumnum_float:
+; MIPS32: # %bb.0:
+; MIPS32-NEXT: addiu $sp, $sp, -24
+; MIPS32-NEXT: .cfi_def_cfa_offset 24
+; MIPS32-NEXT: sw $ra, 20($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: .cfi_offset 31, -4
+; MIPS32-NEXT: jal fminimum_numf
+; MIPS32-NEXT: nop
+; MIPS32-NEXT: lw $ra, 20($sp) # 4-byte Folded Reload
+; MIPS32-NEXT: jr $ra
+; MIPS32-NEXT: addiu $sp, $sp, 24
+ %z = call float @llvm.minimumnum.f32(float %x, float %y)
+ ret float %z
+}
+
+define double @minimumnum_double(double %x, double %y) {
+; MIPS32R6-LABEL: minimumnum_double:
+; MIPS32R6: # %bb.0:
+; MIPS32R6-NEXT: min.d $f0, $f14, $f14
+; MIPS32R6-NEXT: min.d $f1, $f12, $f12
+; MIPS32R6-NEXT: jr $ra
+; MIPS32R6-NEXT: min.d $f0, $f1, $f0
+;
+; MIPS32-LABEL: minimumnum_double:
+; MIPS32: # %bb.0:
+; MIPS32-NEXT: addiu $sp, $sp, -24
+; MIPS32-NEXT: .cfi_def_cfa_offset 24
+; MIPS32-NEXT: sw $ra, 20($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: .cfi_offset 31, -4
+; MIPS32-NEXT: jal fminimum_num
+; MIPS32-NEXT: nop
+; MIPS32-NEXT: lw $ra, 20($sp) # 4-byte Folded Reload
+; MIPS32-NEXT: jr $ra
+; MIPS32-NEXT: addiu $sp, $sp, 24
+ %z = call double @llvm.minimumnum.f64(double %x, double %y)
+ ret double %z
+}
diff --git a/llvm/test/CodeGen/RISCV/double-intrinsics.ll b/llvm/test/CodeGen/RISCV/double-intrinsics.ll
index 52c49cfbfb30a..30ee276912bbf 100644
--- a/llvm/test/CodeGen/RISCV/double-intrinsics.ll
+++ b/llvm/test/CodeGen/RISCV/double-intrinsics.ll
@@ -798,6 +798,84 @@ define double @maxnum_f64(double %a, double %b) nounwind {
; ret double %1
; }
+declare double @llvm.minimumnum.f64(double, double)
+
+define double @minimumnum_f64(double %a, double %b) nounwind {
+; CHECKIFD-LABEL: minimumnum_f64:
+; CHECKIFD: # %bb.0:
+; CHECKIFD-NEXT: fmin.d fa0, fa0, fa1
+; CHECKIFD-NEXT: ret
+;
+; RV32IZFINXZDINX-LABEL: minimumnum_f64:
+; RV32IZFINXZDINX: # %bb.0:
+; RV32IZFINXZDINX-NEXT: fmin.d a0, a0, a2
+; RV32IZFINXZDINX-NEXT: ret
+;
+; RV64IZFINXZDINX-LABEL: minimumnum_f64:
+; RV64IZFINXZDINX: # %bb.0:
+; RV64IZFINXZDINX-NEXT: fmin.d a0, a0, a1
+; RV64IZFINXZDINX-NEXT: ret
+;
+; RV32I-LABEL: minimumnum_f64:
+; RV32I: # %bb.0:
+; RV32I-NEXT: addi sp, sp, -16
+; RV32I-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
+; RV32I-NEXT: call fmin
+; RV32I-NEXT: lw ra, 12(sp) # 4-byte Folded Reload
+; RV32I-NEXT: addi sp, sp, 16
+; RV32I-NEXT: ret
+;
+; RV64I-LABEL: minimumnum_f64:
+; RV64I: # %bb.0:
+; RV64I-NEXT: addi sp, sp, -16
+; RV64I-NEXT: sd ra, 8(sp) # 8-byte Folded Spill
+; RV64I-NEXT: call fmin
+; RV64I-NEXT: ld ra, 8(sp) # 8-byte Folded Reload
+; RV64I-NEXT: addi sp, sp, 16
+; RV64I-NEXT: ret
+ %1 = call double @llvm.minimumnum.f64(double %a, double %b)
+ ret double %1
+}
+
+declare double @llvm.maximumnum.f64(double, double)
+
+define double @maximumnum_f64(double %a, double %b) nounwind {
+; CHECKIFD-LABEL: maximumnum_f64:
+; CHECKIFD: # %bb.0:
+; CHECKIFD-NEXT: fmax.d fa0, fa0, fa1
+; CHECKIFD-NEXT: ret
+;
+; RV32IZFINXZDINX-LABEL: maximumnum_f64:
+; RV32IZFINXZDINX: # %bb.0:
+; RV32IZFINXZDINX-NEXT: fmax.d a0, a0, a2
+; RV32IZFINXZDINX-NEXT: ret
+;
+; RV64IZFINXZDINX-LABEL: maximumnum_f64:
+; RV64IZFINXZDINX: # %bb.0:
+; RV64IZFINXZDINX-NEXT: fmax.d a0, a0, a1
+; RV64IZFINXZDINX-NEXT: ret
+;
+; RV32I-LABEL: maximumnum_f64:
+; RV32I: # %bb.0:
+; RV32I-NEXT: addi sp, sp, -16
+; RV32I-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
+; RV32I-NEXT: call fmax
+; RV32I-NEXT: lw ra, 12(sp) # 4-byte Folded Reload
+; RV32I-NEXT: addi sp, sp, 16
+; RV32I-NEXT: ret
+;
+; RV64I-LABEL: maximumnum_f64:
+; RV64I: # %bb.0:
+; RV64I-NEXT: addi sp, sp, -16
+; RV64I-NEXT: sd ra, 8(sp) # 8-byte Folded Spill
+; RV64I-NEXT: call fmax
+; RV64I-NEXT: ld ra, 8(sp) # 8-byte Folded Reload
+; RV64I-NEXT: addi sp, sp, 16
+; RV64I-NEXT: ret
+ %1 = call double @llvm.maximumnum.f64(double %a, double %b)
+ ret double %1
+}
+
declare double @llvm.copysign.f64(double, double)
define double @copysign_f64(double %a, double %b) nounwind {
@@ -846,16 +924,16 @@ define double @floor_f64(double %a) nounwind {
;
; RV64IFD-LABEL: floor_f64:
; RV64IFD: # %bb.0:
-; RV64IFD-NEXT: lui a0, %hi(.LCPI17_0)
-; RV64IFD-NEXT: fld fa5, %lo(.LCPI17_0)(a0)
+; RV64IFD-NEXT: lui a0, %hi(.LCPI19_0)
+; RV64IFD-NEXT: fld fa5, %lo(.LCPI19_0)(a0)
; RV64IFD-NEXT: fabs.d fa4, fa0
; RV64IFD-NEXT: flt.d a0, fa4, fa5
-; RV64IFD-NEXT: beqz a0, .LBB17_2
+; RV64IFD-NEXT: beqz a0, .LBB19_2
; RV64IFD-NEXT: # %bb.1:
; RV64IFD-NEXT: fcvt.l.d a0, fa0, rdn
; RV64IFD-NEXT: fcvt.d.l fa5, a0, rdn
; RV64IFD-NEXT: fsgnj.d fa0, fa5, fa0
-; RV64IFD-NEXT: .LBB17_2:
+; RV64IFD-NEXT: .LBB19_2:
; RV64IFD-NEXT: ret
;
; RV32IZFINXZDINX-LABEL: floor_f64:
@@ -869,16 +947,16 @@ define double @floor_f64(double %a) nounwind {
;
; RV64IZFINXZDINX-LABEL: floor_f64:
; RV64IZFINXZDINX: # %bb.0:
-; RV64IZFINXZDINX-NEXT: lui a1, %hi(.LCPI17_0)
-; RV64IZFINXZDINX-NEXT: ld a1, %lo(.LCPI17_0)(a1)
+; RV64IZFINXZDINX-NEXT: lui a1, %hi(.LCPI19_0)
+; RV64IZFINXZDINX-NEXT: ld a1, %lo(.LCPI19_0)(a1)
; RV64IZFINXZDINX-NEXT: fabs.d a2, a0
; RV64IZFINXZDINX-NEXT: flt.d a1, a2, a1
-; RV64IZFINXZDINX-NEXT: beqz a1, .LBB17_2
+; RV64IZFINXZDINX-NEXT: beqz a1, .LBB19_2
; RV64IZFINXZDINX-NEXT: # %bb.1:
; RV64IZFINXZDINX-NEXT: fcvt.l.d a1, a0, rdn
; RV64IZFINXZDINX-NEXT: fcvt.d.l a1, a1, rdn
; RV64IZFINXZDINX-NEXT: fsgnj.d a0, a1, a0
-; RV64IZFINXZDINX-NEXT: .LBB17_2:
+; RV64IZFINXZDINX-NEXT: .LBB19_2:
; RV64IZFINXZDINX-NEXT: ret
;
; RV32I-LABEL: floor_f64:
@@ -911,16 +989,16 @@ define double @ceil_f64(double %a) nounwind {
;
; RV64IFD-LABEL: ceil_f64:
; RV64IFD: # %bb.0:
-; RV64IFD-NEXT: lui a0, %hi(.LCPI18_0)
-; RV64IFD-NEXT: fld fa5, %lo(.LCPI18_0)(a0)
+; RV64IFD-NEXT: lui a0, %hi(.LCPI20_0)
+; RV64IFD-NEXT: fld fa5, %lo(.LCPI20_0)(a0)
; RV64IFD-NEXT: fabs.d fa4, fa0
; RV64IFD-NEXT: flt.d a0, fa4, fa5
-; RV64IFD-NEXT: beqz a0, .LBB18_2
+; RV64IFD-NEXT: beqz a0, .LBB20_2
; RV64IFD-NEXT: # %bb.1:
; RV64IFD-NEXT: fcvt.l.d a0, fa0, rup
; RV64IFD-NEXT: fcvt.d.l fa5, a0, rup
; RV64IFD-NEXT: fsgnj.d fa0, fa5, fa0
-; RV64IFD-NEXT: .LBB18_2:
+; RV64IFD-NEXT: .LBB20_2:
; RV64IFD-NEXT: ret
;
; RV32IZFINXZDINX-LABEL: ceil_f64:
@@ -934,16 +1012,16 @@ define double @ceil_f64(double %a) nounwind {
;
; RV64IZFINXZDINX-LABEL: ceil_f64:
; RV64IZFINXZDINX: # %bb.0:
-; RV64IZFINXZDINX-NEXT: lui a1, %hi(.LCPI18_0)
-; RV64IZFINXZDINX-NEXT: ld a1, %lo(.LCPI18_0)(a1)
+; RV64IZFINXZDINX-NEXT: lui a1, %hi(.LCPI20_0)
+; RV64IZFINXZDINX-NEXT: ld a1, %lo(.LCPI20_0)(a1)
; RV64IZFINXZDINX-NEXT: fabs.d a2, a0
; RV64IZFINXZDINX-NEXT: flt.d a1, a2, a1
-; RV64IZFINXZDINX-NEXT: beqz a1, .LBB18_2
+; RV64IZFINXZDINX-NEXT: beqz a1, .LBB20_2
; RV64IZFINXZDINX-NEXT: # %bb.1:
; RV64IZFINXZDINX-NEXT: fcvt.l.d a1, a0, rup
; RV64IZFINXZDINX-NEXT: fcvt.d.l a1, a1, rup
; RV64IZFINXZDINX-NEXT: fsgnj.d a0, a1, a0
-; RV64IZFINXZDINX-NEXT: .LBB18_2:
+; RV64IZFINXZDINX-NEXT: .LBB20_2:
; RV64IZFINXZDINX-NEXT: ret
;
; RV32I-LABEL: ceil_f64:
@@ -976,16 +1054,16 @@ define double @trunc_f64(double %a) nounwind {
;
; RV64IFD-LABEL: trunc_f64:
; RV64IFD: # %bb.0:
-; RV64IFD-NEXT: lui a0, %hi(.LCPI19_0)
-; RV64IFD-NEXT: fld fa5, %lo(.LCPI19_0)(a0)
+; RV64IFD-NEXT: lui a0, %hi(.LCPI21_0)
+; RV64IFD-NEXT: fld fa5, %lo(.LCPI21_0)(a0)
; RV64IFD-NEXT: fabs.d fa4, fa0
; RV64IFD-NEXT: flt.d a0, fa4, fa5
-; RV64IFD-NEXT: beqz a0, .LBB19_2
+; RV64IFD-NEXT: beqz a0, .LBB21_2
; RV64IFD-NEXT: # %bb.1:
; RV64IFD-NEXT: fcvt.l.d a0, fa0, rtz
; RV64IFD-NEXT: fcvt.d.l fa5, a0, rtz
; RV64IFD-NEXT: fsgnj.d fa0, fa5, fa0
-; RV64IFD-NEXT: .LBB19_2:
+; RV64IFD-NEXT: .LBB21_2:
; RV64IFD-NEXT: ret
;
; RV32IZFINXZDINX-LABEL: trunc_f64:
@@ -999,16 +1077,16 @@ define double @trunc_f64(double %a) nounwind {
;
; RV64IZFINXZDINX-LABEL: trunc_f64:
; RV64IZFINXZDINX: # %bb.0:
-; RV64IZFINXZDINX-NEXT: lui a1, %hi(.LCPI19_0)
-; RV64IZFINXZDINX-NEXT: ld a1, %lo(.LCPI19_0)(a1)
+; RV64IZFINXZDINX-NEXT: lui a1, %hi(.LCPI21_0)
+; RV64IZFINXZDINX-NEXT: ld a1, %lo(.LCPI21_0)(a1)
; RV64IZFINXZDINX-NEXT: fabs.d a2, a0
; RV64IZFINXZDINX-NEXT: flt.d a1, a2, a1
-; RV64IZFINXZDINX-NEXT: beqz a1, .LBB19_2
+; RV64IZFINXZDINX-NEXT: beqz a1, .LBB21_2
; RV64IZFINXZDINX-NEXT: # %bb.1:
; RV64IZFINXZDINX-NEXT: fcvt.l.d a1, a0, rtz
; RV64IZFINXZDINX-NEXT: fcvt.d.l a1, a1, rtz
; RV64IZFINXZDINX-NEXT: fsgnj.d a0, a1, a0
-; RV64IZFINXZDINX-NEXT: .LBB19_2:
+; RV64IZFINXZDINX-NEXT: .LBB21_2:
; RV64IZFINXZDINX-NEXT: ret
;
; RV32I-LABEL: trunc_f64:
@@ -1041,16 +1119,16 @@ define double @rint_f64(double %a) nounwind {
;
; RV64IFD-LABEL: rint_f64:
; RV64IFD: # %bb.0:
-; RV64IFD-NEXT: lui a0, %hi(.LCPI20_0)
-; RV64IFD-NEXT: fld fa5, %lo(.LCPI20_0)(a0)
+; RV64IFD-NEXT: lui a0, %hi(.LCPI22_0)
+; RV64IFD-NEXT: fld fa5, %lo(.LCPI22_0)(a0)
; RV64IFD-NEXT: fabs.d fa4, fa0
; RV64IFD-NEXT: flt.d a0, fa4, fa5
-; RV64IFD-NEXT: beqz a0, .LBB20_2
+; RV64IFD-NEXT: beqz a0, .LBB22_2
; RV64IFD-NEXT: # %bb.1:
; RV64IFD-NEXT: fcvt.l.d a0, fa0
; RV64IFD-NEXT: fcvt.d.l fa5, a0
; RV64IFD-NEXT: fsgnj.d fa0, fa5, fa0
-; RV64IFD-NEXT: .LBB20_2:
+; RV64IFD-NEXT: .LBB22_2:
; RV64IFD-NEXT: ret
;
; RV32IZFINXZDINX-LABEL: rint_f64:
@@ -1064,16 +1142,16 @@ define double @rint_f64(double %a) nounwind {
;
; RV64IZFINXZDINX-LABEL: rint_f64:
; RV64IZFINXZDINX: # %bb.0:
-; RV64IZFINXZDINX-NEXT: lui a1, %hi(.LCPI20_0)
-; RV64IZFINXZDINX-NEXT: ld a1, %lo(.LCPI20_0)(a1)
+; RV64IZFINXZDINX-NEXT: lui a1, %hi(.LCPI22_0)
+; RV64IZFINXZDINX-NEXT: ld a1, %lo(.LCPI22_0)(a1)
; RV64IZFINXZDINX-NEXT: fabs.d a2, a0
; RV64IZFINXZDINX-NEXT: flt.d a1, a2, a1
-; RV64IZFINXZDINX-NEXT: beqz a1, .LBB20_2
+; RV64IZFINXZDINX-NEXT: beqz a1, .LBB22_2
; RV64IZFINXZDINX-NEXT: # %bb.1:
; RV64IZFINXZDINX-NEXT: fcvt.l.d a1, a0
; RV64IZFINXZDINX-NEXT: fcvt.d.l a1, a1
; RV64IZFINXZDINX-NEXT: fsgnj.d a0, a1, a0
-; RV64IZFINXZDINX-NEXT: .LBB20_2:
+; RV64IZFINXZDINX-NEXT: .LBB22_2:
; RV64IZFINXZDINX-NEXT: ret
;
; RV32I-LABEL: rint_f64:
@@ -1147,16 +1225,16 @@ define double @round_f64(double %a) nounwind {
;
; RV64IFD-LABEL: round_f64:
; RV64IFD: # %bb.0:
-; RV64IFD-NEXT: lui a0, %hi(.LCPI22_0)
-; RV64IFD-NEXT: fld fa5, %lo(.LCPI22_0)(a0)
+; RV64IFD-NEXT: lui a0, %hi(.LCPI24_0)
+; RV64IFD-NEXT: fld fa5, %lo(.LCPI24_0)(a0)
; RV64IFD-NEXT: fabs.d fa4, fa0
; RV64IFD-NEXT: flt.d a0, fa4, fa5
-; RV64IFD-NEXT: beqz a0, .LBB22_2
+; RV64IFD-NEXT: beqz a0, .LBB24_2
; RV64IFD-NEXT: # %bb.1:
; RV64IFD-NEXT: fcvt.l.d a0, fa0, rmm
; RV64IFD-NEXT: fcvt.d.l fa5, a0, rmm
; RV64IFD-NEXT: fsgnj.d fa0, fa5, fa0
-; RV64IFD-NEXT: .LBB22_2:
+; RV64IFD-NEXT: .LBB24_2:
; RV64IFD-NEXT: ret
;
; RV32IZFINXZDINX-LABEL: round_f64:
@@ -1170,16 +1248,16 @@ define double @round_f64(double %a) nounwind {
;
; RV64IZFINXZDINX-LABEL: round_f64:
; RV64IZFINXZDINX: # %bb.0:
-; RV64IZFINXZDINX-NEXT: lui a1, %hi(.LCPI22_0)
-; RV64IZFINXZDINX-NEXT: ld a1, %lo(.LCPI22_0)(a1)
+; RV64IZFINXZDINX-NEXT: lui a1, %hi(.LCPI24_0)
+; RV64IZFINXZDINX-NEXT: ld a1, %lo(.LCPI24_0)(a1)
; RV64IZFINXZDINX-NEXT: fabs.d a2, a0
; RV64IZFINXZDINX-NEXT: flt.d a1, a2, a1
-; RV64IZFINXZDINX-NEXT: beqz a1, .LBB22_2
+; RV64IZFINXZDINX-NEXT: beqz a1, .LBB24_2
; RV64IZFINXZDINX-NEXT: # %bb.1:
; RV64IZFINXZDINX-NEXT: fcvt.l.d a1, a0, rmm
; RV64IZFINXZDINX-NEXT: fcvt.d.l a1, a1, rmm
; RV64IZFINXZDINX-NEXT: fsgnj.d a0, a1, a0
-; RV64IZFINXZDINX-NEXT: .LBB22_2:
+; RV64IZFINXZDINX-NEXT: .LBB24_2:
; RV64IZFINXZDINX-NEXT: ret
;
; RV32I-LABEL: round_f64:
@@ -1212,16 +1290,16 @@ define double @roundeven_f64(double %a) nounwind {
;
; RV64IFD-LABEL: roundeven_f64:
; RV64IFD: # %bb.0:
-; RV64IFD-NEXT: lui a0, %hi(.LCPI23_0)
-; RV64IFD-NEXT: fld fa5, %lo(.LCPI23_0)(a0)
+; RV64IFD-NEXT: lui a0, %hi(.LCPI25_0)
+; RV64IFD-NEXT: fld fa5, %lo(.LCPI25_0)(a0)
; RV64IFD-NEXT: fabs.d fa4, fa0
; RV64IFD-NEXT: flt.d a0, fa4, fa5
-; RV64IFD-NEXT: beqz a0, .LBB23_2
+; RV64IFD-NEXT: beqz a0, .LBB25_2
; RV64IFD-NEXT: # %bb.1:
; RV64IFD-NEXT: fcvt.l.d a0, fa0, rne
; RV64IFD-NEXT: fcvt.d.l fa5, a0, rne
; RV64IFD-NEXT: fsgnj.d fa0, fa5, fa0
-; RV64IFD-NEXT: .LBB23_2:
+; RV64IFD-NEXT: .LBB25_2:
; RV64IFD-NEXT: ret
;
; RV32IZFINXZDINX-LABEL: roundeven_f64:
@@ -1235,16 +1313,16 @@ define double @roundeven_f64(double %a) nounwind {
;
; RV64IZFINXZDINX-LABEL: roundeven_f64:
; RV64IZFINXZDINX: # %bb.0:
-; RV64IZFINXZDINX-NEXT: lui a1, %hi(.LCPI23_0)
-; RV64IZFINXZDINX-NEXT: ld a1, %lo(.LCPI23_0)(a1)
+; RV64IZFINXZDINX-NEXT: lui a1, %hi(.LCPI25_0)
+; RV64IZFINXZDINX-NEXT: ld a1, %lo(.LCPI25_0)(a1)
; RV64IZFINXZDINX-NEXT: fabs.d a2, a0
; RV64IZFINXZDINX-NEXT: flt.d a1, a2, a1
-; RV64IZFINXZDINX-NEXT: beqz a1, .LBB23_2
+; RV64IZFINXZDINX-NEXT: beqz a1, .LBB25_2
; RV64IZFINXZDINX-NEXT: # %bb.1:
; RV64IZFINXZDINX-NEXT: fcvt.l.d a1, a0, rne
; RV64IZFINXZDINX-NEXT: fcvt.d.l a1, a1, rne
; RV64IZFINXZDINX-NEXT: fsgnj.d a0, a1, a0
-; RV64IZFINXZDINX-NEXT: .LBB23_2:
+; RV64IZFINXZDINX-NEXT: .LBB25_2:
; RV64IZFINXZDINX-NEXT: ret
;
; RV32I-LABEL: roundeven_f64:
@@ -1526,11 +1604,11 @@ define i1 @isnan_d_fpclass(double %x) {
; RV32I-NEXT: slli a1, a1, 1
; RV32I-NEXT: srli a1, a1, 1
; RV32I-NEXT: lui a2, 524032
-; RV32I-NEXT: beq a1, a2, .LBB29_2
+; RV32I-NEXT: beq a1, a2, .LBB31_2
; RV32I-NEXT: # %bb.1:
; RV32I-NEXT: slt a0, a2, a1
; RV32I-NEXT: ret
-; RV32I-NEXT: .LBB29_2:
+; RV32I-NEXT: .LBB31_2:
; RV32I-NEXT: snez a0, a0
; RV32I-NEXT: ret
;
diff --git a/llvm/test/CodeGen/RISCV/float-intrinsics.ll b/llvm/test/CodeGen/RISCV/float-intrinsics.ll
index a00d82942cabe..122cfcb4b868a 100644
--- a/llvm/test/CodeGen/RISCV/float-intrinsics.ll
+++ b/llvm/test/CodeGen/RISCV/float-intrinsics.ll
@@ -823,6 +823,94 @@ define float @maxnum_f32(float %a, float %b) nounwind {
; ret float %1
; }
+declare float @llvm.minimumnum.f32(float, float)
+
+define float @minimumnum_f32(float %a, float %b) nounwind {
+; RV32IF-LABEL: minimumnum_f32:
+; RV32IF: # %bb.0:
+; RV32IF-NEXT: fmin.s fa0, fa0, fa1
+; RV32IF-NEXT: ret
+;
+; RV32IZFINX-LABEL: minimumnum_f32:
+; RV32IZFINX: # %bb.0:
+; RV32IZFINX-NEXT: fmin.s a0, a0, a1
+; RV32IZFINX-NEXT: ret
+;
+; RV64IF-LABEL: minimumnum_f32:
+; RV64IF: # %bb.0:
+; RV64IF-NEXT: fmin.s fa0, fa0, fa1
+; RV64IF-NEXT: ret
+;
+; RV64IZFINX-LABEL: minimumnum_f32:
+; RV64IZFINX: # %bb.0:
+; RV64IZFINX-NEXT: fmin.s a0, a0, a1
+; RV64IZFINX-NEXT: ret
+;
+; RV32I-LABEL: minimumnum_f32:
+; RV32I: # %bb.0:
+; RV32I-NEXT: addi sp, sp, -16
+; RV32I-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
+; RV32I-NEXT: call fminimum_numf
+; RV32I-NEXT: lw ra, 12(sp) # 4-byte Folded Reload
+; RV32I-NEXT: addi sp, sp, 16
+; RV32I-NEXT: ret
+;
+; RV64I-LABEL: minimumnum_f32:
+; RV64I: # %bb.0:
+; RV64I-NEXT: addi sp, sp, -16
+; RV64I-NEXT: sd ra, 8(sp) # 8-byte Folded Spill
+; RV64I-NEXT: call fminimum_numf
+; RV64I-NEXT: ld ra, 8(sp) # 8-byte Folded Reload
+; RV64I-NEXT: addi sp, sp, 16
+; RV64I-NEXT: ret
+ %1 = call float @llvm.minimumnum.f32(float %a, float %b)
+ ret float %1
+}
+
+declare float @llvm.maximumnum.f32(float, float)
+
+define float @maximumnum_f32(float %a, float %b) nounwind {
+; RV32IF-LABEL: maximumnum_f32:
+; RV32IF: # %bb.0:
+; RV32IF-NEXT: fmax.s fa0, fa0, fa1
+; RV32IF-NEXT: ret
+;
+; RV32IZFINX-LABEL: maximumnum_f32:
+; RV32IZFINX: # %bb.0:
+; RV32IZFINX-NEXT: fmax.s a0, a0, a1
+; RV32IZFINX-NEXT: ret
+;
+; RV64IF-LABEL: maximumnum_f32:
+; RV64IF: # %bb.0:
+; RV64IF-NEXT: fmax.s fa0, fa0, fa1
+; RV64IF-NEXT: ret
+;
+; RV64IZFINX-LABEL: maximumnum_f32:
+; RV64IZFINX: # %bb.0:
+; RV64IZFINX-NEXT: fmax.s a0, a0, a1
+; RV64IZFINX-NEXT: ret
+;
+; RV32I-LABEL: maximumnum_f32:
+; RV32I: # %bb.0:
+; RV32I-NEXT: addi sp, sp, -16
+; RV32I-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
+; RV32I-NEXT: call fmaximum_numf
+; RV32I-NEXT: lw ra, 12(sp) # 4-byte Folded Reload
+; RV32I-NEXT: addi sp, sp, 16
+; RV32I-NEXT: ret
+;
+; RV64I-LABEL: maximumnum_f32:
+; RV64I: # %bb.0:
+; RV64I-NEXT: addi sp, sp, -16
+; RV64I-NEXT: sd ra, 8(sp) # 8-byte Folded Spill
+; RV64I-NEXT: call fmaximum_numf
+; RV64I-NEXT: ld ra, 8(sp) # 8-byte Folded Reload
+; RV64I-NEXT: addi sp, sp, 16
+; RV64I-NEXT: ret
+ %1 = call float @llvm.maximumnum.f32(float %a, float %b)
+ ret float %1
+}
+
declare float @llvm.copysign.f32(float, float)
define float @copysign_f32(float %a, float %b) nounwind {
@@ -876,12 +964,12 @@ define float @floor_f32(float %a) nounwind {
; RV32IF-NEXT: fmv.w.x fa5, a0
; RV32IF-NEXT: fabs.s fa4, fa0
; RV32IF-NEXT: flt.s a0, fa4, fa5
-; RV32IF-NEXT: beqz a0, .LBB17_2
+; RV32IF-NEXT: beqz a0, .LBB19_2
; RV32IF-NEXT: # %bb.1:
; RV32IF-NEXT: fcvt.w.s a0, fa0, rdn
; RV32IF-NEXT: fcvt.s.w fa5, a0, rdn
; RV32IF-NEXT: fsgnj.s fa0, fa5, fa0
-; RV32IF-NEXT: .LBB17_2:
+; RV32IF-NEXT: .LBB19_2:
; RV32IF-NEXT: ret
;
; RV32IZFINX-LABEL: floor_f32:
@@ -889,12 +977,12 @@ define float @floor_f32(float %a) nounwind {
; RV32IZFINX-NEXT: lui a1, 307200
; RV32IZFINX-NEXT: fabs.s a2, a0
; RV32IZFINX-NEXT: flt.s a1, a2, a1
-; RV32IZFINX-NEXT: beqz a1, .LBB17_2
+; RV32IZFINX-NEXT: beqz a1, .LBB19_2
; RV32IZFINX-NEXT: # %bb.1:
; RV32IZFINX-NEXT: fcvt.w.s a1, a0, rdn
; RV32IZFINX-NEXT: fcvt.s.w a1, a1, rdn
; RV32IZFINX-NEXT: fsgnj.s a0, a1, a0
-; RV32IZFINX-NEXT: .LBB17_2:
+; RV32IZFINX-NEXT: .LBB19_2:
; RV32IZFINX-NEXT: ret
;
; RV64IF-LABEL: floor_f32:
@@ -903,12 +991,12 @@ define float @floor_f32(float %a) nounwind {
; RV64IF-NEXT: fmv.w.x fa5, a0
; RV64IF-NEXT: fabs.s fa4, fa0
; RV64IF-NEXT: flt.s a0, fa4, fa5
-; RV64IF-NEXT: beqz a0, .LBB17_2
+; RV64IF-NEXT: beqz a0, .LBB19_2
; RV64IF-NEXT: # %bb.1:
; RV64IF-NEXT: fcvt.w.s a0, fa0, rdn
; RV64IF-NEXT: fcvt.s.w fa5, a0, rdn
; RV64IF-NEXT: fsgnj.s fa0, fa5, fa0
-; RV64IF-NEXT: .LBB17_2:
+; RV64IF-NEXT: .LBB19_2:
; RV64IF-NEXT: ret
;
; RV64IZFINX-LABEL: floor_f32:
@@ -916,12 +1004,12 @@ define float @floor_f32(float %a) nounwind {
; RV64IZFINX-NEXT: lui a1, 307200
; RV64IZFINX-NEXT: fabs.s a2, a0
; RV64IZFINX-NEXT: flt.s a1, a2, a1
-; RV64IZFINX-NEXT: beqz a1, .LBB17_2
+; RV64IZFINX-NEXT: beqz a1, .LBB19_2
; RV64IZFINX-NEXT: # %bb.1:
; RV64IZFINX-NEXT: fcvt.w.s a1, a0, rdn
; RV64IZFINX-NEXT: fcvt.s.w a1, a1, rdn
; RV64IZFINX-NEXT: fsgnj.s a0, a1, a0
-; RV64IZFINX-NEXT: .LBB17_2:
+; RV64IZFINX-NEXT: .LBB19_2:
; RV64IZFINX-NEXT: ret
;
; RV32I-LABEL: floor_f32:
@@ -954,12 +1042,12 @@ define float @ceil_f32(float %a) nounwind {
; RV32IF-NEXT: fmv.w.x fa5, a0
; RV32IF-NEXT: fabs.s fa4, fa0
; RV32IF-NEXT: flt.s a0, fa4, fa5
-; RV32IF-NEXT: beqz a0, .LBB18_2
+; RV32IF-NEXT: beqz a0, .LBB20_2
; RV32IF-NEXT: # %bb.1:
; RV32IF-NEXT: fcvt.w.s a0, fa0, rup
; RV32IF-NEXT: fcvt.s.w fa5, a0, rup
; RV32IF-NEXT: fsgnj.s fa0, fa5, fa0
-; RV32IF-NEXT: .LBB18_2:
+; RV32IF-NEXT: .LBB20_2:
; RV32IF-NEXT: ret
;
; RV32IZFINX-LABEL: ceil_f32:
@@ -967,12 +1055,12 @@ define float @ceil_f32(float %a) nounwind {
; RV32IZFINX-NEXT: lui a1, 307200
; RV32IZFINX-NEXT: fabs.s a2, a0
; RV32IZFINX-NEXT: flt.s a1, a2, a1
-; RV32IZFINX-NEXT: beqz a1, .LBB18_2
+; RV32IZFINX-NEXT: beqz a1, .LBB20_2
; RV32IZFINX-NEXT: # %bb.1:
; RV32IZFINX-NEXT: fcvt.w.s a1, a0, rup
; RV32IZFINX-NEXT: fcvt.s.w a1, a1, rup
; RV32IZFINX-NEXT: fsgnj.s a0, a1, a0
-; RV32IZFINX-NEXT: .LBB18_2:
+; RV32IZFINX-NEXT: .LBB20_2:
; RV32IZFINX-NEXT: ret
;
; RV64IF-LABEL: ceil_f32:
@@ -981,12 +1069,12 @@ define float @ceil_f32(float %a) nounwind {
; RV64IF-NEXT: fmv.w.x fa5, a0
; RV64IF-NEXT: fabs.s fa4, fa0
; RV64IF-NEXT: flt.s a0, fa4, fa5
-; RV64IF-NEXT: beqz a0, .LBB18_2
+; RV64IF-NEXT: beqz a0, .LBB20_2
; RV64IF-NEXT: # %bb.1:
; RV64IF-NEXT: fcvt.w.s a0, fa0, rup
; RV64IF-NEXT: fcvt.s.w fa5, a0, rup
; RV64IF-NEXT: fsgnj.s fa0, fa5, fa0
-; RV64IF-NEXT: .LBB18_2:
+; RV64IF-NEXT: .LBB20_2:
; RV64IF-NEXT: ret
;
; RV64IZFINX-LABEL: ceil_f32:
@@ -994,12 +1082,12 @@ define float @ceil_f32(float %a) nounwind {
; RV64IZFINX-NEXT: lui a1, 307200
; RV64IZFINX-NEXT: fabs.s a2, a0
; RV64IZFINX-NEXT: flt.s a1, a2, a1
-; RV64IZFINX-NEXT: beqz a1, .LBB18_2
+; RV64IZFINX-NEXT: beqz a1, .LBB20_2
; RV64IZFINX-NEXT: # %bb.1:
; RV64IZFINX-NEXT: fcvt.w.s a1, a0, rup
; RV64IZFINX-NEXT: fcvt.s.w a1, a1, rup
; RV64IZFINX-NEXT: fsgnj.s a0, a1, a0
-; RV64IZFINX-NEXT: .LBB18_2:
+; RV64IZFINX-NEXT: .LBB20_2:
; RV64IZFINX-NEXT: ret
;
; RV32I-LABEL: ceil_f32:
@@ -1032,12 +1120,12 @@ define float @trunc_f32(float %a) nounwind {
; RV32IF-NEXT: fmv.w.x fa5, a0
; RV32IF-NEXT: fabs.s fa4, fa0
; RV32IF-NEXT: flt.s a0, fa4, fa5
-; RV32IF-NEXT: beqz a0, .LBB19_2
+; RV32IF-NEXT: beqz a0, .LBB21_2
; RV32IF-NEXT: # %bb.1:
; RV32IF-NEXT: fcvt.w.s a0, fa0, rtz
; RV32IF-NEXT: fcvt.s.w fa5, a0, rtz
; RV32IF-NEXT: fsgnj.s fa0, fa5, fa0
-; RV32IF-NEXT: .LBB19_2:
+; RV32IF-NEXT: .LBB21_2:
; RV32IF-NEXT: ret
;
; RV32IZFINX-LABEL: trunc_f32:
@@ -1045,12 +1133,12 @@ define float @trunc_f32(float %a) nounwind {
; RV32IZFINX-NEXT: lui a1, 307200
; RV32IZFINX-NEXT: fabs.s a2, a0
; RV32IZFINX-NEXT: flt.s a1, a2, a1
-; RV32IZFINX-NEXT: beqz a1, .LBB19_2
+; RV32IZFINX-NEXT: beqz a1, .LBB21_2
; RV32IZFINX-NEXT: # %bb.1:
; RV32IZFINX-NEXT: fcvt.w.s a1, a0, rtz
; RV32IZFINX-NEXT: fcvt.s.w a1, a1, rtz
; RV32IZFINX-NEXT: fsgnj.s a0, a1, a0
-; RV32IZFINX-NEXT: .LBB19_2:
+; RV32IZFINX-NEXT: .LBB21_2:
; RV32IZFINX-NEXT: ret
;
; RV64IF-LABEL: trunc_f32:
@@ -1059,12 +1147,12 @@ define float @trunc_f32(float %a) nounwind {
; RV64IF-NEXT: fmv.w.x fa5, a0
; RV64IF-NEXT: fabs.s fa4, fa0
; RV64IF-NEXT: flt.s a0, fa4, fa5
-; RV64IF-NEXT: beqz a0, .LBB19_2
+; RV64IF-NEXT: beqz a0, .LBB21_2
; RV64IF-NEXT: # %bb.1:
; RV64IF-NEXT: fcvt.w.s a0, fa0, rtz
; RV64IF-NEXT: fcvt.s.w fa5, a0, rtz
; RV64IF-NEXT: fsgnj.s fa0, fa5, fa0
-; RV64IF-NEXT: .LBB19_2:
+; RV64IF-NEXT: .LBB21_2:
; RV64IF-NEXT: ret
;
; RV64IZFINX-LABEL: trunc_f32:
@@ -1072,12 +1160,12 @@ define float @trunc_f32(float %a) nounwind {
; RV64IZFINX-NEXT: lui a1, 307200
; RV64IZFINX-NEXT: fabs.s a2, a0
; RV64IZFINX-NEXT: flt.s a1, a2, a1
-; RV64IZFINX-NEXT: beqz a1, .LBB19_2
+; RV64IZFINX-NEXT: beqz a1, .LBB21_2
; RV64IZFINX-NEXT: # %bb.1:
; RV64IZFINX-NEXT: fcvt.w.s a1, a0, rtz
; RV64IZFINX-NEXT: fcvt.s.w a1, a1, rtz
; RV64IZFINX-NEXT: fsgnj.s a0, a1, a0
-; RV64IZFINX-NEXT: .LBB19_2:
+; RV64IZFINX-NEXT: .LBB21_2:
; RV64IZFINX-NEXT: ret
;
; RV32I-LABEL: trunc_f32:
@@ -1110,12 +1198,12 @@ define float @rint_f32(float %a) nounwind {
; RV32IF-NEXT: fmv.w.x fa5, a0
; RV32IF-NEXT: fabs.s fa4, fa0
; RV32IF-NEXT: flt.s a0, fa4, fa5
-; RV32IF-NEXT: beqz a0, .LBB20_2
+; RV32IF-NEXT: beqz a0, .LBB22_2
; RV32IF-NEXT: # %bb.1:
; RV32IF-NEXT: fcvt.w.s a0, fa0
; RV32IF-NEXT: fcvt.s.w fa5, a0
; RV32IF-NEXT: fsgnj.s fa0, fa5, fa0
-; RV32IF-NEXT: .LBB20_2:
+; RV32IF-NEXT: .LBB22_2:
; RV32IF-NEXT: ret
;
; RV32IZFINX-LABEL: rint_f32:
@@ -1123,12 +1211,12 @@ define float @rint_f32(float %a) nounwind {
; RV32IZFINX-NEXT: lui a1, 307200
; RV32IZFINX-NEXT: fabs.s a2, a0
; RV32IZFINX-NEXT: flt.s a1, a2, a1
-; RV32IZFINX-NEXT: beqz a1, .LBB20_2
+; RV32IZFINX-NEXT: beqz a1, .LBB22_2
; RV32IZFINX-NEXT: # %bb.1:
; RV32IZFINX-NEXT: fcvt.w.s a1, a0
; RV32IZFINX-NEXT: fcvt.s.w a1, a1
; RV32IZFINX-NEXT: fsgnj.s a0, a1, a0
-; RV32IZFINX-NEXT: .LBB20_2:
+; RV32IZFINX-NEXT: .LBB22_2:
; RV32IZFINX-NEXT: ret
;
; RV64IF-LABEL: rint_f32:
@@ -1137,12 +1225,12 @@ define float @rint_f32(float %a) nounwind {
; RV64IF-NEXT: fmv.w.x fa5, a0
; RV64IF-NEXT: fabs.s fa4, fa0
; RV64IF-NEXT: flt.s a0, fa4, fa5
-; RV64IF-NEXT: beqz a0, .LBB20_2
+; RV64IF-NEXT: beqz a0, .LBB22_2
; RV64IF-NEXT: # %bb.1:
; RV64IF-NEXT: fcvt.w.s a0, fa0
; RV64IF-NEXT: fcvt.s.w fa5, a0
; RV64IF-NEXT: fsgnj.s fa0, fa5, fa0
-; RV64IF-NEXT: .LBB20_2:
+; RV64IF-NEXT: .LBB22_2:
; RV64IF-NEXT: ret
;
; RV64IZFINX-LABEL: rint_f32:
@@ -1150,12 +1238,12 @@ define float @rint_f32(float %a) nounwind {
; RV64IZFINX-NEXT: lui a1, 307200
; RV64IZFINX-NEXT: fabs.s a2, a0
; RV64IZFINX-NEXT: flt.s a1, a2, a1
-; RV64IZFINX-NEXT: beqz a1, .LBB20_2
+; RV64IZFINX-NEXT: beqz a1, .LBB22_2
; RV64IZFINX-NEXT: # %bb.1:
; RV64IZFINX-NEXT: fcvt.w.s a1, a0
; RV64IZFINX-NEXT: fcvt.s.w a1, a1
; RV64IZFINX-NEXT: fsgnj.s a0, a1, a0
-; RV64IZFINX-NEXT: .LBB20_2:
+; RV64IZFINX-NEXT: .LBB22_2:
; RV64IZFINX-NEXT: ret
;
; RV32I-LABEL: rint_f32:
@@ -1233,12 +1321,12 @@ define float @round_f32(float %a) nounwind {
; RV32IF-NEXT: fmv.w.x fa5, a0
; RV32IF-NEXT: fabs.s fa4, fa0
; RV32IF-NEXT: flt.s a0, fa4, fa5
-; RV32IF-NEXT: beqz a0, .LBB22_2
+; RV32IF-NEXT: beqz a0, .LBB24_2
; RV32IF-NEXT: # %bb.1:
; RV32IF-NEXT: fcvt.w.s a0, fa0, rmm
; RV32IF-NEXT: fcvt.s.w fa5, a0, rmm
; RV32IF-NEXT: fsgnj.s fa0, fa5, fa0
-; RV32IF-NEXT: .LBB22_2:
+; RV32IF-NEXT: .LBB24_2:
; RV32IF-NEXT: ret
;
; RV32IZFINX-LABEL: round_f32:
@@ -1246,12 +1334,12 @@ define float @round_f32(float %a) nounwind {
; RV32IZFINX-NEXT: lui a1, 307200
; RV32IZFINX-NEXT: fabs.s a2, a0
; RV32IZFINX-NEXT: flt.s a1, a2, a1
-; RV32IZFINX-NEXT: beqz a1, .LBB22_2
+; RV32IZFINX-NEXT: beqz a1, .LBB24_2
; RV32IZFINX-NEXT: # %bb.1:
; RV32IZFINX-NEXT: fcvt.w.s a1, a0, rmm
; RV32IZFINX-NEXT: fcvt.s.w a1, a1, rmm
; RV32IZFINX-NEXT: fsgnj.s a0, a1, a0
-; RV32IZFINX-NEXT: .LBB22_2:
+; RV32IZFINX-NEXT: .LBB24_2:
; RV32IZFINX-NEXT: ret
;
; RV64IF-LABEL: round_f32:
@@ -1260,12 +1348,12 @@ define float @round_f32(float %a) nounwind {
; RV64IF-NEXT: fmv.w.x fa5, a0
; RV64IF-NEXT: fabs.s fa4, fa0
; RV64IF-NEXT: flt.s a0, fa4, fa5
-; RV64IF-NEXT: beqz a0, .LBB22_2
+; RV64IF-NEXT: beqz a0, .LBB24_2
; RV64IF-NEXT: # %bb.1:
; RV64IF-NEXT: fcvt.w.s a0, fa0, rmm
; RV64IF-NEXT: fcvt.s.w fa5, a0, rmm
; RV64IF-NEXT: fsgnj.s fa0, fa5, fa0
-; RV64IF-NEXT: .LBB22_2:
+; RV64IF-NEXT: .LBB24_2:
; RV64IF-NEXT: ret
;
; RV64IZFINX-LABEL: round_f32:
@@ -1273,12 +1361,12 @@ define float @round_f32(float %a) nounwind {
; RV64IZFINX-NEXT: lui a1, 307200
; RV64IZFINX-NEXT: fabs.s a2, a0
; RV64IZFINX-NEXT: flt.s a1, a2, a1
-; RV64IZFINX-NEXT: beqz a1, .LBB22_2
+; RV64IZFINX-NEXT: beqz a1, .LBB24_2
; RV64IZFINX-NEXT: # %bb.1:
; RV64IZFINX-NEXT: fcvt.w.s a1, a0, rmm
; RV64IZFINX-NEXT: fcvt.s.w a1, a1, rmm
; RV64IZFINX-NEXT: fsgnj.s a0, a1, a0
-; RV64IZFINX-NEXT: .LBB22_2:
+; RV64IZFINX-NEXT: .LBB24_2:
; RV64IZFINX-NEXT: ret
;
; RV32I-LABEL: round_f32:
@@ -1311,12 +1399,12 @@ define float @roundeven_f32(float %a) nounwind {
; RV32IF-NEXT: fmv.w.x fa5, a0
; RV32IF-NEXT: fabs.s fa4, fa0
; RV32IF-NEXT: flt.s a0, fa4, fa5
-; RV32IF-NEXT: beqz a0, .LBB23_2
+; RV32IF-NEXT: beqz a0, .LBB25_2
; RV32IF-NEXT: # %bb.1:
; RV32IF-NEXT: fcvt.w.s a0, fa0, rne
; RV32IF-NEXT: fcvt.s.w fa5, a0, rne
; RV32IF-NEXT: fsgnj.s fa0, fa5, fa0
-; RV32IF-NEXT: .LBB23_2:
+; RV32IF-NEXT: .LBB25_2:
; RV32IF-NEXT: ret
;
; RV32IZFINX-LABEL: roundeven_f32:
@@ -1324,12 +1412,12 @@ define float @roundeven_f32(float %a) nounwind {
; RV32IZFINX-NEXT: lui a1, 307200
; RV32IZFINX-NEXT: fabs.s a2, a0
; RV32IZFINX-NEXT: flt.s a1, a2, a1
-; RV32IZFINX-NEXT: beqz a1, .LBB23_2
+; RV32IZFINX-NEXT: beqz a1, .LBB25_2
; RV32IZFINX-NEXT: # %bb.1:
; RV32IZFINX-NEXT: fcvt.w.s a1, a0, rne
; RV32IZFINX-NEXT: fcvt.s.w a1, a1, rne
; RV32IZFINX-NEXT: fsgnj.s a0, a1, a0
-; RV32IZFINX-NEXT: .LBB23_2:
+; RV32IZFINX-NEXT: .LBB25_2:
; RV32IZFINX-NEXT: ret
;
; RV64IF-LABEL: roundeven_f32:
@@ -1338,12 +1426,12 @@ define float @roundeven_f32(float %a) nounwind {
; RV64IF-NEXT: fmv.w.x fa5, a0
; RV64IF-NEXT: fabs.s fa4, fa0
; RV64IF-NEXT: flt.s a0, fa4, fa5
-; RV64IF-NEXT: beqz a0, .LBB23_2
+; RV64IF-NEXT: beqz a0, .LBB25_2
; RV64IF-NEXT: # %bb.1:
; RV64IF-NEXT: fcvt.w.s a0, fa0, rne
; RV64IF-NEXT: fcvt.s.w fa5, a0, rne
; RV64IF-NEXT: fsgnj.s fa0, fa5, fa0
-; RV64IF-NEXT: .LBB23_2:
+; RV64IF-NEXT: .LBB25_2:
; RV64IF-NEXT: ret
;
; RV64IZFINX-LABEL: roundeven_f32:
@@ -1351,12 +1439,12 @@ define float @roundeven_f32(float %a) nounwind {
; RV64IZFINX-NEXT: lui a1, 307200
; RV64IZFINX-NEXT: fabs.s a2, a0
; RV64IZFINX-NEXT: flt.s a1, a2, a1
-; RV64IZFINX-NEXT: beqz a1, .LBB23_2
+; RV64IZFINX-NEXT: beqz a1, .LBB25_2
; RV64IZFINX-NEXT: # %bb.1:
; RV64IZFINX-NEXT: fcvt.w.s a1, a0, rne
; RV64IZFINX-NEXT: fcvt.s.w a1, a1, rne
; RV64IZFINX-NEXT: fsgnj.s a0, a1, a0
-; RV64IZFINX-NEXT: .LBB23_2:
+; RV64IZFINX-NEXT: .LBB25_2:
; RV64IZFINX-NEXT: ret
;
; RV32I-LABEL: roundeven_f32:
diff --git a/llvm/test/TableGen/GlobalISelEmitter.td b/llvm/test/TableGen/GlobalISelEmitter.td
index 5d5bf92664a79..e24e88993e2f5 100644
--- a/llvm/test/TableGen/GlobalISelEmitter.td
+++ b/llvm/test/TableGen/GlobalISelEmitter.td
@@ -513,7 +513,7 @@ def : Pat<(frag GPR32:$src1, complex:$src2, complex:$src3),
// R00O-NEXT: GIM_Reject,
// R00O: // Label [[DEFAULT_NUM]]: @[[DEFAULT]]
// R00O-NEXT: GIM_Reject,
-// R00O-NEXT: }; // Size: 1804 bytes
+// R00O-NEXT: }; // Size: 1812 bytes
def INSNBOB : I<(outs GPR32:$dst), (ins GPR32:$src1, GPR32:$src2, GPR32:$src3, GPR32:$src4),
[(set GPR32:$dst,
diff --git a/llvm/test/tools/llvm-tli-checker/ps4-tli-check.yaml b/llvm/test/tools/llvm-tli-checker/ps4-tli-check.yaml
index 95c23007b1a05..e6799f452fa30 100644
--- a/llvm/test/tools/llvm-tli-checker/ps4-tli-check.yaml
+++ b/llvm/test/tools/llvm-tli-checker/ps4-tli-check.yaml
@@ -32,14 +32,20 @@
# RUN: FileCheck %s --check-prefix=AVAIL --input-file %t3.txt
# RUN: FileCheck %s --check-prefix=UNAVAIL --input-file %t3.txt
#
-# CHECK: << Total TLI yes SDK no: 8
+# CHECK: << Total TLI yes SDK no: 14
# CHECK: >> Total TLI no SDK yes: 0
# CHECK: == Total TLI yes SDK yes: 239
#
# WRONG_DETAIL: << TLI yes SDK no : '_ZdaPv' aka operator delete[](void*)
# WRONG_DETAIL: >> TLI no SDK yes: '_ZdaPvj' aka operator delete[](void*, unsigned int)
# WRONG_DETAIL-COUNT-8: << TLI yes SDK no : {{.*}}__hot_cold_t
-# WRONG_SUMMARY: << Total TLI yes SDK no: 9{{$}}
+# WRONG_DETAIL: << TLI yes SDK no : 'fmaximum_num'
+# WRONG_DETAIL: << TLI yes SDK no : 'fmaximum_numf'
+# WRONG_DETAIL: << TLI yes SDK no : 'fmaximum_numl'
+# WRONG_DETAIL: << TLI yes SDK no : 'fminimum_num'
+# WRONG_DETAIL: << TLI yes SDK no : 'fminimum_numf'
+# WRONG_DETAIL: << TLI yes SDK no : 'fminimum_numl'
+# WRONG_SUMMARY: << Total TLI yes SDK no: 15{{$}}
# WRONG_SUMMARY: >> Total TLI no SDK yes: 1{{$}}
# WRONG_SUMMARY: == Total TLI yes SDK yes: 238
#
@@ -47,8 +53,8 @@
## the exact count first; the two directives should add up to that.
## Yes, this means additions to TLI will fail this test, but the argument
## to -COUNT can't be an expression.
-# AVAIL: TLI knows 480 symbols, 247 available
-# AVAIL-COUNT-247: {{^}} available
+# AVAIL: TLI knows 486 symbols, 253 available
+# AVAIL-COUNT-253: {{^}} available
# AVAIL-NOT: {{^}} available
# UNAVAIL-COUNT-233: not available
# UNAVAIL-NOT: not available
diff --git a/llvm/unittests/ADT/APFloatTest.cpp b/llvm/unittests/ADT/APFloatTest.cpp
index f6af4b0e5f651..a105cd81d7431 100644
--- a/llvm/unittests/ADT/APFloatTest.cpp
+++ b/llvm/unittests/ADT/APFloatTest.cpp
@@ -631,6 +631,54 @@ TEST(APFloatTest, Maximum) {
EXPECT_TRUE(std::isnan(maximum(nan, f1).convertToDouble()));
}
+TEST(APFloatTest, MinimumNumber) {
+ APFloat f1(1.0);
+ APFloat f2(2.0);
+ APFloat zp(0.0);
+ APFloat zn(-0.0);
+ APFloat nan = APFloat::getNaN(APFloat::IEEEdouble());
+ APFloat snan = APFloat::getSNaN(APFloat::IEEEdouble());
+
+ EXPECT_EQ(1.0, minimumnum(f1, f2).convertToDouble());
+ EXPECT_EQ(1.0, minimumnum(f2, f1).convertToDouble());
+ EXPECT_EQ(-0.0, minimumnum(zp, zn).convertToDouble());
+ EXPECT_EQ(-0.0, minimumnum(zn, zp).convertToDouble());
+ EXPECT_TRUE(minimumnum(zn, zp).isNegative());
+ EXPECT_TRUE(minimumnum(zp, zn).isNegative());
+ EXPECT_TRUE(minimumnum(zn, zn).isNegative());
+ EXPECT_FALSE(minimumnum(zp, zp).isNegative());
+ EXPECT_FALSE(std::isnan(minimumnum(f1, nan).convertToDouble()));
+ EXPECT_FALSE(std::isnan(minimumnum(nan, f1).convertToDouble()));
+ EXPECT_FALSE(std::isnan(minimumnum(f1, snan).convertToDouble()));
+ EXPECT_FALSE(std::isnan(minimumnum(snan, f1).convertToDouble()));
+ EXPECT_FALSE(minimumnum(snan, nan).isSignaling());
+ EXPECT_FALSE(minimumnum(snan, snan).isSignaling());
+}
+
+TEST(APFloatTest, MaximumNumber) {
+ APFloat f1(1.0);
+ APFloat f2(2.0);
+ APFloat zp(0.0);
+ APFloat zn(-0.0);
+ APFloat nan = APFloat::getNaN(APFloat::IEEEdouble());
+ APFloat snan = APFloat::getSNaN(APFloat::IEEEdouble());
+
+ EXPECT_EQ(2.0, maximumnum(f1, f2).convertToDouble());
+ EXPECT_EQ(2.0, maximumnum(f2, f1).convertToDouble());
+ EXPECT_EQ(0.0, maximumnum(zp, zn).convertToDouble());
+ EXPECT_EQ(0.0, maximumnum(zn, zp).convertToDouble());
+ EXPECT_FALSE(maximumnum(zn, zp).isNegative());
+ EXPECT_FALSE(maximumnum(zp, zn).isNegative());
+ EXPECT_TRUE(maximumnum(zn, zn).isNegative());
+ EXPECT_FALSE(maximumnum(zp, zp).isNegative());
+ EXPECT_FALSE(std::isnan(maximumnum(f1, nan).convertToDouble()));
+ EXPECT_FALSE(std::isnan(maximumnum(nan, f1).convertToDouble()));
+ EXPECT_FALSE(std::isnan(maximumnum(f1, snan).convertToDouble()));
+ EXPECT_FALSE(std::isnan(maximumnum(snan, f1).convertToDouble()));
+ EXPECT_FALSE(maximumnum(snan, nan).isSignaling());
+ EXPECT_FALSE(maximumnum(snan, snan).isSignaling());
+}
+
TEST(APFloatTest, Denormal) {
APFloat::roundingMode rdmd = APFloat::rmNearestTiesToEven;
diff --git a/llvm/unittests/Analysis/TargetLibraryInfoTest.cpp b/llvm/unittests/Analysis/TargetLibraryInfoTest.cpp
index 1fe94e2aae059..5192ebdbe1dd1 100644
--- a/llvm/unittests/Analysis/TargetLibraryInfoTest.cpp
+++ b/llvm/unittests/Analysis/TargetLibraryInfoTest.cpp
@@ -188,6 +188,12 @@ TEST_F(TargetLibraryInfoTest, ValidProto) {
"declare double @fmin(double, double)\n"
"declare float @fminf(float, float)\n"
"declare x86_fp80 @fminl(x86_fp80, x86_fp80)\n"
+ "declare double @fmaximum_num(double, double)\n"
+ "declare float @fmaximum_numf(float, float)\n"
+ "declare x86_fp80 @fmaximum_numl(x86_fp80, x86_fp80)\n"
+ "declare double @fminimum_num(double, double)\n"
+ "declare float @fminimum_numf(float, float)\n"
+ "declare x86_fp80 @fminimum_numl(x86_fp80, x86_fp80)\n"
"declare double @fmod(double, double)\n"
"declare float @fmodf(float, float)\n"
"declare x86_fp80 @fmodl(x86_fp80, x86_fp80)\n"
@@ -660,4 +666,4 @@ class TLITestAarch64 : public ::testing::Test {
TEST_F(TLITestAarch64, TestFrem) {
EXPECT_EQ(getScalarName(Instruction::FRem, Type::getDoubleTy(Ctx)), "fmod");
EXPECT_EQ(getScalarName(Instruction::FRem, Type::getFloatTy(Ctx)), "fmodf");
-}
\ No newline at end of file
+}
diff --git a/llvm/unittests/IR/VPIntrinsicTest.cpp b/llvm/unittests/IR/VPIntrinsicTest.cpp
index d6508abd5197e..e6390678f7329 100644
--- a/llvm/unittests/IR/VPIntrinsicTest.cpp
+++ b/llvm/unittests/IR/VPIntrinsicTest.cpp
@@ -27,8 +27,9 @@ namespace {
static const char *ReductionIntOpcodes[] = {
"add", "mul", "and", "or", "xor", "smin", "smax", "umin", "umax"};
-static const char *ReductionFPOpcodes[] = {"fadd", "fmul", "fmin",
- "fmax", "fminimum", "fmaximum"};
+static const char *ReductionFPOpcodes[] = {
+ "fadd", "fmul", "fmin", "fmax",
+ "fminimum", "fmaximum", "fminimumnum", "fmaximumnum"};
class VPIntrinsicTest : public testing::Test {
protected:
@@ -49,9 +50,9 @@ class VPIntrinsicTest : public testing::Test {
Str << " declare <8 x i32> @llvm.vp." << BinaryIntOpcode
<< ".v8i32(<8 x i32>, <8 x i32>, <8 x i1>, i32) ";
- const char *BinaryFPOpcodes[] = {"fadd", "fsub", "fmul", "fdiv",
- "frem", "minnum", "maxnum", "minimum",
- "maximum", "copysign"};
+ const char *BinaryFPOpcodes[] = {
+ "fadd", "fsub", "fmul", "fdiv", "frem", "minnum",
+ "maxnum", "minimum", "maximum", "minimumnum", "maximumnum", "copysign"};
for (const char *BinaryFPOpcode : BinaryFPOpcodes)
Str << " declare <8 x float> @llvm.vp." << BinaryFPOpcode
<< ".v8f32(<8 x float>, <8 x float>, <8 x i1>, i32) ";
More information about the llvm-commits
mailing list