[llvm] [LLVM] Add `llvm.vector.reduce.{min, max}imumnum` (PR #218295)
Folkert de Vries via llvm-commits
llvm-commits at lists.llvm.org
Sun Aug 23 15:22:00 PDT 2026
https://github.com/folkertdev created https://github.com/llvm/llvm-project/pull/218295
fixes https://github.com/llvm/llvm-project/issues/185827
Really this is just a mechanical addition, using the fallback mechanism for reductions (i.e. no custom lowerings yet).
>From fe54e9b871e98a663aedfb425472406af4e897bf Mon Sep 17 00:00:00 2001
From: Folkert de Vries <folkert at folkertdev.nl>
Date: Sun, 23 Aug 2026 17:46:54 +0200
Subject: [PATCH 1/2] add `reduce_{min, max}imumnum` intrinsic
---
llvm/docs/LangRef.md | 66 ++
llvm/docs/ReleaseNotes.md | 3 +
llvm/include/llvm/CodeGen/BasicTTIImpl.h | 4 +
llvm/include/llvm/CodeGen/ISDOpcodes.h | 4 +
llvm/include/llvm/IR/IRBuilder.h | 10 +
llvm/include/llvm/IR/Intrinsics.td | 6 +
.../include/llvm/Target/TargetSelectionDAG.td | 4 +
llvm/lib/Analysis/ValueTracking.cpp | 4 +-
llvm/lib/CodeGen/ExpandReductions.cpp | 8 +-
llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp | 22 +-
llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp | 10 +-
.../SelectionDAG/LegalizeFloatTypes.cpp | 6 +-
.../SelectionDAG/LegalizeVectorOps.cpp | 6 +
.../SelectionDAG/LegalizeVectorTypes.cpp | 6 +
.../lib/CodeGen/SelectionDAG/SelectionDAG.cpp | 16 +-
.../SelectionDAG/SelectionDAGBuilder.cpp | 8 +
.../SelectionDAG/SelectionDAGDumper.cpp | 2 +
llvm/lib/CodeGen/TargetLoweringBase.cpp | 1 +
llvm/lib/IR/IRBuilder.cpp | 8 +
llvm/lib/Transforms/Utils/LoopUtils.cpp | 10 +
.../CodeGen/AArch64/vecreduce-fmaximumnum.ll | 632 +++++++++++++++++
.../CodeGen/AArch64/vecreduce-fminimumnum.ll | 636 ++++++++++++++++++
.../Generic/expand-experimental-reductions.ll | 30 +
.../test/tools/llubi/intr_fp_vector_reduce.ll | 33 +
llvm/tools/llubi/lib/Interpreter.cpp | 10 +-
25 files changed, 1526 insertions(+), 19 deletions(-)
create mode 100644 llvm/test/CodeGen/AArch64/vecreduce-fmaximumnum.ll
create mode 100644 llvm/test/CodeGen/AArch64/vecreduce-fminimumnum.ll
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index e77076bb68158..c21d0c5efdf8b 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -20389,6 +20389,72 @@ than +0.0. If any element of the vector is a NaN, the result is NaN.
##### Arguments:
The argument to this intrinsic must be a vector of floating-point values.
+(int_vector_reduce_fmaximumnum)=
+
+#### '`llvm.vector.reduce.fmaximumnum.*`' Intrinsic
+
+##### Syntax:
+This is an overloaded intrinsic.
+
+```
+declare float @llvm.vector.reduce.fmaximumnum.v4f32(<4 x float> %a)
+declare double @llvm.vector.reduce.fmaximumnum.v2f64(<2 x double> %a)
+```
+
+##### Overview:
+
+The '`llvm.vector.reduce.fmaximumnum.*`' intrinsics do a floating-point
+`MAX` reduction of a vector, returning the result as a scalar. The return type
+matches the element-type of the vector input.
+
+This instruction has the same comparison and `nsz` semantics as the
+'`llvm.maximumnum.*`' intrinsic:
+
+- this intrinsic does not propagate NaNs: the result is a NaN only if every
+ element of the input vector is a NaN.
+- By default -0.0 is considered less than +0.0, so if the maximum is a zero,
+ the result is `-0.0` when every zero in the vector is `-0.0`, and `+0.0`
+ otherwise.
+- If the `nsz` flag is specified and the result is a zero, its sign is
+ unspecified when the vector contains zeros of both signs: if every zero in
+ the vector has the same sign, the result still has that sign.
+
+##### Arguments:
+The argument to this intrinsic must be a vector of floating-point values.
+
+(int_vector_reduce_fminimumnum)=
+
+#### '`llvm.vector.reduce.fminimumnum.*`' Intrinsic
+
+##### Syntax:
+This is an overloaded intrinsic.
+
+```
+declare float @llvm.vector.reduce.fminimumnum.v4f32(<4 x float> %a)
+declare double @llvm.vector.reduce.fminimumnum.v2f64(<2 x double> %a)
+```
+
+##### Overview:
+
+The '`llvm.vector.reduce.fminimumnum.*`' intrinsics do a floating-point
+`MIN` reduction of a vector, returning the result as a scalar. The return type
+matches the element-type of the vector input.
+
+This instruction has the same comparison and `nsz` semantics as the
+'`llvm.minimumnum.*`' intrinsic:
+
+- this intrinsic does not propagate NaNs: the result is a NaN only if every
+ element of the input vector is a NaN.
+- By default -0.0 is considered less than +0.0, so if the minimum is a zero,
+ the result is `+0.0` when every zero in the vector is `+0.0`, and `-0.0`
+ otherwise.
+- If the `nsz` flag is specified and the result is a zero, its sign is
+ unspecified when the vector contains zeros of both signs: if every zero in
+ the vector has the same sign, the result still has that sign.
+
+##### Arguments:
+The argument to this intrinsic must be a vector of floating-point values.
+
### Vector Partial Reduction Intrinsics
Partial reductions of vectors can be expressed using the intrinsics described in
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index 051d726b7aa8d..e42c6e959a0b3 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -52,6 +52,9 @@ Makes programs 10x faster by doing Special New Thing.
### Changes to the LLVM IR
+* Added `llvm.vector.reduce.fmaximumnum` and `llvm.vector.reduce.fminimumnum`
+ intrinsics, the reduction variants of `llvm.maximumnum` and
+ `llvm.minimumnum`.
* Added `nofreeobj` attribute for attributes and returns, which forbids
freeing the underlying object (as opposed to only frees through that specific
pointer). Renamed `!nofree` metadata to `!nofreeobj`, as it has the same
diff --git a/llvm/include/llvm/CodeGen/BasicTTIImpl.h b/llvm/include/llvm/CodeGen/BasicTTIImpl.h
index 87ad9d6516680..59e52a7e5127b 100644
--- a/llvm/include/llvm/CodeGen/BasicTTIImpl.h
+++ b/llvm/include/llvm/CodeGen/BasicTTIImpl.h
@@ -2086,6 +2086,8 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> {
case Intrinsic::vector_reduce_fmin:
case Intrinsic::vector_reduce_fmaximum:
case Intrinsic::vector_reduce_fminimum:
+ case Intrinsic::vector_reduce_fmaximumnum:
+ case Intrinsic::vector_reduce_fminimumnum:
case Intrinsic::vector_reduce_umax:
case Intrinsic::vector_reduce_umin: {
IntrinsicCostAttributes Attrs(IID, RetTy, Args[0]->getType(), FMF, I, 1);
@@ -2546,6 +2548,8 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> {
case Intrinsic::vector_reduce_fmin:
case Intrinsic::vector_reduce_fmaximum:
case Intrinsic::vector_reduce_fminimum:
+ case Intrinsic::vector_reduce_fmaximumnum:
+ case Intrinsic::vector_reduce_fminimumnum:
return thisT()->getMinMaxReductionCost(getMinMaxReductionIntrinsicOp(IID),
VecOpTy, ICA.getFlags(), CostKind);
case Intrinsic::experimental_vector_match: {
diff --git a/llvm/include/llvm/CodeGen/ISDOpcodes.h b/llvm/include/llvm/CodeGen/ISDOpcodes.h
index 0ce6805ae8f5a..9c3757203299e 100644
--- a/llvm/include/llvm/CodeGen/ISDOpcodes.h
+++ b/llvm/include/llvm/CodeGen/ISDOpcodes.h
@@ -1537,6 +1537,10 @@ enum NodeType {
/// llvm.minimum and llvm.maximum semantics.
VECREDUCE_FMAXIMUM,
VECREDUCE_FMINIMUM,
+ /// FMINIMUMNUM/FMAXIMUMNUM nodes do not propagate NaNs and order signed
+ /// zeroes using the llvm.minimumnum and llvm.maximumnum semantics.
+ VECREDUCE_FMAXIMUMNUM,
+ VECREDUCE_FMINIMUMNUM,
/// Integer reductions may have a result type larger than the vector element
/// type. However, the reduction is performed using the vector element type
/// and the value in the top bits is unspecified.
diff --git a/llvm/include/llvm/IR/IRBuilder.h b/llvm/include/llvm/IR/IRBuilder.h
index f2621bbe298df..03c1c777a5484 100644
--- a/llvm/include/llvm/IR/IRBuilder.h
+++ b/llvm/include/llvm/IR/IRBuilder.h
@@ -789,6 +789,16 @@ class IRBuilderBase {
/// llvm.minimum intrinsic.
LLVM_ABI Value *CreateFPMinimumReduce(Value *Src);
+ /// Create a vector float maximum reduction intrinsic of the source
+ /// vector. This variant follows the NaN and signed zero semantic of
+ /// llvm.maximumnum intrinsic.
+ LLVM_ABI Value *CreateFPMaximumNumReduce(Value *Src);
+
+ /// Create a vector float minimum reduction intrinsic of the source
+ /// vector. This variant follows the NaN and signed zero semantic of
+ /// llvm.minimumnum intrinsic.
+ LLVM_ABI Value *CreateFPMinimumNumReduce(Value *Src);
+
/// Create a lifetime.start intrinsic.
LLVM_ABI CallInst *CreateLifetimeStart(Value *Ptr);
diff --git a/llvm/include/llvm/IR/Intrinsics.td b/llvm/include/llvm/IR/Intrinsics.td
index 12211799e358c..2d721a77189ae 100644
--- a/llvm/include/llvm/IR/Intrinsics.td
+++ b/llvm/include/llvm/IR/Intrinsics.td
@@ -2627,6 +2627,12 @@ let IntrProperties = [IntrNoMem, IntrSpeculatable, IntrNoCreateUndefOrPoison]
def int_vector_reduce_fmaximum: DefaultAttrsIntrinsic<
[LLVMVectorElementType<0>],
[llvm_any_vector_float_ty]>;
+ def int_vector_reduce_fminimumnum: DefaultAttrsIntrinsic<
+ [LLVMVectorElementType<0>],
+ [llvm_any_vector_float_ty]>;
+ def int_vector_reduce_fmaximumnum: DefaultAttrsIntrinsic<
+ [LLVMVectorElementType<0>],
+ [llvm_any_vector_float_ty]>;
}
//===------ Matrix intrinsics ---------------------------------------------===//
diff --git a/llvm/include/llvm/Target/TargetSelectionDAG.td b/llvm/include/llvm/Target/TargetSelectionDAG.td
index 06832d25e7400..ec814cd43f6ee 100644
--- a/llvm/include/llvm/Target/TargetSelectionDAG.td
+++ b/llvm/include/llvm/Target/TargetSelectionDAG.td
@@ -573,6 +573,10 @@ def vecreduce_fmin : SDNode<"ISD::VECREDUCE_FMIN", SDTFPVecReduce>;
def vecreduce_fmax : SDNode<"ISD::VECREDUCE_FMAX", SDTFPVecReduce>;
def vecreduce_fminimum : SDNode<"ISD::VECREDUCE_FMINIMUM", SDTFPVecReduce>;
def vecreduce_fmaximum : SDNode<"ISD::VECREDUCE_FMAXIMUM", SDTFPVecReduce>;
+def vecreduce_fminimumnum : SDNode<"ISD::VECREDUCE_FMINIMUMNUM",
+ SDTFPVecReduce>;
+def vecreduce_fmaximumnum : SDNode<"ISD::VECREDUCE_FMAXIMUMNUM",
+ SDTFPVecReduce>;
def partial_reduce_umla : SDNode<"ISD::PARTIAL_REDUCE_UMLA",
SDTPartialReduceMLA>;
diff --git a/llvm/lib/Analysis/ValueTracking.cpp b/llvm/lib/Analysis/ValueTracking.cpp
index f4ed5e07038da..d31bc794a1b44 100644
--- a/llvm/lib/Analysis/ValueTracking.cpp
+++ b/llvm/lib/Analysis/ValueTracking.cpp
@@ -5496,7 +5496,9 @@ void computeKnownFPClass(const Value *V, const APInt &DemandedElts,
case Intrinsic::vector_reduce_fmax:
case Intrinsic::vector_reduce_fmin:
case Intrinsic::vector_reduce_fmaximum:
- case Intrinsic::vector_reduce_fminimum: {
+ case Intrinsic::vector_reduce_fminimum:
+ case Intrinsic::vector_reduce_fmaximumnum:
+ case Intrinsic::vector_reduce_fminimumnum: {
// reduce min/max will choose an element from one of the vector elements,
// so we can infer and class information that is common to all elements.
Known = computeKnownFPClass(II->getArgOperand(0), II->getFastMathFlags(),
diff --git a/llvm/lib/CodeGen/ExpandReductions.cpp b/llvm/lib/CodeGen/ExpandReductions.cpp
index d075a677e0586..1cc4d16c95546 100644
--- a/llvm/lib/CodeGen/ExpandReductions.cpp
+++ b/llvm/lib/CodeGen/ExpandReductions.cpp
@@ -51,7 +51,9 @@ bool expandReductions(Function &F, const TargetTransformInfo *TTI,
case Intrinsic::vector_reduce_fmax:
case Intrinsic::vector_reduce_fmin:
case Intrinsic::vector_reduce_fmaximum:
- case Intrinsic::vector_reduce_fminimum: {
+ case Intrinsic::vector_reduce_fminimum:
+ case Intrinsic::vector_reduce_fmaximumnum:
+ case Intrinsic::vector_reduce_fminimumnum: {
// Only expand if the target doesn't support this operation natively.
if (TTI->shouldExpandReduction(II))
Worklist.push_back(II);
@@ -163,7 +165,9 @@ bool expandReductions(Function &F, const TargetTransformInfo *TTI,
break;
}
case Intrinsic::vector_reduce_fmaximum:
- case Intrinsic::vector_reduce_fminimum: {
+ case Intrinsic::vector_reduce_fminimum:
+ case Intrinsic::vector_reduce_fmaximumnum:
+ case Intrinsic::vector_reduce_fminimumnum: {
Value *Vec = II->getArgOperand(0);
if (!isPowerOf2_32(
cast<FixedVectorType>(Vec->getType())->getNumElements()))
diff --git a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
index 09e26fe2aa58c..0eaef77519e39 100644
--- a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
@@ -2143,7 +2143,9 @@ SDValue DAGCombiner::visit(SDNode *N) {
case ISD::VECREDUCE_FMAX:
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAXIMUM:
- case ISD::VECREDUCE_FMINIMUM: return visitVECREDUCE(N);
+ case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM: return visitVECREDUCE(N);
#define BEGIN_REGISTER_VP_SDNODE(SDOPC, ...) case ISD::SDOPC:
#include "llvm/IR/VPIntrinsics.def"
return visitVPOp(N);
@@ -21071,15 +21073,17 @@ SDValue DAGCombiner::visitFMinMax(SDNode *N) {
}
}
- // There are no VECREDUCE variants of FMINIMUMNUM or FMAXIMUMNUM
- if (Opc == ISD::FMINIMUMNUM || Opc == ISD::FMAXIMUMNUM)
- return SDValue();
+ unsigned ReduceOpc;
+ if (PropAllNaNsToQNaNs)
+ ReduceOpc = IsMin ? ISD::VECREDUCE_FMINIMUM : ISD::VECREDUCE_FMAXIMUM;
+ else if (PropOnlySNaNsToQNaNs)
+ ReduceOpc = IsMin ? ISD::VECREDUCE_FMIN : ISD::VECREDUCE_FMAX;
+ else
+ ReduceOpc =
+ IsMin ? ISD::VECREDUCE_FMINIMUMNUM : ISD::VECREDUCE_FMAXIMUMNUM;
- if (SDValue SD = reassociateReduction(
- PropAllNaNsToQNaNs
- ? (IsMin ? ISD::VECREDUCE_FMINIMUM : ISD::VECREDUCE_FMAXIMUM)
- : (IsMin ? ISD::VECREDUCE_FMIN : ISD::VECREDUCE_FMAX),
- Opc, SDLoc(N), VT, N0, N1, Flags))
+ if (SDValue SD = reassociateReduction(ReduceOpc, Opc, SDLoc(N), VT, N0, N1,
+ Flags))
return SD;
return SDValue();
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
index 70e094bc0d255..7ce8d9e86516c 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
@@ -1238,6 +1238,8 @@ void SelectionDAGLegalize::LegalizeOp(SDNode *Node) {
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
case ISD::IS_FPCLASS:
Action = TLI.getOperationAction(
Node->getOpcode(), Node->getOperand(0).getValueType());
@@ -4643,6 +4645,8 @@ bool SelectionDAGLegalize::ExpandNode(SDNode *Node) {
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
Results.push_back(TLI.expandVecReduce(Node, DAG));
break;
case ISD::VP_CTTZ_ELTS:
@@ -5465,7 +5469,9 @@ void SelectionDAGLegalize::PromoteNode(SDNode *Node) {
Node->getOpcode() == ISD::VECREDUCE_FMAX ||
Node->getOpcode() == ISD::VECREDUCE_FMIN ||
Node->getOpcode() == ISD::VECREDUCE_FMAXIMUM ||
- Node->getOpcode() == ISD::VECREDUCE_FMINIMUM) {
+ Node->getOpcode() == ISD::VECREDUCE_FMINIMUM ||
+ Node->getOpcode() == ISD::VECREDUCE_FMAXIMUMNUM ||
+ Node->getOpcode() == ISD::VECREDUCE_FMINIMUMNUM) {
OVT = Node->getOperand(0).getSimpleValueType();
}
if (Node->getOpcode() == ISD::ATOMIC_STORE ||
@@ -6302,6 +6308,8 @@ void SelectionDAGLegalize::PromoteNode(SDNode *Node) {
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
case ISD::VP_REDUCE_FMAX:
case ISD::VP_REDUCE_FMIN:
case ISD::VP_REDUCE_FMAXIMUM:
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp
index 9727fef74dafb..17d3c34e75194 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp
@@ -176,7 +176,9 @@ void DAGTypeLegalizer::SoftenFloatResult(SDNode *N, unsigned ResNo) {
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAX:
case ISD::VECREDUCE_FMAXIMUM:
- case ISD::VECREDUCE_FMINIMUM: R = SoftenFloatRes_VECREDUCE(N); break;
+ case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM: R = SoftenFloatRes_VECREDUCE(N); break;
case ISD::VECREDUCE_SEQ_FADD:
case ISD::VECREDUCE_SEQ_FMUL: R = SoftenFloatRes_VECREDUCE_SEQ(N); break;
// clang-format on
@@ -2588,6 +2590,8 @@ void DAGTypeLegalizer::SoftPromoteHalfResult(SDNode *N, unsigned ResNo) {
case ISD::VECREDUCE_FMAX:
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
R = SoftPromoteHalfRes_VECREDUCE(N);
break;
case ISD::VECREDUCE_SEQ_FADD:
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp
index f85e6e58b88c8..ab936ca725976 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp
@@ -525,6 +525,8 @@ SDValue VectorLegalizer::LegalizeOp(SDValue Op) {
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
case ISD::VECREDUCE_FMUL:
case ISD::CTTZ_ELTS:
case ISD::CTTZ_ELTS_ZERO_POISON:
@@ -793,6 +795,8 @@ void VectorLegalizer::Promote(SDNode *Node, SmallVectorImpl<SDValue> &Results) {
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
PromoteFloatVECREDUCE(Node, Results, /*NonArithmetic=*/true);
return;
case ISD::VECTOR_COMPRESS:
@@ -1313,6 +1317,8 @@ void VectorLegalizer::Expand(SDNode *Node, SmallVectorImpl<SDValue> &Results) {
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
Results.push_back(TLI.expandVecReduce(Node, DAG));
return;
case ISD::PARTIAL_REDUCE_UMLA:
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp
index 8a2cf2fd81527..84dfd6fde0221 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp
@@ -967,6 +967,8 @@ bool DAGTypeLegalizer::ScalarizeVectorOperand(SDNode *N, unsigned OpNo) {
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
Res = ScalarizeVecOp_VECREDUCE(N);
break;
case ISD::VECREDUCE_SEQ_FADD:
@@ -3996,6 +3998,8 @@ bool DAGTypeLegalizer::SplitVectorOperand(SDNode *N, unsigned OpNo) {
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
Res = SplitVecOp_VECREDUCE(N, OpNo);
break;
case ISD::VECREDUCE_SEQ_FADD:
@@ -7898,6 +7902,8 @@ bool DAGTypeLegalizer::WidenVectorOperand(SDNode *N, unsigned OpNo) {
case ISD::VECREDUCE_FMIN:
case ISD::VECREDUCE_FMAXIMUM:
case ISD::VECREDUCE_FMINIMUM:
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ case ISD::VECREDUCE_FMINIMUMNUM:
Res = WidenVecOp_VECREDUCE(N);
break;
case ISD::VECREDUCE_SEQ_FADD:
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
index 00cef99069347..d2d6c7db87024 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
@@ -515,6 +515,10 @@ ISD::NodeType ISD::getVecReduceBaseOpcode(unsigned VecReduceOpcode) {
case ISD::VECREDUCE_FMINIMUM:
case ISD::VP_REDUCE_FMINIMUM:
return ISD::FMINIMUM;
+ case ISD::VECREDUCE_FMAXIMUMNUM:
+ return ISD::FMAXIMUMNUM;
+ case ISD::VECREDUCE_FMINIMUMNUM:
+ return ISD::FMINIMUMNUM;
}
}
@@ -13946,14 +13950,16 @@ bool SelectionDAG::isIdentityElement(unsigned Opcode, SDNodeFlags Flags,
case ISD::FDIV:
return OperandNo == 1 && ConstFP->isOne();
case ISD::FMINNUM:
- case ISD::FMAXNUM: {
+ case ISD::FMAXNUM:
+ case ISD::FMINIMUMNUM:
+ case ISD::FMAXIMUMNUM: {
// Neutral element for fminnum is NaN, Inf or FLT_MAX, depending on FMF.
EVT VT = V.getValueType();
const fltSemantics &Semantics = VT.getFltSemantics();
APFloat NeutralAF = !Flags.hasNoNaNs() ? APFloat::getQNaN(Semantics)
: !Flags.hasNoInfs() ? APFloat::getInf(Semantics)
: APFloat::getLargest(Semantics);
- if (Opcode == ISD::FMAXNUM)
+ if (Opcode == ISD::FMAXNUM || Opcode == ISD::FMAXIMUMNUM)
NeutralAF.changeSign();
return ConstFP->isExactlyValue(NeutralAF);
@@ -15296,13 +15302,15 @@ SDValue SelectionDAG::getIdentityElement(unsigned Opcode, const SDLoc &DL,
case ISD::FMUL:
return getConstantFP(1.0, DL, VT);
case ISD::FMINNUM:
- case ISD::FMAXNUM: {
+ case ISD::FMAXNUM:
+ case ISD::FMINIMUMNUM:
+ case ISD::FMAXIMUMNUM: {
// Neutral element for fminnum is NaN, Inf or FLT_MAX, depending on FMF.
const fltSemantics &Semantics = VT.getFltSemantics();
APFloat NeutralAF = !Flags.hasNoNaNs() ? APFloat::getQNaN(Semantics) :
!Flags.hasNoInfs() ? APFloat::getInf(Semantics) :
APFloat::getLargest(Semantics);
- if (Opcode == ISD::FMAXNUM)
+ if (Opcode == ISD::FMAXNUM || Opcode == ISD::FMAXIMUMNUM)
NeutralAF.changeSign();
return getConstantFP(NeutralAF, DL, VT);
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index 81062dd6d3a11..2c754e9c97807 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -8206,6 +8206,8 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
case Intrinsic::vector_reduce_fmin:
case Intrinsic::vector_reduce_fmaximum:
case Intrinsic::vector_reduce_fminimum:
+ case Intrinsic::vector_reduce_fmaximumnum:
+ case Intrinsic::vector_reduce_fminimumnum:
visitVectorReduce(I, Intrinsic);
return;
@@ -11346,6 +11348,12 @@ void SelectionDAGBuilder::visitVectorReduce(const CallInst &I,
case Intrinsic::vector_reduce_fminimum:
Res = DAG.getNode(ISD::VECREDUCE_FMINIMUM, dl, VT, Op1, SDFlags);
break;
+ case Intrinsic::vector_reduce_fmaximumnum:
+ Res = DAG.getNode(ISD::VECREDUCE_FMAXIMUMNUM, dl, VT, Op1, SDFlags);
+ break;
+ case Intrinsic::vector_reduce_fminimumnum:
+ Res = DAG.getNode(ISD::VECREDUCE_FMINIMUMNUM, dl, VT, Op1, SDFlags);
+ break;
default:
llvm_unreachable("Unhandled vector reduce intrinsic");
}
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp
index 5c1b280697262..e5dc0d0f8a1db 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp
@@ -588,6 +588,8 @@ std::string SDNode::getOperationName(const SelectionDAG *G) const {
case ISD::VECREDUCE_FMAX: return "vecreduce_fmax";
case ISD::VECREDUCE_FMIN: return "vecreduce_fmin";
case ISD::VECREDUCE_FMAXIMUM: return "vecreduce_fmaximum";
+ case ISD::VECREDUCE_FMINIMUMNUM: return "vecreduce_fminimumnum";
+ case ISD::VECREDUCE_FMAXIMUMNUM: return "vecreduce_fmaximumnum";
case ISD::VECREDUCE_FMINIMUM: return "vecreduce_fminimum";
case ISD::STACKMAP:
return "stackmap";
diff --git a/llvm/lib/CodeGen/TargetLoweringBase.cpp b/llvm/lib/CodeGen/TargetLoweringBase.cpp
index 58a644094ab8f..58476a5bec66e 100644
--- a/llvm/lib/CodeGen/TargetLoweringBase.cpp
+++ b/llvm/lib/CodeGen/TargetLoweringBase.cpp
@@ -944,6 +944,7 @@ void TargetLoweringBase::initActions() {
ISD::VECREDUCE_XOR, ISD::VECREDUCE_SMAX, ISD::VECREDUCE_SMIN,
ISD::VECREDUCE_UMAX, ISD::VECREDUCE_UMIN, ISD::VECREDUCE_FMAX,
ISD::VECREDUCE_FMIN, ISD::VECREDUCE_FMAXIMUM, ISD::VECREDUCE_FMINIMUM,
+ ISD::VECREDUCE_FMAXIMUMNUM, ISD::VECREDUCE_FMINIMUMNUM,
ISD::VECREDUCE_SEQ_FADD, ISD::VECREDUCE_SEQ_FMUL},
VT, Expand);
diff --git a/llvm/lib/IR/IRBuilder.cpp b/llvm/lib/IR/IRBuilder.cpp
index df738faad7842..42c2a919552aa 100644
--- a/llvm/lib/IR/IRBuilder.cpp
+++ b/llvm/lib/IR/IRBuilder.cpp
@@ -497,6 +497,14 @@ Value *IRBuilderBase::CreateFPMinimumReduce(Value *Src) {
return getReductionIntrinsic(Intrinsic::vector_reduce_fminimum, Src);
}
+Value *IRBuilderBase::CreateFPMaximumNumReduce(Value *Src) {
+ return getReductionIntrinsic(Intrinsic::vector_reduce_fmaximumnum, Src);
+}
+
+Value *IRBuilderBase::CreateFPMinimumNumReduce(Value *Src) {
+ return getReductionIntrinsic(Intrinsic::vector_reduce_fminimumnum, Src);
+}
+
CallInst *IRBuilderBase::CreateLifetimeStart(Value *Ptr) {
assert(isa<PointerType>(Ptr->getType()) &&
"lifetime.start only applies to pointers.");
diff --git a/llvm/lib/Transforms/Utils/LoopUtils.cpp b/llvm/lib/Transforms/Utils/LoopUtils.cpp
index 8ec435f6e3174..af2e838a6c867 100644
--- a/llvm/lib/Transforms/Utils/LoopUtils.cpp
+++ b/llvm/lib/Transforms/Utils/LoopUtils.cpp
@@ -1206,6 +1206,8 @@ unsigned llvm::getArithmeticReductionInstruction(Intrinsic::ID RdxID) {
case Intrinsic::vector_reduce_fmin:
case Intrinsic::vector_reduce_fmaximum:
case Intrinsic::vector_reduce_fminimum:
+ case Intrinsic::vector_reduce_fmaximumnum:
+ case Intrinsic::vector_reduce_fminimumnum:
return Instruction::FCmp;
default:
llvm_unreachable("Unexpected ID");
@@ -1255,6 +1257,10 @@ Intrinsic::ID llvm::getMinMaxReductionIntrinsicOp(Intrinsic::ID RdxID) {
return Intrinsic::minimum;
case Intrinsic::vector_reduce_fmaximum:
return Intrinsic::maximum;
+ case Intrinsic::vector_reduce_fminimumnum:
+ return Intrinsic::minimumnum;
+ case Intrinsic::vector_reduce_fmaximumnum:
+ return Intrinsic::maximumnum;
}
}
@@ -1305,6 +1311,10 @@ RecurKind llvm::getMinMaxReductionRecurKind(Intrinsic::ID RdxID) {
return RecurKind::FMaximum;
case Intrinsic::vector_reduce_fminimum:
return RecurKind::FMinimum;
+ case Intrinsic::vector_reduce_fmaximumnum:
+ return RecurKind::FMaximumNum;
+ case Intrinsic::vector_reduce_fminimumnum:
+ return RecurKind::FMinimumNum;
default:
return RecurKind::None;
}
diff --git a/llvm/test/CodeGen/AArch64/vecreduce-fmaximumnum.ll b/llvm/test/CodeGen/AArch64/vecreduce-fmaximumnum.ll
new file mode 100644
index 0000000000000..42c4e3566ace4
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/vecreduce-fmaximumnum.ll
@@ -0,0 +1,632 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-NOFP
+; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon,+fullfp16 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-FP
+
+define half @test_v1f16(<1 x half> %a) nounwind {
+; CHECK-LABEL: test_v1f16:
+; CHECK: // %bb.0:
+; CHECK-NEXT: ret
+ %b = call half @llvm.vector.reduce.fmaximumnum.v1f16(<1 x half> %a)
+ ret half %b
+}
+
+define float @test_v1f32(<1 x float> %a) nounwind {
+; CHECK-LABEL: test_v1f32:
+; CHECK: // %bb.0:
+; CHECK-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-NEXT: // kill: def $s0 killed $s0 killed $q0
+; CHECK-NEXT: ret
+ %b = call float @llvm.vector.reduce.fmaximumnum.v1f32(<1 x float> %a)
+ ret float %b
+}
+
+define double @test_v1f64(<1 x double> %a) nounwind {
+; CHECK-LABEL: test_v1f64:
+; CHECK: // %bb.0:
+; CHECK-NEXT: ret
+ %b = call double @llvm.vector.reduce.fmaximumnum.v1f64(<1 x double> %a)
+ ret double %b
+}
+
+define fp128 @test_v1f128(<1 x fp128> %a) nounwind {
+; CHECK-LABEL: test_v1f128:
+; CHECK: // %bb.0:
+; CHECK-NEXT: ret
+ %b = call fp128 @llvm.vector.reduce.fmaximumnum.v1f128(<1 x fp128> %a)
+ ret fp128 %b
+}
+
+define half @test_v4f16(<4 x half> %a) nounwind {
+; CHECK-NOFP-LABEL: test_v4f16:
+; CHECK-NOFP: // %bb.0:
+; CHECK-NOFP-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-NOFP-NEXT: mov h1, v0.h[1]
+; CHECK-NOFP-NEXT: fcvt s2, h0
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fmaxnm s1, s2, s1
+; CHECK-NOFP-NEXT: mov h2, v0.h[2]
+; CHECK-NOFP-NEXT: mov h0, v0.h[3]
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fmaxnm s1, s1, s2
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fmaxnm s0, s1, s0
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: ret
+;
+; CHECK-FP-LABEL: test_v4f16:
+; CHECK-FP: // %bb.0:
+; CHECK-FP-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-FP-NEXT: mov h1, v0.h[1]
+; CHECK-FP-NEXT: fminnm h2, h0, h0
+; CHECK-FP-NEXT: mov h3, v0.h[2]
+; CHECK-FP-NEXT: mov h0, v0.h[3]
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fminnm h0, h0, h0
+; CHECK-FP-NEXT: fmaxnm h1, h2, h1
+; CHECK-FP-NEXT: fminnm h2, h3, h3
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fmaxnm h0, h1, h0
+; CHECK-FP-NEXT: ret
+ %b = call half @llvm.vector.reduce.fmaximumnum.v4f16(<4 x half> %a)
+ ret half %b
+}
+
+define half @test_v8f16(<8 x half> %a) nounwind {
+; CHECK-NOFP-LABEL: test_v8f16:
+; CHECK-NOFP: // %bb.0:
+; CHECK-NOFP-NEXT: mov h1, v0.h[1]
+; CHECK-NOFP-NEXT: fcvt s2, h0
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fmaxnm s1, s2, s1
+; CHECK-NOFP-NEXT: mov h2, v0.h[2]
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fmaxnm s1, s1, s2
+; CHECK-NOFP-NEXT: mov h2, v0.h[3]
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fmaxnm s1, s1, s2
+; CHECK-NOFP-NEXT: mov h2, v0.h[4]
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fmaxnm s1, s1, s2
+; CHECK-NOFP-NEXT: mov h2, v0.h[5]
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fmaxnm s1, s1, s2
+; CHECK-NOFP-NEXT: mov h2, v0.h[6]
+; CHECK-NOFP-NEXT: mov h0, v0.h[7]
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fmaxnm s1, s1, s2
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fmaxnm s0, s1, s0
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: ret
+;
+; CHECK-FP-LABEL: test_v8f16:
+; CHECK-FP: // %bb.0:
+; CHECK-FP-NEXT: mov h1, v0.h[1]
+; CHECK-FP-NEXT: fminnm h2, h0, h0
+; CHECK-FP-NEXT: mov h3, v0.h[2]
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fmaxnm h1, h2, h1
+; CHECK-FP-NEXT: fminnm h2, h3, h3
+; CHECK-FP-NEXT: mov h3, v0.h[3]
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: fminnm h2, h3, h3
+; CHECK-FP-NEXT: mov h3, v0.h[4]
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: fminnm h2, h3, h3
+; CHECK-FP-NEXT: mov h3, v0.h[5]
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: fminnm h2, h3, h3
+; CHECK-FP-NEXT: mov h3, v0.h[6]
+; CHECK-FP-NEXT: mov h0, v0.h[7]
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fminnm h0, h0, h0
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: fminnm h2, h3, h3
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fmaxnm h0, h1, h0
+; CHECK-FP-NEXT: ret
+ %b = call half @llvm.vector.reduce.fmaximumnum.v8f16(<8 x half> %a)
+ ret half %b
+}
+
+define half @test_v16f16(<16 x half> %a) nounwind {
+; CHECK-NOFP-LABEL: test_v16f16:
+; CHECK-NOFP: // %bb.0:
+; CHECK-NOFP-NEXT: mov h2, v1.h[1]
+; CHECK-NOFP-NEXT: mov h3, v0.h[1]
+; CHECK-NOFP-NEXT: fcvt s4, h1
+; CHECK-NOFP-NEXT: fcvt s5, h0
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s3, h3
+; CHECK-NOFP-NEXT: fmaxnm s4, s5, s4
+; CHECK-NOFP-NEXT: mov h5, v0.h[2]
+; CHECK-NOFP-NEXT: fmaxnm s2, s3, s2
+; CHECK-NOFP-NEXT: mov h3, v1.h[2]
+; CHECK-NOFP-NEXT: fcvt h4, s4
+; CHECK-NOFP-NEXT: fcvt s5, h5
+; CHECK-NOFP-NEXT: fcvt h2, s2
+; CHECK-NOFP-NEXT: fcvt s3, h3
+; CHECK-NOFP-NEXT: fcvt s4, h4
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fmaxnm s3, s5, s3
+; CHECK-NOFP-NEXT: mov h5, v0.h[3]
+; CHECK-NOFP-NEXT: fmaxnm s2, s4, s2
+; CHECK-NOFP-NEXT: mov h4, v1.h[3]
+; CHECK-NOFP-NEXT: fcvt h3, s3
+; CHECK-NOFP-NEXT: fcvt s5, h5
+; CHECK-NOFP-NEXT: fcvt h2, s2
+; CHECK-NOFP-NEXT: fcvt s4, h4
+; CHECK-NOFP-NEXT: fcvt s3, h3
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fmaxnm s4, s5, s4
+; CHECK-NOFP-NEXT: mov h5, v0.h[4]
+; CHECK-NOFP-NEXT: fmaxnm s2, s2, s3
+; CHECK-NOFP-NEXT: mov h3, v1.h[4]
+; CHECK-NOFP-NEXT: fcvt h4, s4
+; CHECK-NOFP-NEXT: fcvt s5, h5
+; CHECK-NOFP-NEXT: fcvt h2, s2
+; CHECK-NOFP-NEXT: fcvt s3, h3
+; CHECK-NOFP-NEXT: fcvt s4, h4
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fmaxnm s3, s5, s3
+; CHECK-NOFP-NEXT: mov h5, v0.h[5]
+; CHECK-NOFP-NEXT: fmaxnm s2, s2, s4
+; CHECK-NOFP-NEXT: mov h4, v1.h[5]
+; CHECK-NOFP-NEXT: fcvt h3, s3
+; CHECK-NOFP-NEXT: fcvt s5, h5
+; CHECK-NOFP-NEXT: fcvt h2, s2
+; CHECK-NOFP-NEXT: fcvt s4, h4
+; CHECK-NOFP-NEXT: fcvt s3, h3
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fmaxnm s4, s5, s4
+; CHECK-NOFP-NEXT: mov h5, v0.h[6]
+; CHECK-NOFP-NEXT: mov h0, v0.h[7]
+; CHECK-NOFP-NEXT: fmaxnm s2, s2, s3
+; CHECK-NOFP-NEXT: fcvt h3, s4
+; CHECK-NOFP-NEXT: mov h4, v1.h[6]
+; CHECK-NOFP-NEXT: fcvt s5, h5
+; CHECK-NOFP-NEXT: mov h1, v1.h[7]
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fcvt h2, s2
+; CHECK-NOFP-NEXT: fcvt s3, h3
+; CHECK-NOFP-NEXT: fcvt s4, h4
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fmaxnm s0, s0, s1
+; CHECK-NOFP-NEXT: fmaxnm s2, s2, s3
+; CHECK-NOFP-NEXT: fmaxnm s3, s5, s4
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt h2, s2
+; CHECK-NOFP-NEXT: fcvt h3, s3
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s3, h3
+; CHECK-NOFP-NEXT: fmaxnm s2, s2, s3
+; CHECK-NOFP-NEXT: fcvt h1, s2
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fmaxnm s0, s1, s0
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: ret
+;
+; CHECK-FP-LABEL: test_v16f16:
+; CHECK-FP: // %bb.0:
+; CHECK-FP-NEXT: fminnm v1.8h, v1.8h, v1.8h
+; CHECK-FP-NEXT: fminnm v0.8h, v0.8h, v0.8h
+; CHECK-FP-NEXT: fmaxnm v0.8h, v0.8h, v1.8h
+; CHECK-FP-NEXT: mov h1, v0.h[1]
+; CHECK-FP-NEXT: mov h2, v0.h[2]
+; CHECK-FP-NEXT: fmaxnm h1, h0, h1
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[3]
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[4]
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[5]
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[6]
+; CHECK-FP-NEXT: mov h0, v0.h[7]
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: fmaxnm h0, h1, h0
+; CHECK-FP-NEXT: ret
+ %b = call half @llvm.vector.reduce.fmaximumnum.v16f16(<16 x half> %a)
+ ret half %b
+}
+
+define float @test_v2f32(<2 x float> %a) nounwind {
+; CHECK-LABEL: test_v2f32:
+; CHECK: // %bb.0:
+; CHECK-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: fminnm s0, s0, s0
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fmaxnm s0, s0, s1
+; CHECK-NEXT: ret
+ %b = call float @llvm.vector.reduce.fmaximumnum.v2f32(<2 x float> %a)
+ ret float %b
+}
+
+define float @test_v4f32(<4 x float> %a) nounwind {
+; CHECK-LABEL: test_v4f32:
+; CHECK: // %bb.0:
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: fminnm s2, s0, s0
+; CHECK-NEXT: mov s3, v0.s[2]
+; CHECK-NEXT: mov s0, v0.s[3]
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fminnm s0, s0, s0
+; CHECK-NEXT: fmaxnm s1, s2, s1
+; CHECK-NEXT: fminnm s2, s3, s3
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fmaxnm s1, s1, s2
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fmaxnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call float @llvm.vector.reduce.fmaximumnum.v4f32(<4 x float> %a)
+ ret float %b
+}
+
+define float @test_v8f32(<8 x float> %a) nounwind {
+; CHECK-LABEL: test_v8f32:
+; CHECK: // %bb.0:
+; CHECK-NEXT: fminnm v1.4s, v1.4s, v1.4s
+; CHECK-NEXT: fminnm v0.4s, v0.4s, v0.4s
+; CHECK-NEXT: fmaxnm v0.4s, v0.4s, v1.4s
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: mov s2, v0.s[2]
+; CHECK-NEXT: fmaxnm s1, s0, s1
+; CHECK-NEXT: mov s0, v0.s[3]
+; CHECK-NEXT: fmaxnm s1, s1, s2
+; CHECK-NEXT: fmaxnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call float @llvm.vector.reduce.fmaximumnum.v8f32(<8 x float> %a)
+ ret float %b
+}
+
+define float @test_v16f32(<16 x float> %a) nounwind {
+; CHECK-LABEL: test_v16f32:
+; CHECK: // %bb.0:
+; CHECK-NEXT: fminnm v3.4s, v3.4s, v3.4s
+; CHECK-NEXT: fminnm v1.4s, v1.4s, v1.4s
+; CHECK-NEXT: fminnm v2.4s, v2.4s, v2.4s
+; CHECK-NEXT: fminnm v0.4s, v0.4s, v0.4s
+; CHECK-NEXT: fmaxnm v1.4s, v1.4s, v3.4s
+; CHECK-NEXT: fmaxnm v0.4s, v0.4s, v2.4s
+; CHECK-NEXT: fmaxnm v0.4s, v0.4s, v1.4s
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: mov s2, v0.s[2]
+; CHECK-NEXT: fmaxnm s1, s0, s1
+; CHECK-NEXT: mov s0, v0.s[3]
+; CHECK-NEXT: fmaxnm s1, s1, s2
+; CHECK-NEXT: fmaxnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call float @llvm.vector.reduce.fmaximumnum.v16f32(<16 x float> %a)
+ ret float %b
+}
+
+define double @test_v2f64(<2 x double> %a) nounwind {
+; CHECK-LABEL: test_v2f64:
+; CHECK: // %bb.0:
+; CHECK-NEXT: mov d1, v0.d[1]
+; CHECK-NEXT: fminnm d0, d0, d0
+; CHECK-NEXT: fminnm d1, d1, d1
+; CHECK-NEXT: fmaxnm d0, d0, d1
+; CHECK-NEXT: ret
+ %b = call double @llvm.vector.reduce.fmaximumnum.v2f64(<2 x double> %a)
+ ret double %b
+}
+
+define double @test_v4f64(<4 x double> %a) nounwind {
+; CHECK-LABEL: test_v4f64:
+; CHECK: // %bb.0:
+; CHECK-NEXT: fminnm v1.2d, v1.2d, v1.2d
+; CHECK-NEXT: fminnm v0.2d, v0.2d, v0.2d
+; CHECK-NEXT: fmaxnm v0.2d, v0.2d, v1.2d
+; CHECK-NEXT: mov d1, v0.d[1]
+; CHECK-NEXT: fmaxnm d0, d0, d1
+; CHECK-NEXT: ret
+ %b = call double @llvm.vector.reduce.fmaximumnum.v4f64(<4 x double> %a)
+ ret double %b
+}
+
+define half @test_v11f16(<11 x half> %a) nounwind {
+; CHECK-NOFP-LABEL: test_v11f16:
+; CHECK-NOFP: // %bb.0:
+; CHECK-NOFP-NEXT: ldr h16, [sp, #8]
+; CHECK-NOFP-NEXT: ldr h17, [sp]
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s16, h16
+; CHECK-NOFP-NEXT: fcvt s17, h17
+; CHECK-NOFP-NEXT: fmaxnm s1, s1, s16
+; CHECK-NOFP-NEXT: fmaxnm s0, s0, s17
+; CHECK-NOFP-NEXT: ldr h16, [sp, #16]
+; CHECK-NOFP-NEXT: fcvt s16, h16
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fmaxnm s0, s0, s1
+; CHECK-NOFP-NEXT: fmaxnm s1, s2, s16
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fmaxnm s0, s0, s1
+; CHECK-NOFP-NEXT: fcvt s1, h3
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fmaxnm s0, s0, s1
+; CHECK-NOFP-NEXT: fcvt s1, h4
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fmaxnm s0, s0, s1
+; CHECK-NOFP-NEXT: fcvt s1, h5
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fmaxnm s0, s0, s1
+; CHECK-NOFP-NEXT: fcvt s1, h6
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fmaxnm s0, s0, s1
+; CHECK-NOFP-NEXT: fcvt s1, h7
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fmaxnm s0, s0, s1
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: ret
+;
+; CHECK-FP-LABEL: test_v11f16:
+; CHECK-FP: // %bb.0:
+; CHECK-FP-NEXT: // kill: def $h0 killed $h0 def $q0
+; CHECK-FP-NEXT: // kill: def $h1 killed $h1 def $q1
+; CHECK-FP-NEXT: // kill: def $h2 killed $h2 def $q2
+; CHECK-FP-NEXT: // kill: def $h3 killed $h3 def $q3
+; CHECK-FP-NEXT: // kill: def $h4 killed $h4 def $q4
+; CHECK-FP-NEXT: // kill: def $h5 killed $h5 def $q5
+; CHECK-FP-NEXT: mov x8, sp
+; CHECK-FP-NEXT: // kill: def $h6 killed $h6 def $q6
+; CHECK-FP-NEXT: // kill: def $h7 killed $h7 def $q7
+; CHECK-FP-NEXT: mov v0.h[1], v1.h[0]
+; CHECK-FP-NEXT: movi v1.8h, #254, lsl #8
+; CHECK-FP-NEXT: mov v0.h[2], v2.h[0]
+; CHECK-FP-NEXT: ld1 { v1.h }[0], [x8]
+; CHECK-FP-NEXT: add x8, sp, #8
+; CHECK-FP-NEXT: ld1 { v1.h }[1], [x8]
+; CHECK-FP-NEXT: add x8, sp, #16
+; CHECK-FP-NEXT: mov v0.h[3], v3.h[0]
+; CHECK-FP-NEXT: ld1 { v1.h }[2], [x8]
+; CHECK-FP-NEXT: mov v0.h[4], v4.h[0]
+; CHECK-FP-NEXT: fminnm v1.8h, v1.8h, v1.8h
+; CHECK-FP-NEXT: mov v0.h[5], v5.h[0]
+; CHECK-FP-NEXT: mov v0.h[6], v6.h[0]
+; CHECK-FP-NEXT: mov v0.h[7], v7.h[0]
+; CHECK-FP-NEXT: fminnm v0.8h, v0.8h, v0.8h
+; CHECK-FP-NEXT: fmaxnm v0.8h, v0.8h, v1.8h
+; CHECK-FP-NEXT: mov h1, v0.h[1]
+; CHECK-FP-NEXT: mov h2, v0.h[2]
+; CHECK-FP-NEXT: fmaxnm h1, h0, h1
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[3]
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[4]
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[5]
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[6]
+; CHECK-FP-NEXT: mov h0, v0.h[7]
+; CHECK-FP-NEXT: fmaxnm h1, h1, h2
+; CHECK-FP-NEXT: fmaxnm h0, h1, h0
+; CHECK-FP-NEXT: ret
+ %b = call half @llvm.vector.reduce.fmaximumnum.v11f16(<11 x half> %a)
+ ret half %b
+}
+
+define float @test_v3f32(<3 x float> %a) nounwind {
+; CHECK-LABEL: test_v3f32:
+; CHECK: // %bb.0:
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: fminnm s2, s0, s0
+; CHECK-NEXT: mov s0, v0.s[2]
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fminnm s0, s0, s0
+; CHECK-NEXT: fmaxnm s1, s2, s1
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fmaxnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call float @llvm.vector.reduce.fmaximumnum.v3f32(<3 x float> %a)
+ ret float %b
+}
+
+define float @test_v3f32_ninf(<3 x float> %a) nounwind {
+; CHECK-LABEL: test_v3f32_ninf:
+; CHECK: // %bb.0:
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: fminnm s2, s0, s0
+; CHECK-NEXT: mov s0, v0.s[2]
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fminnm s0, s0, s0
+; CHECK-NEXT: fmaxnm s1, s2, s1
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fmaxnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call ninf float @llvm.vector.reduce.fmaximumnum.v3f32(<3 x float> %a)
+ ret float %b
+}
+
+define fp128 @test_v2f128(<2 x fp128> %a) nounwind {
+; CHECK-LABEL: test_v2f128:
+; CHECK: // %bb.0:
+; CHECK-NEXT: sub sp, sp, #48
+; CHECK-NEXT: stp q0, q1, [sp] // 32-byte Folded Spill
+; CHECK-NEXT: mov v1.16b, v0.16b
+; CHECK-NEXT: str x30, [sp, #32] // 8-byte Spill
+; CHECK-NEXT: bl __unordtf2
+; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-NEXT: cmp w0, #0
+; CHECK-NEXT: b.eq .LBB16_2
+; CHECK-NEXT: // %bb.1:
+; CHECK-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-NEXT: .LBB16_2:
+; CHECK-NEXT: mov v1.16b, v0.16b
+; CHECK-NEXT: bl __unordtf2
+; CHECK-NEXT: ldp q0, q1, [sp] // 32-byte Folded Reload
+; CHECK-NEXT: cmp w0, #0
+; CHECK-NEXT: b.eq .LBB16_4
+; CHECK-NEXT: // %bb.3:
+; CHECK-NEXT: mov v1.16b, v0.16b
+; CHECK-NEXT: .LBB16_4:
+; CHECK-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-NEXT: bl __gttf2
+; CHECK-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-NEXT: cmp w0, #0
+; CHECK-NEXT: b.le .LBB16_6
+; CHECK-NEXT: // %bb.5:
+; CHECK-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-NEXT: .LBB16_6:
+; CHECK-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-NEXT: bl __trunctfsf2
+; CHECK-NEXT: fmov w8, s0
+; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-NEXT: mov v1.16b, v0.16b
+; CHECK-NEXT: cmp w8, #0
+; CHECK-NEXT: b.ne .LBB16_8
+; CHECK-NEXT: // %bb.7:
+; CHECK-NEXT: ldr q1, [sp] // 16-byte Reload
+; CHECK-NEXT: .LBB16_8:
+; CHECK-NEXT: adrp x8, .LCPI16_0
+; CHECK-NEXT: str q1, [sp] // 16-byte Spill
+; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI16_0]
+; CHECK-NEXT: bl __eqtf2
+; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-NEXT: cmp w0, #0
+; CHECK-NEXT: b.ne .LBB16_10
+; CHECK-NEXT: // %bb.9:
+; CHECK-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-NEXT: .LBB16_10:
+; CHECK-NEXT: ldr x30, [sp, #32] // 8-byte Reload
+; CHECK-NEXT: add sp, sp, #48
+; CHECK-NEXT: ret
+ %b = call fp128 @llvm.vector.reduce.fmaximumnum.v2f128(<2 x fp128> %a)
+ ret fp128 %b
+}
+
+; Needs widening rather than scalarizing.
+; The mvni constructs a NaN that is used as the identity element.
+define float @test_v7f32(<7 x float> %a) nounwind {
+; CHECK-LABEL: test_v7f32:
+; CHECK: // %bb.0:
+; CHECK-NEXT: mvni v7.4s, #63, msl #16
+; CHECK-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-NEXT: // kill: def $s4 killed $s4 def $q4
+; CHECK-NEXT: // kill: def $s1 killed $s1 def $q1
+; CHECK-NEXT: // kill: def $s5 killed $s5 def $q5
+; CHECK-NEXT: // kill: def $s2 killed $s2 def $q2
+; CHECK-NEXT: // kill: def $s6 killed $s6 def $q6
+; CHECK-NEXT: // kill: def $s3 killed $s3 def $q3
+; CHECK-NEXT: mov v0.s[1], v1.s[0]
+; CHECK-NEXT: mov v7.s[0], v4.s[0]
+; CHECK-NEXT: mov v0.s[2], v2.s[0]
+; CHECK-NEXT: mov v7.s[1], v5.s[0]
+; CHECK-NEXT: mov v0.s[3], v3.s[0]
+; CHECK-NEXT: mov v7.s[2], v6.s[0]
+; CHECK-NEXT: fminnm v0.4s, v0.4s, v0.4s
+; CHECK-NEXT: fminnm v1.4s, v7.4s, v7.4s
+; CHECK-NEXT: fmaxnm v0.4s, v0.4s, v1.4s
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: mov s2, v0.s[2]
+; CHECK-NEXT: fmaxnm s1, s0, s1
+; CHECK-NEXT: mov s0, v0.s[3]
+; CHECK-NEXT: fmaxnm s1, s1, s2
+; CHECK-NEXT: fmaxnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call float @llvm.vector.reduce.fmaximumnum.v7f32(<7 x float> %a)
+ ret float %b
+}
+
+; The mvni constructs Inf that is used as the identity element.
+; That is valid because nnan means we don't need to consider NaN.
+define float @test_v7f32_nnan(<7 x float> %a) nounwind {
+; CHECK-LABEL: test_v7f32_nnan:
+; CHECK: // %bb.0:
+; CHECK-NEXT: mvni v7.4s, #127, msl #16
+; CHECK-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-NEXT: // kill: def $s4 killed $s4 def $q4
+; CHECK-NEXT: // kill: def $s1 killed $s1 def $q1
+; CHECK-NEXT: // kill: def $s5 killed $s5 def $q5
+; CHECK-NEXT: // kill: def $s2 killed $s2 def $q2
+; CHECK-NEXT: // kill: def $s6 killed $s6 def $q6
+; CHECK-NEXT: // kill: def $s3 killed $s3 def $q3
+; CHECK-NEXT: mov v0.s[1], v1.s[0]
+; CHECK-NEXT: mov v7.s[0], v4.s[0]
+; CHECK-NEXT: mov v0.s[2], v2.s[0]
+; CHECK-NEXT: mov v7.s[1], v5.s[0]
+; CHECK-NEXT: mov v0.s[3], v3.s[0]
+; CHECK-NEXT: mov v7.s[2], v6.s[0]
+; CHECK-NEXT: fmaxnm v0.4s, v0.4s, v7.4s
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: mov s2, v0.s[2]
+; CHECK-NEXT: fmaxnm s1, s0, s1
+; CHECK-NEXT: mov s0, v0.s[3]
+; CHECK-NEXT: fmaxnm s1, s1, s2
+; CHECK-NEXT: fmaxnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call nnan float @llvm.vector.reduce.fmaximumnum.v7f32(<7 x float> %a)
+ ret float %b
+}
+
+; The mvni constructs -FLT_MAX that is used as the identity element.
+; That is valid because nnan ninf means we don't need to consider NaN or Inf.
+define float @test_v7f32_nnan_ninf(<7 x float> %a) nounwind {
+; CHECK-LABEL: test_v7f32_nnan_ninf:
+; CHECK: // %bb.0:
+; CHECK-NEXT: mvni v7.4s, #128, lsl #16
+; CHECK-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-NEXT: // kill: def $s4 killed $s4 def $q4
+; CHECK-NEXT: // kill: def $s1 killed $s1 def $q1
+; CHECK-NEXT: // kill: def $s5 killed $s5 def $q5
+; CHECK-NEXT: // kill: def $s2 killed $s2 def $q2
+; CHECK-NEXT: // kill: def $s6 killed $s6 def $q6
+; CHECK-NEXT: // kill: def $s3 killed $s3 def $q3
+; CHECK-NEXT: mov v0.s[1], v1.s[0]
+; CHECK-NEXT: mov v7.s[0], v4.s[0]
+; CHECK-NEXT: mov v0.s[2], v2.s[0]
+; CHECK-NEXT: mov v7.s[1], v5.s[0]
+; CHECK-NEXT: mov v0.s[3], v3.s[0]
+; CHECK-NEXT: mov v7.s[2], v6.s[0]
+; CHECK-NEXT: fmaxnm v0.4s, v0.4s, v7.4s
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: mov s2, v0.s[2]
+; CHECK-NEXT: fmaxnm s1, s0, s1
+; CHECK-NEXT: mov s0, v0.s[3]
+; CHECK-NEXT: fmaxnm s1, s1, s2
+; CHECK-NEXT: fmaxnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call nnan ninf float @llvm.vector.reduce.fmaximumnum.v7f32(<7 x float> %a)
+ ret float %b
+}
+
diff --git a/llvm/test/CodeGen/AArch64/vecreduce-fminimumnum.ll b/llvm/test/CodeGen/AArch64/vecreduce-fminimumnum.ll
new file mode 100644
index 0000000000000..0881f6ec76c14
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/vecreduce-fminimumnum.ll
@@ -0,0 +1,636 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-NOFP
+; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon,+fullfp16 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-FP
+
+define half @test_v1f16(<1 x half> %a) nounwind {
+; CHECK-LABEL: test_v1f16:
+; CHECK: // %bb.0:
+; CHECK-NEXT: ret
+ %b = call half @llvm.vector.reduce.fminimumnum.v1f16(<1 x half> %a)
+ ret half %b
+}
+
+define float @test_v1f32(<1 x float> %a) nounwind {
+; CHECK-LABEL: test_v1f32:
+; CHECK: // %bb.0:
+; CHECK-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-NEXT: // kill: def $s0 killed $s0 killed $q0
+; CHECK-NEXT: ret
+ %b = call float @llvm.vector.reduce.fminimumnum.v1f32(<1 x float> %a)
+ ret float %b
+}
+
+define double @test_v1f64(<1 x double> %a) nounwind {
+; CHECK-LABEL: test_v1f64:
+; CHECK: // %bb.0:
+; CHECK-NEXT: ret
+ %b = call double @llvm.vector.reduce.fminimumnum.v1f64(<1 x double> %a)
+ ret double %b
+}
+
+define fp128 @test_v1f128(<1 x fp128> %a) nounwind {
+; CHECK-LABEL: test_v1f128:
+; CHECK: // %bb.0:
+; CHECK-NEXT: ret
+ %b = call fp128 @llvm.vector.reduce.fminimumnum.v1f128(<1 x fp128> %a)
+ ret fp128 %b
+}
+
+define half @test_v4f16(<4 x half> %a) nounwind {
+; CHECK-NOFP-LABEL: test_v4f16:
+; CHECK-NOFP: // %bb.0:
+; CHECK-NOFP-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-NOFP-NEXT: mov h1, v0.h[1]
+; CHECK-NOFP-NEXT: fcvt s2, h0
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fminnm s1, s2, s1
+; CHECK-NOFP-NEXT: mov h2, v0.h[2]
+; CHECK-NOFP-NEXT: mov h0, v0.h[3]
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fminnm s1, s1, s2
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fminnm s0, s1, s0
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: ret
+;
+; CHECK-FP-LABEL: test_v4f16:
+; CHECK-FP: // %bb.0:
+; CHECK-FP-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-FP-NEXT: mov h1, v0.h[1]
+; CHECK-FP-NEXT: fminnm h2, h0, h0
+; CHECK-FP-NEXT: mov h3, v0.h[2]
+; CHECK-FP-NEXT: mov h0, v0.h[3]
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fminnm h0, h0, h0
+; CHECK-FP-NEXT: fminnm h1, h2, h1
+; CHECK-FP-NEXT: fminnm h2, h3, h3
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fminnm h0, h1, h0
+; CHECK-FP-NEXT: ret
+ %b = call half @llvm.vector.reduce.fminimumnum.v4f16(<4 x half> %a)
+ ret half %b
+}
+
+define half @test_v8f16(<8 x half> %a) nounwind {
+; CHECK-NOFP-LABEL: test_v8f16:
+; CHECK-NOFP: // %bb.0:
+; CHECK-NOFP-NEXT: mov h1, v0.h[1]
+; CHECK-NOFP-NEXT: fcvt s2, h0
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fminnm s1, s2, s1
+; CHECK-NOFP-NEXT: mov h2, v0.h[2]
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fminnm s1, s1, s2
+; CHECK-NOFP-NEXT: mov h2, v0.h[3]
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fminnm s1, s1, s2
+; CHECK-NOFP-NEXT: mov h2, v0.h[4]
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fminnm s1, s1, s2
+; CHECK-NOFP-NEXT: mov h2, v0.h[5]
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fminnm s1, s1, s2
+; CHECK-NOFP-NEXT: mov h2, v0.h[6]
+; CHECK-NOFP-NEXT: mov h0, v0.h[7]
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fminnm s1, s1, s2
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fminnm s0, s1, s0
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: ret
+;
+; CHECK-FP-LABEL: test_v8f16:
+; CHECK-FP: // %bb.0:
+; CHECK-FP-NEXT: mov h1, v0.h[1]
+; CHECK-FP-NEXT: fminnm h2, h0, h0
+; CHECK-FP-NEXT: mov h3, v0.h[2]
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fminnm h1, h2, h1
+; CHECK-FP-NEXT: fminnm h2, h3, h3
+; CHECK-FP-NEXT: mov h3, v0.h[3]
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: fminnm h2, h3, h3
+; CHECK-FP-NEXT: mov h3, v0.h[4]
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: fminnm h2, h3, h3
+; CHECK-FP-NEXT: mov h3, v0.h[5]
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: fminnm h2, h3, h3
+; CHECK-FP-NEXT: mov h3, v0.h[6]
+; CHECK-FP-NEXT: mov h0, v0.h[7]
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fminnm h0, h0, h0
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: fminnm h2, h3, h3
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: fminnm h1, h1, h1
+; CHECK-FP-NEXT: fminnm h0, h1, h0
+; CHECK-FP-NEXT: ret
+ %b = call half @llvm.vector.reduce.fminimumnum.v8f16(<8 x half> %a)
+ ret half %b
+}
+
+define half @test_v16f16(<16 x half> %a) nounwind {
+; CHECK-NOFP-LABEL: test_v16f16:
+; CHECK-NOFP: // %bb.0:
+; CHECK-NOFP-NEXT: mov h2, v1.h[1]
+; CHECK-NOFP-NEXT: mov h3, v0.h[1]
+; CHECK-NOFP-NEXT: fcvt s4, h1
+; CHECK-NOFP-NEXT: fcvt s5, h0
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s3, h3
+; CHECK-NOFP-NEXT: fminnm s4, s5, s4
+; CHECK-NOFP-NEXT: mov h5, v0.h[2]
+; CHECK-NOFP-NEXT: fminnm s2, s3, s2
+; CHECK-NOFP-NEXT: mov h3, v1.h[2]
+; CHECK-NOFP-NEXT: fcvt h4, s4
+; CHECK-NOFP-NEXT: fcvt s5, h5
+; CHECK-NOFP-NEXT: fcvt h2, s2
+; CHECK-NOFP-NEXT: fcvt s3, h3
+; CHECK-NOFP-NEXT: fcvt s4, h4
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fminnm s3, s5, s3
+; CHECK-NOFP-NEXT: mov h5, v0.h[3]
+; CHECK-NOFP-NEXT: fminnm s2, s4, s2
+; CHECK-NOFP-NEXT: mov h4, v1.h[3]
+; CHECK-NOFP-NEXT: fcvt h3, s3
+; CHECK-NOFP-NEXT: fcvt s5, h5
+; CHECK-NOFP-NEXT: fcvt h2, s2
+; CHECK-NOFP-NEXT: fcvt s4, h4
+; CHECK-NOFP-NEXT: fcvt s3, h3
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fminnm s4, s5, s4
+; CHECK-NOFP-NEXT: mov h5, v0.h[4]
+; CHECK-NOFP-NEXT: fminnm s2, s2, s3
+; CHECK-NOFP-NEXT: mov h3, v1.h[4]
+; CHECK-NOFP-NEXT: fcvt h4, s4
+; CHECK-NOFP-NEXT: fcvt s5, h5
+; CHECK-NOFP-NEXT: fcvt h2, s2
+; CHECK-NOFP-NEXT: fcvt s3, h3
+; CHECK-NOFP-NEXT: fcvt s4, h4
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fminnm s3, s5, s3
+; CHECK-NOFP-NEXT: mov h5, v0.h[5]
+; CHECK-NOFP-NEXT: fminnm s2, s2, s4
+; CHECK-NOFP-NEXT: mov h4, v1.h[5]
+; CHECK-NOFP-NEXT: fcvt h3, s3
+; CHECK-NOFP-NEXT: fcvt s5, h5
+; CHECK-NOFP-NEXT: fcvt h2, s2
+; CHECK-NOFP-NEXT: fcvt s4, h4
+; CHECK-NOFP-NEXT: fcvt s3, h3
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fminnm s4, s5, s4
+; CHECK-NOFP-NEXT: mov h5, v0.h[6]
+; CHECK-NOFP-NEXT: mov h0, v0.h[7]
+; CHECK-NOFP-NEXT: fminnm s2, s2, s3
+; CHECK-NOFP-NEXT: fcvt h3, s4
+; CHECK-NOFP-NEXT: mov h4, v1.h[6]
+; CHECK-NOFP-NEXT: fcvt s5, h5
+; CHECK-NOFP-NEXT: mov h1, v1.h[7]
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fcvt h2, s2
+; CHECK-NOFP-NEXT: fcvt s3, h3
+; CHECK-NOFP-NEXT: fcvt s4, h4
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fminnm s0, s0, s1
+; CHECK-NOFP-NEXT: fminnm s2, s2, s3
+; CHECK-NOFP-NEXT: fminnm s3, s5, s4
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt h2, s2
+; CHECK-NOFP-NEXT: fcvt h3, s3
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s3, h3
+; CHECK-NOFP-NEXT: fminnm s2, s2, s3
+; CHECK-NOFP-NEXT: fcvt h1, s2
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fminnm s0, s1, s0
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: ret
+;
+; CHECK-FP-LABEL: test_v16f16:
+; CHECK-FP: // %bb.0:
+; CHECK-FP-NEXT: fminnm v1.8h, v1.8h, v1.8h
+; CHECK-FP-NEXT: fminnm v0.8h, v0.8h, v0.8h
+; CHECK-FP-NEXT: fminnm v0.8h, v0.8h, v1.8h
+; CHECK-FP-NEXT: mov h1, v0.h[1]
+; CHECK-FP-NEXT: mov h2, v0.h[2]
+; CHECK-FP-NEXT: fminnm h1, h0, h1
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[3]
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[4]
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[5]
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[6]
+; CHECK-FP-NEXT: mov h0, v0.h[7]
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: fminnm h0, h1, h0
+; CHECK-FP-NEXT: ret
+ %b = call half @llvm.vector.reduce.fminimumnum.v16f16(<16 x half> %a)
+ ret half %b
+}
+
+define float @test_v2f32(<2 x float> %a) nounwind {
+; CHECK-LABEL: test_v2f32:
+; CHECK: // %bb.0:
+; CHECK-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: fminnm s0, s0, s0
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fminnm s0, s0, s1
+; CHECK-NEXT: ret
+ %b = call float @llvm.vector.reduce.fminimumnum.v2f32(<2 x float> %a)
+ ret float %b
+}
+
+define float @test_v4f32(<4 x float> %a) nounwind {
+; CHECK-LABEL: test_v4f32:
+; CHECK: // %bb.0:
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: fminnm s2, s0, s0
+; CHECK-NEXT: mov s3, v0.s[2]
+; CHECK-NEXT: mov s0, v0.s[3]
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fminnm s0, s0, s0
+; CHECK-NEXT: fminnm s1, s2, s1
+; CHECK-NEXT: fminnm s2, s3, s3
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fminnm s1, s1, s2
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fminnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call float @llvm.vector.reduce.fminimumnum.v4f32(<4 x float> %a)
+ ret float %b
+}
+
+define float @test_v8f32(<8 x float> %a) nounwind {
+; CHECK-LABEL: test_v8f32:
+; CHECK: // %bb.0:
+; CHECK-NEXT: fminnm v1.4s, v1.4s, v1.4s
+; CHECK-NEXT: fminnm v0.4s, v0.4s, v0.4s
+; CHECK-NEXT: fminnm v0.4s, v0.4s, v1.4s
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: mov s2, v0.s[2]
+; CHECK-NEXT: fminnm s1, s0, s1
+; CHECK-NEXT: mov s0, v0.s[3]
+; CHECK-NEXT: fminnm s1, s1, s2
+; CHECK-NEXT: fminnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call float @llvm.vector.reduce.fminimumnum.v8f32(<8 x float> %a)
+ ret float %b
+}
+
+define float @test_v16f32(<16 x float> %a) nounwind {
+; CHECK-LABEL: test_v16f32:
+; CHECK: // %bb.0:
+; CHECK-NEXT: fminnm v3.4s, v3.4s, v3.4s
+; CHECK-NEXT: fminnm v1.4s, v1.4s, v1.4s
+; CHECK-NEXT: fminnm v2.4s, v2.4s, v2.4s
+; CHECK-NEXT: fminnm v0.4s, v0.4s, v0.4s
+; CHECK-NEXT: fminnm v1.4s, v1.4s, v3.4s
+; CHECK-NEXT: fminnm v0.4s, v0.4s, v2.4s
+; CHECK-NEXT: fminnm v0.4s, v0.4s, v1.4s
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: mov s2, v0.s[2]
+; CHECK-NEXT: fminnm s1, s0, s1
+; CHECK-NEXT: mov s0, v0.s[3]
+; CHECK-NEXT: fminnm s1, s1, s2
+; CHECK-NEXT: fminnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call float @llvm.vector.reduce.fminimumnum.v16f32(<16 x float> %a)
+ ret float %b
+}
+
+define double @test_v2f64(<2 x double> %a) nounwind {
+; CHECK-LABEL: test_v2f64:
+; CHECK: // %bb.0:
+; CHECK-NEXT: mov d1, v0.d[1]
+; CHECK-NEXT: fminnm d0, d0, d0
+; CHECK-NEXT: fminnm d1, d1, d1
+; CHECK-NEXT: fminnm d0, d0, d1
+; CHECK-NEXT: ret
+ %b = call double @llvm.vector.reduce.fminimumnum.v2f64(<2 x double> %a)
+ ret double %b
+}
+
+define double @test_v4f64(<4 x double> %a) nounwind {
+; CHECK-LABEL: test_v4f64:
+; CHECK: // %bb.0:
+; CHECK-NEXT: fminnm v1.2d, v1.2d, v1.2d
+; CHECK-NEXT: fminnm v0.2d, v0.2d, v0.2d
+; CHECK-NEXT: fminnm v0.2d, v0.2d, v1.2d
+; CHECK-NEXT: mov d1, v0.d[1]
+; CHECK-NEXT: fminnm d0, d0, d1
+; CHECK-NEXT: ret
+ %b = call double @llvm.vector.reduce.fminimumnum.v4f64(<4 x double> %a)
+ ret double %b
+}
+
+define half @test_v11f16(<11 x half> %a) nounwind {
+; CHECK-NOFP-LABEL: test_v11f16:
+; CHECK-NOFP: // %bb.0:
+; CHECK-NOFP-NEXT: ldr h16, [sp, #8]
+; CHECK-NOFP-NEXT: ldr h17, [sp]
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fcvt s2, h2
+; CHECK-NOFP-NEXT: fcvt s16, h16
+; CHECK-NOFP-NEXT: fcvt s17, h17
+; CHECK-NOFP-NEXT: fminnm s1, s1, s16
+; CHECK-NOFP-NEXT: fminnm s0, s0, s17
+; CHECK-NOFP-NEXT: ldr h16, [sp, #16]
+; CHECK-NOFP-NEXT: fcvt s16, h16
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fminnm s0, s0, s1
+; CHECK-NOFP-NEXT: fminnm s1, s2, s16
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt h1, s1
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fcvt s1, h1
+; CHECK-NOFP-NEXT: fminnm s0, s0, s1
+; CHECK-NOFP-NEXT: fcvt s1, h3
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fminnm s0, s0, s1
+; CHECK-NOFP-NEXT: fcvt s1, h4
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fminnm s0, s0, s1
+; CHECK-NOFP-NEXT: fcvt s1, h5
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fminnm s0, s0, s1
+; CHECK-NOFP-NEXT: fcvt s1, h6
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fminnm s0, s0, s1
+; CHECK-NOFP-NEXT: fcvt s1, h7
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: fcvt s0, h0
+; CHECK-NOFP-NEXT: fminnm s0, s0, s1
+; CHECK-NOFP-NEXT: fcvt h0, s0
+; CHECK-NOFP-NEXT: ret
+;
+; CHECK-FP-LABEL: test_v11f16:
+; CHECK-FP: // %bb.0:
+; CHECK-FP-NEXT: // kill: def $h0 killed $h0 def $q0
+; CHECK-FP-NEXT: // kill: def $h1 killed $h1 def $q1
+; CHECK-FP-NEXT: // kill: def $h2 killed $h2 def $q2
+; CHECK-FP-NEXT: // kill: def $h3 killed $h3 def $q3
+; CHECK-FP-NEXT: // kill: def $h4 killed $h4 def $q4
+; CHECK-FP-NEXT: // kill: def $h5 killed $h5 def $q5
+; CHECK-FP-NEXT: mov x8, sp
+; CHECK-FP-NEXT: // kill: def $h6 killed $h6 def $q6
+; CHECK-FP-NEXT: // kill: def $h7 killed $h7 def $q7
+; CHECK-FP-NEXT: mov v0.h[1], v1.h[0]
+; CHECK-FP-NEXT: movi v1.8h, #126, lsl #8
+; CHECK-FP-NEXT: mov v0.h[2], v2.h[0]
+; CHECK-FP-NEXT: ld1 { v1.h }[0], [x8]
+; CHECK-FP-NEXT: add x8, sp, #8
+; CHECK-FP-NEXT: ld1 { v1.h }[1], [x8]
+; CHECK-FP-NEXT: add x8, sp, #16
+; CHECK-FP-NEXT: mov v0.h[3], v3.h[0]
+; CHECK-FP-NEXT: ld1 { v1.h }[2], [x8]
+; CHECK-FP-NEXT: mov v0.h[4], v4.h[0]
+; CHECK-FP-NEXT: fminnm v1.8h, v1.8h, v1.8h
+; CHECK-FP-NEXT: mov v0.h[5], v5.h[0]
+; CHECK-FP-NEXT: mov v0.h[6], v6.h[0]
+; CHECK-FP-NEXT: mov v0.h[7], v7.h[0]
+; CHECK-FP-NEXT: fminnm v0.8h, v0.8h, v0.8h
+; CHECK-FP-NEXT: fminnm v0.8h, v0.8h, v1.8h
+; CHECK-FP-NEXT: mov h1, v0.h[1]
+; CHECK-FP-NEXT: mov h2, v0.h[2]
+; CHECK-FP-NEXT: fminnm h1, h0, h1
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[3]
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[4]
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[5]
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: mov h2, v0.h[6]
+; CHECK-FP-NEXT: mov h0, v0.h[7]
+; CHECK-FP-NEXT: fminnm h1, h1, h2
+; CHECK-FP-NEXT: fminnm h0, h1, h0
+; CHECK-FP-NEXT: ret
+ %b = call half @llvm.vector.reduce.fminimumnum.v11f16(<11 x half> %a)
+ ret half %b
+}
+
+define float @test_v3f32(<3 x float> %a) nounwind {
+; CHECK-LABEL: test_v3f32:
+; CHECK: // %bb.0:
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: fminnm s2, s0, s0
+; CHECK-NEXT: mov s0, v0.s[2]
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fminnm s0, s0, s0
+; CHECK-NEXT: fminnm s1, s2, s1
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fminnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call float @llvm.vector.reduce.fminimumnum.v3f32(<3 x float> %a)
+ ret float %b
+}
+
+define float @test_v3f32_ninf(<3 x float> %a) nounwind {
+; CHECK-LABEL: test_v3f32_ninf:
+; CHECK: // %bb.0:
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: fminnm s2, s0, s0
+; CHECK-NEXT: mov s0, v0.s[2]
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fminnm s0, s0, s0
+; CHECK-NEXT: fminnm s1, s2, s1
+; CHECK-NEXT: fminnm s1, s1, s1
+; CHECK-NEXT: fminnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call ninf float @llvm.vector.reduce.fminimumnum.v3f32(<3 x float> %a)
+ ret float %b
+}
+
+define fp128 @test_v2f128(<2 x fp128> %a) nounwind {
+; CHECK-LABEL: test_v2f128:
+; CHECK: // %bb.0:
+; CHECK-NEXT: sub sp, sp, #48
+; CHECK-NEXT: stp q0, q1, [sp] // 32-byte Folded Spill
+; CHECK-NEXT: mov v1.16b, v0.16b
+; CHECK-NEXT: str x30, [sp, #32] // 8-byte Spill
+; CHECK-NEXT: bl __unordtf2
+; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-NEXT: cmp w0, #0
+; CHECK-NEXT: b.eq .LBB16_2
+; CHECK-NEXT: // %bb.1:
+; CHECK-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-NEXT: .LBB16_2:
+; CHECK-NEXT: mov v1.16b, v0.16b
+; CHECK-NEXT: bl __unordtf2
+; CHECK-NEXT: ldp q0, q1, [sp] // 32-byte Folded Reload
+; CHECK-NEXT: cmp w0, #0
+; CHECK-NEXT: b.eq .LBB16_4
+; CHECK-NEXT: // %bb.3:
+; CHECK-NEXT: mov v1.16b, v0.16b
+; CHECK-NEXT: .LBB16_4:
+; CHECK-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-NEXT: bl __lttf2
+; CHECK-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-NEXT: cmp w0, #0
+; CHECK-NEXT: b.pl .LBB16_6
+; CHECK-NEXT: // %bb.5:
+; CHECK-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-NEXT: .LBB16_6:
+; CHECK-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-NEXT: bl __trunctfsf2
+; CHECK-NEXT: fmov w9, s0
+; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-NEXT: mov w8, #-2147483648 // =0x80000000
+; CHECK-NEXT: mov v1.16b, v0.16b
+; CHECK-NEXT: cmp w9, w8
+; CHECK-NEXT: b.ne .LBB16_8
+; CHECK-NEXT: // %bb.7:
+; CHECK-NEXT: ldr q1, [sp] // 16-byte Reload
+; CHECK-NEXT: .LBB16_8:
+; CHECK-NEXT: adrp x8, .LCPI16_0
+; CHECK-NEXT: str q1, [sp] // 16-byte Spill
+; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI16_0]
+; CHECK-NEXT: bl __eqtf2
+; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-NEXT: cmp w0, #0
+; CHECK-NEXT: b.ne .LBB16_10
+; CHECK-NEXT: // %bb.9:
+; CHECK-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-NEXT: .LBB16_10:
+; CHECK-NEXT: ldr x30, [sp, #32] // 8-byte Reload
+; CHECK-NEXT: add sp, sp, #48
+; CHECK-NEXT: ret
+ %b = call fp128 @llvm.vector.reduce.fminimumnum.v2f128(<2 x fp128> %a)
+ ret fp128 %b
+}
+
+; Needs widening rather than scalarizing.
+; The mvni constructs a NaN that is used as the identity element.
+define float @test_v7f32(<7 x float> %a) nounwind {
+; CHECK-LABEL: test_v7f32:
+; CHECK: // %bb.0:
+; CHECK-NEXT: mvni v7.4s, #63, msl #16
+; CHECK-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-NEXT: // kill: def $s4 killed $s4 def $q4
+; CHECK-NEXT: // kill: def $s1 killed $s1 def $q1
+; CHECK-NEXT: // kill: def $s5 killed $s5 def $q5
+; CHECK-NEXT: // kill: def $s2 killed $s2 def $q2
+; CHECK-NEXT: // kill: def $s6 killed $s6 def $q6
+; CHECK-NEXT: // kill: def $s3 killed $s3 def $q3
+; CHECK-NEXT: mov v0.s[1], v1.s[0]
+; CHECK-NEXT: fneg v7.4s, v7.4s
+; CHECK-NEXT: mov v0.s[2], v2.s[0]
+; CHECK-NEXT: mov v7.s[0], v4.s[0]
+; CHECK-NEXT: mov v0.s[3], v3.s[0]
+; CHECK-NEXT: mov v7.s[1], v5.s[0]
+; CHECK-NEXT: fminnm v0.4s, v0.4s, v0.4s
+; CHECK-NEXT: mov v7.s[2], v6.s[0]
+; CHECK-NEXT: fminnm v1.4s, v7.4s, v7.4s
+; CHECK-NEXT: fminnm v0.4s, v0.4s, v1.4s
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: mov s2, v0.s[2]
+; CHECK-NEXT: fminnm s1, s0, s1
+; CHECK-NEXT: mov s0, v0.s[3]
+; CHECK-NEXT: fminnm s1, s1, s2
+; CHECK-NEXT: fminnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call float @llvm.vector.reduce.fminimumnum.v7f32(<7 x float> %a)
+ ret float %b
+}
+
+; The mvni constructs Inf that is used as the identity element.
+; That is valid because nnan means we don't need to consider NaN.
+define float @test_v7f32_nnan(<7 x float> %a) nounwind {
+; CHECK-LABEL: test_v7f32_nnan:
+; CHECK: // %bb.0:
+; CHECK-NEXT: mvni v7.4s, #127, msl #16
+; CHECK-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-NEXT: // kill: def $s4 killed $s4 def $q4
+; CHECK-NEXT: // kill: def $s1 killed $s1 def $q1
+; CHECK-NEXT: // kill: def $s5 killed $s5 def $q5
+; CHECK-NEXT: // kill: def $s2 killed $s2 def $q2
+; CHECK-NEXT: // kill: def $s6 killed $s6 def $q6
+; CHECK-NEXT: // kill: def $s3 killed $s3 def $q3
+; CHECK-NEXT: mov v0.s[1], v1.s[0]
+; CHECK-NEXT: fneg v7.4s, v7.4s
+; CHECK-NEXT: mov v0.s[2], v2.s[0]
+; CHECK-NEXT: mov v7.s[0], v4.s[0]
+; CHECK-NEXT: mov v0.s[3], v3.s[0]
+; CHECK-NEXT: mov v7.s[1], v5.s[0]
+; CHECK-NEXT: mov v7.s[2], v6.s[0]
+; CHECK-NEXT: fminnm v0.4s, v0.4s, v7.4s
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: mov s2, v0.s[2]
+; CHECK-NEXT: fminnm s1, s0, s1
+; CHECK-NEXT: mov s0, v0.s[3]
+; CHECK-NEXT: fminnm s1, s1, s2
+; CHECK-NEXT: fminnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call nnan float @llvm.vector.reduce.fminimumnum.v7f32(<7 x float> %a)
+ ret float %b
+}
+
+; The mvni constructs -FLT_MAX that is negated and used as the identity element.
+; That is valid because nnan ninf means we don't need to consider NaN or Inf.
+define float @test_v7f32_nnan_ninf(<7 x float> %a) nounwind {
+; CHECK-LABEL: test_v7f32_nnan_ninf:
+; CHECK: // %bb.0:
+; CHECK-NEXT: mvni v7.4s, #128, lsl #16
+; CHECK-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-NEXT: // kill: def $s4 killed $s4 def $q4
+; CHECK-NEXT: // kill: def $s1 killed $s1 def $q1
+; CHECK-NEXT: // kill: def $s5 killed $s5 def $q5
+; CHECK-NEXT: // kill: def $s2 killed $s2 def $q2
+; CHECK-NEXT: // kill: def $s6 killed $s6 def $q6
+; CHECK-NEXT: // kill: def $s3 killed $s3 def $q3
+; CHECK-NEXT: mov v0.s[1], v1.s[0]
+; CHECK-NEXT: fneg v7.4s, v7.4s
+; CHECK-NEXT: mov v0.s[2], v2.s[0]
+; CHECK-NEXT: mov v7.s[0], v4.s[0]
+; CHECK-NEXT: mov v0.s[3], v3.s[0]
+; CHECK-NEXT: mov v7.s[1], v5.s[0]
+; CHECK-NEXT: mov v7.s[2], v6.s[0]
+; CHECK-NEXT: fminnm v0.4s, v0.4s, v7.4s
+; CHECK-NEXT: mov s1, v0.s[1]
+; CHECK-NEXT: mov s2, v0.s[2]
+; CHECK-NEXT: fminnm s1, s0, s1
+; CHECK-NEXT: mov s0, v0.s[3]
+; CHECK-NEXT: fminnm s1, s1, s2
+; CHECK-NEXT: fminnm s0, s1, s0
+; CHECK-NEXT: ret
+ %b = call nnan ninf float @llvm.vector.reduce.fminimumnum.v7f32(<7 x float> %a)
+ ret float %b
+}
+
diff --git a/llvm/test/CodeGen/Generic/expand-experimental-reductions.ll b/llvm/test/CodeGen/Generic/expand-experimental-reductions.ll
index b8e473cbaa4c8..241d98d3d6b9d 100644
--- a/llvm/test/CodeGen/Generic/expand-experimental-reductions.ll
+++ b/llvm/test/CodeGen/Generic/expand-experimental-reductions.ll
@@ -299,6 +299,36 @@ entry:
ret double %r
}
+define double @fmaximum_num_f64(<4 x double> %vec) {
+; CHECK-LABEL: @fmaximum_num_f64(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: [[RDX_SHUF:%.*]] = shufflevector <4 x double> [[VEC:%.*]], <4 x double> poison, <4 x i32> <i32 2, i32 3, i32 poison, i32 poison>
+; CHECK-NEXT: [[RDX_MINMAX:%.*]] = call <4 x double> @llvm.maximumnum.v4f64(<4 x double> [[VEC]], <4 x double> [[RDX_SHUF]])
+; CHECK-NEXT: [[RDX_SHUF1:%.*]] = shufflevector <4 x double> [[RDX_MINMAX]], <4 x double> poison, <4 x i32> <i32 1, i32 poison, i32 poison, i32 poison>
+; CHECK-NEXT: [[RDX_MINMAX2:%.*]] = call <4 x double> @llvm.maximumnum.v4f64(<4 x double> [[RDX_MINMAX]], <4 x double> [[RDX_SHUF1]])
+; CHECK-NEXT: [[TMP0:%.*]] = extractelement <4 x double> [[RDX_MINMAX2]], i32 0
+; CHECK-NEXT: ret double [[TMP0]]
+;
+entry:
+ %r = call double @llvm.vector.reduce.fmaximumnum.v4f64(<4 x double> %vec)
+ ret double %r
+}
+
+define double @fminimum_num_f64(<4 x double> %vec) {
+; CHECK-LABEL: @fminimum_num_f64(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: [[RDX_SHUF:%.*]] = shufflevector <4 x double> [[VEC:%.*]], <4 x double> poison, <4 x i32> <i32 2, i32 3, i32 poison, i32 poison>
+; CHECK-NEXT: [[RDX_MINMAX:%.*]] = call <4 x double> @llvm.minimumnum.v4f64(<4 x double> [[VEC]], <4 x double> [[RDX_SHUF]])
+; CHECK-NEXT: [[RDX_SHUF1:%.*]] = shufflevector <4 x double> [[RDX_MINMAX]], <4 x double> poison, <4 x i32> <i32 1, i32 poison, i32 poison, i32 poison>
+; CHECK-NEXT: [[RDX_MINMAX2:%.*]] = call <4 x double> @llvm.minimumnum.v4f64(<4 x double> [[RDX_MINMAX]], <4 x double> [[RDX_SHUF1]])
+; CHECK-NEXT: [[TMP0:%.*]] = extractelement <4 x double> [[RDX_MINMAX2]], i32 0
+; CHECK-NEXT: ret double [[TMP0]]
+;
+entry:
+ %r = call double @llvm.vector.reduce.fminimumnum.v4f64(<4 x double> %vec)
+ ret double %r
+}
+
; FIXME: Why is this not expanded?
; Test when the vector size is not power of two.
diff --git a/llvm/test/tools/llubi/intr_fp_vector_reduce.ll b/llvm/test/tools/llubi/intr_fp_vector_reduce.ll
index feccc03b7f1c4..b41a8d640c28c 100644
--- a/llvm/test/tools/llubi/intr_fp_vector_reduce.ll
+++ b/llvm/test/tools/llubi/intr_fp_vector_reduce.ll
@@ -24,6 +24,18 @@ define void @main() {
%fminimum = call float @llvm.vector.reduce.fminimum.v4f32(<4 x float> <float 1.0, float 2.0, float 4.0, float 5.0>)
%fminimum_poison = call float @llvm.vector.reduce.fminimum.v4f32(<4 x float> <float 1.0, float poison, float 4.0, float 5.0>)
+ %fmaximum_num = call float @llvm.vector.reduce.fmaximumnum.v4f32(<4 x float> <float 1.0, float 2.0, float 4.0, float 5.0>)
+ %fmaximum_num_poison = call float @llvm.vector.reduce.fmaximumnum.v4f32(<4 x float> <float 1.0, float poison, float 4.0, float 5.0>)
+ %fmaximum_num_nan = call float @llvm.vector.reduce.fmaximumnum.v4f32(<4 x float> <float 1.0, float 0x7FF8000000000000, float 4.0, float 5.0>)
+ %fmaximum_num_all_nan = call float @llvm.vector.reduce.fmaximumnum.v2f32(<2 x float> <float 0x7FF8000000000000, float 0x7FF8000000000000>)
+ %fmaximum_num_zeros = call float @llvm.vector.reduce.fmaximumnum.v2f32(<2 x float> <float -0.0, float 0.0>)
+
+ %fminimum_num = call float @llvm.vector.reduce.fminimumnum.v4f32(<4 x float> <float 1.0, float 2.0, float 4.0, float 5.0>)
+ %fminimum_num_poison = call float @llvm.vector.reduce.fminimumnum.v4f32(<4 x float> <float 1.0, float poison, float 4.0, float 5.0>)
+ %fminimum_num_nan = call float @llvm.vector.reduce.fminimumnum.v4f32(<4 x float> <float 0x7FF8000000000000, float 2.0, float 4.0, float 5.0>)
+ %fminimum_num_all_nan = call float @llvm.vector.reduce.fminimumnum.v2f32(<2 x float> <float 0x7FF8000000000000, float 0x7FF8000000000000>)
+ %fminimum_num_zeros = call float @llvm.vector.reduce.fminimumnum.v2f32(<2 x float> <float -0.0, float 0.0>)
+
%sv_poison = insertelement <vscale x 4 x float> splat (float 2.0), float poison, i64 1
%sv_fadd = call float @llvm.vector.reduce.fadd.nxv4f32(float 1.0, <vscale x 4 x float> splat (float 2.0))
%sv_fadd_poison_acc = call float @llvm.vector.reduce.fadd.nxv4f32(float poison, <vscale x 4 x float> splat (float 2.0))
@@ -44,6 +56,13 @@ define void @main() {
%sv_fminimum = call float @llvm.vector.reduce.fminimum.nxv4f32(<vscale x 4 x float> splat (float 2.0))
%sv_fminimum_poison = call float @llvm.vector.reduce.fminimum.nxv4f32(<vscale x 4 x float> %sv_poison)
+
+ %sv_fmaximum_num = call float @llvm.vector.reduce.fmaximumnum.nxv4f32(<vscale x 4 x float> splat (float 2.0))
+ %sv_fmaximum_num_poison = call float @llvm.vector.reduce.fmaximumnum.nxv4f32(<vscale x 4 x float> %sv_poison)
+
+ %sv_fminimum_num = call float @llvm.vector.reduce.fminimumnum.nxv4f32(<vscale x 4 x float> splat (float 2.0))
+ %sv_fminimum_num_poison = call float @llvm.vector.reduce.fminimumnum.nxv4f32(<vscale x 4 x float> %sv_poison)
+
ret void
}
; CHECK: Entering function: main
@@ -63,6 +82,16 @@ define void @main() {
; CHECK-NEXT: %fmaximum_poison = call float @llvm.vector.reduce.fmaximum.v4f32(<4 x float> <float 1.000000e+00, float poison, float 4.000000e+00, float 5.000000e+00>) => poison
; CHECK-NEXT: %fminimum = call float @llvm.vector.reduce.fminimum.v4f32(<4 x float> <float 1.000000e+00, float 2.000000e+00, float 4.000000e+00, float 5.000000e+00>) => float 1.000000e+00
; CHECK-NEXT: %fminimum_poison = call float @llvm.vector.reduce.fminimum.v4f32(<4 x float> <float 1.000000e+00, float poison, float 4.000000e+00, float 5.000000e+00>) => poison
+; CHECK-NEXT: %fmaximum_num = call float @llvm.vector.reduce.fmaximumnum.v4f32(<4 x float> <float 1.000000e+00, float 2.000000e+00, float 4.000000e+00, float 5.000000e+00>) => float 5.000000e+00
+; CHECK-NEXT: %fmaximum_num_poison = call float @llvm.vector.reduce.fmaximumnum.v4f32(<4 x float> <float 1.000000e+00, float poison, float 4.000000e+00, float 5.000000e+00>) => poison
+; CHECK-NEXT: %fmaximum_num_nan = call float @llvm.vector.reduce.fmaximumnum.v4f32(<4 x float> <float 1.000000e+00, float +qnan, float 4.000000e+00, float 5.000000e+00>) => float 5.000000e+00
+; CHECK-NEXT: %fmaximum_num_all_nan = call float @llvm.vector.reduce.fmaximumnum.v2f32(<2 x float> splat (float +qnan)) => float 0xFFC00000
+; CHECK-NEXT: %fmaximum_num_zeros = call float @llvm.vector.reduce.fmaximumnum.v2f32(<2 x float> <float -0.000000e+00, float 0.000000e+00>) => float 0.000000e+00
+; CHECK-NEXT: %fminimum_num = call float @llvm.vector.reduce.fminimumnum.v4f32(<4 x float> <float 1.000000e+00, float 2.000000e+00, float 4.000000e+00, float 5.000000e+00>) => float 1.000000e+00
+; CHECK-NEXT: %fminimum_num_poison = call float @llvm.vector.reduce.fminimumnum.v4f32(<4 x float> <float 1.000000e+00, float poison, float 4.000000e+00, float 5.000000e+00>) => poison
+; CHECK-NEXT: %fminimum_num_nan = call float @llvm.vector.reduce.fminimumnum.v4f32(<4 x float> <float +qnan, float 2.000000e+00, float 4.000000e+00, float 5.000000e+00>) => float 2.000000e+00
+; CHECK-NEXT: %fminimum_num_all_nan = call float @llvm.vector.reduce.fminimumnum.v2f32(<2 x float> splat (float +qnan)) => float 0xFFC00000
+; CHECK-NEXT: %fminimum_num_zeros = call float @llvm.vector.reduce.fminimumnum.v2f32(<2 x float> <float -0.000000e+00, float 0.000000e+00>) => float -0.000000e+00
; CHECK-NEXT: %sv_poison = insertelement <vscale x 4 x float> splat (float 2.000000e+00), float poison, i64 1 => { float 2.000000e+00, poison, float 2.000000e+00, float 2.000000e+00, float 2.000000e+00, float 2.000000e+00, float 2.000000e+00, float 2.000000e+00, float 2.000000e+00, float 2.000000e+00, float 2.000000e+00, float 2.000000e+00, float 2.000000e+00, float 2.000000e+00, float 2.000000e+00, float 2.000000e+00 }
; CHECK-NEXT: %sv_fadd = call float @llvm.vector.reduce.fadd.nxv4f32(float 1.000000e+00, <vscale x 4 x float> splat (float 2.000000e+00)) => float 3.300000e+01
; CHECK-NEXT: %sv_fadd_poison_acc = call float @llvm.vector.reduce.fadd.nxv4f32(float poison, <vscale x 4 x float> splat (float 2.000000e+00)) => poison
@@ -78,5 +107,9 @@ define void @main() {
; CHECK-NEXT: %sv_fmaximum_poison = call float @llvm.vector.reduce.fmaximum.nxv4f32(<vscale x 4 x float> %sv_poison) => poison
; CHECK-NEXT: %sv_fminimum = call float @llvm.vector.reduce.fminimum.nxv4f32(<vscale x 4 x float> splat (float 2.000000e+00)) => float 2.000000e+00
; CHECK-NEXT: %sv_fminimum_poison = call float @llvm.vector.reduce.fminimum.nxv4f32(<vscale x 4 x float> %sv_poison) => poison
+; CHECK-NEXT: %sv_fmaximum_num = call float @llvm.vector.reduce.fmaximumnum.nxv4f32(<vscale x 4 x float> splat (float 2.000000e+00)) => float 2.000000e+00
+; CHECK-NEXT: %sv_fmaximum_num_poison = call float @llvm.vector.reduce.fmaximumnum.nxv4f32(<vscale x 4 x float> %sv_poison) => poison
+; CHECK-NEXT: %sv_fminimum_num = call float @llvm.vector.reduce.fminimumnum.nxv4f32(<vscale x 4 x float> splat (float 2.000000e+00)) => float 2.000000e+00
+; CHECK-NEXT: %sv_fminimum_num_poison = call float @llvm.vector.reduce.fminimumnum.nxv4f32(<vscale x 4 x float> %sv_poison) => poison
; CHECK-NEXT: ret void
; CHECK-NEXT: Exiting function: main
diff --git a/llvm/tools/llubi/lib/Interpreter.cpp b/llvm/tools/llubi/lib/Interpreter.cpp
index 387e6aa062803..311d6aebbabaf 100644
--- a/llvm/tools/llubi/lib/Interpreter.cpp
+++ b/llvm/tools/llubi/lib/Interpreter.cpp
@@ -1518,7 +1518,9 @@ class InstExecutor : public InstVisitor<InstExecutor, void>,
case Intrinsic::vector_reduce_fadd:
case Intrinsic::vector_reduce_fmul:
case Intrinsic::vector_reduce_fmaximum:
- case Intrinsic::vector_reduce_fminimum: {
+ case Intrinsic::vector_reduce_fminimum:
+ case Intrinsic::vector_reduce_fmaximumnum:
+ case Intrinsic::vector_reduce_fminimumnum: {
const auto DenormMode = getCurrentDenormalMode(RetTy);
const bool HasStart = IID == Intrinsic::vector_reduce_fadd ||
IID == Intrinsic::vector_reduce_fmul;
@@ -1559,6 +1561,12 @@ class InstExecutor : public InstVisitor<InstExecutor, void>,
case Intrinsic::vector_reduce_fminimum:
*Res = minimum(*Res, Op);
break;
+ case Intrinsic::vector_reduce_fmaximumnum:
+ *Res = maximumnum(*Res, Op);
+ break;
+ case Intrinsic::vector_reduce_fminimumnum:
+ *Res = minimumnum(*Res, Op);
+ break;
default:
llvm_unreachable("Unexpected intrinsic ID");
}
>From 4bb5f0a35861812cbde9c510b37cc8c934b715ce Mon Sep 17 00:00:00 2001
From: Folkert de Vries <folkert at folkertdev.nl>
Date: Sun, 23 Aug 2026 22:09:56 +0200
Subject: [PATCH 2/2] fix gisel
---
llvm/docs/GlobalISel/GenericOpcode.rst | 7 ++-
.../CodeGen/GlobalISel/GenericMachineInstrs.h | 8 +++
llvm/include/llvm/CodeGen/GlobalISel/Utils.h | 4 ++
llvm/include/llvm/Support/TargetOpcodes.def | 2 +
llvm/include/llvm/Target/GenericOpcodes.td | 2 +
.../Target/GlobalISel/SelectionDAGCompat.td | 2 +
.../CodeGen/GlobalISel/GISelValueTracking.cpp | 4 +-
llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp | 4 ++
llvm/lib/CodeGen/MachineVerifier.cpp | 2 +
.../GlobalISel/legalizer-info-validation.mir | 6 ++
.../CodeGen/AArch64/vecreduce-fmaximumnum.ll | 57 ++++++++++++++++++-
.../CodeGen/AArch64/vecreduce-fminimumnum.ll | 57 ++++++++++++++++++-
.../GlobalISel/legalizer-info-validation.mir | 6 ++
llvm/test/TableGen/get-named-operand-idx.td | 3 +-
14 files changed, 153 insertions(+), 11 deletions(-)
diff --git a/llvm/docs/GlobalISel/GenericOpcode.rst b/llvm/docs/GlobalISel/GenericOpcode.rst
index d2552b6c3a3f3..6b54132461bb5 100644
--- a/llvm/docs/GlobalISel/GenericOpcode.rst
+++ b/llvm/docs/GlobalISel/GenericOpcode.rst
@@ -900,10 +900,11 @@ G_VECREDUCE_FADD, G_VECREDUCE_FMUL
These reductions are relaxed variants which may reduce the elements in any order.
-G_VECREDUCE_FMAX, G_VECREDUCE_FMIN, G_VECREDUCE_FMAXIMUM, G_VECREDUCE_FMINIMUM
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+G_VECREDUCE_FMAX, G_VECREDUCE_FMIN, G_VECREDUCE_FMAXIMUM, G_VECREDUCE_FMINIMUM, G_VECREDUCE_FMAXIMUMNUM, G_VECREDUCE_FMINIMUMNUM
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-FMIN/FMAX/FMINIMUM/FMAXIMUM nodes can have flags, for NaN/NoNaN variants.
+FMIN/FMAX/FMINIMUM/FMAXIMUM/FMINIMUMNUM/FMAXIMUMNUM nodes can have flags, for
+NaN/NoNaN variants.
Integer/bitwise reductions
diff --git a/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h b/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h
index 6daa7f3808f43..0a34fb2726423 100644
--- a/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h
+++ b/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h
@@ -615,6 +615,8 @@ class GVecReduce : public GenericMachineInstr {
case TargetOpcode::G_VECREDUCE_FMIN:
case TargetOpcode::G_VECREDUCE_FMAXIMUM:
case TargetOpcode::G_VECREDUCE_FMINIMUM:
+ case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM:
+ case TargetOpcode::G_VECREDUCE_FMINIMUMNUM:
case TargetOpcode::G_VECREDUCE_ADD:
case TargetOpcode::G_VECREDUCE_MUL:
case TargetOpcode::G_VECREDUCE_AND:
@@ -653,6 +655,12 @@ class GVecReduce : public GenericMachineInstr {
case TargetOpcode::G_VECREDUCE_FMINIMUM:
ScalarOpc = TargetOpcode::G_FMINIMUM;
break;
+ case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM:
+ ScalarOpc = TargetOpcode::G_FMAXIMUMNUM;
+ break;
+ case TargetOpcode::G_VECREDUCE_FMINIMUMNUM:
+ ScalarOpc = TargetOpcode::G_FMINIMUMNUM;
+ break;
case TargetOpcode::G_VECREDUCE_ADD:
ScalarOpc = TargetOpcode::G_ADD;
break;
diff --git a/llvm/include/llvm/CodeGen/GlobalISel/Utils.h b/llvm/include/llvm/CodeGen/GlobalISel/Utils.h
index 2feaf76aaeaa6..7b339211cc3ca 100644
--- a/llvm/include/llvm/CodeGen/GlobalISel/Utils.h
+++ b/llvm/include/llvm/CodeGen/GlobalISel/Utils.h
@@ -64,6 +64,8 @@ class APFloat;
case TargetOpcode::G_VECREDUCE_FMIN: \
case TargetOpcode::G_VECREDUCE_FMAXIMUM: \
case TargetOpcode::G_VECREDUCE_FMINIMUM: \
+ case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM: \
+ case TargetOpcode::G_VECREDUCE_FMINIMUMNUM: \
case TargetOpcode::G_VECREDUCE_ADD: \
case TargetOpcode::G_VECREDUCE_MUL: \
case TargetOpcode::G_VECREDUCE_AND: \
@@ -81,6 +83,8 @@ class APFloat;
case TargetOpcode::G_VECREDUCE_FMIN: \
case TargetOpcode::G_VECREDUCE_FMAXIMUM: \
case TargetOpcode::G_VECREDUCE_FMINIMUM: \
+ case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM: \
+ case TargetOpcode::G_VECREDUCE_FMINIMUMNUM: \
case TargetOpcode::G_VECREDUCE_ADD: \
case TargetOpcode::G_VECREDUCE_MUL: \
case TargetOpcode::G_VECREDUCE_AND: \
diff --git a/llvm/include/llvm/Support/TargetOpcodes.def b/llvm/include/llvm/Support/TargetOpcodes.def
index 051cea764e5a0..549a93b7bec3a 100644
--- a/llvm/include/llvm/Support/TargetOpcodes.def
+++ b/llvm/include/llvm/Support/TargetOpcodes.def
@@ -976,6 +976,8 @@ HANDLE_TARGET_OPCODE(G_VECREDUCE_FMAX)
HANDLE_TARGET_OPCODE(G_VECREDUCE_FMIN)
HANDLE_TARGET_OPCODE(G_VECREDUCE_FMAXIMUM)
HANDLE_TARGET_OPCODE(G_VECREDUCE_FMINIMUM)
+HANDLE_TARGET_OPCODE(G_VECREDUCE_FMAXIMUMNUM)
+HANDLE_TARGET_OPCODE(G_VECREDUCE_FMINIMUMNUM)
HANDLE_TARGET_OPCODE(G_VECREDUCE_ADD)
HANDLE_TARGET_OPCODE(G_VECREDUCE_MUL)
HANDLE_TARGET_OPCODE(G_VECREDUCE_AND)
diff --git a/llvm/include/llvm/Target/GenericOpcodes.td b/llvm/include/llvm/Target/GenericOpcodes.td
index b7b216b552a04..a95f297660923 100644
--- a/llvm/include/llvm/Target/GenericOpcodes.td
+++ b/llvm/include/llvm/Target/GenericOpcodes.td
@@ -1801,6 +1801,8 @@ def G_VECREDUCE_FMAX : VectorReduction;
def G_VECREDUCE_FMIN : VectorReduction;
def G_VECREDUCE_FMAXIMUM : VectorReduction;
def G_VECREDUCE_FMINIMUM : VectorReduction;
+def G_VECREDUCE_FMAXIMUMNUM : VectorReduction;
+def G_VECREDUCE_FMINIMUMNUM : VectorReduction;
def G_VECREDUCE_ADD : VectorReduction;
def G_VECREDUCE_MUL : VectorReduction;
diff --git a/llvm/include/llvm/Target/GlobalISel/SelectionDAGCompat.td b/llvm/include/llvm/Target/GlobalISel/SelectionDAGCompat.td
index e06c96fa837df..a91ef6058b7e4 100644
--- a/llvm/include/llvm/Target/GlobalISel/SelectionDAGCompat.td
+++ b/llvm/include/llvm/Target/GlobalISel/SelectionDAGCompat.td
@@ -215,6 +215,8 @@ def : GINodeEquiv<G_VECREDUCE_FMAX, vecreduce_fmax>;
def : GINodeEquiv<G_VECREDUCE_FMIN, vecreduce_fmin>;
def : GINodeEquiv<G_VECREDUCE_FMAXIMUM, vecreduce_fmaximum>;
def : GINodeEquiv<G_VECREDUCE_FMINIMUM, vecreduce_fminimum>;
+def : GINodeEquiv<G_VECREDUCE_FMAXIMUMNUM, vecreduce_fmaximumnum>;
+def : GINodeEquiv<G_VECREDUCE_FMINIMUMNUM, vecreduce_fminimumnum>;
def : GINodeEquiv<G_VECREDUCE_UMIN, vecreduce_umin>;
def : GINodeEquiv<G_VECREDUCE_UMAX, vecreduce_umax>;
def : GINodeEquiv<G_VECREDUCE_SMIN, vecreduce_smin>;
diff --git a/llvm/lib/CodeGen/GlobalISel/GISelValueTracking.cpp b/llvm/lib/CodeGen/GlobalISel/GISelValueTracking.cpp
index 0672cfffc6eac..31748e545f3c6 100644
--- a/llvm/lib/CodeGen/GlobalISel/GISelValueTracking.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/GISelValueTracking.cpp
@@ -1559,7 +1559,9 @@ void GISelValueTracking::computeKnownFPClass(Register R,
case TargetOpcode::G_VECREDUCE_FMAX:
case TargetOpcode::G_VECREDUCE_FMIN:
case TargetOpcode::G_VECREDUCE_FMAXIMUM:
- case TargetOpcode::G_VECREDUCE_FMINIMUM: {
+ case TargetOpcode::G_VECREDUCE_FMINIMUM:
+ case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM:
+ case TargetOpcode::G_VECREDUCE_FMINIMUMNUM: {
Register Val = MI.getOperand(1).getReg();
// reduce min/max will choose an element from one of the vector elements,
// so we can infer and class information that is common to all elements.
diff --git a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
index baac98561f25b..3441214206561 100644
--- a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
@@ -2824,6 +2824,10 @@ unsigned IRTranslatorImpl::getSimpleIntrinsicOpcode(Intrinsic::ID ID) {
return TargetOpcode::G_VECREDUCE_FMINIMUM;
case Intrinsic::vector_reduce_fmaximum:
return TargetOpcode::G_VECREDUCE_FMAXIMUM;
+ case Intrinsic::vector_reduce_fminimumnum:
+ return TargetOpcode::G_VECREDUCE_FMINIMUMNUM;
+ case Intrinsic::vector_reduce_fmaximumnum:
+ return TargetOpcode::G_VECREDUCE_FMAXIMUMNUM;
case Intrinsic::vector_reduce_add:
return TargetOpcode::G_VECREDUCE_ADD;
case Intrinsic::vector_reduce_mul:
diff --git a/llvm/lib/CodeGen/MachineVerifier.cpp b/llvm/lib/CodeGen/MachineVerifier.cpp
index 8c22487a1a213..0d3015e098cf2 100644
--- a/llvm/lib/CodeGen/MachineVerifier.cpp
+++ b/llvm/lib/CodeGen/MachineVerifier.cpp
@@ -2198,6 +2198,8 @@ void MachineVerifier::verifyPreISelGenericInstruction(const MachineInstr *MI) {
case TargetOpcode::G_VECREDUCE_FMIN:
case TargetOpcode::G_VECREDUCE_FMAXIMUM:
case TargetOpcode::G_VECREDUCE_FMINIMUM:
+ case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM:
+ case TargetOpcode::G_VECREDUCE_FMINIMUMNUM:
case TargetOpcode::G_VECREDUCE_ADD:
case TargetOpcode::G_VECREDUCE_MUL:
case TargetOpcode::G_VECREDUCE_AND:
diff --git a/llvm/test/CodeGen/AArch64/GlobalISel/legalizer-info-validation.mir b/llvm/test/CodeGen/AArch64/GlobalISel/legalizer-info-validation.mir
index eb8293e1b4169..e45efc026d397 100644
--- a/llvm/test/CodeGen/AArch64/GlobalISel/legalizer-info-validation.mir
+++ b/llvm/test/CodeGen/AArch64/GlobalISel/legalizer-info-validation.mir
@@ -954,6 +954,12 @@
# DEBUG-NEXT: .. opcode {{[0-9]+}} is aliased to {{[0-9]+}}
# DEBUG-NEXT: .. type index coverage check SKIPPED: user-defined predicate detected
# DEBUG-NEXT: .. imm index coverage check SKIPPED: user-defined predicate detected
+# DEBUG-NEXT: G_VECREDUCE_FMAXIMUMNUM (opcode {{[0-9]+}}): 2 type indices, 0 imm indices
+# DEBUG-NEXT: .. type index coverage check SKIPPED: no rules defined
+# DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined
+# DEBUG-NEXT: G_VECREDUCE_FMINIMUMNUM (opcode {{[0-9]+}}): 2 type indices, 0 imm indices
+# DEBUG-NEXT: .. type index coverage check SKIPPED: no rules defined
+# DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined
# DEBUG-NEXT: G_VECREDUCE_ADD (opcode {{[0-9]+}}): 2 type indices, 0 imm indices
# DEBUG-NEXT: .. the first uncovered type index: 2, OK
# DEBUG-NEXT: .. the first uncovered imm index: 0, OK
diff --git a/llvm/test/CodeGen/AArch64/vecreduce-fmaximumnum.ll b/llvm/test/CodeGen/AArch64/vecreduce-fmaximumnum.ll
index 42c4e3566ace4..75468cb45a2ad 100644
--- a/llvm/test/CodeGen/AArch64/vecreduce-fmaximumnum.ll
+++ b/llvm/test/CodeGen/AArch64/vecreduce-fmaximumnum.ll
@@ -1,6 +1,53 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-NOFP
-; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon,+fullfp16 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-FP
+; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-NOFP --check-prefix=CHECK-NOFP-SD
+; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon,+fullfp16 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-FP --check-prefix=CHECK-FP-SD
+; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon -global-isel -global-isel-abort=2 2>&1 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-NOFP --check-prefix=CHECK-NOFP-GI
+; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon,+fullfp16 -global-isel -global-isel-abort=2 2>&1 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-FP --check-prefix=CHECK-FP-GI
+
+; GlobalISel has no legalizer rules for G_VECREDUCE_F{MAX,MIN}IMUMNUM yet, so
+; every function below falls back to SelectionDAG.
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v1f16
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v1f32
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v1f64
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v1f128
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v4f16
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v8f16
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v16f16
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v2f32
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v4f32
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v8f32
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v16f32
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v2f64
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v4f64
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v11f16
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v3f32
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v3f32_ninf
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v2f128
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v7f32
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v7f32_nnan
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v7f32_nnan_ninf
+;
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v1f16
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v1f32
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v1f64
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v1f128
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v4f16
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v8f16
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v16f16
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v2f32
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v4f32
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v8f32
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v16f32
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v2f64
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v4f64
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v11f16
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v3f32
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v3f32_ninf
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v2f128
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v7f32
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v7f32_nnan
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v7f32_nnan_ninf
+;
define half @test_v1f16(<1 x half> %a) nounwind {
; CHECK-LABEL: test_v1f16:
@@ -629,4 +676,8 @@ define float @test_v7f32_nnan_ninf(<7 x float> %a) nounwind {
%b = call nnan ninf float @llvm.vector.reduce.fmaximumnum.v7f32(<7 x float> %a)
ret float %b
}
-
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; CHECK-FP-GI: {{.*}}
+; CHECK-FP-SD: {{.*}}
+; CHECK-NOFP-GI: {{.*}}
+; CHECK-NOFP-SD: {{.*}}
diff --git a/llvm/test/CodeGen/AArch64/vecreduce-fminimumnum.ll b/llvm/test/CodeGen/AArch64/vecreduce-fminimumnum.ll
index 0881f6ec76c14..bbbefffc06758 100644
--- a/llvm/test/CodeGen/AArch64/vecreduce-fminimumnum.ll
+++ b/llvm/test/CodeGen/AArch64/vecreduce-fminimumnum.ll
@@ -1,6 +1,53 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-NOFP
-; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon,+fullfp16 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-FP
+; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-NOFP --check-prefix=CHECK-NOFP-SD
+; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon,+fullfp16 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-FP --check-prefix=CHECK-FP-SD
+; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon -global-isel -global-isel-abort=2 2>&1 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-NOFP --check-prefix=CHECK-NOFP-GI
+; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+neon,+fullfp16 -global-isel -global-isel-abort=2 2>&1 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-FP --check-prefix=CHECK-FP-GI
+
+; GlobalISel has no legalizer rules for G_VECREDUCE_F{MAX,MIN}IMUMNUM yet, so
+; every function below falls back to SelectionDAG.
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v1f16
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v1f32
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v1f64
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v1f128
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v4f16
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v8f16
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v16f16
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v2f32
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v4f32
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v8f32
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v16f32
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v2f64
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v4f64
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v11f16
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v3f32
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v3f32_ninf
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v2f128
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v7f32
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v7f32_nnan
+; CHECK-NOFP-GI: warning: Instruction selection used fallback path for test_v7f32_nnan_ninf
+;
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v1f16
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v1f32
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v1f64
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v1f128
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v4f16
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v8f16
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v16f16
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v2f32
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v4f32
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v8f32
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v16f32
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v2f64
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v4f64
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v11f16
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v3f32
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v3f32_ninf
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v2f128
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v7f32
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v7f32_nnan
+; CHECK-FP-GI: warning: Instruction selection used fallback path for test_v7f32_nnan_ninf
+;
define half @test_v1f16(<1 x half> %a) nounwind {
; CHECK-LABEL: test_v1f16:
@@ -633,4 +680,8 @@ define float @test_v7f32_nnan_ninf(<7 x float> %a) nounwind {
%b = call nnan ninf float @llvm.vector.reduce.fminimumnum.v7f32(<7 x float> %a)
ret float %b
}
-
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; CHECK-FP-GI: {{.*}}
+; CHECK-FP-SD: {{.*}}
+; CHECK-NOFP-GI: {{.*}}
+; CHECK-NOFP-SD: {{.*}}
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/legalizer-info-validation.mir b/llvm/test/CodeGen/RISCV/GlobalISel/legalizer-info-validation.mir
index 1770bc6430eb9..90b1c8fc4c225 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/legalizer-info-validation.mir
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/legalizer-info-validation.mir
@@ -947,6 +947,12 @@
# DEBUG-NEXT: G_VECREDUCE_FMINIMUM (opcode {{[0-9]+}}): 2 type indices, 0 imm indices
# DEBUG-NEXT: .. type index coverage check SKIPPED: no rules defined
# DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined
+# DEBUG-NEXT: G_VECREDUCE_FMAXIMUMNUM (opcode {{[0-9]+}}): 2 type indices, 0 imm indices
+# DEBUG-NEXT: .. type index coverage check SKIPPED: no rules defined
+# DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined
+# DEBUG-NEXT: G_VECREDUCE_FMINIMUMNUM (opcode {{[0-9]+}}): 2 type indices, 0 imm indices
+# DEBUG-NEXT: .. type index coverage check SKIPPED: no rules defined
+# DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined
# DEBUG-NEXT: G_VECREDUCE_ADD (opcode {{[0-9]+}}): 2 type indices, 0 imm indices
# DEBUG-NEXT: .. type index coverage check SKIPPED: no rules defined
# DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined
diff --git a/llvm/test/TableGen/get-named-operand-idx.td b/llvm/test/TableGen/get-named-operand-idx.td
index 5e6a9859f3603..31de6faec0da4 100644
--- a/llvm/test/TableGen/get-named-operand-idx.td
+++ b/llvm/test/TableGen/get-named-operand-idx.td
@@ -98,7 +98,8 @@ defm : RemapAllTargetPseudoPointerOperands<RegClass>;
// CHECK-NEXT: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
// CHECK-NEXT: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
// CHECK-NEXT: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-// CHECK-NEXT: 1, 2, 2, 0,
+// CHECK-NEXT: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 2,
+// CHECK-NEXT: 0,
// CHECK-NEXT: };
// CHECK-NEXT: return InstructionIndex[Opcode];
// CHECK-NEXT: }
More information about the llvm-commits
mailing list