[llvm] Add new llvm.cbrt intrinsic (PR #178396)
David Sherwood via llvm-commits
llvm-commits at lists.llvm.org
Wed Jan 28 02:46:56 PST 2026
https://github.com/david-arm created https://github.com/llvm/llvm-project/pull/178396
In CodeGen we already have support for the FCBRT DAG node that gets lowered to a library call on targets where the libcall cbrt is available. This node gets created during a DAG combine that converts
fpow(float x, float 1.f/3.f) -> fcbrt(float x)
and
fpow(double x, double 1./3.) -> fcbrt(double x)
when the appropriate fast-math flags are set on the fpow.
I would like to add an equivalent LLVM intrinsic for cbrt for two reasons:
1. The cbrt math call seems to be a commonly available math call on most systems. Adding a cbrt intrinsic would be consistent with similar intrinsics such as pow, sqrt, etc. There is currently no builtin support for cbrt in clang, but this can be added in a later patch.
2. It permits the use of a similar IR level libcall simplification:
llvm.pow.f32(float x, float 1.0f/3.0f) ->
llvm.cbrt.f32(float x)
There is already a precedence for this in
Transforms/Utils/SimplifyLibCalls.cpp where we currently simplify
llvm.pow.f32(float x, float 0.25) ->
llvm.sqrt.f32(float x)
provided the intrinsic has the appropriate flags. There is already an existing test in test/Transforms/InstCombine/pow-cbrt.ll and presumably this was added with the intention of adding such a transformation. Doing this unlocks performance gains on targets that have vector math library support for cbrt, since both the loop and SLP vectorisers are able to take advantage of them.
>From e5553c0fb83d7819df400ddedd225a4ad6062a31 Mon Sep 17 00:00:00 2001
From: David Sherwood <david.sherwood at arm.com>
Date: Wed, 28 Jan 2026 10:33:11 +0000
Subject: [PATCH] Add new llvm.cbrt intrinsic
In CodeGen we already have support for the FCBRT DAG node that
gets lowered to a library call on targets where the libcall cbrt
is available. This node gets created during a DAG combine that
converts
fpow(float x, float 1.f/3.f) -> fcbrt(float x)
and
fpow(double x, double 1./3.) -> fcbrt(double x)
when the appropriate fast-math flags are set on the fpow.
I would like to add an equivalent LLVM intrinsic for cbrt for
two reasons:
1. The cbrt math call seems to be a commonly available math call
on most systems. Adding a cbrt intrinsic would be consistent with
similar intrinsics such as pow, sqrt, etc. There is currently no
builtin support for cbrt in clang, but this can be added in a
later patch.
2. It permits the use of a similar IR level libcall simplification:
llvm.pow.f32(float x, float 1.0f/3.0f) ->
llvm.cbrt.f32(float x)
There is already a precedence for this in
Transforms/Utils/SimplifyLibCalls.cpp where we currently simplify
llvm.pow.f32(float x, float 0.25) ->
llvm.sqrt.f32(float x)
provided the intrinsic has the appropriate flags. Doing so unlocks
performance gains on targets that have vector math library support
for cbrt, since both the loop and SLP vectorisers are able to take
advantage of them.
---
llvm/docs/LangRef.rst | 40 +
llvm/include/llvm/IR/Intrinsics.td | 1 +
llvm/include/llvm/Support/TargetOpcodes.def | 3 +
llvm/include/llvm/Target/GenericOpcodes.td | 7 +
llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp | 2 +
.../CodeGen/GlobalISel/LegalizerHelper.cpp | 6 +
llvm/lib/CodeGen/GlobalISel/Utils.cpp | 1 +
llvm/lib/CodeGen/IntrinsicLowering.cpp | 4 +
llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp | 1 +
.../SelectionDAG/LegalizeVectorOps.cpp | 2 +
.../SelectionDAG/LegalizeVectorTypes.cpp | 3 +
.../SelectionDAG/SelectionDAGBuilder.cpp | 8 +
.../Target/AArch64/AArch64ISelLowering.cpp | 14 +-
.../AArch64/GISel/AArch64LegalizerInfo.cpp | 2 +-
.../GlobalISel/legalizer-info-validation.mir | 4 +
llvm/test/CodeGen/AArch64/cbrt.ll | 1305 +++++++++++++++++
.../AArch64/fast-isel-runtime-libcall.ll | 24 +
.../AArch64/replace-with-veclib-armpl.ll | 45 +
.../replace-with-veclib-libmvec-scalable.ll | 23 +-
.../AArch64/replace-with-veclib-libmvec.ll | 21 +-
.../replace-with-veclib-sleef-scalable.ll | 23 +-
.../AArch64/replace-with-veclib-sleef.ll | 23 +-
llvm/test/CodeGen/ARM/cbrt.ll | 131 ++
.../GlobalISel/legalizer-info-validation.mir | 3 +
llvm/test/CodeGen/RISCV/double-intrinsics.ll | 40 +
llvm/test/CodeGen/RISCV/float-intrinsics.ll | 42 +
llvm/test/CodeGen/X86/cbrt.ll | 78 +
27 files changed, 1844 insertions(+), 12 deletions(-)
create mode 100644 llvm/test/CodeGen/AArch64/cbrt.ll
create mode 100644 llvm/test/CodeGen/ARM/cbrt.ll
create mode 100644 llvm/test/CodeGen/X86/cbrt.ll
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index a6881099f81f4..1814d29feca1b 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -16331,6 +16331,46 @@ matches a conforming libm implementation.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
+.. _int_cbrt:
+
+'``llvm.cbrt.*``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+This is an overloaded intrinsic. You can use ``llvm.cbrt`` on any
+floating-point or vector of floating-point type. Not all targets support
+all types however.
+
+::
+
+ declare float @llvm.cbrt.f32(float %Val)
+ declare double @llvm.cbrt.f64(double %Val)
+ declare x86_fp80 @llvm.cbrt.f80(x86_fp80 %Val)
+ declare fp128 @llvm.cbrt.f128(fp128 %Val)
+ declare ppc_fp128 @llvm.cbrt.ppcf128(ppc_fp128 %Val)
+
+Overview:
+"""""""""
+
+The '``llvm.cbrt``' intrinsics return the square root of the specified value.
+
+Arguments:
+""""""""""
+
+The argument and return value are floating-point numbers of the same type.
+
+Semantics:
+""""""""""
+
+Return the same value as a corresponding libm '``cbrt``' function but without
+trapping or setting ``errno``. For types specified by IEEE-754, the result
+matches a conforming libm implementation.
+
+When specified with the fast-math-flag 'afn', the result may be approximated
+using a less accurate calculation.
+
'``llvm.powi.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/llvm/include/llvm/IR/Intrinsics.td b/llvm/include/llvm/IR/Intrinsics.td
index 24d51bdaf67fb..9981dc47a1760 100644
--- a/llvm/include/llvm/IR/Intrinsics.td
+++ b/llvm/include/llvm/IR/Intrinsics.td
@@ -1107,6 +1107,7 @@ let IntrProperties = [IntrNoMem, IntrSpeculatable, IntrNoCreateUndefOrPoison] in
// rounding mode. LLVM purposely does not model changes to the FP
// environment so they can be treated as readnone.
def int_sqrt : DefaultAttrsIntrinsic<[llvm_anyfloat_ty], [LLVMMatchType<0>]>;
+ def int_cbrt : DefaultAttrsIntrinsic<[llvm_anyfloat_ty], [LLVMMatchType<0>]>;
def int_powi : DefaultAttrsIntrinsic<[llvm_anyfloat_ty], [LLVMMatchType<0>, llvm_anyint_ty]>;
def int_sin : DefaultAttrsIntrinsic<[llvm_anyfloat_ty], [LLVMMatchType<0>]>;
def int_cos : DefaultAttrsIntrinsic<[llvm_anyfloat_ty], [LLVMMatchType<0>]>;
diff --git a/llvm/include/llvm/Support/TargetOpcodes.def b/llvm/include/llvm/Support/TargetOpcodes.def
index 3217ffafc235a..ff8ac18c9c677 100644
--- a/llvm/include/llvm/Support/TargetOpcodes.def
+++ b/llvm/include/llvm/Support/TargetOpcodes.def
@@ -890,6 +890,9 @@ HANDLE_TARGET_OPCODE(G_FTANH)
/// Floating point square root.
HANDLE_TARGET_OPCODE(G_FSQRT)
+/// Floating point cube root.
+HANDLE_TARGET_OPCODE(G_FCBRT)
+
/// Floating point floor.
HANDLE_TARGET_OPCODE(G_FFLOOR)
diff --git a/llvm/include/llvm/Target/GenericOpcodes.td b/llvm/include/llvm/Target/GenericOpcodes.td
index b785847b53f0f..e91ad748d7f74 100644
--- a/llvm/include/llvm/Target/GenericOpcodes.td
+++ b/llvm/include/llvm/Target/GenericOpcodes.td
@@ -1186,6 +1186,13 @@ def G_FSQRT : GenericInstruction {
let hasSideEffects = false;
}
+// Floating point cube root.
+def G_FCBRT : GenericInstruction {
+ let OutOperandList = (outs type0:$dst);
+ let InOperandList = (ins type0:$src1);
+ let hasSideEffects = true;
+}
+
// Floating point floor of a value.
def G_FFLOOR : GenericInstruction {
let OutOperandList = (outs type0:$dst);
diff --git a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
index 63ca973bc224e..87c6ab328eaf3 100644
--- a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
@@ -1931,6 +1931,8 @@ unsigned IRTranslator::getSimpleIntrinsicOpcode(Intrinsic::ID ID) {
return TargetOpcode::G_FSHL;
case Intrinsic::fshr:
return TargetOpcode::G_FSHR;
+ case Intrinsic::cbrt:
+ return TargetOpcode::G_FCBRT;
case Intrinsic::ceil:
return TargetOpcode::G_FCEIL;
case Intrinsic::cos:
diff --git a/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp b/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
index 260fc6c4050bd..5e554ea03749c 100644
--- a/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
@@ -497,6 +497,8 @@ static RTLIB::Libcall getRTLibDesc(unsigned Opcode, unsigned Size) {
RTLIBCASE(FMAXIMUM_NUM_F);
case TargetOpcode::G_FSQRT:
RTLIBCASE(SQRT_F);
+ case TargetOpcode::G_FCBRT:
+ RTLIBCASE(CBRT_F);
case TargetOpcode::G_FRINT:
RTLIBCASE(RINT_F);
case TargetOpcode::G_FNEARBYINT:
@@ -1361,6 +1363,7 @@ LegalizerHelper::libcall(MachineInstr &MI, LostDebugLocObserver &LocObserver) {
case TargetOpcode::G_FMINIMUMNUM:
case TargetOpcode::G_FMAXIMUMNUM:
case TargetOpcode::G_FSQRT:
+ case TargetOpcode::G_FCBRT:
case TargetOpcode::G_FRINT:
case TargetOpcode::G_FNEARBYINT:
case TargetOpcode::G_INTRINSIC_TRUNC:
@@ -3387,6 +3390,7 @@ LegalizerHelper::widenScalar(MachineInstr &MI, unsigned TypeIdx, LLT WideTy) {
case TargetOpcode::G_FRINT:
case TargetOpcode::G_FNEARBYINT:
case TargetOpcode::G_FSQRT:
+ case TargetOpcode::G_FCBRT:
case TargetOpcode::G_FEXP:
case TargetOpcode::G_FEXP2:
case TargetOpcode::G_FEXP10:
@@ -5579,6 +5583,7 @@ LegalizerHelper::fewerElementsVector(MachineInstr &MI, unsigned TypeIdx,
case G_FSINH:
case G_FTANH:
case G_FSQRT:
+ case G_FCBRT:
case G_BSWAP:
case G_BITREVERSE:
case G_SDIV:
@@ -6745,6 +6750,7 @@ LegalizerHelper::moreElementsVector(MachineInstr &MI, unsigned TypeIdx,
case TargetOpcode::G_FNEG:
case TargetOpcode::G_FABS:
case TargetOpcode::G_FSQRT:
+ case TargetOpcode::G_FCBRT:
case TargetOpcode::G_FCEIL:
case TargetOpcode::G_FFLOOR:
case TargetOpcode::G_FNEARBYINT:
diff --git a/llvm/lib/CodeGen/GlobalISel/Utils.cpp b/llvm/lib/CodeGen/GlobalISel/Utils.cpp
index 658774ec3fcb9..2edfee33b2cc7 100644
--- a/llvm/lib/CodeGen/GlobalISel/Utils.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/Utils.cpp
@@ -1791,6 +1791,7 @@ bool llvm::isPreISelGenericFloatingPointOpcode(unsigned Opc) {
case TargetOpcode::G_FSINH:
case TargetOpcode::G_FTANH:
case TargetOpcode::G_FSQRT:
+ case TargetOpcode::G_FCBRT:
case TargetOpcode::G_FSUB:
case TargetOpcode::G_INTRINSIC_ROUND:
case TargetOpcode::G_INTRINSIC_ROUNDEVEN:
diff --git a/llvm/lib/CodeGen/IntrinsicLowering.cpp b/llvm/lib/CodeGen/IntrinsicLowering.cpp
index 1518ead7698be..b1a568561564b 100644
--- a/llvm/lib/CodeGen/IntrinsicLowering.cpp
+++ b/llvm/lib/CodeGen/IntrinsicLowering.cpp
@@ -387,6 +387,10 @@ void IntrinsicLowering::LowerIntrinsicCall(CallInst *CI) {
ReplaceFPIntrinsicWithCall(CI, "sqrtf", "sqrt", "sqrtl");
break;
}
+ case Intrinsic::cbrt: {
+ ReplaceFPIntrinsicWithCall(CI, "cbrtf", "cbrt", "cbrtl");
+ break;
+ }
case Intrinsic::log: {
ReplaceFPIntrinsicWithCall(CI, "logf", "log", "logl");
break;
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
index 09d79ae208a8f..f9e2f7642e234 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
@@ -5922,6 +5922,7 @@ void SelectionDAGLegalize::PromoteNode(SDNode *Node) {
case ISD::FTRUNC:
case ISD::FNEG:
case ISD::FSQRT:
+ case ISD::FCBRT:
case ISD::FSIN:
case ISD::FCOS:
case ISD::FTAN:
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp
index 8f6eddefa57ac..53de9a215ef56 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp
@@ -409,6 +409,7 @@ SDValue VectorLegalizer::LegalizeOp(SDValue Op) {
case ISD::FMINIMUMNUM:
case ISD::FMAXIMUMNUM:
case ISD::FCOPYSIGN:
+ case ISD::FCBRT:
case ISD::FSQRT:
case ISD::FSIN:
case ISD::FCOS:
@@ -1354,6 +1355,7 @@ void VectorLegalizer::Expand(SDNode *Node, SmallVectorImpl<SDValue> &Results) {
case ISD::FROUNDEVEN:
case ISD::FTRUNC:
case ISD::FSQRT:
+ case ISD::FCBRT:
if (SDValue Expanded = TLI.expandVectorNaryOpBySplitting(Node, DAG)) {
Results.push_back(Expanded);
return;
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp
index 0b15d5915d9d0..61d7383836d05 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp
@@ -128,6 +128,7 @@ void DAGTypeLegalizer::ScalarizeVectorResult(SDNode *N, unsigned ResNo) {
case ISD::FSIN:
case ISD::FSINH:
case ISD::FSQRT:
+ case ISD::FCBRT:
case ISD::FTAN:
case ISD::FTANH:
case ISD::FTRUNC:
@@ -1353,6 +1354,7 @@ void DAGTypeLegalizer::SplitVectorResult(SDNode *N, unsigned ResNo) {
case ISD::LLROUND:
case ISD::FSIN:
case ISD::FSINH:
+ case ISD::FCBRT:
case ISD::FSQRT: case ISD::VP_SQRT:
case ISD::FTAN:
case ISD::FTANH:
@@ -5129,6 +5131,7 @@ void DAGTypeLegalizer::WidenVectorResult(SDNode *N, unsigned ResNo) {
case ISD::FSIN:
case ISD::FSINH:
case ISD::FSQRT:
+ case ISD::FCBRT:
case ISD::FTAN:
case ISD::FTANH:
case ISD::FTRUNC:
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index 7471a6c856bcf..0aecb7f4adec1 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -6889,6 +6889,7 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
getValue(I.getArgOperand(1)), DAG, TLI, Flags));
return;
case Intrinsic::sqrt:
+ case Intrinsic::cbrt:
case Intrinsic::fabs:
case Intrinsic::sin:
case Intrinsic::cos:
@@ -6913,6 +6914,7 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
switch (Intrinsic) {
default: llvm_unreachable("Impossible intrinsic"); // Can't reach here.
case Intrinsic::sqrt: Opcode = ISD::FSQRT; break;
+ case Intrinsic::cbrt: Opcode = ISD::FCBRT; break;
case Intrinsic::fabs: Opcode = ISD::FABS; break;
case Intrinsic::sin: Opcode = ISD::FSIN; break;
case Intrinsic::cos: Opcode = ISD::FCOS; break;
@@ -9686,6 +9688,12 @@ void SelectionDAGBuilder::visitCall(const CallInst &I) {
if (visitUnaryFloatCall(I, ISD::FTANH))
return;
break;
+ case LibFunc_cbrt:
+ case LibFunc_cbrtf:
+ case LibFunc_cbrtl:
+ if (visitUnaryFloatCall(I, ISD::FCBRT))
+ return;
+ break;
case LibFunc_sqrt:
case LibFunc_sqrtf:
case LibFunc_sqrtl:
diff --git a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
index d550dbaf40a4d..f689e31e81c31 100644
--- a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
+++ b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
@@ -776,13 +776,13 @@ AArch64TargetLowering::AArch64TargetLowering(const TargetMachine &TM,
ISD::FCOSH, ISD::FSINH, ISD::FTANH,
ISD::FTAN, ISD::FEXP, ISD::FEXP2,
ISD::FEXP10, ISD::FLOG, ISD::FLOG2,
- ISD::FLOG10, ISD::STRICT_FREM, ISD::STRICT_FPOW,
- ISD::STRICT_FPOWI, ISD::STRICT_FCOS, ISD::STRICT_FSIN,
- ISD::STRICT_FACOS, ISD::STRICT_FASIN, ISD::STRICT_FATAN,
- ISD::STRICT_FATAN2, ISD::STRICT_FCOSH, ISD::STRICT_FSINH,
- ISD::STRICT_FTANH, ISD::STRICT_FEXP, ISD::STRICT_FEXP2,
- ISD::STRICT_FLOG, ISD::STRICT_FLOG2, ISD::STRICT_FLOG10,
- ISD::STRICT_FTAN}) {
+ ISD::FLOG10, ISD::FCBRT, ISD::STRICT_FREM,
+ ISD::STRICT_FPOW, ISD::STRICT_FPOWI, ISD::STRICT_FCOS,
+ ISD::STRICT_FSIN, ISD::STRICT_FACOS, ISD::STRICT_FASIN,
+ ISD::STRICT_FATAN, ISD::STRICT_FATAN2,ISD::STRICT_FCOSH,
+ ISD::STRICT_FSINH, ISD::STRICT_FTANH, ISD::STRICT_FEXP,
+ ISD::STRICT_FEXP2, ISD::STRICT_FLOG, ISD::STRICT_FLOG2,
+ ISD::STRICT_FLOG10, ISD::STRICT_FTAN}) {
setOperationAction(Op, MVT::f16, Promote);
setOperationAction(Op, MVT::v4f16, Expand);
setOperationAction(Op, MVT::v8f16, Expand);
diff --git a/llvm/lib/Target/AArch64/GISel/AArch64LegalizerInfo.cpp b/llvm/lib/Target/AArch64/GISel/AArch64LegalizerInfo.cpp
index 131c72a24964a..f6f6aac651026 100644
--- a/llvm/lib/Target/AArch64/GISel/AArch64LegalizerInfo.cpp
+++ b/llvm/lib/Target/AArch64/GISel/AArch64LegalizerInfo.cpp
@@ -435,7 +435,7 @@ AArch64LegalizerInfo::AArch64LegalizerInfo(const AArch64Subtarget &ST)
getActionDefinitionsBuilder({G_FCOS, G_FSIN, G_FPOW, G_FLOG, G_FLOG2,
G_FLOG10, G_FTAN, G_FEXP, G_FEXP2, G_FEXP10,
G_FACOS, G_FASIN, G_FATAN, G_FATAN2, G_FCOSH,
- G_FSINH, G_FTANH, G_FMODF})
+ G_FSINH, G_FTANH, G_FMODF, G_FCBRT})
// We need a call for these, so we always need to scalarize.
.scalarize(0)
// Regardless of FP16 support, widen 16-bit elements to 32-bits.
diff --git a/llvm/test/CodeGen/AArch64/GlobalISel/legalizer-info-validation.mir b/llvm/test/CodeGen/AArch64/GlobalISel/legalizer-info-validation.mir
index 655e793f55c59..5cfb32051ca0b 100644
--- a/llvm/test/CodeGen/AArch64/GlobalISel/legalizer-info-validation.mir
+++ b/llvm/test/CodeGen/AArch64/GlobalISel/legalizer-info-validation.mir
@@ -809,6 +809,10 @@
# DEBUG-NEXT: .. opcode {{[0-9]+}} is aliased to {{[0-9]+}}
# DEBUG-NEXT: .. the first uncovered type index: 1, OK
# DEBUG-NEXT: .. the first uncovered imm index: 0, OK
+# DEBUG-NEXT: G_FCBRT (opcode 277): 1 type index, 0 imm indices
+# DEBUG-NEXT: .. opcode 277 is aliased to 265
+# DEBUG-NEXT: .. the first uncovered type index: 1, OK
+# DEBUG-NEXT: .. the first uncovered imm index: 0, OK
# DEBUG-NEXT: G_FFLOOR (opcode {{[0-9]+}}): 1 type index, 0 imm indices
# DEBUG-NEXT: .. opcode {{[0-9]+}} is aliased to {{[0-9]+}}
# DEBUG-NEXT: .. the first uncovered type index: 1, OK
diff --git a/llvm/test/CodeGen/AArch64/cbrt.ll b/llvm/test/CodeGen/AArch64/cbrt.ll
new file mode 100644
index 0000000000000..815b4a2900bf6
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/cbrt.ll
@@ -0,0 +1,1305 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 6
+; RUN: llc -mtriple=aarch64 %s -o - | FileCheck %s --check-prefixes=CHECK,CHECK-SD
+; RUN: llc -mtriple=aarch64 -global-isel %s -o - | FileCheck %s --check-prefixes=CHECK,CHECK-GI
+
+define double @cbrt_f64(double %a) {
+; CHECK-LABEL: cbrt_f64:
+; CHECK: // %bb.0: // %entry
+; CHECK-NEXT: b cbrt
+entry:
+ %c = call double @llvm.cbrt.f64(double %a)
+ ret double %c
+}
+
+define float @cbrt_f32(float %a) {
+; CHECK-LABEL: cbrt_f32:
+; CHECK: // %bb.0: // %entry
+; CHECK-NEXT: b cbrtf
+entry:
+ %c = call float @llvm.cbrt.f32(float %a)
+ ret float %c
+}
+
+define half @cbrt_f16(half %a) {
+; CHECK-LABEL: cbrt_f16:
+; CHECK: // %bb.0: // %entry
+; CHECK-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
+; CHECK-NEXT: .cfi_def_cfa_offset 16
+; CHECK-NEXT: .cfi_offset w30, -16
+; CHECK-NEXT: fcvt s0, h0
+; CHECK-NEXT: bl cbrtf
+; CHECK-NEXT: fcvt h0, s0
+; CHECK-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT: ret
+entry:
+ %c = call half @llvm.cbrt.f16(half %a)
+ ret half %c
+}
+
+define fp128 @cbrt_fp128(fp128 %a) {
+; CHECK-LABEL: cbrt_fp128:
+; CHECK: // %bb.0: // %entry
+; CHECK-NEXT: b cbrtl
+entry:
+ %c = call fp128 @llvm.cbrt.fp128(fp128 %a)
+ ret fp128 %c
+}
+
+define <1 x double> @cbrt_v1f64(<1 x double> %x) {
+; CHECK-LABEL: cbrt_v1f64:
+; CHECK: // %bb.0:
+; CHECK-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
+; CHECK-NEXT: .cfi_def_cfa_offset 16
+; CHECK-NEXT: .cfi_offset w30, -16
+; CHECK-NEXT: bl cbrt
+; CHECK-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT: ret
+ %c = call <1 x double> @llvm.cbrt.v1f64(<1 x double> %x)
+ ret <1 x double> %c
+}
+
+define <2 x double> @cbrt_v2f64(<2 x double> %a) {
+; CHECK-SD-LABEL: cbrt_v2f64:
+; CHECK-SD: // %bb.0: // %entry
+; CHECK-SD-NEXT: sub sp, sp, #48
+; CHECK-SD-NEXT: str x30, [sp, #32] // 8-byte Spill
+; CHECK-SD-NEXT: .cfi_def_cfa_offset 48
+; CHECK-SD-NEXT: .cfi_offset w30, -16
+; CHECK-SD-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: mov d0, v0.d[1]
+; CHECK-SD-NEXT: bl cbrt
+; CHECK-SD-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-SD-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $d0 killed $d0 killed $q0
+; CHECK-SD-NEXT: bl cbrt
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-SD-NEXT: ldr x30, [sp, #32] // 8-byte Reload
+; CHECK-SD-NEXT: mov v0.d[1], v1.d[0]
+; CHECK-SD-NEXT: add sp, sp, #48
+; CHECK-SD-NEXT: ret
+;
+; CHECK-GI-LABEL: cbrt_v2f64:
+; CHECK-GI: // %bb.0: // %entry
+; CHECK-GI-NEXT: sub sp, sp, #32
+; CHECK-GI-NEXT: str d8, [sp, #16] // 8-byte Spill
+; CHECK-GI-NEXT: str x30, [sp, #24] // 8-byte Spill
+; CHECK-GI-NEXT: .cfi_def_cfa_offset 32
+; CHECK-GI-NEXT: .cfi_offset w30, -8
+; CHECK-GI-NEXT: .cfi_offset b8, -16
+; CHECK-GI-NEXT: mov d8, v0.d[1]
+; CHECK-GI-NEXT: // kill: def $d0 killed $d0 killed $q0
+; CHECK-GI-NEXT: bl cbrt
+; CHECK-GI-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-GI-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-GI-NEXT: fmov d0, d8
+; CHECK-GI-NEXT: bl cbrt
+; CHECK-GI-NEXT: ldr q1, [sp] // 16-byte Reload
+; CHECK-GI-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-GI-NEXT: ldr x30, [sp, #24] // 8-byte Reload
+; CHECK-GI-NEXT: ldr d8, [sp, #16] // 8-byte Reload
+; CHECK-GI-NEXT: mov v1.d[1], v0.d[0]
+; CHECK-GI-NEXT: mov v0.16b, v1.16b
+; CHECK-GI-NEXT: add sp, sp, #32
+; CHECK-GI-NEXT: ret
+entry:
+ %c = call <2 x double> @llvm.cbrt.v2f64(<2 x double> %a)
+ ret <2 x double> %c
+}
+
+define <3 x double> @cbrt_v3f64(<3 x double> %a) {
+; CHECK-SD-LABEL: cbrt_v3f64:
+; CHECK-SD: // %bb.0: // %entry
+; CHECK-SD-NEXT: str d10, [sp, #-32]! // 8-byte Folded Spill
+; CHECK-SD-NEXT: stp d9, d8, [sp, #8] // 16-byte Folded Spill
+; CHECK-SD-NEXT: str x30, [sp, #24] // 8-byte Spill
+; CHECK-SD-NEXT: .cfi_def_cfa_offset 32
+; CHECK-SD-NEXT: .cfi_offset w30, -8
+; CHECK-SD-NEXT: .cfi_offset b8, -16
+; CHECK-SD-NEXT: .cfi_offset b9, -24
+; CHECK-SD-NEXT: .cfi_offset b10, -32
+; CHECK-SD-NEXT: fmov d8, d2
+; CHECK-SD-NEXT: fmov d9, d1
+; CHECK-SD-NEXT: bl cbrt
+; CHECK-SD-NEXT: fmov d10, d0
+; CHECK-SD-NEXT: fmov d0, d9
+; CHECK-SD-NEXT: bl cbrt
+; CHECK-SD-NEXT: fmov d9, d0
+; CHECK-SD-NEXT: fmov d0, d8
+; CHECK-SD-NEXT: bl cbrt
+; CHECK-SD-NEXT: fmov d1, d9
+; CHECK-SD-NEXT: ldp d9, d8, [sp, #8] // 16-byte Folded Reload
+; CHECK-SD-NEXT: ldr x30, [sp, #24] // 8-byte Reload
+; CHECK-SD-NEXT: fmov d2, d0
+; CHECK-SD-NEXT: fmov d0, d10
+; CHECK-SD-NEXT: ldr d10, [sp], #32 // 8-byte Folded Reload
+; CHECK-SD-NEXT: ret
+;
+; CHECK-GI-LABEL: cbrt_v3f64:
+; CHECK-GI: // %bb.0: // %entry
+; CHECK-GI-NEXT: str d10, [sp, #-32]! // 8-byte Folded Spill
+; CHECK-GI-NEXT: stp d9, d8, [sp, #8] // 16-byte Folded Spill
+; CHECK-GI-NEXT: str x30, [sp, #24] // 8-byte Spill
+; CHECK-GI-NEXT: .cfi_def_cfa_offset 32
+; CHECK-GI-NEXT: .cfi_offset w30, -8
+; CHECK-GI-NEXT: .cfi_offset b8, -16
+; CHECK-GI-NEXT: .cfi_offset b9, -24
+; CHECK-GI-NEXT: .cfi_offset b10, -32
+; CHECK-GI-NEXT: fmov d8, d1
+; CHECK-GI-NEXT: fmov d9, d2
+; CHECK-GI-NEXT: bl cbrt
+; CHECK-GI-NEXT: fmov d10, d0
+; CHECK-GI-NEXT: fmov d0, d8
+; CHECK-GI-NEXT: bl cbrt
+; CHECK-GI-NEXT: fmov d8, d0
+; CHECK-GI-NEXT: fmov d0, d9
+; CHECK-GI-NEXT: bl cbrt
+; CHECK-GI-NEXT: fmov d1, d8
+; CHECK-GI-NEXT: ldp d9, d8, [sp, #8] // 16-byte Folded Reload
+; CHECK-GI-NEXT: ldr x30, [sp, #24] // 8-byte Reload
+; CHECK-GI-NEXT: fmov d2, d0
+; CHECK-GI-NEXT: fmov d0, d10
+; CHECK-GI-NEXT: ldr d10, [sp], #32 // 8-byte Folded Reload
+; CHECK-GI-NEXT: ret
+entry:
+ %c = call <3 x double> @llvm.cbrt.v3f64(<3 x double> %a)
+ ret <3 x double> %c
+}
+
+define <4 x double> @cbrt_v4f64(<4 x double> %a) {
+; CHECK-SD-LABEL: cbrt_v4f64:
+; CHECK-SD: // %bb.0: // %entry
+; CHECK-SD-NEXT: sub sp, sp, #64
+; CHECK-SD-NEXT: str x30, [sp, #48] // 8-byte Spill
+; CHECK-SD-NEXT: .cfi_def_cfa_offset 64
+; CHECK-SD-NEXT: .cfi_offset w30, -16
+; CHECK-SD-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: mov d0, v0.d[1]
+; CHECK-SD-NEXT: str q1, [sp, #32] // 16-byte Spill
+; CHECK-SD-NEXT: bl cbrt
+; CHECK-SD-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-SD-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $d0 killed $d0 killed $q0
+; CHECK-SD-NEXT: bl cbrt
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-SD-NEXT: mov v0.d[1], v1.d[0]
+; CHECK-SD-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp, #32] // 16-byte Reload
+; CHECK-SD-NEXT: mov d0, v0.d[1]
+; CHECK-SD-NEXT: bl cbrt
+; CHECK-SD-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-SD-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp, #32] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $d0 killed $d0 killed $q0
+; CHECK-SD-NEXT: bl cbrt
+; CHECK-SD-NEXT: fmov d1, d0
+; CHECK-SD-NEXT: ldp q2, q0, [sp] // 32-byte Folded Reload
+; CHECK-SD-NEXT: ldr x30, [sp, #48] // 8-byte Reload
+; CHECK-SD-NEXT: mov v1.d[1], v2.d[0]
+; CHECK-SD-NEXT: add sp, sp, #64
+; CHECK-SD-NEXT: ret
+;
+; CHECK-GI-LABEL: cbrt_v4f64:
+; CHECK-GI: // %bb.0: // %entry
+; CHECK-GI-NEXT: sub sp, sp, #80
+; CHECK-GI-NEXT: stp d9, d8, [sp, #48] // 16-byte Folded Spill
+; CHECK-GI-NEXT: str x30, [sp, #64] // 8-byte Spill
+; CHECK-GI-NEXT: .cfi_def_cfa_offset 80
+; CHECK-GI-NEXT: .cfi_offset w30, -16
+; CHECK-GI-NEXT: .cfi_offset b8, -24
+; CHECK-GI-NEXT: .cfi_offset b9, -32
+; CHECK-GI-NEXT: str q1, [sp] // 16-byte Spill
+; CHECK-GI-NEXT: mov d8, v0.d[1]
+; CHECK-GI-NEXT: mov d9, v1.d[1]
+; CHECK-GI-NEXT: // kill: def $d0 killed $d0 killed $q0
+; CHECK-GI-NEXT: bl cbrt
+; CHECK-GI-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-GI-NEXT: str q0, [sp, #32] // 16-byte Spill
+; CHECK-GI-NEXT: fmov d0, d8
+; CHECK-GI-NEXT: bl cbrt
+; CHECK-GI-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-GI-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-GI-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-GI-NEXT: // kill: def $d0 killed $d0 killed $q0
+; CHECK-GI-NEXT: bl cbrt
+; CHECK-GI-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-GI-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-GI-NEXT: fmov d0, d9
+; CHECK-GI-NEXT: bl cbrt
+; CHECK-GI-NEXT: ldp q1, q2, [sp, #16] // 32-byte Folded Reload
+; CHECK-GI-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-GI-NEXT: ldr x30, [sp, #64] // 8-byte Reload
+; CHECK-GI-NEXT: ldp d9, d8, [sp, #48] // 16-byte Folded Reload
+; CHECK-GI-NEXT: mov v2.d[1], v1.d[0]
+; CHECK-GI-NEXT: ldr q1, [sp] // 16-byte Reload
+; CHECK-GI-NEXT: mov v1.d[1], v0.d[0]
+; CHECK-GI-NEXT: mov v0.16b, v2.16b
+; CHECK-GI-NEXT: add sp, sp, #80
+; CHECK-GI-NEXT: ret
+entry:
+ %c = call <4 x double> @llvm.cbrt.v4f64(<4 x double> %a)
+ ret <4 x double> %c
+}
+
+define <2 x float> @cbrt_v2f32(<2 x float> %a) {
+; CHECK-SD-LABEL: cbrt_v2f32:
+; CHECK-SD: // %bb.0: // %entry
+; CHECK-SD-NEXT: sub sp, sp, #48
+; CHECK-SD-NEXT: str x30, [sp, #32] // 8-byte Spill
+; CHECK-SD-NEXT: .cfi_def_cfa_offset 48
+; CHECK-SD-NEXT: .cfi_offset w30, -16
+; CHECK-SD-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-SD-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: mov s0, v0.s[1]
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 killed $q0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: ldr x30, [sp, #32] // 8-byte Reload
+; CHECK-SD-NEXT: mov v0.s[1], v1.s[0]
+; CHECK-SD-NEXT: // kill: def $d0 killed $d0 killed $q0
+; CHECK-SD-NEXT: add sp, sp, #48
+; CHECK-SD-NEXT: ret
+;
+; CHECK-GI-LABEL: cbrt_v2f32:
+; CHECK-GI: // %bb.0: // %entry
+; CHECK-GI-NEXT: sub sp, sp, #32
+; CHECK-GI-NEXT: str d8, [sp, #16] // 8-byte Spill
+; CHECK-GI-NEXT: str x30, [sp, #24] // 8-byte Spill
+; CHECK-GI-NEXT: .cfi_def_cfa_offset 32
+; CHECK-GI-NEXT: .cfi_offset w30, -8
+; CHECK-GI-NEXT: .cfi_offset b8, -16
+; CHECK-GI-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-GI-NEXT: mov s8, v0.s[1]
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 killed $q0
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s8
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: ldr q1, [sp] // 16-byte Reload
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: ldr x30, [sp, #24] // 8-byte Reload
+; CHECK-GI-NEXT: ldr d8, [sp, #16] // 8-byte Reload
+; CHECK-GI-NEXT: mov v1.s[1], v0.s[0]
+; CHECK-GI-NEXT: fmov d0, d1
+; CHECK-GI-NEXT: add sp, sp, #32
+; CHECK-GI-NEXT: ret
+entry:
+ %c = call <2 x float> @llvm.cbrt.v2f32(<2 x float> %a)
+ ret <2 x float> %c
+}
+
+define <3 x float> @cbrt_v3f32(<3 x float> %a) {
+; CHECK-SD-LABEL: cbrt_v3f32:
+; CHECK-SD: // %bb.0: // %entry
+; CHECK-SD-NEXT: sub sp, sp, #48
+; CHECK-SD-NEXT: str x30, [sp, #32] // 8-byte Spill
+; CHECK-SD-NEXT: .cfi_def_cfa_offset 48
+; CHECK-SD-NEXT: .cfi_offset w30, -16
+; CHECK-SD-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: mov s0, v0.s[1]
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 killed $q0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: ldr q1, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: mov v0.s[1], v1.s[0]
+; CHECK-SD-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov s0, v0.s[2]
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: ldr q1, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: ldr x30, [sp, #32] // 8-byte Reload
+; CHECK-SD-NEXT: mov v1.s[2], v0.s[0]
+; CHECK-SD-NEXT: mov v0.16b, v1.16b
+; CHECK-SD-NEXT: add sp, sp, #48
+; CHECK-SD-NEXT: ret
+;
+; CHECK-GI-LABEL: cbrt_v3f32:
+; CHECK-GI: // %bb.0: // %entry
+; CHECK-GI-NEXT: sub sp, sp, #64
+; CHECK-GI-NEXT: stp d9, d8, [sp, #32] // 16-byte Folded Spill
+; CHECK-GI-NEXT: str x30, [sp, #48] // 8-byte Spill
+; CHECK-GI-NEXT: .cfi_def_cfa_offset 64
+; CHECK-GI-NEXT: .cfi_offset w30, -16
+; CHECK-GI-NEXT: .cfi_offset b8, -24
+; CHECK-GI-NEXT: .cfi_offset b9, -32
+; CHECK-GI-NEXT: mov s8, v0.s[1]
+; CHECK-GI-NEXT: mov s9, v0.s[2]
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 killed $q0
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s8
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s9
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: ldp q2, q1, [sp] // 32-byte Folded Reload
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: ldr x30, [sp, #48] // 8-byte Reload
+; CHECK-GI-NEXT: ldp d9, d8, [sp, #32] // 16-byte Folded Reload
+; CHECK-GI-NEXT: mov v1.s[1], v2.s[0]
+; CHECK-GI-NEXT: mov v1.s[2], v0.s[0]
+; CHECK-GI-NEXT: mov v0.16b, v1.16b
+; CHECK-GI-NEXT: add sp, sp, #64
+; CHECK-GI-NEXT: ret
+entry:
+ %c = call <3 x float> @llvm.cbrt.v3f32(<3 x float> %a)
+ ret <3 x float> %c
+}
+
+define <4 x float> @cbrt_v4f32(<4 x float> %a) {
+; CHECK-SD-LABEL: cbrt_v4f32:
+; CHECK-SD: // %bb.0: // %entry
+; CHECK-SD-NEXT: sub sp, sp, #48
+; CHECK-SD-NEXT: str x30, [sp, #32] // 8-byte Spill
+; CHECK-SD-NEXT: .cfi_def_cfa_offset 48
+; CHECK-SD-NEXT: .cfi_offset w30, -16
+; CHECK-SD-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: mov s0, v0.s[1]
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 killed $q0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: ldr q1, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: mov v0.s[1], v1.s[0]
+; CHECK-SD-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov s0, v0.s[2]
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: ldr q1, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: mov v1.s[2], v0.s[0]
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov s0, v0.s[3]
+; CHECK-SD-NEXT: str q1, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: ldr q1, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: ldr x30, [sp, #32] // 8-byte Reload
+; CHECK-SD-NEXT: mov v1.s[3], v0.s[0]
+; CHECK-SD-NEXT: mov v0.16b, v1.16b
+; CHECK-SD-NEXT: add sp, sp, #48
+; CHECK-SD-NEXT: ret
+;
+; CHECK-GI-LABEL: cbrt_v4f32:
+; CHECK-GI: // %bb.0: // %entry
+; CHECK-GI-NEXT: sub sp, sp, #80
+; CHECK-GI-NEXT: str d10, [sp, #48] // 8-byte Spill
+; CHECK-GI-NEXT: stp d9, d8, [sp, #56] // 16-byte Folded Spill
+; CHECK-GI-NEXT: str x30, [sp, #72] // 8-byte Spill
+; CHECK-GI-NEXT: .cfi_def_cfa_offset 80
+; CHECK-GI-NEXT: .cfi_offset w30, -8
+; CHECK-GI-NEXT: .cfi_offset b8, -16
+; CHECK-GI-NEXT: .cfi_offset b9, -24
+; CHECK-GI-NEXT: .cfi_offset b10, -32
+; CHECK-GI-NEXT: mov s8, v0.s[1]
+; CHECK-GI-NEXT: mov s9, v0.s[2]
+; CHECK-GI-NEXT: mov s10, v0.s[3]
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 killed $q0
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: str q0, [sp, #32] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s8
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s9
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s10
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: ldp q2, q1, [sp, #16] // 32-byte Folded Reload
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: ldr x30, [sp, #72] // 8-byte Reload
+; CHECK-GI-NEXT: ldp d9, d8, [sp, #56] // 16-byte Folded Reload
+; CHECK-GI-NEXT: ldr d10, [sp, #48] // 8-byte Reload
+; CHECK-GI-NEXT: mov v1.s[1], v2.s[0]
+; CHECK-GI-NEXT: ldr q2, [sp] // 16-byte Reload
+; CHECK-GI-NEXT: mov v1.s[2], v2.s[0]
+; CHECK-GI-NEXT: mov v1.s[3], v0.s[0]
+; CHECK-GI-NEXT: mov v0.16b, v1.16b
+; CHECK-GI-NEXT: add sp, sp, #80
+; CHECK-GI-NEXT: ret
+entry:
+ %c = call <4 x float> @llvm.cbrt.v4f32(<4 x float> %a)
+ ret <4 x float> %c
+}
+
+define <8 x float> @cbrt_v8f32(<8 x float> %a) {
+; CHECK-SD-LABEL: cbrt_v8f32:
+; CHECK-SD: // %bb.0: // %entry
+; CHECK-SD-NEXT: sub sp, sp, #64
+; CHECK-SD-NEXT: str x30, [sp, #48] // 8-byte Spill
+; CHECK-SD-NEXT: .cfi_def_cfa_offset 64
+; CHECK-SD-NEXT: .cfi_offset w30, -16
+; CHECK-SD-NEXT: stp q0, q1, [sp] // 32-byte Folded Spill
+; CHECK-SD-NEXT: mov s0, v0.s[1]
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: str q0, [sp, #32] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 killed $q0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: ldr q1, [sp, #32] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: mov v0.s[1], v1.s[0]
+; CHECK-SD-NEXT: str q0, [sp, #32] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov s0, v0.s[2]
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: ldr q1, [sp, #32] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: mov v1.s[2], v0.s[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov s0, v0.s[3]
+; CHECK-SD-NEXT: str q1, [sp, #32] // 16-byte Spill
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: ldr q1, [sp, #32] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: mov v1.s[3], v0.s[0]
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov s0, v0.s[1]
+; CHECK-SD-NEXT: str q1, [sp, #32] // 16-byte Spill
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 killed $q0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: ldr q1, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: mov v0.s[1], v1.s[0]
+; CHECK-SD-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov s0, v0.s[2]
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: ldr q1, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-SD-NEXT: mov v1.s[2], v0.s[0]
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov s0, v0.s[3]
+; CHECK-SD-NEXT: str q1, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fmov s2, s0
+; CHECK-SD-NEXT: ldr q1, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: ldr q0, [sp, #32] // 16-byte Reload
+; CHECK-SD-NEXT: ldr x30, [sp, #48] // 8-byte Reload
+; CHECK-SD-NEXT: mov v1.s[3], v2.s[0]
+; CHECK-SD-NEXT: add sp, sp, #64
+; CHECK-SD-NEXT: ret
+;
+; CHECK-GI-LABEL: cbrt_v8f32:
+; CHECK-GI: // %bb.0: // %entry
+; CHECK-GI-NEXT: sub sp, sp, #176
+; CHECK-GI-NEXT: stp d13, d12, [sp, #112] // 16-byte Folded Spill
+; CHECK-GI-NEXT: stp d11, d10, [sp, #128] // 16-byte Folded Spill
+; CHECK-GI-NEXT: stp d9, d8, [sp, #144] // 16-byte Folded Spill
+; CHECK-GI-NEXT: str x30, [sp, #160] // 8-byte Spill
+; CHECK-GI-NEXT: .cfi_def_cfa_offset 176
+; CHECK-GI-NEXT: .cfi_offset w30, -16
+; CHECK-GI-NEXT: .cfi_offset b8, -24
+; CHECK-GI-NEXT: .cfi_offset b9, -32
+; CHECK-GI-NEXT: .cfi_offset b10, -40
+; CHECK-GI-NEXT: .cfi_offset b11, -48
+; CHECK-GI-NEXT: .cfi_offset b12, -56
+; CHECK-GI-NEXT: .cfi_offset b13, -64
+; CHECK-GI-NEXT: str q1, [sp, #32] // 16-byte Spill
+; CHECK-GI-NEXT: mov s8, v0.s[1]
+; CHECK-GI-NEXT: mov s9, v0.s[2]
+; CHECK-GI-NEXT: mov s10, v0.s[3]
+; CHECK-GI-NEXT: mov s11, v1.s[1]
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 killed $q0
+; CHECK-GI-NEXT: mov s12, v1.s[2]
+; CHECK-GI-NEXT: mov s13, v1.s[3]
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: str q0, [sp, #64] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s8
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: str q0, [sp, #48] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s9
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: str q0, [sp, #96] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s10
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: str q0, [sp, #80] // 16-byte Spill
+; CHECK-GI-NEXT: ldr q0, [sp, #32] // 16-byte Reload
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 killed $q0
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: str q0, [sp, #32] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s11
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s12
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s13
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: ldp q2, q1, [sp, #48] // 32-byte Folded Reload
+; CHECK-GI-NEXT: // kill: def $s0 killed $s0 def $q0
+; CHECK-GI-NEXT: ldr x30, [sp, #160] // 8-byte Reload
+; CHECK-GI-NEXT: ldp d9, d8, [sp, #144] // 16-byte Folded Reload
+; CHECK-GI-NEXT: ldp d11, d10, [sp, #128] // 16-byte Folded Reload
+; CHECK-GI-NEXT: mov v1.s[1], v2.s[0]
+; CHECK-GI-NEXT: ldp q2, q3, [sp, #16] // 32-byte Folded Reload
+; CHECK-GI-NEXT: ldp d13, d12, [sp, #112] // 16-byte Folded Reload
+; CHECK-GI-NEXT: mov v3.s[1], v2.s[0]
+; CHECK-GI-NEXT: ldr q2, [sp, #96] // 16-byte Reload
+; CHECK-GI-NEXT: mov v1.s[2], v2.s[0]
+; CHECK-GI-NEXT: ldr q2, [sp] // 16-byte Reload
+; CHECK-GI-NEXT: mov v3.s[2], v2.s[0]
+; CHECK-GI-NEXT: ldr q2, [sp, #80] // 16-byte Reload
+; CHECK-GI-NEXT: mov v1.s[3], v2.s[0]
+; CHECK-GI-NEXT: mov v3.s[3], v0.s[0]
+; CHECK-GI-NEXT: mov v2.16b, v1.16b
+; CHECK-GI-NEXT: mov v1.16b, v3.16b
+; CHECK-GI-NEXT: mov v0.16b, v2.16b
+; CHECK-GI-NEXT: add sp, sp, #176
+; CHECK-GI-NEXT: ret
+entry:
+ %c = call <8 x float> @llvm.cbrt.v8f32(<8 x float> %a)
+ ret <8 x float> %c
+}
+
+define <7 x half> @cbrt_v7f16(<7 x half> %a) {
+; CHECK-SD-LABEL: cbrt_v7f16:
+; CHECK-SD: // %bb.0: // %entry
+; CHECK-SD-NEXT: sub sp, sp, #48
+; CHECK-SD-NEXT: str x30, [sp, #32] // 8-byte Spill
+; CHECK-SD-NEXT: .cfi_def_cfa_offset 48
+; CHECK-SD-NEXT: .cfi_offset w30, -16
+; CHECK-SD-NEXT: mov h1, v0.h[1]
+; CHECK-SD-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h1
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v0.h[1], v1.h[0]
+; CHECK-SD-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[2]
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[2], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[3]
+; CHECK-SD-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[3], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[4]
+; CHECK-SD-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[4], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[5]
+; CHECK-SD-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[5], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[6]
+; CHECK-SD-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[6], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[7]
+; CHECK-SD-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h1, s0
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: ldr x30, [sp, #32] // 8-byte Reload
+; CHECK-SD-NEXT: mov v0.h[7], v1.h[0]
+; CHECK-SD-NEXT: add sp, sp, #48
+; CHECK-SD-NEXT: ret
+;
+; CHECK-GI-LABEL: cbrt_v7f16:
+; CHECK-GI: // %bb.0: // %entry
+; CHECK-GI-NEXT: sub sp, sp, #160
+; CHECK-GI-NEXT: stp d13, d12, [sp, #96] // 16-byte Folded Spill
+; CHECK-GI-NEXT: stp d11, d10, [sp, #112] // 16-byte Folded Spill
+; CHECK-GI-NEXT: stp d9, d8, [sp, #128] // 16-byte Folded Spill
+; CHECK-GI-NEXT: str x30, [sp, #144] // 8-byte Spill
+; CHECK-GI-NEXT: .cfi_def_cfa_offset 160
+; CHECK-GI-NEXT: .cfi_offset w30, -16
+; CHECK-GI-NEXT: .cfi_offset b8, -24
+; CHECK-GI-NEXT: .cfi_offset b9, -32
+; CHECK-GI-NEXT: .cfi_offset b10, -40
+; CHECK-GI-NEXT: .cfi_offset b11, -48
+; CHECK-GI-NEXT: .cfi_offset b12, -56
+; CHECK-GI-NEXT: .cfi_offset b13, -64
+; CHECK-GI-NEXT: mov h8, v0.h[1]
+; CHECK-GI-NEXT: mov h9, v0.h[2]
+; CHECK-GI-NEXT: mov h10, v0.h[3]
+; CHECK-GI-NEXT: mov h11, v0.h[4]
+; CHECK-GI-NEXT: mov h12, v0.h[5]
+; CHECK-GI-NEXT: mov h13, v0.h[6]
+; CHECK-GI-NEXT: fcvt s0, h0
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h8
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #80] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h9
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #48] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h10
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #64] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h11
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #32] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h12
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h13
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: ldp q3, q2, [sp, #48] // 32-byte Folded Reload
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: ldr q1, [sp, #80] // 16-byte Reload
+; CHECK-GI-NEXT: ldp d9, d8, [sp, #128] // 16-byte Folded Reload
+; CHECK-GI-NEXT: ldp d11, d10, [sp, #112] // 16-byte Folded Reload
+; CHECK-GI-NEXT: ldr x30, [sp, #144] // 8-byte Reload
+; CHECK-GI-NEXT: mov v1.h[1], v3.h[0]
+; CHECK-GI-NEXT: ldp d13, d12, [sp, #96] // 16-byte Folded Reload
+; CHECK-GI-NEXT: mov v1.h[2], v2.h[0]
+; CHECK-GI-NEXT: ldp q2, q3, [sp, #16] // 32-byte Folded Reload
+; CHECK-GI-NEXT: mov v1.h[3], v3.h[0]
+; CHECK-GI-NEXT: mov v1.h[4], v2.h[0]
+; CHECK-GI-NEXT: ldr q2, [sp] // 16-byte Reload
+; CHECK-GI-NEXT: mov v1.h[5], v2.h[0]
+; CHECK-GI-NEXT: mov v1.h[6], v0.h[0]
+; CHECK-GI-NEXT: mov v0.16b, v1.16b
+; CHECK-GI-NEXT: add sp, sp, #160
+; CHECK-GI-NEXT: ret
+entry:
+ %c = call <7 x half> @llvm.cbrt.v7f16(<7 x half> %a)
+ ret <7 x half> %c
+}
+
+define <4 x half> @cbrt_v4f16(<4 x half> %a) {
+; CHECK-SD-LABEL: cbrt_v4f16:
+; CHECK-SD: // %bb.0: // %entry
+; CHECK-SD-NEXT: sub sp, sp, #48
+; CHECK-SD-NEXT: str x30, [sp, #32] // 8-byte Spill
+; CHECK-SD-NEXT: .cfi_def_cfa_offset 48
+; CHECK-SD-NEXT: .cfi_offset w30, -16
+; CHECK-SD-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-SD-NEXT: mov h1, v0.h[1]
+; CHECK-SD-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h1
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: fcvt s1, h1
+; CHECK-SD-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: fmov s0, s1
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: fcvt h2, s0
+; CHECK-SD-NEXT: mov h1, v1.h[2]
+; CHECK-SD-NEXT: fcvt s0, h1
+; CHECK-SD-NEXT: ldr q1, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov v2.h[1], v1.h[0]
+; CHECK-SD-NEXT: str q2, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: fcvt h2, s0
+; CHECK-SD-NEXT: mov h1, v1.h[3]
+; CHECK-SD-NEXT: fcvt s0, h1
+; CHECK-SD-NEXT: ldr q1, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[2], v2.h[0]
+; CHECK-SD-NEXT: str q1, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h1, s0
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: ldr x30, [sp, #32] // 8-byte Reload
+; CHECK-SD-NEXT: mov v0.h[3], v1.h[0]
+; CHECK-SD-NEXT: // kill: def $d0 killed $d0 killed $q0
+; CHECK-SD-NEXT: add sp, sp, #48
+; CHECK-SD-NEXT: ret
+;
+; CHECK-GI-LABEL: cbrt_v4f16:
+; CHECK-GI: // %bb.0: // %entry
+; CHECK-GI-NEXT: sub sp, sp, #80
+; CHECK-GI-NEXT: str d10, [sp, #48] // 8-byte Spill
+; CHECK-GI-NEXT: stp d9, d8, [sp, #56] // 16-byte Folded Spill
+; CHECK-GI-NEXT: str x30, [sp, #72] // 8-byte Spill
+; CHECK-GI-NEXT: .cfi_def_cfa_offset 80
+; CHECK-GI-NEXT: .cfi_offset w30, -8
+; CHECK-GI-NEXT: .cfi_offset b8, -16
+; CHECK-GI-NEXT: .cfi_offset b9, -24
+; CHECK-GI-NEXT: .cfi_offset b10, -32
+; CHECK-GI-NEXT: // kill: def $d0 killed $d0 def $q0
+; CHECK-GI-NEXT: mov h8, v0.h[1]
+; CHECK-GI-NEXT: mov h9, v0.h[2]
+; CHECK-GI-NEXT: mov h10, v0.h[3]
+; CHECK-GI-NEXT: fcvt s0, h0
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h8
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #32] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h9
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h10
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: ldp q3, q2, [sp] // 32-byte Folded Reload
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: ldr q1, [sp, #32] // 16-byte Reload
+; CHECK-GI-NEXT: ldp d9, d8, [sp, #56] // 16-byte Folded Reload
+; CHECK-GI-NEXT: ldr x30, [sp, #72] // 8-byte Reload
+; CHECK-GI-NEXT: ldr d10, [sp, #48] // 8-byte Reload
+; CHECK-GI-NEXT: mov v1.h[1], v3.h[0]
+; CHECK-GI-NEXT: mov v1.h[2], v2.h[0]
+; CHECK-GI-NEXT: mov v1.h[3], v0.h[0]
+; CHECK-GI-NEXT: mov v0.16b, v1.16b
+; CHECK-GI-NEXT: // kill: def $d0 killed $d0 killed $q0
+; CHECK-GI-NEXT: add sp, sp, #80
+; CHECK-GI-NEXT: ret
+entry:
+ %c = call <4 x half> @llvm.cbrt.v4f16(<4 x half> %a)
+ ret <4 x half> %c
+}
+
+define <8 x half> @cbrt_v8f16(<8 x half> %a) {
+; CHECK-SD-LABEL: cbrt_v8f16:
+; CHECK-SD: // %bb.0: // %entry
+; CHECK-SD-NEXT: sub sp, sp, #48
+; CHECK-SD-NEXT: str x30, [sp, #32] // 8-byte Spill
+; CHECK-SD-NEXT: .cfi_def_cfa_offset 48
+; CHECK-SD-NEXT: .cfi_offset w30, -16
+; CHECK-SD-NEXT: mov h1, v0.h[1]
+; CHECK-SD-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h1
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v0.h[1], v1.h[0]
+; CHECK-SD-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[2]
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[2], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[3]
+; CHECK-SD-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[3], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[4]
+; CHECK-SD-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[4], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[5]
+; CHECK-SD-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[5], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[6]
+; CHECK-SD-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[6], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[7]
+; CHECK-SD-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h1, s0
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: ldr x30, [sp, #32] // 8-byte Reload
+; CHECK-SD-NEXT: mov v0.h[7], v1.h[0]
+; CHECK-SD-NEXT: add sp, sp, #48
+; CHECK-SD-NEXT: ret
+;
+; CHECK-GI-LABEL: cbrt_v8f16:
+; CHECK-GI: // %bb.0: // %entry
+; CHECK-GI-NEXT: sub sp, sp, #176
+; CHECK-GI-NEXT: str d14, [sp, #112] // 8-byte Spill
+; CHECK-GI-NEXT: stp d13, d12, [sp, #120] // 16-byte Folded Spill
+; CHECK-GI-NEXT: stp d11, d10, [sp, #136] // 16-byte Folded Spill
+; CHECK-GI-NEXT: stp d9, d8, [sp, #152] // 16-byte Folded Spill
+; CHECK-GI-NEXT: str x30, [sp, #168] // 8-byte Spill
+; CHECK-GI-NEXT: .cfi_def_cfa_offset 176
+; CHECK-GI-NEXT: .cfi_offset w30, -8
+; CHECK-GI-NEXT: .cfi_offset b8, -16
+; CHECK-GI-NEXT: .cfi_offset b9, -24
+; CHECK-GI-NEXT: .cfi_offset b10, -32
+; CHECK-GI-NEXT: .cfi_offset b11, -40
+; CHECK-GI-NEXT: .cfi_offset b12, -48
+; CHECK-GI-NEXT: .cfi_offset b13, -56
+; CHECK-GI-NEXT: .cfi_offset b14, -64
+; CHECK-GI-NEXT: mov h8, v0.h[1]
+; CHECK-GI-NEXT: mov h9, v0.h[2]
+; CHECK-GI-NEXT: mov h10, v0.h[3]
+; CHECK-GI-NEXT: mov h11, v0.h[4]
+; CHECK-GI-NEXT: mov h12, v0.h[5]
+; CHECK-GI-NEXT: mov h13, v0.h[6]
+; CHECK-GI-NEXT: mov h14, v0.h[7]
+; CHECK-GI-NEXT: fcvt s0, h0
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h8
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #96] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h9
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #64] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h10
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #80] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h11
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #48] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h12
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #32] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h13
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h14
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: ldp q3, q2, [sp, #64] // 32-byte Folded Reload
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: ldr q1, [sp, #96] // 16-byte Reload
+; CHECK-GI-NEXT: ldp d9, d8, [sp, #152] // 16-byte Folded Reload
+; CHECK-GI-NEXT: ldp d11, d10, [sp, #136] // 16-byte Folded Reload
+; CHECK-GI-NEXT: ldr x30, [sp, #168] // 8-byte Reload
+; CHECK-GI-NEXT: mov v1.h[1], v3.h[0]
+; CHECK-GI-NEXT: ldr d14, [sp, #112] // 8-byte Reload
+; CHECK-GI-NEXT: ldp d13, d12, [sp, #120] // 16-byte Folded Reload
+; CHECK-GI-NEXT: mov v1.h[2], v2.h[0]
+; CHECK-GI-NEXT: ldp q2, q3, [sp, #32] // 32-byte Folded Reload
+; CHECK-GI-NEXT: mov v1.h[3], v3.h[0]
+; CHECK-GI-NEXT: mov v1.h[4], v2.h[0]
+; CHECK-GI-NEXT: ldp q2, q3, [sp] // 32-byte Folded Reload
+; CHECK-GI-NEXT: mov v1.h[5], v3.h[0]
+; CHECK-GI-NEXT: mov v1.h[6], v2.h[0]
+; CHECK-GI-NEXT: mov v1.h[7], v0.h[0]
+; CHECK-GI-NEXT: mov v0.16b, v1.16b
+; CHECK-GI-NEXT: add sp, sp, #176
+; CHECK-GI-NEXT: ret
+entry:
+ %c = call <8 x half> @llvm.cbrt.v8f16(<8 x half> %a)
+ ret <8 x half> %c
+}
+
+define <16 x half> @cbrt_v16f16(<16 x half> %a) {
+; CHECK-SD-LABEL: cbrt_v16f16:
+; CHECK-SD: // %bb.0: // %entry
+; CHECK-SD-NEXT: sub sp, sp, #64
+; CHECK-SD-NEXT: str x30, [sp, #48] // 8-byte Spill
+; CHECK-SD-NEXT: .cfi_def_cfa_offset 64
+; CHECK-SD-NEXT: .cfi_offset w30, -16
+; CHECK-SD-NEXT: stp q1, q0, [sp] // 32-byte Folded Spill
+; CHECK-SD-NEXT: mov h1, v0.h[1]
+; CHECK-SD-NEXT: fcvt s0, h1
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: str q0, [sp, #32] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #32] // 16-byte Reload
+; CHECK-SD-NEXT: mov v0.h[1], v1.h[0]
+; CHECK-SD-NEXT: str q0, [sp, #32] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[2]
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #32] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[2], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[3]
+; CHECK-SD-NEXT: str q1, [sp, #32] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #32] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[3], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[4]
+; CHECK-SD-NEXT: str q1, [sp, #32] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #32] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[4], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[5]
+; CHECK-SD-NEXT: str q1, [sp, #32] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #32] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[5], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[6]
+; CHECK-SD-NEXT: str q1, [sp, #32] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #32] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[6], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[7]
+; CHECK-SD-NEXT: str q1, [sp, #32] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #32] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[7], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[1]
+; CHECK-SD-NEXT: str q1, [sp, #32] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v0.h[1], v1.h[0]
+; CHECK-SD-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[2]
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[2], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[3]
+; CHECK-SD-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[3], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[4]
+; CHECK-SD-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[4], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[5]
+; CHECK-SD-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[5], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[6]
+; CHECK-SD-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fcvt h0, s0
+; CHECK-SD-NEXT: ldr q1, [sp, #16] // 16-byte Reload
+; CHECK-SD-NEXT: mov v1.h[6], v0.h[0]
+; CHECK-SD-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-SD-NEXT: mov h0, v0.h[7]
+; CHECK-SD-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-SD-NEXT: fcvt s0, h0
+; CHECK-SD-NEXT: bl cbrtf
+; CHECK-SD-NEXT: fmov s1, s0
+; CHECK-SD-NEXT: ldr x30, [sp, #48] // 8-byte Reload
+; CHECK-SD-NEXT: fcvt h2, s1
+; CHECK-SD-NEXT: ldp q1, q0, [sp, #16] // 32-byte Folded Reload
+; CHECK-SD-NEXT: mov v1.h[7], v2.h[0]
+; CHECK-SD-NEXT: add sp, sp, #64
+; CHECK-SD-NEXT: ret
+;
+; CHECK-GI-LABEL: cbrt_v16f16:
+; CHECK-GI: // %bb.0: // %entry
+; CHECK-GI-NEXT: sub sp, sp, #320
+; CHECK-GI-NEXT: stp d15, d14, [sp, #240] // 16-byte Folded Spill
+; CHECK-GI-NEXT: stp d13, d12, [sp, #256] // 16-byte Folded Spill
+; CHECK-GI-NEXT: stp d11, d10, [sp, #272] // 16-byte Folded Spill
+; CHECK-GI-NEXT: stp d9, d8, [sp, #288] // 16-byte Folded Spill
+; CHECK-GI-NEXT: stp x29, x30, [sp, #304] // 16-byte Folded Spill
+; CHECK-GI-NEXT: .cfi_def_cfa_offset 320
+; CHECK-GI-NEXT: .cfi_offset w30, -8
+; CHECK-GI-NEXT: .cfi_offset w29, -16
+; CHECK-GI-NEXT: .cfi_offset b8, -24
+; CHECK-GI-NEXT: .cfi_offset b9, -32
+; CHECK-GI-NEXT: .cfi_offset b10, -40
+; CHECK-GI-NEXT: .cfi_offset b11, -48
+; CHECK-GI-NEXT: .cfi_offset b12, -56
+; CHECK-GI-NEXT: .cfi_offset b13, -64
+; CHECK-GI-NEXT: .cfi_offset b14, -72
+; CHECK-GI-NEXT: .cfi_offset b15, -80
+; CHECK-GI-NEXT: mov v2.16b, v1.16b
+; CHECK-GI-NEXT: str q1, [sp, #80] // 16-byte Spill
+; CHECK-GI-NEXT: mov h14, v1.h[1]
+; CHECK-GI-NEXT: mov h1, v1.h[2]
+; CHECK-GI-NEXT: mov h15, v0.h[1]
+; CHECK-GI-NEXT: mov h8, v0.h[2]
+; CHECK-GI-NEXT: mov h9, v0.h[3]
+; CHECK-GI-NEXT: mov h10, v0.h[4]
+; CHECK-GI-NEXT: mov h11, v0.h[5]
+; CHECK-GI-NEXT: mov h12, v0.h[6]
+; CHECK-GI-NEXT: mov h13, v0.h[7]
+; CHECK-GI-NEXT: fcvt s0, h0
+; CHECK-GI-NEXT: str h1, [sp, #16] // 2-byte Spill
+; CHECK-GI-NEXT: mov h1, v2.h[3]
+; CHECK-GI-NEXT: str h1, [sp, #32] // 2-byte Spill
+; CHECK-GI-NEXT: mov h1, v2.h[4]
+; CHECK-GI-NEXT: str h1, [sp, #48] // 2-byte Spill
+; CHECK-GI-NEXT: mov h1, v2.h[5]
+; CHECK-GI-NEXT: str h1, [sp, #64] // 2-byte Spill
+; CHECK-GI-NEXT: mov h1, v2.h[6]
+; CHECK-GI-NEXT: str h1, [sp, #96] // 2-byte Spill
+; CHECK-GI-NEXT: mov h1, v2.h[7]
+; CHECK-GI-NEXT: str h1, [sp, #160] // 2-byte Spill
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h15
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #192] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h8
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #112] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h9
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #224] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h10
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #208] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h11
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #176] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h12
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #144] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h13
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp, #128] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: ldr q1, [sp, #80] // 16-byte Reload
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: fcvt s1, h1
+; CHECK-GI-NEXT: str q0, [sp, #80] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: fcvt s1, h14
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: ldr h1, [sp, #16] // 2-byte Reload
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: fcvt s1, h1
+; CHECK-GI-NEXT: str q0, [sp, #16] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: ldr h1, [sp, #32] // 2-byte Reload
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: fcvt s1, h1
+; CHECK-GI-NEXT: str q0, [sp, #32] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: ldr h1, [sp, #48] // 2-byte Reload
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: fcvt s1, h1
+; CHECK-GI-NEXT: str q0, [sp, #48] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: ldr h1, [sp, #64] // 2-byte Reload
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: fcvt s1, h1
+; CHECK-GI-NEXT: str q0, [sp, #64] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: ldr h1, [sp, #96] // 2-byte Reload
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: fcvt s1, h1
+; CHECK-GI-NEXT: str q0, [sp, #96] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: ldr h1, [sp, #160] // 2-byte Reload
+; CHECK-GI-NEXT: fcvt h0, s0
+; CHECK-GI-NEXT: fcvt s1, h1
+; CHECK-GI-NEXT: str q0, [sp, #160] // 16-byte Spill
+; CHECK-GI-NEXT: fmov s0, s1
+; CHECK-GI-NEXT: bl cbrtf
+; CHECK-GI-NEXT: ldr q3, [sp, #192] // 16-byte Reload
+; CHECK-GI-NEXT: ldr q2, [sp, #112] // 16-byte Reload
+; CHECK-GI-NEXT: ldp x29, x30, [sp, #304] // 16-byte Folded Reload
+; CHECK-GI-NEXT: mov v3.h[1], v2.h[0]
+; CHECK-GI-NEXT: ldp q1, q2, [sp] // 32-byte Folded Reload
+; CHECK-GI-NEXT: ldp d9, d8, [sp, #288] // 16-byte Folded Reload
+; CHECK-GI-NEXT: ldp d11, d10, [sp, #272] // 16-byte Folded Reload
+; CHECK-GI-NEXT: mov v1.h[1], v2.h[0]
+; CHECK-GI-NEXT: ldr q2, [sp, #224] // 16-byte Reload
+; CHECK-GI-NEXT: ldp d13, d12, [sp, #256] // 16-byte Folded Reload
+; CHECK-GI-NEXT: mov v3.h[2], v2.h[0]
+; CHECK-GI-NEXT: ldr q2, [sp, #32] // 16-byte Reload
+; CHECK-GI-NEXT: ldp d15, d14, [sp, #240] // 16-byte Folded Reload
+; CHECK-GI-NEXT: mov v1.h[2], v2.h[0]
+; CHECK-GI-NEXT: ldr q2, [sp, #208] // 16-byte Reload
+; CHECK-GI-NEXT: mov v3.h[3], v2.h[0]
+; CHECK-GI-NEXT: ldr q2, [sp, #48] // 16-byte Reload
+; CHECK-GI-NEXT: mov v1.h[3], v2.h[0]
+; CHECK-GI-NEXT: ldr q2, [sp, #176] // 16-byte Reload
+; CHECK-GI-NEXT: mov v3.h[4], v2.h[0]
+; CHECK-GI-NEXT: ldr q2, [sp, #64] // 16-byte Reload
+; CHECK-GI-NEXT: mov v1.h[4], v2.h[0]
+; CHECK-GI-NEXT: ldr q2, [sp, #144] // 16-byte Reload
+; CHECK-GI-NEXT: mov v3.h[5], v2.h[0]
+; CHECK-GI-NEXT: ldr q2, [sp, #96] // 16-byte Reload
+; CHECK-GI-NEXT: mov v1.h[5], v2.h[0]
+; CHECK-GI-NEXT: fcvt h2, s0
+; CHECK-GI-NEXT: ldr q0, [sp, #128] // 16-byte Reload
+; CHECK-GI-NEXT: mov v3.h[6], v0.h[0]
+; CHECK-GI-NEXT: ldr q0, [sp, #160] // 16-byte Reload
+; CHECK-GI-NEXT: mov v1.h[6], v0.h[0]
+; CHECK-GI-NEXT: ldr q0, [sp, #80] // 16-byte Reload
+; CHECK-GI-NEXT: mov v3.h[7], v0.h[0]
+; CHECK-GI-NEXT: mov v1.h[7], v2.h[0]
+; CHECK-GI-NEXT: mov v0.16b, v3.16b
+; CHECK-GI-NEXT: add sp, sp, #320
+; CHECK-GI-NEXT: ret
+entry:
+ %c = call <16 x half> @llvm.cbrt.v16f16(<16 x half> %a)
+ ret <16 x half> %c
+}
+
+define <2 x fp128> @cbrt_v2fp128(<2 x fp128> %a) {
+; CHECK-LABEL: cbrt_v2fp128:
+; CHECK: // %bb.0: // %entry
+; CHECK-NEXT: sub sp, sp, #48
+; CHECK-NEXT: str x30, [sp, #32] // 8-byte Spill
+; CHECK-NEXT: .cfi_def_cfa_offset 48
+; CHECK-NEXT: .cfi_offset w30, -16
+; CHECK-NEXT: str q1, [sp, #16] // 16-byte Spill
+; CHECK-NEXT: bl cbrtl
+; CHECK-NEXT: str q0, [sp] // 16-byte Spill
+; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Reload
+; CHECK-NEXT: bl cbrtl
+; CHECK-NEXT: mov v1.16b, v0.16b
+; CHECK-NEXT: ldr q0, [sp] // 16-byte Reload
+; CHECK-NEXT: ldr x30, [sp, #32] // 8-byte Reload
+; CHECK-NEXT: add sp, sp, #48
+; CHECK-NEXT: ret
+entry:
+ %c = call <2 x fp128> @llvm.cbrt.v2fp128(<2 x fp128> %a)
+ ret <2 x fp128> %c
+}
diff --git a/llvm/test/CodeGen/AArch64/fast-isel-runtime-libcall.ll b/llvm/test/CodeGen/AArch64/fast-isel-runtime-libcall.ll
index 7ff4dfca3f4c3..82e06684daace 100644
--- a/llvm/test/CodeGen/AArch64/fast-isel-runtime-libcall.ll
+++ b/llvm/test/CodeGen/AArch64/fast-isel-runtime-libcall.ll
@@ -110,6 +110,28 @@ define double @pow_f64(double %a, double %b) {
%1 = call double @llvm.pow.f64(double %a, double %b)
ret double %1
}
+
+define float @cbrt_f32(float %a) {
+; SMALL-LABEL: cbrt_f32
+; SMALL: bl _cbrtf
+; LARGE-LABEL: cbrt_f32
+; LARGE: adrp [[REG:x[0-9]+]], _cbrtf at GOTPAGE
+; LARGE: ldr [[REG]], [[[REG]], _cbrtf at GOTPAGEOFF]
+; LARGE-NEXT: blr [[REG]]
+ %1 = call float @llvm.cbrt.f32(float %a)
+ ret float %1
+}
+
+define double @cbrt_f64(double %a) {
+; SMALL-LABEL: cbrt_f64
+; SMALL: bl _cbrt
+; LARGE-LABEL: cbrt_f64
+; LARGE: adrp [[REG:x[0-9]+]], _cbrt at GOTPAGE
+; LARGE: ldr [[REG]], [[[REG]], _cbrt at GOTPAGEOFF]
+; LARGE-NEXT: blr [[REG]]
+ %1 = call double @llvm.cbrt.f64(double %a)
+ ret double %1
+}
declare float @llvm.sin.f32(float)
declare double @llvm.sin.f64(double)
declare float @llvm.cos.f32(float)
@@ -118,3 +140,5 @@ declare float @llvm.tan.f32(float)
declare double @llvm.tan.f64(double)
declare float @llvm.pow.f32(float, float)
declare double @llvm.pow.f64(double, double)
+declare float @llvm.cbrt.f32(float)
+declare double @llvm.cbrt.f64(double)
diff --git a/llvm/test/CodeGen/AArch64/replace-with-veclib-armpl.ll b/llvm/test/CodeGen/AArch64/replace-with-veclib-armpl.ll
index 8a0ac6d4ace7a..02f1edced0b19 100644
--- a/llvm/test/CodeGen/AArch64/replace-with-veclib-armpl.ll
+++ b/llvm/test/CodeGen/AArch64/replace-with-veclib-armpl.ll
@@ -778,6 +778,51 @@ define <vscale x 4 x float> @llvm_tanh_vscale_f32(<vscale x 4 x float> %in) #0 {
ret <vscale x 4 x float> %1
}
+declare <2 x double> @llvm.cbrt.v2f64(<2 x double>)
+declare <4 x float> @llvm.cbrt.v4f32(<4 x float>)
+declare <vscale x 2 x double> @llvm.cbrt.nxv2f64(<vscale x 2 x double>)
+declare <vscale x 4 x float> @llvm.cbrt.nxv4f32(<vscale x 4 x float>)
+
+define <2 x double> @llvm_cbrt_f64(<2 x double> %in) {
+; CHECK-LABEL: define <2 x double> @llvm_cbrt_f64
+; CHECK-SAME: (<2 x double> [[IN:%.*]]) {
+; CHECK-NEXT: [[TMP1:%.*]] = call fast <2 x double> @llvm.cbrt.v2f64(<2 x double> [[IN]])
+; CHECK-NEXT: ret <2 x double> [[TMP1]]
+;
+ %1 = call fast <2 x double> @llvm.cbrt.v2f64(<2 x double> %in)
+ ret <2 x double> %1
+}
+
+define <4 x float> @llvm_cbrt_f32(<4 x float> %in) {
+; CHECK-LABEL: define <4 x float> @llvm_cbrt_f32
+; CHECK-SAME: (<4 x float> [[IN:%.*]]) {
+; CHECK-NEXT: [[TMP1:%.*]] = call fast <4 x float> @llvm.cbrt.v4f32(<4 x float> [[IN]])
+; CHECK-NEXT: ret <4 x float> [[TMP1]]
+;
+ %1 = call fast <4 x float> @llvm.cbrt.v4f32(<4 x float> %in)
+ ret <4 x float> %1
+}
+
+define <vscale x 2 x double> @llvm_cbrt_vscale_f64(<vscale x 2 x double> %in) #0 {
+; CHECK-LABEL: define <vscale x 2 x double> @llvm_cbrt_vscale_f64
+; CHECK-SAME: (<vscale x 2 x double> [[IN:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT: [[TMP1:%.*]] = call fast <vscale x 2 x double> @llvm.cbrt.nxv2f64(<vscale x 2 x double> [[IN]])
+; CHECK-NEXT: ret <vscale x 2 x double> [[TMP1]]
+;
+ %1 = call fast <vscale x 2 x double> @llvm.cbrt.nxv2f64(<vscale x 2 x double> %in)
+ ret <vscale x 2 x double> %1
+}
+
+define <vscale x 4 x float> @llvm_cbrt_vscale_f32(<vscale x 4 x float> %in) #0 {
+; CHECK-LABEL: define <vscale x 4 x float> @llvm_cbrt_vscale_f32
+; CHECK-SAME: (<vscale x 4 x float> [[IN:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT: [[TMP1:%.*]] = call fast <vscale x 4 x float> @llvm.cbrt.nxv4f32(<vscale x 4 x float> [[IN]])
+; CHECK-NEXT: ret <vscale x 4 x float> [[TMP1]]
+;
+ %1 = call fast <vscale x 4 x float> @llvm.cbrt.nxv4f32(<vscale x 4 x float> %in)
+ ret <vscale x 4 x float> %1
+}
+
attributes #0 = { "target-features"="+sve" }
;.
; CHECK: attributes #[[ATTR0:[0-9]+]] = { nocallback nocreateundeforpoison nofree nosync nounwind speculatable willreturn memory(none) }
diff --git a/llvm/test/CodeGen/AArch64/replace-with-veclib-libmvec-scalable.ll b/llvm/test/CodeGen/AArch64/replace-with-veclib-libmvec-scalable.ll
index 1b541d1330aae..7a75b0733d5d9 100644
--- a/llvm/test/CodeGen/AArch64/replace-with-veclib-libmvec-scalable.ll
+++ b/llvm/test/CodeGen/AArch64/replace-with-veclib-libmvec-scalable.ll
@@ -529,6 +529,24 @@ define <vscale x 4 x float> @llvm_trunc_vscale_f32(<vscale x 4 x float> %in) {
ret <vscale x 4 x float> %1
}
+define <vscale x 2 x double> @llvm_cbrt_vscale_f64(<vscale x 2 x double> %in) {
+; CHECK-LABEL: @llvm_cbrt_vscale_f64(
+; CHECK-NEXT: [[TMP1:%.*]] = call fast <vscale x 2 x double> @llvm.cbrt.nxv2f64(<vscale x 2 x double> [[IN:%.*]])
+; CHECK-NEXT: ret <vscale x 2 x double> [[TMP1]]
+;
+ %1 = call fast <vscale x 2 x double> @llvm.cbrt.nxv2f64(<vscale x 2 x double> %in)
+ ret <vscale x 2 x double> %1
+}
+
+define <vscale x 4 x float> @llvm_cbrt_vscale_f32(<vscale x 4 x float> %in) {
+; CHECK-LABEL: @llvm_cbrt_vscale_f32(
+; CHECK-NEXT: [[TMP1:%.*]] = call fast <vscale x 4 x float> @llvm.cbrt.nxv4f32(<vscale x 4 x float> [[IN:%.*]])
+; CHECK-NEXT: ret <vscale x 4 x float> [[TMP1]]
+;
+ %1 = call fast <vscale x 4 x float> @llvm.cbrt.nxv4f32(<vscale x 4 x float> %in)
+ ret <vscale x 4 x float> %1
+}
+
declare <vscale x 2 x double> @llvm.ceil.nxv2f64(<vscale x 2 x double>)
declare <vscale x 4 x float> @llvm.ceil.nxv4f32(<vscale x 4 x float>)
declare <vscale x 2 x double> @llvm.copysign.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>)
@@ -573,7 +591,10 @@ declare <vscale x 2 x double> @llvm.tan.nxv2f64(<vscale x 2 x double>)
declare <vscale x 4 x float> @llvm.tan.nxv4f32(<vscale x 4 x float>)
declare <vscale x 2 x double> @llvm.trunc.nxv2f64(<vscale x 2 x double>)
declare <vscale x 4 x float> @llvm.trunc.nxv4f32(<vscale x 4 x float>)
+declare <vscale x 2 x double> @llvm.cbrt.nxv2f64(<vscale x 2 x double>)
+declare <vscale x 4 x float> @llvm.cbrt.nxv4f32(<vscale x 4 x float>)
;.
; CHECK: attributes #[[ATTR0:[0-9]+]] = { "target-features"="+sve" }
-; CHECK: attributes #[[ATTR1:[0-9]+]] = { nocallback nofree nosync nounwind speculatable willreturn memory(none) "target-features"="+sve" }
+; CHECK: attributes #[[ATTR1:[0-9]+]] = { nocallback nocreateundeforpoison nofree nosync nounwind speculatable willreturn memory(none) "target-features"="+sve" }
+; CHECK: attributes #[[ATTR2:[0-9]+]] = { nocallback nofree nosync nounwind speculatable willreturn memory(none) "target-features"="+sve" }
;.
diff --git a/llvm/test/CodeGen/AArch64/replace-with-veclib-libmvec.ll b/llvm/test/CodeGen/AArch64/replace-with-veclib-libmvec.ll
index 6323d942a08e7..09f0caaaaf947 100644
--- a/llvm/test/CodeGen/AArch64/replace-with-veclib-libmvec.ll
+++ b/llvm/test/CodeGen/AArch64/replace-with-veclib-libmvec.ll
@@ -528,6 +528,24 @@ define <4 x float> @llvm_trunc_f32(<4 x float> %in) {
ret <4 x float> %1
}
+define <2 x double> @llvm_cbrt_f64(<2 x double> %in) {
+; CHECK-LABEL: @llvm_cbrt_f64(
+; CHECK-NEXT: [[TMP1:%.*]] = call fast <2 x double> @llvm.cbrt.v2f64(<2 x double> [[IN:%.*]])
+; CHECK-NEXT: ret <2 x double> [[TMP1]]
+;
+ %1 = call fast <2 x double> @llvm.cbrt.v2f64(<2 x double> %in)
+ ret <2 x double> %1
+}
+
+define <4 x float> @llvm_cbrt_f32(<4 x float> %in) {
+; CHECK-LABEL: @llvm_cbrt_f32(
+; CHECK-NEXT: [[TMP1:%.*]] = call fast <4 x float> @llvm.cbrt.v4f32(<4 x float> [[IN:%.*]])
+; CHECK-NEXT: ret <4 x float> [[TMP1]]
+;
+ %1 = call fast <4 x float> @llvm.cbrt.v4f32(<4 x float> %in)
+ ret <4 x float> %1
+}
+
declare <2 x double> @llvm.ceil.v2f64(<2 x double>)
declare <4 x float> @llvm.ceil.v4f32(<4 x float>)
declare <2 x double> @llvm.copysign.v2f64(<2 x double>, <2 x double>)
@@ -573,5 +591,6 @@ declare <4 x float> @llvm.tan.v4f32(<4 x float>)
declare <2 x double> @llvm.trunc.v2f64(<2 x double>)
declare <4 x float> @llvm.trunc.v4f32(<4 x float>)
;.
-; CHECK: attributes #[[ATTR0:[0-9]+]] = { nocallback nofree nosync nounwind speculatable willreturn memory(none) }
+; CHECK: attributes #[[ATTR0:[0-9]+]] = { nocallback nocreateundeforpoison nofree nosync nounwind speculatable willreturn memory(none) }
+; CHECK: attributes #[[ATTR1:[0-9]+]] = { nocallback nofree nosync nounwind speculatable willreturn memory(none) }
;.
diff --git a/llvm/test/CodeGen/AArch64/replace-with-veclib-sleef-scalable.ll b/llvm/test/CodeGen/AArch64/replace-with-veclib-sleef-scalable.ll
index 1d429ece6f810..6e39a84780e62 100644
--- a/llvm/test/CodeGen/AArch64/replace-with-veclib-sleef-scalable.ll
+++ b/llvm/test/CodeGen/AArch64/replace-with-veclib-sleef-scalable.ll
@@ -529,6 +529,24 @@ define <vscale x 4 x float> @llvm_trunc_vscale_f32(<vscale x 4 x float> %in) {
ret <vscale x 4 x float> %1
}
+define <vscale x 2 x double> @llvm_cbrt_vscale_f64(<vscale x 2 x double> %in) {
+; CHECK-LABEL: @llvm_cbrt_vscale_f64(
+; CHECK-NEXT: [[TMP1:%.*]] = call fast <vscale x 2 x double> @llvm.cbrt.nxv2f64(<vscale x 2 x double> [[IN:%.*]])
+; CHECK-NEXT: ret <vscale x 2 x double> [[TMP1]]
+;
+ %1 = call fast <vscale x 2 x double> @llvm.cbrt.nxv2f64(<vscale x 2 x double> %in)
+ ret <vscale x 2 x double> %1
+}
+
+define <vscale x 4 x float> @llvm_cbrt_vscale_f32(<vscale x 4 x float> %in) {
+; CHECK-LABEL: @llvm_cbrt_vscale_f32(
+; CHECK-NEXT: [[TMP1:%.*]] = call fast <vscale x 4 x float> @llvm.cbrt.nxv4f32(<vscale x 4 x float> [[IN:%.*]])
+; CHECK-NEXT: ret <vscale x 4 x float> [[TMP1]]
+;
+ %1 = call fast <vscale x 4 x float> @llvm.cbrt.nxv4f32(<vscale x 4 x float> %in)
+ ret <vscale x 4 x float> %1
+}
+
declare <vscale x 2 x double> @llvm.ceil.nxv2f64(<vscale x 2 x double>)
declare <vscale x 4 x float> @llvm.ceil.nxv4f32(<vscale x 4 x float>)
declare <vscale x 2 x double> @llvm.copysign.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>)
@@ -573,7 +591,10 @@ declare <vscale x 2 x double> @llvm.tan.nxv2f64(<vscale x 2 x double>)
declare <vscale x 4 x float> @llvm.tan.nxv4f32(<vscale x 4 x float>)
declare <vscale x 2 x double> @llvm.trunc.nxv2f64(<vscale x 2 x double>)
declare <vscale x 4 x float> @llvm.trunc.nxv4f32(<vscale x 4 x float>)
+declare <vscale x 2 x double> @llvm.cbrt.nxv2f64(<vscale x 2 x double>)
+declare <vscale x 4 x float> @llvm.cbrt.nxv4f32(<vscale x 4 x float>)
;.
; CHECK: attributes #[[ATTR0:[0-9]+]] = { "target-features"="+sve" }
-; CHECK: attributes #[[ATTR1:[0-9]+]] = { nocallback nofree nosync nounwind speculatable willreturn memory(none) "target-features"="+sve" }
+; CHECK: attributes #[[ATTR1:[0-9]+]] = { nocallback nocreateundeforpoison nofree nosync nounwind speculatable willreturn memory(none) "target-features"="+sve" }
+; CHECK: attributes #[[ATTR2:[0-9]+]] = { nocallback nofree nosync nounwind speculatable willreturn memory(none) "target-features"="+sve" }
;.
diff --git a/llvm/test/CodeGen/AArch64/replace-with-veclib-sleef.ll b/llvm/test/CodeGen/AArch64/replace-with-veclib-sleef.ll
index ebe5e78667680..8cb1c7bd3da10 100644
--- a/llvm/test/CodeGen/AArch64/replace-with-veclib-sleef.ll
+++ b/llvm/test/CodeGen/AArch64/replace-with-veclib-sleef.ll
@@ -528,6 +528,24 @@ define <4 x float> @llvm_trunc_f32(<4 x float> %in) {
ret <4 x float> %1
}
+define <2 x double> @llvm_cbrt_f64(<2 x double> %in) {
+; CHECK-LABEL: @llvm_cbrt_f64(
+; CHECK-NEXT: [[TMP1:%.*]] = call fast <2 x double> @llvm.cbrt.v2f64(<2 x double> [[IN:%.*]])
+; CHECK-NEXT: ret <2 x double> [[TMP1]]
+;
+ %1 = call fast <2 x double> @llvm.cbrt.v2f64(<2 x double> %in)
+ ret <2 x double> %1
+}
+
+define <4 x float> @llvm_cbrt_f32(<4 x float> %in) {
+; CHECK-LABEL: @llvm_cbrt_f32(
+; CHECK-NEXT: [[TMP1:%.*]] = call fast <4 x float> @llvm.cbrt.v4f32(<4 x float> [[IN:%.*]])
+; CHECK-NEXT: ret <4 x float> [[TMP1]]
+;
+ %1 = call fast <4 x float> @llvm.cbrt.v4f32(<4 x float> %in)
+ ret <4 x float> %1
+}
+
declare <2 x double> @llvm.ceil.v2f64(<2 x double>)
declare <4 x float> @llvm.ceil.v4f32(<4 x float>)
declare <2 x double> @llvm.copysign.v2f64(<2 x double>, <2 x double>)
@@ -572,6 +590,9 @@ declare <2 x double> @llvm.tan.v2f64(<2 x double>)
declare <4 x float> @llvm.tan.v4f32(<4 x float>)
declare <2 x double> @llvm.trunc.v2f64(<2 x double>)
declare <4 x float> @llvm.trunc.v4f32(<4 x float>)
+declare <2 x double> @llvm.cbrt.v2f64(<2 x double>)
+declare <4 x float> @llvm.cbrt.v4f32(<4 x float>)
;.
-; CHECK: attributes #[[ATTR0:[0-9]+]] = { nocallback nofree nosync nounwind speculatable willreturn memory(none) }
+; CHECK: attributes #[[ATTR0:[0-9]+]] = { nocallback nocreateundeforpoison nofree nosync nounwind speculatable willreturn memory(none) }
+; CHECK: attributes #[[ATTR1:[0-9]+]] = { nocallback nofree nosync nounwind speculatable willreturn memory(none) }
;.
diff --git a/llvm/test/CodeGen/ARM/cbrt.ll b/llvm/test/CodeGen/ARM/cbrt.ll
new file mode 100644
index 0000000000000..8b650f61eace5
--- /dev/null
+++ b/llvm/test/CodeGen/ARM/cbrt.ll
@@ -0,0 +1,131 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=thumbv7m-linux-gnueabi | FileCheck %s --check-prefixes=ANY,SOFTFLOAT
+; RUN: llc < %s -mtriple=thumbv8-linux-gnueabihf -mattr=neon | FileCheck %s --check-prefixes=ANY,HARDFLOAT
+
+define float @cbrt_f32_fmf(float %x) nounwind {
+; SOFTFLOAT-LABEL: cbrt_f32_fmf:
+; SOFTFLOAT: @ %bb.0:
+; SOFTFLOAT-NEXT: .save {r7, lr}
+; SOFTFLOAT-NEXT: push {r7, lr}
+; SOFTFLOAT-NEXT: bl cbrtf
+; SOFTFLOAT-NEXT: pop {r7, pc}
+;
+; HARDFLOAT-LABEL: cbrt_f32_fmf:
+; HARDFLOAT: @ %bb.0:
+; HARDFLOAT-NEXT: b cbrtf
+ %r = call nsz ninf afn float @llvm.cbrt.f32(float %x)
+ ret float %r
+}
+
+define double @cbrt_f64_fmf(double %x) nounwind {
+; SOFTFLOAT-LABEL: cbrt_f64_fmf:
+; SOFTFLOAT: @ %bb.0:
+; SOFTFLOAT-NEXT: .save {r7, lr}
+; SOFTFLOAT-NEXT: push {r7, lr}
+; SOFTFLOAT-NEXT: bl cbrt
+; SOFTFLOAT-NEXT: pop {r7, pc}
+;
+; HARDFLOAT-LABEL: cbrt_f64_fmf:
+; HARDFLOAT: @ %bb.0:
+; HARDFLOAT-NEXT: b cbrt
+ %r = call nsz ninf afn double @llvm.cbrt.f64(double %x)
+ ret double %r
+}
+
+define <4 x float> @cbrt_v4f32_fmf(<4 x float> %x) nounwind {
+; SOFTFLOAT-LABEL: cbrt_v4f32_fmf:
+; SOFTFLOAT: @ %bb.0:
+; SOFTFLOAT-NEXT: .save {r4, r5, r6, r7, lr}
+; SOFTFLOAT-NEXT: push {r4, r5, r6, r7, lr}
+; SOFTFLOAT-NEXT: .pad #4
+; SOFTFLOAT-NEXT: sub sp, #4
+; SOFTFLOAT-NEXT: mov r4, r3
+; SOFTFLOAT-NEXT: mov r5, r2
+; SOFTFLOAT-NEXT: mov r6, r1
+; SOFTFLOAT-NEXT: bl cbrtf
+; SOFTFLOAT-NEXT: mov r7, r0
+; SOFTFLOAT-NEXT: mov r0, r6
+; SOFTFLOAT-NEXT: bl cbrtf
+; SOFTFLOAT-NEXT: mov r6, r0
+; SOFTFLOAT-NEXT: mov r0, r5
+; SOFTFLOAT-NEXT: bl cbrtf
+; SOFTFLOAT-NEXT: mov r5, r0
+; SOFTFLOAT-NEXT: mov r0, r4
+; SOFTFLOAT-NEXT: bl cbrtf
+; SOFTFLOAT-NEXT: mov r3, r0
+; SOFTFLOAT-NEXT: mov r0, r7
+; SOFTFLOAT-NEXT: mov r1, r6
+; SOFTFLOAT-NEXT: mov r2, r5
+; SOFTFLOAT-NEXT: add sp, #4
+; SOFTFLOAT-NEXT: pop {r4, r5, r6, r7, pc}
+;
+; HARDFLOAT-LABEL: cbrt_v4f32_fmf:
+; HARDFLOAT: @ %bb.0:
+; HARDFLOAT-NEXT: .save {r7, lr}
+; HARDFLOAT-NEXT: push {r7, lr}
+; HARDFLOAT-NEXT: .vsave {d8, d9, d10, d11}
+; HARDFLOAT-NEXT: vpush {d8, d9, d10, d11}
+; HARDFLOAT-NEXT: vorr q5, q0, q0
+; HARDFLOAT-NEXT: vmov.f32 s0, s23
+; HARDFLOAT-NEXT: bl cbrtf
+; HARDFLOAT-NEXT: vmov.f32 s19, s0
+; HARDFLOAT-NEXT: vmov.f32 s0, s22
+; HARDFLOAT-NEXT: bl cbrtf
+; HARDFLOAT-NEXT: vmov.f32 s18, s0
+; HARDFLOAT-NEXT: vmov.f32 s0, s21
+; HARDFLOAT-NEXT: bl cbrtf
+; HARDFLOAT-NEXT: vmov.f32 s17, s0
+; HARDFLOAT-NEXT: vmov.f32 s0, s20
+; HARDFLOAT-NEXT: bl cbrtf
+; HARDFLOAT-NEXT: vmov.f32 s16, s0
+; HARDFLOAT-NEXT: vorr q0, q4, q4
+; HARDFLOAT-NEXT: vpop {d8, d9, d10, d11}
+; HARDFLOAT-NEXT: pop {r7, pc}
+ %r = call fast <4 x float> @llvm.cbrt.v4f32(<4 x float> %x)
+ ret <4 x float> %r
+}
+
+define <2 x double> @cbrt_v2f64_fmf(<2 x double> %x) nounwind {
+; SOFTFLOAT-LABEL: cbrt_v2f64_fmf:
+; SOFTFLOAT: @ %bb.0:
+; SOFTFLOAT-NEXT: .save {r4, r5, r6, r7, lr}
+; SOFTFLOAT-NEXT: push {r4, r5, r6, r7, lr}
+; SOFTFLOAT-NEXT: .pad #4
+; SOFTFLOAT-NEXT: sub sp, #4
+; SOFTFLOAT-NEXT: mov r4, r3
+; SOFTFLOAT-NEXT: mov r5, r2
+; SOFTFLOAT-NEXT: bl cbrt
+; SOFTFLOAT-NEXT: mov r6, r0
+; SOFTFLOAT-NEXT: mov r7, r1
+; SOFTFLOAT-NEXT: mov r0, r5
+; SOFTFLOAT-NEXT: mov r1, r4
+; SOFTFLOAT-NEXT: bl cbrt
+; SOFTFLOAT-NEXT: mov r2, r0
+; SOFTFLOAT-NEXT: mov r3, r1
+; SOFTFLOAT-NEXT: mov r0, r6
+; SOFTFLOAT-NEXT: mov r1, r7
+; SOFTFLOAT-NEXT: add sp, #4
+; SOFTFLOAT-NEXT: pop {r4, r5, r6, r7, pc}
+;
+; HARDFLOAT-LABEL: cbrt_v2f64_fmf:
+; HARDFLOAT: @ %bb.0:
+; HARDFLOAT-NEXT: .save {r7, lr}
+; HARDFLOAT-NEXT: push {r7, lr}
+; HARDFLOAT-NEXT: .vsave {d8, d9, d10, d11}
+; HARDFLOAT-NEXT: vpush {d8, d9, d10, d11}
+; HARDFLOAT-NEXT: vorr q4, q0, q0
+; HARDFLOAT-NEXT: vorr d0, d9, d9
+; HARDFLOAT-NEXT: bl cbrt
+; HARDFLOAT-NEXT: vorr d11, d0, d0
+; HARDFLOAT-NEXT: vorr d0, d8, d8
+; HARDFLOAT-NEXT: bl cbrt
+; HARDFLOAT-NEXT: vorr d10, d0, d0
+; HARDFLOAT-NEXT: vorr q0, q5, q5
+; HARDFLOAT-NEXT: vpop {d8, d9, d10, d11}
+; HARDFLOAT-NEXT: pop {r7, pc}
+ %r = call fast <2 x double> @llvm.cbrt.v2f64(<2 x double> %x)
+ ret <2 x double> %r
+}
+
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; ANY: {{.*}}
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/legalizer-info-validation.mir b/llvm/test/CodeGen/RISCV/GlobalISel/legalizer-info-validation.mir
index 43ef36da6a11a..b29d468260fdc 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/legalizer-info-validation.mir
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/legalizer-info-validation.mir
@@ -797,6 +797,9 @@
# DEBUG-NEXT: .. opcode {{[0-9]+}} is aliased to {{[0-9]+}}
# DEBUG-NEXT: .. the first uncovered type index: 1, OK
# DEBUG-NEXT: .. the first uncovered imm index: 0, OK
+# DEBUG-NEXT: G_FCBRT (opcode 277): 1 type index, 0 imm indices
+# DEBUG-NEXT: .. type index coverage check SKIPPED: no rules defined
+# DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined
# DEBUG-NEXT: G_FFLOOR (opcode {{[0-9]+}}): 1 type index, 0 imm indices
# DEBUG-NEXT: .. opcode {{[0-9]+}} is aliased to {{[0-9]+}}
# DEBUG-NEXT: .. the first uncovered type index: 1, OK
diff --git a/llvm/test/CodeGen/RISCV/double-intrinsics.ll b/llvm/test/CodeGen/RISCV/double-intrinsics.ll
index 81e6d84af17cb..3bfe5955a8660 100644
--- a/llvm/test/CodeGen/RISCV/double-intrinsics.ll
+++ b/llvm/test/CodeGen/RISCV/double-intrinsics.ll
@@ -2131,3 +2131,43 @@ define { double, double } @test_modf_f64(double %a) nounwind {
%result = call { double, double } @llvm.modf.f64(double %a)
ret { double, double } %result
}
+
+define double @cbrt_f64(double %a) nounwind {
+; CHECKIFD-LABEL: cbrt_f64:
+; CHECKIFD: # %bb.0:
+; CHECKIFD-NEXT: tail cbrt
+;
+; RV32IZFINXZDINX-LABEL: cbrt_f64:
+; RV32IZFINXZDINX: # %bb.0:
+; RV32IZFINXZDINX-NEXT: addi sp, sp, -16
+; RV32IZFINXZDINX-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
+; RV32IZFINXZDINX-NEXT: call cbrt
+; RV32IZFINXZDINX-NEXT: lw ra, 12(sp) # 4-byte Folded Reload
+; RV32IZFINXZDINX-NEXT: addi sp, sp, 16
+; RV32IZFINXZDINX-NEXT: ret
+;
+; RV64IZFINXZDINX-LABEL: cbrt_f64:
+; RV64IZFINXZDINX: # %bb.0:
+; RV64IZFINXZDINX-NEXT: tail cbrt
+;
+; RV32I-LABEL: cbrt_f64:
+; RV32I: # %bb.0:
+; RV32I-NEXT: addi sp, sp, -16
+; RV32I-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
+; RV32I-NEXT: call cbrt
+; RV32I-NEXT: lw ra, 12(sp) # 4-byte Folded Reload
+; RV32I-NEXT: addi sp, sp, 16
+; RV32I-NEXT: ret
+;
+; RV64I-LABEL: cbrt_f64:
+; RV64I: # %bb.0:
+; RV64I-NEXT: addi sp, sp, -16
+; RV64I-NEXT: sd ra, 8(sp) # 8-byte Folded Spill
+; RV64I-NEXT: call cbrt
+; RV64I-NEXT: ld ra, 8(sp) # 8-byte Folded Reload
+; RV64I-NEXT: addi sp, sp, 16
+; RV64I-NEXT: ret
+ %1 = call double @llvm.cbrt.f64(double %a)
+ ret double %1
+}
+
diff --git a/llvm/test/CodeGen/RISCV/float-intrinsics.ll b/llvm/test/CodeGen/RISCV/float-intrinsics.ll
index 069e20da7b908..1586ae6a02247 100644
--- a/llvm/test/CodeGen/RISCV/float-intrinsics.ll
+++ b/llvm/test/CodeGen/RISCV/float-intrinsics.ll
@@ -3071,3 +3071,45 @@ define { float, float } @test_modf_f32(float %a) nounwind {
%result = call { float, float } @llvm.modf.f32(float %a)
ret { float, float } %result
}
+
+define float @cbrt_f32(float %a) nounwind {
+; RV32IF-LABEL: cbrt_f32:
+; RV32IF: # %bb.0:
+; RV32IF-NEXT: tail cbrtf
+;
+; RV32IZFINX-LABEL: cbrt_f32:
+; RV32IZFINX: # %bb.0:
+; RV32IZFINX-NEXT: tail cbrtf
+;
+; RV64IF-LABEL: cbrt_f32:
+; RV64IF: # %bb.0:
+; RV64IF-NEXT: tail cbrtf
+;
+; RV64IZFINX-LABEL: cbrt_f32:
+; RV64IZFINX: # %bb.0:
+; RV64IZFINX-NEXT: tail cbrtf
+;
+; RV64IFD-LABEL: cbrt_f32:
+; RV64IFD: # %bb.0:
+; RV64IFD-NEXT: tail cbrtf
+;
+; RV32I-LABEL: cbrt_f32:
+; RV32I: # %bb.0:
+; RV32I-NEXT: addi sp, sp, -16
+; RV32I-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
+; RV32I-NEXT: call cbrtf
+; RV32I-NEXT: lw ra, 12(sp) # 4-byte Folded Reload
+; RV32I-NEXT: addi sp, sp, 16
+; RV32I-NEXT: ret
+;
+; RV64I-LABEL: cbrt_f32:
+; RV64I: # %bb.0:
+; RV64I-NEXT: addi sp, sp, -16
+; RV64I-NEXT: sd ra, 8(sp) # 8-byte Folded Spill
+; RV64I-NEXT: call cbrtf
+; RV64I-NEXT: ld ra, 8(sp) # 8-byte Folded Reload
+; RV64I-NEXT: addi sp, sp, 16
+; RV64I-NEXT: ret
+ %1 = call float @llvm.cbrt.f32(float %a)
+ ret float %1
+}
diff --git a/llvm/test/CodeGen/X86/cbrt.ll b/llvm/test/CodeGen/X86/cbrt.ll
new file mode 100644
index 0000000000000..fc296821f770c
--- /dev/null
+++ b/llvm/test/CodeGen/X86/cbrt.ll
@@ -0,0 +1,78 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=x86_64-- -mcpu=x86-64 | FileCheck %s
+
+define float @cbrt_f32_fmf_ieee(float %x) nounwind {
+; CHECK-LABEL: cbrt_f32_fmf_ieee:
+; CHECK: # %bb.0:
+; CHECK-NEXT: jmp cbrtf at PLT # TAILCALL
+ %r = call nsz ninf afn float @llvm.cbrt.f32(float %x)
+ ret float %r
+}
+
+define float @cbrt_f32_fmf_daz(float %x) #0 {
+; CHECK-LABEL: cbrt_f32_fmf_daz:
+; CHECK: # %bb.0:
+; CHECK-NEXT: jmp cbrtf at PLT # TAILCALL
+ %r = call nsz ninf afn float @llvm.cbrt.f32(float %x)
+ ret float %r
+}
+
+define double @cbrt_f64_fmf(double %x) nounwind {
+; CHECK-LABEL: cbrt_f64_fmf:
+; CHECK: # %bb.0:
+; CHECK-NEXT: jmp cbrt at PLT # TAILCALL
+ %r = call nsz ninf afn double @llvm.cbrt.f64(double %x)
+ ret double %r
+}
+
+define <4 x float> @cbrt_v4f32_fmf(<4 x float> %x) nounwind {
+; CHECK-LABEL: cbrt_v4f32_fmf:
+; CHECK: # %bb.0:
+; CHECK-NEXT: subq $56, %rsp
+; CHECK-NEXT: movaps %xmm0, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
+; CHECK-NEXT: shufps {{.*#+}} xmm0 = xmm0[3,3,3,3]
+; CHECK-NEXT: callq cbrtf at PLT
+; CHECK-NEXT: movaps %xmm0, (%rsp) # 16-byte Spill
+; CHECK-NEXT: movaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm0 # 16-byte Reload
+; CHECK-NEXT: movhlps {{.*#+}} xmm0 = xmm0[1,1]
+; CHECK-NEXT: callq cbrtf at PLT
+; CHECK-NEXT: unpcklps (%rsp), %xmm0 # 16-byte Folded Reload
+; CHECK-NEXT: # xmm0 = xmm0[0],mem[0],xmm0[1],mem[1]
+; CHECK-NEXT: movaps %xmm0, (%rsp) # 16-byte Spill
+; CHECK-NEXT: movaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm0 # 16-byte Reload
+; CHECK-NEXT: callq cbrtf at PLT
+; CHECK-NEXT: movaps %xmm0, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
+; CHECK-NEXT: movaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm0 # 16-byte Reload
+; CHECK-NEXT: shufps {{.*#+}} xmm0 = xmm0[1,1,1,1]
+; CHECK-NEXT: callq cbrtf at PLT
+; CHECK-NEXT: movaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm1 # 16-byte Reload
+; CHECK-NEXT: unpcklps {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1]
+; CHECK-NEXT: unpcklpd (%rsp), %xmm1 # 16-byte Folded Reload
+; CHECK-NEXT: # xmm1 = xmm1[0],mem[0]
+; CHECK-NEXT: movaps %xmm1, %xmm0
+; CHECK-NEXT: addq $56, %rsp
+; CHECK-NEXT: retq
+ %r = call fast <4 x float> @llvm.cbrt.v4f32(<4 x float> %x)
+ ret <4 x float> %r
+}
+
+define <2 x double> @cbrt_v2f64_fmf(<2 x double> %x) nounwind {
+; CHECK-LABEL: cbrt_v2f64_fmf:
+; CHECK: # %bb.0:
+; CHECK-NEXT: subq $40, %rsp
+; CHECK-NEXT: movaps %xmm0, (%rsp) # 16-byte Spill
+; CHECK-NEXT: callq cbrt at PLT
+; CHECK-NEXT: movaps %xmm0, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
+; CHECK-NEXT: movaps (%rsp), %xmm0 # 16-byte Reload
+; CHECK-NEXT: movhlps {{.*#+}} xmm0 = xmm0[1,1]
+; CHECK-NEXT: callq cbrt at PLT
+; CHECK-NEXT: movaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm1 # 16-byte Reload
+; CHECK-NEXT: movlhps {{.*#+}} xmm1 = xmm1[0],xmm0[0]
+; CHECK-NEXT: movaps %xmm1, %xmm0
+; CHECK-NEXT: addq $40, %rsp
+; CHECK-NEXT: retq
+ %r = call fast <2 x double> @llvm.cbrt.v2f64(<2 x double> %x)
+ ret <2 x double> %r
+}
+
+attributes #0 = { nounwind "denormal-fp-math"="ieee,preserve-sign" }
More information about the llvm-commits
mailing list