[llvm] r173755 - Teach SDISel to combine fsin / fcos into a fsincos node if the following
Stephen Canon
scanon at apple.com
Mon Feb 4 10:59:23 PST 2013
For a variety of historical reasons, cexp(0 + i*x) is not guaranteed to be identical to cos(x) + i*sin(x), so the transform would be limited to fast-math contexts. __sincos_stret(x) makes that guarantee and can be used freely.
Also, __sincos_stret(x) can go faster, since it doesn't need to compute exp(0) or check that the real part of an argument is zero and branch. The whole point of this exercise is performance, after all.
- Steve
On Feb 4, 2013, at 10:51 AM, Benjamin Kramer <benny.kra at gmail.com> wrote:
> On 29.01.2013, at 03:32, Evan Cheng <evan.cheng at apple.com> wrote:
>
>> Author: evancheng
>> Date: Mon Jan 28 20:32:37 2013
>> New Revision: 173755
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=173755&view=rev
>> Log:
>> Teach SDISel to combine fsin / fcos into a fsincos node if the following
>> conditions are met:
>> 1. They share the same operand and are in the same BB.
>> 2. Both outputs are used.
>> 3. The target has a native instruction that maps to ISD::FSINCOS node or
>> the target provides a sincos library call.
>>
>> Implemented the generic optimization in sdisel and enabled it for
>> Mac OSX. Also added an additional optimization for x86_64 Mac OSX by
>> using an alternative entry point __sincos_stret which returns the two
>> results in xmm0 / xmm1.
>
> Before I forget about it: How is __sincos_stret different from cexp(0 + i*x), which was available on osx for a long time (and on all other C99 targets too)?
>
> The signature on x86_64 is:
>
> declare { double, double } @cexp(double, double)
> so the complex number is splat onto two registers making this just as simple.
>
> Looking at the source it adds a couple of extra checks that sincos doesn't need, but it looks like a more portable (and standard) alternative.
>
> - Ben
>
>
>>
>> rdar://13087969
>> PR13204
>>
>> Modified:
>> llvm/trunk/include/llvm/CodeGen/ISDOpcodes.h
>> llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h
>> llvm/trunk/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
>> llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp
>> llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp
>> llvm/trunk/lib/Target/ARM/ARMISelLowering.cpp
>> llvm/trunk/lib/Target/Hexagon/HexagonISelLowering.cpp
>> llvm/trunk/lib/Target/MBlaze/MBlazeISelLowering.cpp
>> llvm/trunk/lib/Target/Mips/MipsISelLowering.cpp
>> llvm/trunk/lib/Target/PowerPC/PPCISelLowering.cpp
>> llvm/trunk/lib/Target/Sparc/SparcISelLowering.cpp
>> llvm/trunk/lib/Target/X86/X86ISelLowering.cpp
>> llvm/trunk/lib/Target/X86/X86ISelLowering.h
>> llvm/trunk/lib/Target/X86/X86Subtarget.cpp
>> llvm/trunk/lib/Target/X86/X86Subtarget.h
>>
>> Modified: llvm/trunk/include/llvm/CodeGen/ISDOpcodes.h
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/ISDOpcodes.h?rev=173755&r1=173754&r2=173755&view=diff
>> ==============================================================================
>> --- llvm/trunk/include/llvm/CodeGen/ISDOpcodes.h (original)
>> +++ llvm/trunk/include/llvm/CodeGen/ISDOpcodes.h Mon Jan 28 20:32:37 2013
>> @@ -455,6 +455,9 @@ namespace ISD {
>> FNEG, FABS, FSQRT, FSIN, FCOS, FPOWI, FPOW,
>> FLOG, FLOG2, FLOG10, FEXP, FEXP2,
>> FCEIL, FTRUNC, FRINT, FNEARBYINT, FFLOOR,
>> +
>> + /// FSINCOS - Compute both fsin and fcos as a single operation.
>> + FSINCOS,
>>
>> /// LOAD and STORE have token chains as their first operand, then the same
>> /// operands as an LLVM load/store instruction, then an offset node that
>>
>> Modified: llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h?rev=173755&r1=173754&r2=173755&view=diff
>> ==============================================================================
>> --- llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h (original)
>> +++ llvm/trunk/include/llvm/CodeGen/RuntimeLibcalls.h Mon Jan 28 20:32:37 2013
>> @@ -158,6 +158,11 @@ namespace RTLIB {
>> COS_F80,
>> COS_F128,
>> COS_PPCF128,
>> + SINCOS_F32,
>> + SINCOS_F64,
>> + SINCOS_F80,
>> + SINCOS_F128,
>> + SINCOS_PPCF128,
>> POW_F32,
>> POW_F64,
>> POW_F80,
>>
>> Modified: llvm/trunk/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp?rev=173755&r1=173754&r2=173755&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp (original)
>> +++ llvm/trunk/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp Mon Jan 28 20:32:37 2013
>> @@ -102,7 +102,8 @@ private:
>> SDNode *Node, bool isSigned);
>> SDValue ExpandFPLibCall(SDNode *Node, RTLIB::Libcall Call_F32,
>> RTLIB::Libcall Call_F64, RTLIB::Libcall Call_F80,
>> - RTLIB::Libcall Call_F128, RTLIB::Libcall Call_PPCF128);
>> + RTLIB::Libcall Call_F128,
>> + RTLIB::Libcall Call_PPCF128);
>> SDValue ExpandIntLibCall(SDNode *Node, bool isSigned,
>> RTLIB::Libcall Call_I8,
>> RTLIB::Libcall Call_I16,
>> @@ -110,6 +111,7 @@ private:
>> RTLIB::Libcall Call_I64,
>> RTLIB::Libcall Call_I128);
>> void ExpandDivRemLibCall(SDNode *Node, SmallVectorImpl<SDValue> &Results);
>> + void ExpandSinCosLibCall(SDNode *Node, SmallVectorImpl<SDValue> &Results);
>>
>> SDValue EmitStackConvert(SDValue SrcOp, EVT SlotVT, EVT DestVT, DebugLoc dl);
>> SDValue ExpandBUILD_VECTOR(SDNode *Node);
>> @@ -2095,6 +2097,106 @@ SelectionDAGLegalize::ExpandDivRemLibCal
>> Results.push_back(Rem);
>> }
>>
>> +/// isSinCosLibcallAvailable - Return true if sincos libcall is available.
>> +static bool isSinCosLibcallAvailable(SDNode *Node, const TargetLowering &TLI) {
>> + RTLIB::Libcall LC;
>> + switch (Node->getValueType(0).getSimpleVT().SimpleTy) {
>> + default: llvm_unreachable("Unexpected request for libcall!");
>> + case MVT::f32: LC = RTLIB::SINCOS_F32; break;
>> + case MVT::f64: LC = RTLIB::SINCOS_F64; break;
>> + case MVT::f80: LC = RTLIB::SINCOS_F80; break;
>> + case MVT::f128: LC = RTLIB::SINCOS_F128; break;
>> + case MVT::ppcf128: LC = RTLIB::SINCOS_PPCF128; break;
>> + }
>> + return TLI.getLibcallName(LC) != 0;
>> +}
>> +
>> +/// useSinCos - Only issue sincos libcall if both sin and cos are
>> +/// needed.
>> +static bool useSinCos(SDNode *Node) {
>> + unsigned OtherOpcode = Node->getOpcode() == ISD::FSIN
>> + ? ISD::FCOS : ISD::FSIN;
>> +
>> + SDValue Op0 = Node->getOperand(0);
>> + for (SDNode::use_iterator UI = Op0.getNode()->use_begin(),
>> + UE = Op0.getNode()->use_end(); UI != UE; ++UI) {
>> + SDNode *User = *UI;
>> + if (User == Node)
>> + continue;
>> + // The other user might have been turned into sincos already.
>> + if (User->getOpcode() == OtherOpcode || User->getOpcode() == ISD::FSINCOS)
>> + return true;
>> + }
>> + return false;
>> +}
>> +
>> +/// ExpandSinCosLibCall - Issue libcalls to sincos to compute sin / cos
>> +/// pairs.
>> +void
>> +SelectionDAGLegalize::ExpandSinCosLibCall(SDNode *Node,
>> + SmallVectorImpl<SDValue> &Results) {
>> + RTLIB::Libcall LC;
>> + switch (Node->getValueType(0).getSimpleVT().SimpleTy) {
>> + default: llvm_unreachable("Unexpected request for libcall!");
>> + case MVT::f32: LC = RTLIB::SINCOS_F32; break;
>> + case MVT::f64: LC = RTLIB::SINCOS_F64; break;
>> + case MVT::f80: LC = RTLIB::SINCOS_F80; break;
>> + case MVT::f128: LC = RTLIB::SINCOS_F128; break;
>> + case MVT::ppcf128: LC = RTLIB::SINCOS_PPCF128; break;
>> + }
>> +
>> + // The input chain to this libcall is the entry node of the function.
>> + // Legalizing the call will automatically add the previous call to the
>> + // dependence.
>> + SDValue InChain = DAG.getEntryNode();
>> +
>> + EVT RetVT = Node->getValueType(0);
>> + Type *RetTy = RetVT.getTypeForEVT(*DAG.getContext());
>> +
>> + TargetLowering::ArgListTy Args;
>> + TargetLowering::ArgListEntry Entry;
>> +
>> + // Pass the argument.
>> + Entry.Node = Node->getOperand(0);
>> + Entry.Ty = RetTy;
>> + Entry.isSExt = false;
>> + Entry.isZExt = false;
>> + Args.push_back(Entry);
>> +
>> + // Pass the return address of sin.
>> + SDValue SinPtr = DAG.CreateStackTemporary(RetVT);
>> + Entry.Node = SinPtr;
>> + Entry.Ty = RetTy->getPointerTo();
>> + Entry.isSExt = false;
>> + Entry.isZExt = false;
>> + Args.push_back(Entry);
>> +
>> + // Also pass the return address of the cos.
>> + SDValue CosPtr = DAG.CreateStackTemporary(RetVT);
>> + Entry.Node = CosPtr;
>> + Entry.Ty = RetTy->getPointerTo();
>> + Entry.isSExt = false;
>> + Entry.isZExt = false;
>> + Args.push_back(Entry);
>> +
>> + SDValue Callee = DAG.getExternalSymbol(TLI.getLibcallName(LC),
>> + TLI.getPointerTy());
>> +
>> + DebugLoc dl = Node->getDebugLoc();
>> + TargetLowering::
>> + CallLoweringInfo CLI(InChain, Type::getVoidTy(*DAG.getContext()),
>> + false, false, false, false,
>> + 0, TLI.getLibcallCallingConv(LC), /*isTailCall=*/false,
>> + /*doesNotReturn=*/false, /*isReturnValueUsed=*/true,
>> + Callee, Args, DAG, dl);
>> + std::pair<SDValue, SDValue> CallInfo = TLI.LowerCallTo(CLI);
>> +
>> + Results.push_back(DAG.getLoad(RetVT, dl, CallInfo.second, SinPtr,
>> + MachinePointerInfo(), false, false, false, 0));
>> + Results.push_back(DAG.getLoad(RetVT, dl, CallInfo.second, CosPtr,
>> + MachinePointerInfo(), false, false, false, 0));
>> +}
>> +
>> /// ExpandLegalINT_TO_FP - This function is responsible for legalizing a
>> /// INT_TO_FP operation of the specified operand when the target requests that
>> /// we expand it. At this point, we know that the result and operand types are
>> @@ -3041,14 +3143,33 @@ void SelectionDAGLegalize::ExpandNode(SD
>> RTLIB::SQRT_PPCF128));
>> break;
>> case ISD::FSIN:
>> - Results.push_back(ExpandFPLibCall(Node, RTLIB::SIN_F32, RTLIB::SIN_F64,
>> - RTLIB::SIN_F80, RTLIB::SIN_F128,
>> - RTLIB::SIN_PPCF128));
>> - break;
>> - case ISD::FCOS:
>> - Results.push_back(ExpandFPLibCall(Node, RTLIB::COS_F32, RTLIB::COS_F64,
>> - RTLIB::COS_F80, RTLIB::COS_F128,
>> - RTLIB::COS_PPCF128));
>> + case ISD::FCOS: {
>> + EVT VT = Node->getValueType(0);
>> + bool isSIN = Node->getOpcode() == ISD::FSIN;
>> + // Turn fsin / fcos into ISD::FSINCOS node if there are a pair of fsin /
>> + // fcos which share the same operand and both are used.
>> + if ((TLI.isOperationLegalOrCustom(ISD::FSINCOS, VT) ||
>> + isSinCosLibcallAvailable(Node, TLI))
>> + && useSinCos(Node)) {
>> + SDVTList VTs = DAG.getVTList(VT, VT);
>> + Tmp1 = DAG.getNode(ISD::FSINCOS, dl, VTs, Node->getOperand(0));
>> + if (!isSIN)
>> + Tmp1 = Tmp1.getValue(1);
>> + Results.push_back(Tmp1);
>> + } else if (isSIN) {
>> + Results.push_back(ExpandFPLibCall(Node, RTLIB::SIN_F32, RTLIB::SIN_F64,
>> + RTLIB::SIN_F80, RTLIB::SIN_F128,
>> + RTLIB::SIN_PPCF128));
>> + } else {
>> + Results.push_back(ExpandFPLibCall(Node, RTLIB::COS_F32, RTLIB::COS_F64,
>> + RTLIB::COS_F80, RTLIB::COS_F128,
>> + RTLIB::COS_PPCF128));
>> + }
>> + break;
>> + }
>> + case ISD::FSINCOS:
>> + // Expand into sincos libcall.
>> + ExpandSinCosLibCall(Node, Results);
>> break;
>> case ISD::FLOG:
>> Results.push_back(ExpandFPLibCall(Node, RTLIB::LOG_F32, RTLIB::LOG_F64,
>> @@ -3181,7 +3302,6 @@ void SelectionDAGLegalize::ExpandNode(SD
>> case ISD::UREM:
>> case ISD::SREM: {
>> EVT VT = Node->getValueType(0);
>> - SDVTList VTs = DAG.getVTList(VT, VT);
>> bool isSigned = Node->getOpcode() == ISD::SREM;
>> unsigned DivOpc = isSigned ? ISD::SDIV : ISD::UDIV;
>> unsigned DivRemOpc = isSigned ? ISD::SDIVREM : ISD::UDIVREM;
>> @@ -3192,6 +3312,7 @@ void SelectionDAGLegalize::ExpandNode(SD
>> // If div is legal, it's better to do the normal expansion
>> !TLI.isOperationLegalOrCustom(DivOpc, Node->getValueType(0)) &&
>> useDivRem(Node, isSigned, false))) {
>> + SDVTList VTs = DAG.getVTList(VT, VT);
>> Tmp1 = DAG.getNode(DivRemOpc, dl, VTs, Tmp2, Tmp3).getValue(1);
>> } else if (TLI.isOperationLegalOrCustom(DivOpc, VT)) {
>> // X % Y -> X-X/Y*Y
>>
>> Modified: llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp?rev=173755&r1=173754&r2=173755&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp (original)
>> +++ llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp Mon Jan 28 20:32:37 2013
>> @@ -140,6 +140,7 @@ std::string SDNode::getOperationName(con
>> case ISD::FSQRT: return "fsqrt";
>> case ISD::FSIN: return "fsin";
>> case ISD::FCOS: return "fcos";
>> + case ISD::FSINCOS: return "fsincos";
>> case ISD::FTRUNC: return "ftrunc";
>> case ISD::FFLOOR: return "ffloor";
>> case ISD::FCEIL: return "fceil";
>>
>> Modified: llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp?rev=173755&r1=173754&r2=173755&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp (original)
>> +++ llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp Mon Jan 28 20:32:37 2013
>> @@ -340,6 +340,13 @@ static void InitLibcallNames(const char
>> Names[RTLIB::SYNC_FETCH_AND_NAND_2] = "__sync_fetch_and_nand_2";
>> Names[RTLIB::SYNC_FETCH_AND_NAND_4] = "__sync_fetch_and_nand_4";
>> Names[RTLIB::SYNC_FETCH_AND_NAND_8] = "__sync_fetch_and_nand_8";
>> +
>> + // These are generally not available.
>> + Names[RTLIB::SINCOS_F32] = 0;
>> + Names[RTLIB::SINCOS_F64] = 0;
>> + Names[RTLIB::SINCOS_F80] = 0;
>> + Names[RTLIB::SINCOS_F128] = 0;
>> + Names[RTLIB::SINCOS_PPCF128] = 0;
>> }
>>
>> /// InitLibcallCallingConvs - Set default libcall CallingConvs.
>>
>> Modified: llvm/trunk/lib/Target/ARM/ARMISelLowering.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/ARM/ARMISelLowering.cpp?rev=173755&r1=173754&r2=173755&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/Target/ARM/ARMISelLowering.cpp (original)
>> +++ llvm/trunk/lib/Target/ARM/ARMISelLowering.cpp Mon Jan 28 20:32:37 2013
>> @@ -781,6 +781,8 @@ ARMTargetLowering::ARMTargetLowering(Tar
>> setOperationAction(ISD::FSIN, MVT::f32, Expand);
>> setOperationAction(ISD::FCOS, MVT::f32, Expand);
>> setOperationAction(ISD::FCOS, MVT::f64, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f64, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f32, Expand);
>> setOperationAction(ISD::FREM, MVT::f64, Expand);
>> setOperationAction(ISD::FREM, MVT::f32, Expand);
>> if (!TM.Options.UseSoftFloat && Subtarget->hasVFP2() &&
>>
>> Modified: llvm/trunk/lib/Target/Hexagon/HexagonISelLowering.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/Hexagon/HexagonISelLowering.cpp?rev=173755&r1=173754&r2=173755&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/Target/Hexagon/HexagonISelLowering.cpp (original)
>> +++ llvm/trunk/lib/Target/Hexagon/HexagonISelLowering.cpp Mon Jan 28 20:32:37 2013
>> @@ -1364,6 +1364,8 @@ HexagonTargetLowering::HexagonTargetLowe
>> setOperationAction(ISD::FSIN , MVT::f32, Expand);
>> setOperationAction(ISD::FCOS , MVT::f32, Expand);
>> setOperationAction(ISD::FREM , MVT::f32, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f64, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f32, Expand);
>> setOperationAction(ISD::CTPOP, MVT::i32, Expand);
>> setOperationAction(ISD::CTTZ , MVT::i32, Expand);
>> setOperationAction(ISD::CTTZ_ZERO_UNDEF, MVT::i32, Expand);
>>
>> Modified: llvm/trunk/lib/Target/MBlaze/MBlazeISelLowering.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/MBlaze/MBlazeISelLowering.cpp?rev=173755&r1=173754&r2=173755&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/Target/MBlaze/MBlazeISelLowering.cpp (original)
>> +++ llvm/trunk/lib/Target/MBlaze/MBlazeISelLowering.cpp Mon Jan 28 20:32:37 2013
>> @@ -81,6 +81,7 @@ MBlazeTargetLowering::MBlazeTargetLoweri
>> setOperationAction(ISD::FCOPYSIGN, MVT::f64, Expand);
>> setOperationAction(ISD::FSIN, MVT::f32, Expand);
>> setOperationAction(ISD::FCOS, MVT::f32, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f32, Expand);
>> setOperationAction(ISD::FPOWI, MVT::f32, Expand);
>> setOperationAction(ISD::FPOW, MVT::f32, Expand);
>> setOperationAction(ISD::FLOG, MVT::f32, Expand);
>>
>> Modified: llvm/trunk/lib/Target/Mips/MipsISelLowering.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/Mips/MipsISelLowering.cpp?rev=173755&r1=173754&r2=173755&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/Target/Mips/MipsISelLowering.cpp (original)
>> +++ llvm/trunk/lib/Target/Mips/MipsISelLowering.cpp Mon Jan 28 20:32:37 2013
>> @@ -421,6 +421,8 @@ MipsTargetLowering(MipsTargetMachine &TM
>> setOperationAction(ISD::FSIN, MVT::f64, Expand);
>> setOperationAction(ISD::FCOS, MVT::f32, Expand);
>> setOperationAction(ISD::FCOS, MVT::f64, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f32, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f64, Expand);
>> setOperationAction(ISD::FPOWI, MVT::f32, Expand);
>> setOperationAction(ISD::FPOW, MVT::f32, Expand);
>> setOperationAction(ISD::FPOW, MVT::f64, Expand);
>>
>> Modified: llvm/trunk/lib/Target/PowerPC/PPCISelLowering.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/PowerPC/PPCISelLowering.cpp?rev=173755&r1=173754&r2=173755&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/Target/PowerPC/PPCISelLowering.cpp (original)
>> +++ llvm/trunk/lib/Target/PowerPC/PPCISelLowering.cpp Mon Jan 28 20:32:37 2013
>> @@ -132,11 +132,13 @@ PPCTargetLowering::PPCTargetLowering(PPC
>> // We don't support sin/cos/sqrt/fmod/pow
>> setOperationAction(ISD::FSIN , MVT::f64, Expand);
>> setOperationAction(ISD::FCOS , MVT::f64, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f64, Expand);
>> setOperationAction(ISD::FREM , MVT::f64, Expand);
>> setOperationAction(ISD::FPOW , MVT::f64, Expand);
>> setOperationAction(ISD::FMA , MVT::f64, Legal);
>> setOperationAction(ISD::FSIN , MVT::f32, Expand);
>> setOperationAction(ISD::FCOS , MVT::f32, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f32, Expand);
>> setOperationAction(ISD::FREM , MVT::f32, Expand);
>> setOperationAction(ISD::FPOW , MVT::f32, Expand);
>> setOperationAction(ISD::FMA , MVT::f32, Legal);
>>
>> Modified: llvm/trunk/lib/Target/Sparc/SparcISelLowering.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/Sparc/SparcISelLowering.cpp?rev=173755&r1=173754&r2=173755&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/Target/Sparc/SparcISelLowering.cpp (original)
>> +++ llvm/trunk/lib/Target/Sparc/SparcISelLowering.cpp Mon Jan 28 20:32:37 2013
>> @@ -759,10 +759,12 @@ SparcTargetLowering::SparcTargetLowering
>>
>> setOperationAction(ISD::FSIN , MVT::f64, Expand);
>> setOperationAction(ISD::FCOS , MVT::f64, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f64, Expand);
>> setOperationAction(ISD::FREM , MVT::f64, Expand);
>> setOperationAction(ISD::FMA , MVT::f64, Expand);
>> setOperationAction(ISD::FSIN , MVT::f32, Expand);
>> setOperationAction(ISD::FCOS , MVT::f32, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f32, Expand);
>> setOperationAction(ISD::FREM , MVT::f32, Expand);
>> setOperationAction(ISD::FMA , MVT::f32, Expand);
>> setOperationAction(ISD::CTPOP, MVT::i32, Expand);
>>
>> Modified: llvm/trunk/lib/Target/X86/X86ISelLowering.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86ISelLowering.cpp?rev=173755&r1=173754&r2=173755&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/Target/X86/X86ISelLowering.cpp (original)
>> +++ llvm/trunk/lib/Target/X86/X86ISelLowering.cpp Mon Jan 28 20:32:37 2013
>> @@ -605,10 +605,12 @@ X86TargetLowering::X86TargetLowering(X86
>> setOperationAction(ISD::FGETSIGN, MVT::i32, Custom);
>>
>> // We don't support sin/cos/fmod
>> - setOperationAction(ISD::FSIN , MVT::f64, Expand);
>> - setOperationAction(ISD::FCOS , MVT::f64, Expand);
>> - setOperationAction(ISD::FSIN , MVT::f32, Expand);
>> - setOperationAction(ISD::FCOS , MVT::f32, Expand);
>> + setOperationAction(ISD::FSIN , MVT::f64, Expand);
>> + setOperationAction(ISD::FCOS , MVT::f64, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f64, Expand);
>> + setOperationAction(ISD::FSIN , MVT::f32, Expand);
>> + setOperationAction(ISD::FCOS , MVT::f32, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f32, Expand);
>>
>> // Expand FP immediates into loads from the stack, except for the special
>> // cases we handle.
>> @@ -633,8 +635,9 @@ X86TargetLowering::X86TargetLowering(X86
>> setOperationAction(ISD::FCOPYSIGN, MVT::f32, Custom);
>>
>> // We don't support sin/cos/fmod
>> - setOperationAction(ISD::FSIN , MVT::f32, Expand);
>> - setOperationAction(ISD::FCOS , MVT::f32, Expand);
>> + setOperationAction(ISD::FSIN , MVT::f32, Expand);
>> + setOperationAction(ISD::FCOS , MVT::f32, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f32, Expand);
>>
>> // Special cases we handle for FP constants.
>> addLegalFPImmediate(APFloat(+0.0f)); // xorps
>> @@ -644,8 +647,9 @@ X86TargetLowering::X86TargetLowering(X86
>> addLegalFPImmediate(APFloat(-1.0)); // FLD1/FCHS
>>
>> if (!TM.Options.UnsafeFPMath) {
>> - setOperationAction(ISD::FSIN , MVT::f64 , Expand);
>> - setOperationAction(ISD::FCOS , MVT::f64 , Expand);
>> + setOperationAction(ISD::FSIN , MVT::f64, Expand);
>> + setOperationAction(ISD::FCOS , MVT::f64, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f64, Expand);
>> }
>> } else if (!TM.Options.UseSoftFloat) {
>> // f32 and f64 in x87.
>> @@ -659,10 +663,12 @@ X86TargetLowering::X86TargetLowering(X86
>> setOperationAction(ISD::FCOPYSIGN, MVT::f32, Expand);
>>
>> if (!TM.Options.UnsafeFPMath) {
>> - setOperationAction(ISD::FSIN , MVT::f32 , Expand);
>> - setOperationAction(ISD::FSIN , MVT::f64 , Expand);
>> - setOperationAction(ISD::FCOS , MVT::f32 , Expand);
>> - setOperationAction(ISD::FCOS , MVT::f64 , Expand);
>> + setOperationAction(ISD::FSIN , MVT::f64, Expand);
>> + setOperationAction(ISD::FSIN , MVT::f32, Expand);
>> + setOperationAction(ISD::FCOS , MVT::f64, Expand);
>> + setOperationAction(ISD::FCOS , MVT::f32, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f64, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f32, Expand);
>> }
>> addLegalFPImmediate(APFloat(+0.0)); // FLD0
>> addLegalFPImmediate(APFloat(+1.0)); // FLD1
>> @@ -699,8 +705,9 @@ X86TargetLowering::X86TargetLowering(X86
>> }
>>
>> if (!TM.Options.UnsafeFPMath) {
>> - setOperationAction(ISD::FSIN , MVT::f80 , Expand);
>> - setOperationAction(ISD::FCOS , MVT::f80 , Expand);
>> + setOperationAction(ISD::FSIN , MVT::f80, Expand);
>> + setOperationAction(ISD::FCOS , MVT::f80, Expand);
>> + setOperationAction(ISD::FSINCOS, MVT::f80, Expand);
>> }
>>
>> setOperationAction(ISD::FFLOOR, MVT::f80, Expand);
>> @@ -748,7 +755,9 @@ X86TargetLowering::X86TargetLowering(X86
>> setOperationAction(ISD::INSERT_SUBVECTOR, VT,Expand);
>> setOperationAction(ISD::FABS, VT, Expand);
>> setOperationAction(ISD::FSIN, VT, Expand);
>> + setOperationAction(ISD::FSINCOS, VT, Expand);
>> setOperationAction(ISD::FCOS, VT, Expand);
>> + setOperationAction(ISD::FSINCOS, VT, Expand);
>> setOperationAction(ISD::FREM, VT, Expand);
>> setOperationAction(ISD::FMA, VT, Expand);
>> setOperationAction(ISD::FPOWI, VT, Expand);
>> @@ -1281,6 +1290,19 @@ X86TargetLowering::X86TargetLowering(X86
>> setLibcallName(RTLIB::SRA_I128, 0);
>> }
>>
>> + // Combine sin / cos into one node or libcall if possible.
>> + if (Subtarget->hasSinCos()) {
>> + setLibcallName(RTLIB::SINCOS_F32, "sincosf");
>> + setLibcallName(RTLIB::SINCOS_F64, "sincos");
>> + if (Subtarget->isTargetDarwin() && Subtarget->is64Bit()) {
>> + // For MacOSX, we don't want to the normal expansion of a libcall to
>> + // sincos. We want to issue a libcall to __sincos_stret to avoid memory
>> + // traffic.
>> + setOperationAction(ISD::FSINCOS, MVT::f64, Custom);
>> + setOperationAction(ISD::FSINCOS, MVT::f32, Custom);
>> + }
>> + }
>> +
>> // We have target-specific dag combine patterns for the following nodes:
>> setTargetDAGCombine(ISD::VECTOR_SHUFFLE);
>> setTargetDAGCombine(ISD::EXTRACT_VECTOR_ELT);
>> @@ -12014,6 +12036,50 @@ static SDValue LowerADDC_ADDE_SUBC_SUBE(
>> Op.getOperand(1), Op.getOperand(2));
>> }
>>
>> +SDValue X86TargetLowering::LowerFSINCOS(SDValue Op, SelectionDAG &DAG) const {
>> + assert(Subtarget->isTargetDarwin());
>> +
>> + // For MacOSX, we want to call an alternative entry point: __sincos_stret,
>> + // which returns the values in two XMM registers.
>> + DebugLoc dl = Op.getDebugLoc();
>> + SDValue Arg = Op.getOperand(0);
>> + EVT ArgVT = Arg.getValueType();
>> + Type *ArgTy = ArgVT.getTypeForEVT(*DAG.getContext());
>> +
>> + ArgListTy Args;
>> + ArgListEntry Entry;
>> +
>> + Entry.Node = Arg;
>> + Entry.Ty = ArgTy;
>> + Entry.isSExt = false;
>> + Entry.isZExt = false;
>> + Args.push_back(Entry);
>> +
>> + const char *LibcallName = (ArgVT == MVT::f64)
>> + ? "__sincos_stret" : "__sincosf_stret";
>> + SDValue Callee = DAG.getExternalSymbol(LibcallName, getPointerTy());
>> +
>> + StructType *RetTy = StructType::get(ArgTy, ArgTy, NULL);
>> + TargetLowering::
>> + CallLoweringInfo CLI(DAG.getEntryNode(), RetTy,
>> + false, false, false, false, 0,
>> + CallingConv::C, /*isTaillCall=*/false,
>> + /*doesNotRet=*/false, /*isReturnValueUsed*/true,
>> + Callee, Args, DAG, dl);
>> + std::pair<SDValue, SDValue> CallResult = LowerCallTo(CLI);
>> +#if 1
>> + return CallResult.first;
>> +#else
>> + SDValue RetSin = DAG.getNode(ISD::EXTRACT_ELEMENT, dl, ArgVT,
>> + CallResult.first, DAG.getIntPtrConstant(0));
>> + SDValue RetCos = DAG.getNode(ISD::EXTRACT_ELEMENT, dl, ArgVT,
>> + CallResult.first, DAG.getIntPtrConstant(1));
>> +
>> + SDVTList Tys = DAG.getVTList(ArgVT, ArgVT);
>> + return DAG.getNode(ISD::MERGE_VALUES, dl, Tys, RetSin, RetCos);
>> +#endif
>> +}
>> +
>> /// LowerOperation - Provide custom lowering hooks for some operations.
>> ///
>> SDValue X86TargetLowering::LowerOperation(SDValue Op, SelectionDAG &DAG) const {
>> @@ -12096,6 +12162,7 @@ SDValue X86TargetLowering::LowerOperatio
>> case ISD::ADD: return LowerADD(Op, DAG);
>> case ISD::SUB: return LowerSUB(Op, DAG);
>> case ISD::SDIV: return LowerSDIV(Op, DAG);
>> + case ISD::FSINCOS: return LowerFSINCOS(Op, DAG);
>> }
>> }
>>
>>
>> Modified: llvm/trunk/lib/Target/X86/X86ISelLowering.h
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86ISelLowering.h?rev=173755&r1=173754&r2=173755&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/Target/X86/X86ISelLowering.h (original)
>> +++ llvm/trunk/lib/Target/X86/X86ISelLowering.h Mon Jan 28 20:32:37 2013
>> @@ -838,8 +838,8 @@ namespace llvm {
>> SDValue LowerFLT_ROUNDS_(SDValue Op, SelectionDAG &DAG) const;
>> SDValue LowerShift(SDValue Op, SelectionDAG &DAG) const;
>> SDValue LowerSDIV(SDValue Op, SelectionDAG &DAG) const;
>> -
>> SDValue LowerSIGN_EXTEND_INREG(SDValue Op, SelectionDAG &DAG) const;
>> + SDValue LowerFSINCOS(SDValue Op, SelectionDAG &DAG) const;
>>
>> // Utility functions to help LowerVECTOR_SHUFFLE & LowerBUILD_VECTOR
>> SDValue LowerVectorBroadcast(SDValue Op, SelectionDAG &DAG) const;
>>
>> Modified: llvm/trunk/lib/Target/X86/X86Subtarget.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86Subtarget.cpp?rev=173755&r1=173754&r2=173755&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/Target/X86/X86Subtarget.cpp (original)
>> +++ llvm/trunk/lib/Target/X86/X86Subtarget.cpp Mon Jan 28 20:32:37 2013
>> @@ -155,6 +155,11 @@ const char *X86Subtarget::getBZeroEntry(
>> return 0;
>> }
>>
>> +bool X86Subtarget::hasSinCos() const {
>> + return getTargetTriple().isMacOSX() &&
>> + !getTargetTriple().isMacOSXVersionLT(10, 9);
>> +}
>> +
>> /// IsLegalToCallImmediateAddr - Return true if the subtarget allows calls
>> /// to immediate address.
>> bool X86Subtarget::IsLegalToCallImmediateAddr(const TargetMachine &TM) const {
>>
>> Modified: llvm/trunk/lib/Target/X86/X86Subtarget.h
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86Subtarget.h?rev=173755&r1=173754&r2=173755&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/Target/X86/X86Subtarget.h (original)
>> +++ llvm/trunk/lib/Target/X86/X86Subtarget.h Mon Jan 28 20:32:37 2013
>> @@ -328,6 +328,10 @@ public:
>> /// memset with zero passed as the second argument. Otherwise it
>> /// returns null.
>> const char *getBZeroEntry() const;
>> +
>> + /// This function returns true if the target has sincos() routine in its
>> + /// compiler runtime or math libraries.
>> + bool hasSinCos() const;
>>
>> /// enablePostRAScheduler - run for Atom optimization.
>> bool enablePostRAScheduler(CodeGenOpt::Level OptLevel,
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
More information about the llvm-commits
mailing list