[llvm-commits] [llvm] r47593 - in /llvm/trunk/lib/Target/X86: X86ISelLowering.cpp X86ISelLowering.h
Tanya Lattner
lattner at apple.com
Tue Feb 26 10:45:54 PST 2008
On Feb 26, 2008, at 9:46 AM, Arnold Schwaighofer wrote:
> Will correct. I always get them wrong. actually looked before
> commiting and many functions in that file had 2 so i followed :).
>
Thanks!
Feel free to update any that you see. Some always manage to get
missed or they were there before we really started using doxygen.
-Tanya
> On 26 Feb 2008, at 18:24, Tanya Lattner wrote:
>
>> Please use doxygen style comments for functions (///)
>>
>> -Tanya
>>
>> On Feb 26, 2008, at 1:20 AM, Arnold Schwaighofer wrote:
>>
>>> Author: arnolds
>>> Date: Tue Feb 26 03:19:59 2008
>>> New Revision: 47593
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=47593&view=rev
>>> Log:
>>> Change the lowering of arguments for tail call optimized
>>> calls. Before arguments that could overwrite each other were
>>> explicitly lowered to a stack slot, not giving the register
>>> allocator
>>> a chance to optimize. Now a sequence of copyto/copyfrom virtual
>>> registers ensures that arguments are loaded in (virtual) registers
>>> before they are lowered to the stack slot (and might overwrite each
>>> other). Also parameter stack slots are marked mutable for
>>> (potentially) tail calling functions.
>>>
>>> Modified:
>>> llvm/trunk/lib/Target/X86/X86ISelLowering.cpp
>>> llvm/trunk/lib/Target/X86/X86ISelLowering.h
>>>
>>> Modified: llvm/trunk/lib/Target/X86/X86ISelLowering.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/
>>> X86ISelLowering.cpp?rev=47593&r1=47592&r2=47593&view=diff
>>>
>>> ====================================================================
>>> =
>>> =
>>> ========
>>> --- llvm/trunk/lib/Target/X86/X86ISelLowering.cpp (original)
>>> +++ llvm/trunk/lib/Target/X86/X86ISelLowering.cpp Tue Feb 26
>>> 03:19:59 2008
>>> @@ -1057,26 +1057,55 @@
>>> return None;
>>> }
>>>
>>> -
>>> // IsPossiblyOverwrittenArgumentOfTailCall - Check if the operand
>>> could possibly
>>> // be overwritten when lowering the outgoing arguments in a tail
>>> call. Currently
>>> // the implementation of this call is very conservative and
>>> assumes all
>>> // arguments sourcing from FORMAL_ARGUMENTS or a CopyFromReg with
>>> virtual
>>> // registers would be overwritten by direct lowering.
>>> -// Possible improvement:
>>> -// Check FORMAL_ARGUMENTS corresponding MERGE_VALUES for
>>> CopyFromReg nodes
>>> -// indicating inreg passed arguments which also need not be
>>> lowered to a safe
>>> -// stack slot.
>>> -static bool IsPossiblyOverwrittenArgumentOfTailCall(SDOperand Op) {
>>> +static bool IsPossiblyOverwrittenArgumentOfTailCall(SDOperand Op,
>>> +
>>> MachineFrameInfo * MFI) {
>>> RegisterSDNode * OpReg = NULL;
>>> + FrameIndexSDNode * FrameIdxNode = NULL;
>>> + int FrameIdx = 0;
>>> if (Op.getOpcode() == ISD::FORMAL_ARGUMENTS ||
>>> (Op.getOpcode()== ISD::CopyFromReg &&
>>> - (OpReg = cast<RegisterSDNode>(Op.getOperand(1))) &&
>>> - OpReg->getReg() >=
>>> TargetRegisterInfo::FirstVirtualRegister))
>>> + (OpReg = dyn_cast<RegisterSDNode>(Op.getOperand(1))) &&
>>> + (OpReg->getReg() >=
>>> TargetRegisterInfo::FirstVirtualRegister)) ||
>>> + (Op.getOpcode() == ISD::LOAD &&
>>> + (FrameIdxNode = dyn_cast<FrameIndexSDNode>(Op.getOperand
>>> (1))) &&
>>> + (MFI->isFixedObjectIndex((FrameIdx = FrameIdxNode->getIndex
>>> ()))) &&
>>> + (MFI->getObjectOffset(FrameIdx) >= 0)))
>>> return true;
>>> return false;
>>> }
>>>
>>> +// CopyTailCallClobberedArgumentsToVRegs - Create virtual
>>> registers for all
>>> +// arguments to force loading and guarantee that arguments
>>> sourcing from
>>> +// incomming parameters are not overwriting each other.
>>> +static SDOperand
>>> +CopyTailCallClobberedArgumentsToVRegs(SDOperand Chain,
>>> + SmallVector<std::pair<unsigned, SDOperand>, 8>
>>> &TailCallClobberedVRegs,
>>> + SelectionDAG &DAG,
>>> + MachineFunction &MF,
>>> + const TargetLowering * TL) {
>>> +
>>> + SDOperand InFlag;
>>> + for (unsigned i = 0, e = TailCallClobberedVRegs.size(); i != e; i
>>> ++) {
>>> + SDOperand Arg = TailCallClobberedVRegs[i].second;
>>> + unsigned Idx = TailCallClobberedVRegs[i].first;
>>> + unsigned VReg =
>>> + MF.getRegInfo().
>>> + createVirtualRegister(TL->getRegClassFor(Arg.getValueType
>>> ()));
>>> + Chain = DAG.getCopyToReg(Chain, VReg, Arg, InFlag);
>>> + InFlag = Chain.getValue(1);
>>> + Arg = DAG.getCopyFromReg(Chain, VReg, Arg.getValueType(),
>>> InFlag);
>>> + TailCallClobberedVRegs[i] = std::make_pair(Idx, Arg);
>>> + Chain = Arg.getValue(1);
>>> + InFlag = Arg.getValue(2);
>>> + }
>>> + return Chain;
>>> +}
>>> +
>>> // CreateCopyOfByValArgument - Make a copy of an aggregate at
>>> address specified
>>> // by "Src" to address "Dst" with size and alignment information
>>> specified by
>>> // the specific parameter attribute. The copy will be passed as a
>>> byval function
>>> @@ -1097,15 +1126,20 @@
>>> SDOperand X86TargetLowering::LowerMemArgument(SDOperand Op,
>>> SelectionDAG &DAG,
>>> const CCValAssign
>>> &VA,
>>> MachineFrameInfo
>>> *MFI,
>>> + unsigned CC,
>>> SDOperand Root,
>>> unsigned i) {
>>> // Create the nodes corresponding to a load from this parameter
>>> slot.
>>> unsigned Flags = cast<ConstantSDNode>(Op.getOperand(3 + i))-
>>>> getValue();
>>> + bool AlwaysUseMutable = (CC==CallingConv::Fast) &&
>>> PerformTailCallOpt;
>>> bool isByVal = Flags & ISD::ParamFlags::ByVal;
>>> + bool isImmutable = !AlwaysUseMutable && !isByVal;
>>>
>>> - // FIXME: For now, all byval parameter objects are marked
>>> mutable. This
>>> - // can be changed with more analysis.
>>> + // FIXME: For now, all byval parameter objects are marked
>>> mutable. This can be
>>> + // changed with more analysis.
>>> + // In case of tail call optimization mark all arguments mutable.
>>> Since they
>>> + // could be overwritten by lowering of arguments in case of a
>>> tail call.
>>> int FI = MFI->CreateFixedObject(MVT::getSizeInBits(VA.getValVT
>>> ())/8,
>>> - VA.getLocMemOffset(), !isByVal);
>>> + VA.getLocMemOffset(),
>>> isImmutable);
>>> SDOperand FIN = DAG.getFrameIndex(FI, getPointerTy());
>>> if (isByVal)
>>> return FIN;
>>> @@ -1195,7 +1229,7 @@
>>> ArgValues.push_back(ArgValue);
>>> } else {
>>> assert(VA.isMemLoc());
>>> - ArgValues.push_back(LowerMemArgument(Op, DAG, VA, MFI, Root,
>>> i));
>>> + ArgValues.push_back(LowerMemArgument(Op, DAG, VA, MFI, CC,
>>> Root, i));
>>> }
>>> }
>>>
>>> @@ -1381,6 +1415,7 @@
>>>
>>> SDOperand X86TargetLowering::LowerCALL(SDOperand Op, SelectionDAG
>>> &DAG) {
>>> MachineFunction &MF = DAG.getMachineFunction();
>>> + MachineFrameInfo * MFI = MF.getFrameInfo();
>>> SDOperand Chain = Op.getOperand(0);
>>> unsigned CC = cast<ConstantSDNode>(Op.getOperand(1))-
>>>> getValue();
>>> bool isVarArg = cast<ConstantSDNode>(Op.getOperand(2))-
>>>> getValue() != 0;
>>> @@ -1442,7 +1477,7 @@
>>>
>>> Chain = DAG.getCALLSEQ_START(Chain, DAG.getIntPtrConstant
>>> (NumBytes));
>>>
>>> - SDOperand RetAddrFrIdx, NewRetAddrFrIdx;
>>> + SDOperand RetAddrFrIdx;
>>> if (IsTailCall) {
>>> // Adjust the Return address stack slot.
>>> if (FPDiff) {
>>> @@ -1451,23 +1486,18 @@
>>> // Load the "old" Return address.
>>> RetAddrFrIdx =
>>> DAG.getLoad(VT, Chain,RetAddrFrIdx, NULL, 0);
>>> - // Calculate the new stack slot for the return address.
>>> - int SlotSize = Is64Bit ? 8 : 4;
>>> - int NewReturnAddrFI =
>>> - MF.getFrameInfo()->CreateFixedObject(SlotSize, FPDiff-
>>> SlotSize);
>>> - NewRetAddrFrIdx = DAG.getFrameIndex(NewReturnAddrFI, VT);
>>> Chain = SDOperand(RetAddrFrIdx.Val, 1);
>>> }
>>> }
>>>
>>> SmallVector<std::pair<unsigned, SDOperand>, 8> RegsToPass;
>>> + SmallVector<std::pair<unsigned, SDOperand>, 8>
>>> TailCallClobberedVRegs;
>>> SmallVector<SDOperand, 8> MemOpChains;
>>>
>>> SDOperand StackPtr;
>>>
>>> // Walk the register/memloc assignments, inserting copies/
>>> loads. For tail
>>> - // calls, lower arguments which could otherwise be possibly
>>> overwritten to the
>>> - // stack slot where they would go on normal function calls.
>>> + // calls, remember all arguments for later special lowering.
>>> for (unsigned i = 0, e = ArgLocs.size(); i != e; ++i) {
>>> CCValAssign &VA = ArgLocs[i];
>>> SDOperand Arg = Op.getOperand(5+2*VA.getValNo());
>>> @@ -1490,13 +1520,15 @@
>>> if (VA.isRegLoc()) {
>>> RegsToPass.push_back(std::make_pair(VA.getLocReg(), Arg));
>>> } else {
>>> - if (!IsTailCall || IsPossiblyOverwrittenArgumentOfTailCall
>>> (Arg)) {
>>> + if (!IsTailCall) {
>>> assert(VA.isMemLoc());
>>> if (StackPtr.Val == 0)
>>> StackPtr = DAG.getCopyFromReg(Chain, X86StackPtr,
>>> getPointerTy());
>>>
>>> MemOpChains.push_back(LowerMemOpCallTo(Op, DAG, StackPtr,
>>> VA, Chain,
>>> Arg));
>>> + } else if (IsPossiblyOverwrittenArgumentOfTailCall(Arg,
>>> MFI)) {
>>> + TailCallClobberedVRegs.push_back(std::make_pair(i,Arg));
>>> }
>>> }
>>> }
>>> @@ -1514,9 +1546,6 @@
>>> InFlag = Chain.getValue(1);
>>> }
>>>
>>> - if (IsTailCall)
>>> - InFlag = SDOperand(); // ??? Isn't this nuking the preceding
>>> loop's output?
>>> -
>>> // ELF / PIC requires GOT in the EBX register before function
>>> calls via PLT
>>> // GOT pointer.
>>> // Does not work with tail call since ebx is not restored
>>> correctly by
>>> @@ -1551,11 +1580,18 @@
>>> InFlag = Chain.getValue(1);
>>> }
>>>
>>> +
>>> // For tail calls lower the arguments to the 'real' stack slot.
>>> if (IsTailCall) {
>>> SmallVector<SDOperand, 8> MemOpChains2;
>>> SDOperand FIN;
>>> int FI = 0;
>>> + // Do not flag preceeding copytoreg stuff together with the
>>> following stuff.
>>> + InFlag = SDOperand();
>>> +
>>> + Chain = CopyTailCallClobberedArgumentsToVRegs(Chain,
>>> TailCallClobberedVRegs,
>>> + DAG, MF, this);
>>> +
>>> for (unsigned i = 0, e = ArgLocs.size(); i != e; ++i) {
>>> CCValAssign &VA = ArgLocs[i];
>>> if (!VA.isRegLoc()) {
>>> @@ -1568,28 +1604,26 @@
>>> uint32_t OpSize = (MVT::getSizeInBits(VA.getLocVT())+7)/8;
>>> FI = MF.getFrameInfo()->CreateFixedObject(OpSize, Offset);
>>> FIN = DAG.getFrameIndex(FI, MVT::i32);
>>> - SDOperand Source = Arg;
>>> - if (IsPossiblyOverwrittenArgumentOfTailCall(Arg)) {
>>> - // Copy from stack slots to stack slot of a tail called
>>> function. This
>>> - // needs to be done because if we would lower the
>>> arguments directly
>>> - // to their real stack slot we might end up overwriting
>>> each other.
>>> - // Get source stack slot.
>>> - Source = DAG.getIntPtrConstant(VA.getLocMemOffset());
>>> - if (StackPtr.Val == 0)
>>> - StackPtr = DAG.getCopyFromReg(Chain, X86StackPtr,
>>> getPointerTy());
>>> - Source = DAG.getNode(ISD::ADD, getPointerTy(), StackPtr,
>>> Source);
>>> - if ((Flags & ISD::ParamFlags::ByVal)==0)
>>> - Source = DAG.getLoad(VA.getValVT(), Chain, Source,
>>> NULL, 0);
>>> - }
>>>
>>> + // Find virtual register for this argument.
>>> + bool Found=false;
>>> + for (unsigned idx=0, e= TailCallClobberedVRegs.size(); idx
>>> < e; idx++)
>>> + if (TailCallClobberedVRegs[idx].first==i) {
>>> + Arg = TailCallClobberedVRegs[idx].second;
>>> + Found=true;
>>> + break;
>>> + }
>>> + assert(IsPossiblyOverwrittenArgumentOfTailCall(Arg, MFI)
>>> ==false ||
>>> + (Found==true && "No corresponding Argument was found"));
>>> +
>>> if (Flags & ISD::ParamFlags::ByVal) {
>>> // Copy relative to framepointer.
>>> - MemOpChains2.push_back(CreateCopyOfByValArgument(Source,
>>> FIN, Chain,
>>> + MemOpChains2.push_back(CreateCopyOfByValArgument(Arg,
>>> FIN, Chain,
>>> Flags,
>>> DAG));
>>> } else {
>>> // Store relative to framepointer.
>>> MemOpChains2.push_back(
>>> - DAG.getStore(Chain, Source, FIN,
>>> + DAG.getStore(Chain, Arg, FIN,
>>> PseudoSourceValue::getFixedStack(), FI));
>>> }
>>> }
>>> @@ -1600,8 +1634,16 @@
>>> &MemOpChains2[0], MemOpChains2.size());
>>>
>>> // Store the return address to the appropriate stack slot.
>>> - if (FPDiff)
>>> - Chain = DAG.getStore(Chain,RetAddrFrIdx, NewRetAddrFrIdx,
>>> NULL, 0);
>>> + if (FPDiff) {
>>> + // Calculate the new stack slot for the return address.
>>> + int SlotSize = Is64Bit ? 8 : 4;
>>> + int NewReturnAddrFI =
>>> + MF.getFrameInfo()->CreateFixedObject(SlotSize, FPDiff-
>>> SlotSize);
>>> + MVT::ValueType VT = Is64Bit ? MVT::i64 : MVT::i32;
>>> + SDOperand NewRetAddrFrIdx = DAG.getFrameIndex
>>> (NewReturnAddrFI, VT);
>>> + Chain = DAG.getStore(Chain, RetAddrFrIdx, NewRetAddrFrIdx,
>>> + PseudoSourceValue::getFixedStack(),
>>> NewReturnAddrFI);
>>> + }
>>> }
>>>
>>> // If the callee is a GlobalAddress node (quite common, every
>>> direct call is)
>>>
>>> Modified: llvm/trunk/lib/Target/X86/X86ISelLowering.h
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/
>>> X86ISelLowering.h?rev=47593&r1=47592&r2=47593&view=diff
>>>
>>> ====================================================================
>>> =
>>> =
>>> ========
>>> --- llvm/trunk/lib/Target/X86/X86ISelLowering.h (original)
>>> +++ llvm/trunk/lib/Target/X86/X86ISelLowering.h Tue Feb 26 03:19:59
>>> 2008
>>> @@ -482,7 +482,7 @@
>>>
>>> SDOperand LowerMemArgument(SDOperand Op, SelectionDAG &DAG,
>>> const CCValAssign &VA,
>>> MachineFrameInfo *MFI,
>>> - SDOperand Root, unsigned i);
>>> + unsigned CC, SDOperand Root,
>>> unsigned i);
>>>
>>> SDOperand LowerMemOpCallTo(SDOperand Op, SelectionDAG &DAG,
>>> const SDOperand &StackPtr,
>>>
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
More information about the llvm-commits
mailing list