<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text --><style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<meta content="text/html; charset=UTF-8">
<style type="text/css" style="">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Arial,Helvetica,sans-serif">
<p>I think this is caused by the optimization in eliminateRedundantSpills. It decides to eliminate a redundant spill and replaces the store opcode with KILL. This doesn't need the implicit def of m0 on the spill pseudo anymore, but nothing removes it.</p>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Nicolai Hähnle <nhaehnle@gmail.com><br>
<b>Sent:</b> Tuesday, November 15, 2016 2:18:03 AM<br>
<b>To:</b> Arsenault, Matthew; llvm-commits@lists.llvm.org<br>
<b>Subject:</b> Re: [llvm] r286766 - AMDGPU: Implement SGPR spilling with scalar stores</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">Hi Matt,<br>
<br>
this change causes a regression. Compiling the attached shader with<br>
<br>
llc -march=amdgcn -mattr=+vgpr-spilling -mcpu=tonga<br>
<br>
triggers an assertion:<br>
<br>
llc: /home/nha/amd/llvm/llvm/lib/CodeGen/LiveRangeEdit.cpp:248: void <br>
llvm::LiveRangeEdit::eliminateDeadDef(llvm::MachineInstr*, <br>
llvm::LiveRangeEdit::ToShrinkSet&, llvm::AliasAnalysis*): Assertion <br>
`MI->allDefsAreDead() && "Def isn't really dead"' failed.<br>
<br>
Any idea why?<br>
<br>
Thanks,<br>
Nicolai<br>
<br>
On 13.11.2016 19:20, Matt Arsenault via llvm-commits wrote:<br>
> Author: arsenm<br>
> Date: Sun Nov 13 12:20:54 2016<br>
> New Revision: 286766<br>
><br>
> URL: <a href="http://llvm.org/viewvc/llvm-project?rev=286766&view=rev">http://llvm.org/viewvc/llvm-project?rev=286766&view=rev</a><br>
> Log:<br>
> AMDGPU: Implement SGPR spilling with scalar stores<br>
><br>
> nThis avoids the nasty problems caused by using<br>
> memory instructions that read the exec mask while<br>
> spilling / restoring registers used for control flow<br>
> masking, but only for VI when these were added.<br>
><br>
> This always uses the scalar stores when enabled currently,<br>
> but it may be better to still try to spill to a VGPR<br>
> and use this on the fallback memory path.<br>
><br>
> The cache also needs to be flushed before wave termination<br>
> if a scalar store is used.<br>
><br>
> Added:<br>
> llvm/trunk/test/CodeGen/MIR/AMDGPU/scalar-store-cache-flush.mir<br>
> Modified:<br>
> llvm/trunk/lib/Target/AMDGPU/SIInsertWaits.cpp<br>
> llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.cpp<br>
> llvm/trunk/lib/Target/AMDGPU/SIRegisterInfo.cpp<br>
> llvm/trunk/test/CodeGen/AMDGPU/attr-amdgpu-num-sgpr.ll<br>
> llvm/trunk/test/CodeGen/AMDGPU/basic-branch.ll<br>
> llvm/trunk/test/CodeGen/AMDGPU/si-spill-sgpr-stack.ll<br>
> llvm/trunk/test/CodeGen/AMDGPU/spill-m0.ll<br>
><br>
> Modified: llvm/trunk/lib/Target/AMDGPU/SIInsertWaits.cpp<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInsertWaits.cpp?rev=286766&r1=286765&r2=286766&view=diff">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInsertWaits.cpp?rev=286766&r1=286765&r2=286766&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/lib/Target/AMDGPU/SIInsertWaits.cpp (original)<br>
> +++ llvm/trunk/lib/Target/AMDGPU/SIInsertWaits.cpp Sun Nov 13 12:20:54 2016<br>
> @@ -532,6 +532,7 @@ bool SIInsertWaits::runOnMachineFunction<br>
> TRI = &TII->getRegisterInfo();<br>
> MRI = &MF.getRegInfo();<br>
> IV = getIsaVersion(ST->getFeatureBits());<br>
> + const SIMachineFunctionInfo *MFI = MF.getInfo<SIMachineFunctionInfo>();<br>
><br>
> HardwareLimits.Named.VM = getVmcntBitMask(IV);<br>
> HardwareLimits.Named.EXP = getExpcntBitMask(IV);<br>
> @@ -543,20 +544,27 @@ bool SIInsertWaits::runOnMachineFunction<br>
> LastOpcodeType = OTHER;<br>
> LastInstWritesM0 = false;<br>
> IsFlatOutstanding = false;<br>
> - ReturnsVoid = MF.getInfo<SIMachineFunctionInfo>()->returnsVoid();<br>
> + ReturnsVoid = MFI->returnsVoid();<br>
><br>
> memset(&UsedRegs, 0, sizeof(UsedRegs));<br>
> memset(&DefinedRegs, 0, sizeof(DefinedRegs));<br>
><br>
> SmallVector<MachineInstr *, 4> RemoveMI;<br>
> + SmallVector<MachineBasicBlock *, 4> EndPgmBlocks;<br>
> +<br>
> + bool HaveScalarStores = false;<br>
><br>
> for (MachineFunction::iterator BI = MF.begin(), BE = MF.end();<br>
> BI != BE; ++BI) {<br>
><br>
> MachineBasicBlock &MBB = *BI;<br>
> +<br>
> for (MachineBasicBlock::iterator I = MBB.begin(), E = MBB.end();<br>
> I != E; ++I) {<br>
><br>
> + if (!HaveScalarStores && TII->isScalarStore(*I))<br>
> + HaveScalarStores = true;<br>
> +<br>
> if (ST->getGeneration() <= SISubtarget::SEA_ISLANDS) {<br>
> // There is a hardware bug on CI/SI where SMRD instruction may corrupt<br>
> // vccz bit, so when we detect that an instruction may read from a<br>
> @@ -625,12 +633,45 @@ bool SIInsertWaits::runOnMachineFunction<br>
><br>
> pushInstruction(MBB, I, Increment);<br>
> handleSendMsg(MBB, I);<br>
> +<br>
> + if (I->getOpcode() == AMDGPU::S_ENDPGM ||<br>
> + I->getOpcode() == AMDGPU::SI_RETURN)<br>
> + EndPgmBlocks.push_back(&MBB);<br>
> }<br>
><br>
> // Wait for everything at the end of the MBB<br>
> Changes |= insertWait(MBB, MBB.getFirstTerminator(), LastIssued);<br>
> }<br>
><br>
> + if (HaveScalarStores) {<br>
> + // If scalar writes are used, the cache must be flushed or else the next<br>
> + // wave to reuse the same scratch memory can be clobbered.<br>
> + //<br>
> + // Insert s_dcache_wb at wave termination points if there were any scalar<br>
> + // stores, and only if the cache hasn't already been flushed. This could be<br>
> + // improved by looking across blocks for flushes in postdominating blocks<br>
> + // from the stores but an explicitly requested flush is probably very rare.<br>
> + for (MachineBasicBlock *MBB : EndPgmBlocks) {<br>
> + bool SeenDCacheWB = false;<br>
> +<br>
> + for (MachineBasicBlock::iterator I = MBB->begin(), E = MBB->end();<br>
> + I != E; ++I) {<br>
> +<br>
> + if (I->getOpcode() == AMDGPU::S_DCACHE_WB)<br>
> + SeenDCacheWB = true;<br>
> + else if (TII->isScalarStore(*I))<br>
> + SeenDCacheWB = false;<br>
> +<br>
> + // FIXME: It would be better to insert this before a waitcnt if any.<br>
> + if ((I->getOpcode() == AMDGPU::S_ENDPGM ||<br>
> + I->getOpcode() == AMDGPU::SI_RETURN) && !SeenDCacheWB) {<br>
> + Changes = true;<br>
> + BuildMI(*MBB, I, I->getDebugLoc(), TII->get(AMDGPU::S_DCACHE_WB));<br>
> + }<br>
> + }<br>
> + }<br>
> + }<br>
> +<br>
> for (MachineInstr *I : RemoveMI)<br>
> I->eraseFromParent();<br>
><br>
><br>
> Modified: llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.cpp<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.cpp?rev=286766&r1=286765&r2=286766&view=diff">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.cpp?rev=286766&r1=286765&r2=286766&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.cpp (original)<br>
> +++ llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.cpp Sun Nov 13 12:20:54 2016<br>
> @@ -539,7 +539,7 @@ void SIInstrInfo::storeRegToStackSlot(Ma<br>
> MRI.constrainRegClass(SrcReg, &AMDGPU::SReg_32_XM0RegClass);<br>
> }<br>
><br>
> - BuildMI(MBB, MI, DL, OpDesc)<br>
> + MachineInstrBuilder Spill = BuildMI(MBB, MI, DL, OpDesc)<br>
> .addReg(SrcReg, getKillRegState(isKill)) // data<br>
> .addFrameIndex(FrameIndex) // addr<br>
> .addMemOperand(MMO)<br>
> @@ -549,6 +549,11 @@ void SIInstrInfo::storeRegToStackSlot(Ma<br>
> // needing them, and need to ensure that the reserved registers are<br>
> // correctly handled.<br>
><br>
> + if (ST.hasScalarStores()) {<br>
> + // m0 is used for offset to scalar stores if used to spill.<br>
> + Spill.addReg(AMDGPU::M0, RegState::ImplicitDefine);<br>
> + }<br>
> +<br>
> return;<br>
> }<br>
><br>
> @@ -638,12 +643,17 @@ void SIInstrInfo::loadRegFromStackSlot(M<br>
> MRI.constrainRegClass(DestReg, &AMDGPU::SReg_32_XM0RegClass);<br>
> }<br>
><br>
> - BuildMI(MBB, MI, DL, OpDesc, DestReg)<br>
> + MachineInstrBuilder Spill = BuildMI(MBB, MI, DL, OpDesc, DestReg)<br>
> .addFrameIndex(FrameIndex) // addr<br>
> .addMemOperand(MMO)<br>
> .addReg(MFI->getScratchRSrcReg(), RegState::Implicit)<br>
> .addReg(MFI->getScratchWaveOffsetReg(), RegState::Implicit);<br>
><br>
> + if (ST.hasScalarStores()) {<br>
> + // m0 is used for offset to scalar stores if used to spill.<br>
> + Spill.addReg(AMDGPU::M0, RegState::ImplicitDefine);<br>
> + }<br>
> +<br>
> return;<br>
> }<br>
><br>
><br>
> Modified: llvm/trunk/lib/Target/AMDGPU/SIRegisterInfo.cpp<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIRegisterInfo.cpp?rev=286766&r1=286765&r2=286766&view=diff">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIRegisterInfo.cpp?rev=286766&r1=286765&r2=286766&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/lib/Target/AMDGPU/SIRegisterInfo.cpp (original)<br>
> +++ llvm/trunk/lib/Target/AMDGPU/SIRegisterInfo.cpp Sun Nov 13 12:20:54 2016<br>
> @@ -24,6 +24,12 @@<br>
><br>
> using namespace llvm;<br>
><br>
> +static cl::opt<bool> EnableSpillSGPRToSMEM(<br>
> + "amdgpu-spill-sgpr-to-smem",<br>
> + cl::desc("Use scalar stores to spill SGPRs if supported by subtarget"),<br>
> + cl::init(true));<br>
> +<br>
> +<br>
> static bool hasPressureSet(const int *PSets, unsigned PSetID) {<br>
> for (unsigned i = 0; PSets[i] != -1; ++i) {<br>
> if (PSets[i] == (int)PSetID)<br>
> @@ -475,18 +481,21 @@ void SIRegisterInfo::buildSpillLoadStore<br>
> void SIRegisterInfo::spillSGPR(MachineBasicBlock::iterator MI,<br>
> int Index,<br>
> RegScavenger *RS) const {<br>
> - MachineFunction *MF = MI->getParent()->getParent();<br>
> - MachineRegisterInfo &MRI = MF->getRegInfo();<br>
> MachineBasicBlock *MBB = MI->getParent();<br>
> - SIMachineFunctionInfo *MFI = MF->getInfo<SIMachineFunctionInfo>();<br>
> - MachineFrameInfo &FrameInfo = MF->getFrameInfo();<br>
> + MachineFunction *MF = MBB->getParent();<br>
> + MachineRegisterInfo &MRI = MF->getRegInfo();<br>
> const SISubtarget &ST = MF->getSubtarget<SISubtarget>();<br>
> const SIInstrInfo *TII = ST.getInstrInfo();<br>
> - const DebugLoc &DL = MI->getDebugLoc();<br>
><br>
> unsigned NumSubRegs = getNumSubRegsForSpillOp(MI->getOpcode());<br>
> unsigned SuperReg = MI->getOperand(0).getReg();<br>
> bool IsKill = MI->getOperand(0).isKill();<br>
> + const DebugLoc &DL = MI->getDebugLoc();<br>
> +<br>
> + SIMachineFunctionInfo *MFI = MF->getInfo<SIMachineFunctionInfo>();<br>
> + MachineFrameInfo &FrameInfo = MF->getFrameInfo();<br>
> +<br>
> + bool SpillToSMEM = ST.hasScalarStores() && EnableSpillSGPRToSMEM;<br>
><br>
> // SubReg carries the "Kill" flag when SubReg == SuperReg.<br>
> unsigned SubKillState = getKillRegState((NumSubRegs == 1) && IsKill);<br>
> @@ -494,6 +503,55 @@ void SIRegisterInfo::spillSGPR(MachineBa<br>
> unsigned SubReg = NumSubRegs == 1 ?<br>
> SuperReg : getSubReg(SuperReg, getSubRegFromChannel(i));<br>
><br>
> + if (SpillToSMEM) {<br>
> + if (SuperReg == AMDGPU::M0) {<br>
> + assert(NumSubRegs == 1);<br>
> + unsigned CopyM0<br>
> + = MRI.createVirtualRegister(&AMDGPU::SReg_32_XM0RegClass);<br>
> +<br>
> + BuildMI(*MBB, MI, DL, TII->get(AMDGPU::COPY), CopyM0)<br>
> + .addReg(AMDGPU::M0, getKillRegState(IsKill));<br>
> +<br>
> + // The real spill now kills the temp copy.<br>
> + SubReg = SuperReg = CopyM0;<br>
> + IsKill = true;<br>
> + }<br>
> +<br>
> + int64_t FrOffset = FrameInfo.getObjectOffset(Index);<br>
> + unsigned Size = FrameInfo.getObjectSize(Index);<br>
> + unsigned Align = FrameInfo.getObjectAlignment(Index);<br>
> + MachinePointerInfo PtrInfo<br>
> + = MachinePointerInfo::getFixedStack(*MF, Index);<br>
> + MachineMemOperand *MMO<br>
> + = MF->getMachineMemOperand(PtrInfo, MachineMemOperand::MOStore,<br>
> + Size, Align);<br>
> +<br>
> + unsigned OffsetReg = AMDGPU::M0;<br>
> + // Add i * 4 wave offset.<br>
> + //<br>
> + // SMEM instructions only support a single offset, so increment the wave<br>
> + // offset.<br>
> +<br>
> + int64_t Offset = ST.getWavefrontSize() * (FrOffset + 4 * i);<br>
> + if (Offset != 0) {<br>
> + BuildMI(*MBB, MI, DL, TII->get(AMDGPU::S_ADD_U32), OffsetReg)<br>
> + .addReg(MFI->getScratchWaveOffsetReg())<br>
> + .addImm(Offset);<br>
> + } else {<br>
> + BuildMI(*MBB, MI, DL, TII->get(AMDGPU::S_MOV_B32), OffsetReg)<br>
> + .addReg(MFI->getScratchWaveOffsetReg());<br>
> + }<br>
> +<br>
> + BuildMI(*MBB, MI, DL, TII->get(AMDGPU::S_BUFFER_STORE_DWORD_SGPR))<br>
> + .addReg(SubReg, getKillRegState(IsKill)) // sdata<br>
> + .addReg(MFI->getScratchRSrcReg()) // sbase<br>
> + .addReg(OffsetReg) // soff<br>
> + .addImm(0) // glc<br>
> + .addMemOperand(MMO);<br>
> +<br>
> + continue;<br>
> + }<br>
> +<br>
> struct SIMachineFunctionInfo::SpilledReg Spill =<br>
> MFI->getSpilledReg(MF, Index, i);<br>
> if (Spill.hasReg()) {<br>
> @@ -520,10 +578,9 @@ void SIRegisterInfo::spillSGPR(MachineBa<br>
> // it are fixed.<br>
> } else {<br>
> // Spill SGPR to a frame index.<br>
> - // FIXME we should use S_STORE_DWORD here for VI.<br>
> -<br>
> // TODO: Should VI try to spill to VGPR and then spill to SMEM?<br>
> unsigned TmpReg = MRI.createVirtualRegister(&AMDGPU::VGPR_32RegClass);<br>
> + // TODO: Should VI try to spill to VGPR and then spill to SMEM?<br>
><br>
> MachineInstrBuilder Mov<br>
> = BuildMI(*MBB, MI, DL, TII->get(AMDGPU::V_MOV_B32_e32), TmpReg)<br>
> @@ -575,6 +632,7 @@ void SIRegisterInfo::restoreSGPR(Machine<br>
><br>
> unsigned NumSubRegs = getNumSubRegsForSpillOp(MI->getOpcode());<br>
> unsigned SuperReg = MI->getOperand(0).getReg();<br>
> + bool SpillToSMEM = ST.hasScalarStores() && EnableSpillSGPRToSMEM;<br>
><br>
> // m0 is not allowed as with readlane/writelane, so a temporary SGPR and<br>
> // extra copy is needed.<br>
> @@ -584,10 +642,44 @@ void SIRegisterInfo::restoreSGPR(Machine<br>
> SuperReg = MRI.createVirtualRegister(&AMDGPU::SReg_32_XM0RegClass);<br>
> }<br>
><br>
> + int64_t FrOffset = FrameInfo.getObjectOffset(Index);<br>
> +<br>
> for (unsigned i = 0, e = NumSubRegs; i < e; ++i) {<br>
> unsigned SubReg = NumSubRegs == 1 ?<br>
> SuperReg : getSubReg(SuperReg, getSubRegFromChannel(i));<br>
><br>
> + if (SpillToSMEM) {<br>
> + unsigned Size = FrameInfo.getObjectSize(Index);<br>
> + unsigned Align = FrameInfo.getObjectAlignment(Index);<br>
> + MachinePointerInfo PtrInfo<br>
> + = MachinePointerInfo::getFixedStack(*MF, Index);<br>
> + MachineMemOperand *MMO<br>
> + = MF->getMachineMemOperand(PtrInfo, MachineMemOperand::MOLoad,<br>
> + Size, Align);<br>
> +<br>
> + unsigned OffsetReg = AMDGPU::M0;<br>
> +<br>
> + // Add i * 4 offset<br>
> + int64_t Offset = ST.getWavefrontSize() * (FrOffset + 4 * i);<br>
> + if (Offset != 0) {<br>
> + BuildMI(*MBB, MI, DL, TII->get(AMDGPU::S_ADD_U32), OffsetReg)<br>
> + .addReg(MFI->getScratchWaveOffsetReg())<br>
> + .addImm(Offset);<br>
> + } else {<br>
> + BuildMI(*MBB, MI, DL, TII->get(AMDGPU::S_MOV_B32), OffsetReg)<br>
> + .addReg(MFI->getScratchWaveOffsetReg());<br>
> + }<br>
> +<br>
> + BuildMI(*MBB, MI, DL, TII->get(AMDGPU::S_BUFFER_LOAD_DWORD_SGPR), SubReg)<br>
> + .addReg(MFI->getScratchRSrcReg()) // sbase<br>
> + .addReg(OffsetReg) // soff<br>
> + .addImm(0) // glc<br>
> + .addMemOperand(MMO)<br>
> + .addReg(MI->getOperand(0).getReg(), RegState::ImplicitDefine);<br>
> +<br>
> + continue;<br>
> + }<br>
> +<br>
> SIMachineFunctionInfo::SpilledReg Spill<br>
> = MFI->getSpilledReg(MF, Index, i);<br>
><br>
><br>
> Modified: llvm/trunk/test/CodeGen/AMDGPU/attr-amdgpu-num-sgpr.ll<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/attr-amdgpu-num-sgpr.ll?rev=286766&r1=286765&r2=286766&view=diff">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/attr-amdgpu-num-sgpr.ll?rev=286766&r1=286765&r2=286766&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/test/CodeGen/AMDGPU/attr-amdgpu-num-sgpr.ll (original)<br>
> +++ llvm/trunk/test/CodeGen/AMDGPU/attr-amdgpu-num-sgpr.ll Sun Nov 13 12:20:54 2016<br>
> @@ -1,16 +1,20 @@<br>
> -; RUN: llc -mtriple=amdgcn--amdhsa -mcpu=fiji -verify-machineinstrs < %s | FileCheck %s<br>
> +; RUN: llc -mtriple=amdgcn--amdhsa -mcpu=fiji -amdgpu-spill-sgpr-to-smem=0 -verify-machineinstrs < %s | FileCheck -check-prefix=TOSGPR -check-prefix=ALL %s<br>
> +; RUN: llc -mtriple=amdgcn--amdhsa -mcpu=fiji -amdgpu-spill-sgpr-to-smem=1 -verify-machineinstrs < %s | FileCheck -check-prefix=TOSMEM -check-prefix=ALL %s<br>
><br>
> -; CHECK-LABEL: {{^}}max_14_sgprs:<br>
> +; If spilling to smem, additional registers are used for the resource<br>
> +; descriptor.<br>
> +<br>
> +; ALL-LABEL: {{^}}max_14_sgprs:<br>
><br>
> ; FIXME: Should be ablo to skip this copying of the private segment<br>
> ; buffer because all the SGPR spills are to VGPRs.<br>
><br>
> -; CHECK: s_mov_b64 s[6:7], s[2:3]<br>
> -; CHECK: s_mov_b64 s[4:5], s[0:1]<br>
> -<br>
> -; CHECK: SGPRBlocks: 1<br>
> -; CHECK: NumSGPRsForWavesPerEU: 14<br>
> +; ALL: s_mov_b64 s[6:7], s[2:3]<br>
> +; ALL: s_mov_b64 s[4:5], s[0:1]<br>
> +; ALL: SGPRBlocks: 1<br>
> +; ALL: NumSGPRsForWavesPerEU: 14<br>
> define void @max_14_sgprs(i32 addrspace(1)* %out1,<br>
> +<br>
> i32 addrspace(1)* %out2,<br>
> i32 addrspace(1)* %out3,<br>
> i32 addrspace(1)* %out4,<br>
> @@ -31,7 +35,7 @@ define void @max_14_sgprs(i32 addrspace(<br>
> ; ---------------------<br>
> ; total: 14<br>
><br>
> -; + reserved vcc, flat_scratch = 18<br>
> +; + reserved vcc, xnack, flat_scratch = 20<br>
><br>
> ; Because we can't handle re-using the last few input registers as the<br>
> ; special vcc etc. registers (as well as decide to not use the unused<br>
> @@ -40,14 +44,14 @@ define void @max_14_sgprs(i32 addrspace(<br>
><br>
> ; ALL-LABEL: {{^}}max_12_sgprs_14_input_sgprs:<br>
> ; TOSGPR: SGPRBlocks: 2<br>
> -; TOSGPR: NumSGPRsForWavesPerEU: 18<br>
> +; TOSGPR: NumSGPRsForWavesPerEU: 20<br>
><br>
> ; TOSMEM: s_mov_b64 s[6:7], s[2:3]<br>
> -; TOSMEM: s_mov_b32 s9, s13<br>
> ; TOSMEM: s_mov_b64 s[4:5], s[0:1]<br>
> +; TOSMEM: s_mov_b32 s3, s13<br>
><br>
> ; TOSMEM: SGPRBlocks: 2<br>
> -; TOSMEM: NumSGPRsForWavesPerEU: 18<br>
> +; TOSMEM: NumSGPRsForWavesPerEU: 20<br>
> define void @max_12_sgprs_14_input_sgprs(i32 addrspace(1)* %out1,<br>
> i32 addrspace(1)* %out2,<br>
> i32 addrspace(1)* %out3,<br>
> @@ -79,12 +83,12 @@ define void @max_12_sgprs_14_input_sgprs<br>
> ; ; swapping the order the registers are copied from what normally<br>
> ; ; happens.<br>
><br>
> -; TOSMEM: s_mov_b64 s[6:7], s[2:3]<br>
> -; TOSMEM: s_mov_b64 s[4:5], s[0:1]<br>
> -; TOSMEM: s_mov_b32 s3, s11<br>
> +; TOSMEM: s_mov_b32 s5, s11<br>
> +; TOSMEM: s_add_u32 m0, s5,<br>
> +; TOSMEM: s_buffer_store_dword vcc_lo, s[0:3], m0<br>
><br>
> -; ALL: SGPRBlocks: 1<br>
> -; ALL: NumSGPRsForWavesPerEU: 16<br>
> +; ALL: SGPRBlocks: 2<br>
> +; ALL: NumSGPRsForWavesPerEU: 18<br>
> define void @max_12_sgprs_12_input_sgprs(i32 addrspace(1)* %out1,<br>
> i32 addrspace(1)* %out2,<br>
> i32 addrspace(1)* %out3,<br>
><br>
> Modified: llvm/trunk/test/CodeGen/AMDGPU/basic-branch.ll<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/basic-branch.ll?rev=286766&r1=286765&r2=286766&view=diff">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/basic-branch.ll?rev=286766&r1=286765&r2=286766&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/test/CodeGen/AMDGPU/basic-branch.ll (original)<br>
> +++ llvm/trunk/test/CodeGen/AMDGPU/basic-branch.ll Sun Nov 13 12:20:54 2016<br>
> @@ -1,5 +1,5 @@<br>
> ; RUN: llc -O0 -march=amdgcn -verify-machineinstrs < %s | FileCheck -check-prefix=GCNNOOPT -check-prefix=GCN %s<br>
> -; RUN: llc -O0 -march=amdgcn -mcpu=tonga -verify-machineinstrs < %s | FileCheck -check-prefix=GCNNOOPT -check-prefix=GCN %s<br>
> +; RUN: llc -O0 -march=amdgcn -mcpu=tonga -amdgpu-spill-sgpr-to-smem=0 -verify-machineinstrs < %s | FileCheck -check-prefix=GCNNOOPT -check-prefix=GCN %s<br>
> ; RUN: llc -march=amdgcn -verify-machineinstrs < %s | FileCheck -check-prefix=GCNOPT -check-prefix=GCN %s<br>
> ; RUN: llc -march=amdgcn -mcpu=tonga -verify-machineinstrs < %s | FileCheck -check-prefix=GCNOPT -check-prefix=GCN %s<br>
><br>
><br>
> Modified: llvm/trunk/test/CodeGen/AMDGPU/si-spill-sgpr-stack.ll<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/si-spill-sgpr-stack.ll?rev=286766&r1=286765&r2=286766&view=diff">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/si-spill-sgpr-stack.ll?rev=286766&r1=286765&r2=286766&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/test/CodeGen/AMDGPU/si-spill-sgpr-stack.ll (original)<br>
> +++ llvm/trunk/test/CodeGen/AMDGPU/si-spill-sgpr-stack.ll Sun Nov 13 12:20:54 2016<br>
> @@ -1,14 +1,44 @@<br>
> -; RUN: llc -march=amdgcn -mcpu=fiji -verify-machineinstrs < %s | FileCheck %s<br>
> +; RUN: llc -march=amdgcn -mcpu=fiji -amdgpu-spill-sgpr-to-smem=0 -verify-machineinstrs < %s | FileCheck -check-prefix=ALL -check-prefix=SGPR %s<br>
> +; RUN: llc -march=amdgcn -mcpu=fiji -amdgpu-spill-sgpr-to-smem=1 -verify-machineinstrs < %s | FileCheck -check-prefix=ALL -check-prefix=SMEM %s<br>
><br>
> ; Make sure this doesn't crash.<br>
> -; CHECK: {{^}}test:<br>
> +; ALL-LABEL: {{^}}test:<br>
> +; ALL: s_mov_b32 s92, SCRATCH_RSRC_DWORD0<br>
> +; ALL: s_mov_b32 s91, s3<br>
> +<br>
> ; Make sure we are handling hazards correctly.<br>
> -; CHECK: buffer_load_dword [[VHI:v[0-9]+]], off, s[{{[0-9]+:[0-9]+}}], s{{[0-9]+}} offset:12<br>
> -; CHECK-NEXT: s_waitcnt vmcnt(0)<br>
> -; CHECK-NEXT: v_readfirstlane_b32 s[[HI:[0-9]+]], [[VHI]]<br>
> -; CHECK-NEXT: s_nop 4<br>
> -; CHECK-NEXT: buffer_store_dword v0, off, s[0:[[HI]]{{\]}}, 0<br>
> -; CHECK: s_endpgm<br>
> +; SGPR: buffer_load_dword [[VHI:v[0-9]+]], off, s[{{[0-9]+:[0-9]+}}], s{{[0-9]+}} offset:12<br>
> +; SGPR-NEXT: s_waitcnt vmcnt(0)<br>
> +; SGPR-NEXT: v_readfirstlane_b32 s[[HI:[0-9]+]], [[VHI]]<br>
> +; SGPR-NEXT: s_nop 4<br>
> +; SGPR-NEXT: buffer_store_dword v0, off, s[0:[[HI]]{{\]}}, 0<br>
> +<br>
> +<br>
> +; Make sure scratch wave offset register is correctly incremented and<br>
> +; then restored.<br>
> +; SMEM: s_mov_b32 m0, s91{{$}}<br>
> +; SMEM: s_buffer_store_dword s{{[0-9]+}}, s[92:95], m0 ; 16-byte Folded Spill<br>
> +; SMEM: s_add_u32 m0, s91, 0x100{{$}}<br>
> +; SMEM: s_buffer_store_dword s{{[0-9]+}}, s[92:95], m0 ; 16-byte Folded Spill<br>
> +; SMEM: s_add_u32 m0, s91, 0x200{{$}}<br>
> +; SMEM: s_buffer_store_dword s{{[0-9]+}}, s[92:95], m0 ; 16-byte Folded Spill<br>
> +; SMEM: s_add_u32 m0, s91, 0x300{{$}}<br>
> +; SMEM: s_buffer_store_dword s{{[0-9]+}}, s[92:95], m0 ; 16-byte Folded Spill<br>
> +<br>
> +<br>
> +; SMEM: s_mov_b32 m0, s91{{$}}<br>
> +; SMEM: s_buffer_load_dword s{{[0-9]+}}, s[92:95], m0 ; 16-byte Folded Reload<br>
> +; SMEM: s_add_u32 m0, s91, 0x100{{$}}<br>
> +; SMEM: s_waitcnt lgkmcnt(0)<br>
> +; SMEM: s_buffer_load_dword s{{[0-9]+}}, s[92:95], m0 ; 16-byte Folded Reload<br>
> +; SMEM: s_add_u32 m0, s91, 0x200{{$}}<br>
> +; SMEM: s_waitcnt lgkmcnt(0)<br>
> +; SMEM: s_buffer_load_dword s{{[0-9]+}}, s[92:95], m0 ; 16-byte Folded Reload<br>
> +; SMEM: s_add_u32 m0, s91, 0x300{{$}}<br>
> +; SMEM: s_waitcnt lgkmcnt(0)<br>
> +; SMEM: s_buffer_load_dword s{{[0-9]+}}, s[92:95], m0 ; 16-byte Folded Reload<br>
> +<br>
> +; ALL: s_endpgm<br>
> define void @test(i32 addrspace(1)* %out, i32 %in) {<br>
> call void asm sideeffect "", "~{SGPR0_SGPR1_SGPR2_SGPR3_SGPR4_SGPR5_SGPR6_SGPR7}" ()<br>
> call void asm sideeffect "", "~{SGPR8_SGPR9_SGPR10_SGPR11_SGPR12_SGPR13_SGPR14_SGPR15}" ()<br>
><br>
> Modified: llvm/trunk/test/CodeGen/AMDGPU/spill-m0.ll<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/spill-m0.ll?rev=286766&r1=286765&r2=286766&view=diff">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/spill-m0.ll?rev=286766&r1=286765&r2=286766&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/test/CodeGen/AMDGPU/spill-m0.ll (original)<br>
> +++ llvm/trunk/test/CodeGen/AMDGPU/spill-m0.ll Sun Nov 13 12:20:54 2016<br>
> @@ -1,12 +1,13 @@<br>
> ; RUN: llc -O0 -amdgpu-spill-sgpr-to-vgpr=1 -march=amdgcn -mattr=+vgpr-spilling -verify-machineinstrs < %s | FileCheck -check-prefix=TOVGPR -check-prefix=GCN %s<br>
> -; RUN: llc -O0 -amdgpu-spill-sgpr-to-vgpr=1 -march=amdgcn -mcpu=tonga -mattr=+vgpr-spilling -verify-machineinstrs < %s | FileCheck -check-prefix=TOVGPR -check-prefix=GCN %s<br>
> +; RUN: llc -O0 -amdgpu-spill-sgpr-to-vgpr=1 -amdgpu-spill-sgpr-to-smem=0 -march=amdgcn -mcpu=tonga -mattr=+vgpr-spilling -verify-machineinstrs < %s | FileCheck -check-prefix=TOVGPR -check-prefix=GCN %s<br>
> ; RUN: llc -O0 -amdgpu-spill-sgpr-to-vgpr=0 -march=amdgcn -mattr=+vgpr-spilling -verify-machineinstrs < %s | FileCheck -check-prefix=TOVMEM -check-prefix=GCN %s<br>
> -; RUN: llc -O0 -amdgpu-spill-sgpr-to-vgpr=0 -march=amdgcn -mattr=+vgpr-spilling -mcpu=tonga -verify-machineinstrs < %s | FileCheck -check-prefix=TOVMEM -check-prefix=GCN %s<br>
> +; RUN: llc -O0 -amdgpu-spill-sgpr-to-vgpr=0 -amdgpu-spill-sgpr-to-smem=0 -march=amdgcn -mcpu=tonga -mattr=+vgpr-spilling -verify-machineinstrs < %s | FileCheck -check-prefix=TOVMEM -check-prefix=GCN %s<br>
> +; RUN: llc -O0 -amdgpu-spill-sgpr-to-vgpr=0 -amdgpu-spill-sgpr-to-smem=1 -march=amdgcn -mcpu=tonga -mattr=+vgpr-spilling -verify-machineinstrs < %s | FileCheck -check-prefix=TOSMEM -check-prefix=GCN %s<br>
><br>
> ; XXX - Why does it like to use vcc?<br>
><br>
> ; GCN-LABEL: {{^}}spill_m0:<br>
> -; TOSMEM: s_mov_b32 s88, SCRATCH_RSRC_DWORD0<br>
> +; TOSMEM: s_mov_b32 s84, SCRATCH_RSRC_DWORD0<br>
><br>
> ; GCN: s_cmp_lg_u32<br>
><br>
> @@ -16,6 +17,13 @@<br>
> ; TOVMEM: v_mov_b32_e32 [[SPILL_VREG:v[0-9]+]], m0<br>
> ; TOVMEM: buffer_store_dword [[SPILL_VREG]], off, s{{\[[0-9]+:[0-9]+\]}}, s{{[0-9]+}} ; 4-byte Folded Spill<br>
> ; TOVMEM: s_waitcnt vmcnt(0)<br>
> +<br>
> +; TOSMEM: s_mov_b32 vcc_hi, m0<br>
> +; TOSMEM: s_mov_b32 m0, s3{{$}}<br>
> +; TOSMEM-NOT: vcc_hi<br>
> +; TOSMEM: s_buffer_store_dword vcc_hi, s[84:87], m0 ; 4-byte Folded Spill<br>
> +; TOSMEM: s_waitcnt lgkmcnt(0)<br>
> +<br>
> ; GCN: s_cbranch_scc1 [[ENDIF:BB[0-9]+_[0-9]+]]<br>
><br>
> ; GCN: [[ENDIF]]:<br>
> @@ -27,6 +35,11 @@<br>
> ; TOVMEM: v_readfirstlane_b32 vcc_hi, [[RELOAD_VREG]]<br>
> ; TOVMEM: s_mov_b32 m0, vcc_hi<br>
><br>
> +; TOSMEM: s_mov_b32 m0, s3{{$}}<br>
> +; TOSMEM: s_buffer_load_dword vcc_hi, s[84:87], m0 ; 4-byte Folded Reload<br>
> +; TOSMEM-NOT: vcc_hi<br>
> +; TOSMEM: s_mov_b32 m0, vcc_hi<br>
> +<br>
> ; GCN: s_add_i32 m0, m0, 1<br>
> define void @spill_m0(i32 %cond, i32 addrspace(1)* %out) #0 {<br>
> entry:<br>
> @@ -48,6 +61,8 @@ endif:<br>
><br>
> ; GCN-LABEL: {{^}}spill_m0_lds:<br>
> ; GCN-NOT: v_readlane_b32 m0<br>
> +; GCN-NOT: s_buffer_store_dword m0<br>
> +; GCN-NOT: s_buffer_load_dword m0<br>
> define amdgpu_ps void @spill_m0_lds(<16 x i8> addrspace(2)* inreg, <16 x i8> addrspace(2)* inreg, <32 x i8> addrspace(2)* inreg, i32 inreg) #0 {<br>
> main_body:<br>
> %4 = call float @llvm.SI.fs.constant(i32 0, i32 0, i32 %3)<br>
><br>
> Added: llvm/trunk/test/CodeGen/MIR/AMDGPU/scalar-store-cache-flush.mir<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/MIR/AMDGPU/scalar-store-cache-flush.mir?rev=286766&view=auto">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/MIR/AMDGPU/scalar-store-cache-flush.mir?rev=286766&view=auto</a><br>
> ==============================================================================<br>
> --- llvm/trunk/test/CodeGen/MIR/AMDGPU/scalar-store-cache-flush.mir (added)<br>
> +++ llvm/trunk/test/CodeGen/MIR/AMDGPU/scalar-store-cache-flush.mir Sun Nov 13 12:20:54 2016<br>
> @@ -0,0 +1,173 @@<br>
> +# RUN: llc -march=amdgcn -run-pass si-insert-waits %s -o - | FileCheck %s<br>
> +<br>
> +--- |<br>
> + define void @basic_insert_dcache_wb() {<br>
> + ret void<br>
> + }<br>
> +<br>
> + define void @explicit_flush_after() {<br>
> + ret void<br>
> + }<br>
> +<br>
> + define void @explicit_flush_before() {<br>
> + ret void<br>
> + }<br>
> +<br>
> + define void @no_scalar_store() {<br>
> + ret void<br>
> + }<br>
> +<br>
> + define void @multi_block_store() {<br>
> + bb0:<br>
> + br i1 undef, label %bb1, label %bb2<br>
> +<br>
> + bb1:<br>
> + ret void<br>
> +<br>
> + bb2:<br>
> + ret void<br>
> + }<br>
> +<br>
> + define void @one_block_store() {<br>
> + bb0:<br>
> + br i1 undef, label %bb1, label %bb2<br>
> +<br>
> + bb1:<br>
> + ret void<br>
> +<br>
> + bb2:<br>
> + ret void<br>
> + }<br>
> +<br>
> + define amdgpu_ps float @si_return() {<br>
> + ret float undef<br>
> + }<br>
> +<br>
> +...<br>
> +---<br>
> +# CHECK-LABEL: name: basic_insert_dcache_wb<br>
> +# CHECK: bb.0:<br>
> +# CHECK-NEXT: S_STORE_DWORD<br>
> +# CHECK-NEXT: S_DCACHE_WB<br>
> +# CHECK-NEXT: S_ENDPGM<br>
> +<br>
> +name: basic_insert_dcache_wb<br>
> +tracksRegLiveness: false<br>
> +<br>
> +body: |<br>
> + bb.0:<br>
> + S_STORE_DWORD_SGPR undef %sgpr2, undef %sgpr0_sgpr1, undef %m0, 0<br>
> + S_ENDPGM<br>
> +...<br>
> +---<br>
> +# Already has an explicitly requested flush after the last store.<br>
> +# CHECK-LABEL: name: explicit_flush_after<br>
> +# CHECK: bb.0:<br>
> +# CHECK-NEXT: S_STORE_DWORD<br>
> +# CHECK-NEXT: S_DCACHE_WB<br>
> +# CHECK-NEXT: S_ENDPGM<br>
> +<br>
> +name: explicit_flush_after<br>
> +tracksRegLiveness: false<br>
> +<br>
> +body: |<br>
> + bb.0:<br>
> + S_STORE_DWORD_SGPR undef %sgpr2, undef %sgpr0_sgpr1, undef %m0, 0<br>
> + S_DCACHE_WB<br>
> + S_ENDPGM<br>
> +...<br>
> +---<br>
> +# Already has an explicitly requested flush before the last store.<br>
> +# CHECK-LABEL: name: explicit_flush_before<br>
> +# CHECK: bb.0:<br>
> +# CHECK-NEXT: S_DCACHE_WB<br>
> +# CHECK-NEXT: S_STORE_DWORD<br>
> +# CHECK-NEXT: S_DCACHE_WB<br>
> +# CHECK-NEXT: S_ENDPGM<br>
> +<br>
> +name: explicit_flush_before<br>
> +tracksRegLiveness: false<br>
> +<br>
> +body: |<br>
> + bb.0:<br>
> + S_DCACHE_WB<br>
> + S_STORE_DWORD_SGPR undef %sgpr2, undef %sgpr0_sgpr1, undef %m0, 0<br>
> + S_ENDPGM<br>
> +...<br>
> +---<br>
> +# CHECK-LABEL: no_scalar_store<br>
> +# CHECK: bb.0<br>
> +# CHECK-NEXT: S_ENDPGM<br>
> +name: no_scalar_store<br>
> +tracksRegLiveness: false<br>
> +<br>
> +body: |<br>
> + bb.0:<br>
> + S_ENDPGM<br>
> +...<br>
> +<br>
> +# CHECK-LABEL: name: multi_block_store<br>
> +# CHECK: bb.0:<br>
> +# CHECK-NEXT: S_STORE_DWORD<br>
> +# CHECK-NEXT: S_DCACHE_WB<br>
> +# CHECK-NEXT: S_ENDPGM<br>
> +<br>
> +# CHECK: bb.1:<br>
> +# CHECK-NEXT: S_STORE_DWORD<br>
> +# CHECK-NEXT: S_DCACHE_WB<br>
> +# CHECK-NEXT: S_ENDPGM<br>
> +<br>
> +name: multi_block_store<br>
> +tracksRegLiveness: false<br>
> +<br>
> +body: |<br>
> + bb.0:<br>
> + S_STORE_DWORD_SGPR undef %sgpr2, undef %sgpr0_sgpr1, undef %m0, 0<br>
> + S_ENDPGM<br>
> +<br>
> + bb.1:<br>
> + S_STORE_DWORD_SGPR undef %sgpr4, undef %sgpr6_sgpr7, undef %m0, 0<br>
> + S_ENDPGM<br>
> +...<br>
> +...<br>
> +<br>
> +# This one should be able to omit the flush in the storeless block but<br>
> +# this isn't handled now.<br>
> +<br>
> +# CHECK-LABEL: name: one_block_store<br>
> +# CHECK: bb.0:<br>
> +# CHECK-NEXT: S_DCACHE_WB<br>
> +# CHECK-NEXT: S_ENDPGM<br>
> +<br>
> +# CHECK: bb.1:<br>
> +# CHECK-NEXT: S_STORE_DWORD<br>
> +# CHECK-NEXT: S_DCACHE_WB<br>
> +# CHECK-NEXT: S_ENDPGM<br>
> +<br>
> +name: one_block_store<br>
> +tracksRegLiveness: false<br>
> +<br>
> +body: |<br>
> + bb.0:<br>
> + S_ENDPGM<br>
> +<br>
> + bb.1:<br>
> + S_STORE_DWORD_SGPR undef %sgpr4, undef %sgpr6_sgpr7, undef %m0, 0<br>
> + S_ENDPGM<br>
> +...<br>
> +---<br>
> +# CHECK-LABEL: name: si_return<br>
> +# CHECK: bb.0:<br>
> +# CHECK-NEXT: S_STORE_DWORD<br>
> +# CHECK-NEXT: S_WAITCNT<br>
> +# CHECK-NEXT: S_DCACHE_WB<br>
> +# CHECK-NEXT: SI_RETURN<br>
> +<br>
> +name: si_return<br>
> +tracksRegLiveness: false<br>
> +<br>
> +body: |<br>
> + bb.0:<br>
> + S_STORE_DWORD_SGPR undef %sgpr2, undef %sgpr0_sgpr1, undef %m0, 0<br>
> + SI_RETURN undef %vgpr0<br>
> +...<br>
><br>
><br>
> _______________________________________________<br>
> llvm-commits mailing list<br>
> llvm-commits@lists.llvm.org<br>
> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
><br>
</div>
</span></font>
</body>
</html>