<div dir="ltr"><div>Yes, please revert it. Thank you!<br><br></div><div>Thanks</div><div><br></div><div>Galina<br></div></div><br><div class="gmail_quote"><div dir="ltr">On Thu, Sep 6, 2018 at 1:15 PM <<a href="mailto:scott@scottlinder.com">scott@scottlinder.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Galina,<br>
<br>
If this buildbot being red for this long is an issue, I can also revert <br>
the change for now and re-commit the whole thing after completing the <br>
review for the fix. Let me know if you would prefer this.<br>
<br>
Thanks,<br>
Scott<br>
<br>
On 2018-09-06 14:12, <a href="mailto:scott@scottlinder.com" target="_blank">scott@scottlinder.com</a> wrote:<br>
> Hi Galina,<br>
> <br>
> I apologize for the delay; I have a fix for the regression up at<br>
> <a href="https://reviews.llvm.org/D51742" rel="noreferrer" target="_blank">https://reviews.llvm.org/D51742</a> and will get it merged ASAP.<br>
> <br>
> Thanks,<br>
> Scott<br>
> <br>
> On 2018-09-06 14:03, Galina Kistanova wrote:<br>
>> Ping?<br>
>> <br>
>> Failing Tests (2):<br>
>> LLVM :: CodeGen/AMDGPU/mubuf-legalize-operands.ll<br>
>> LLVM :: CodeGen/AMDGPU/mubuf-legalize-operands.mir<br>
>> <br>
>> Thanks<br>
>> <br>
>> Galina<br>
>> <br>
>> On Wed, Sep 5, 2018 at 1:54 PM Galina Kistanova <<a href="mailto:gkistanova@gmail.com" target="_blank">gkistanova@gmail.com</a>><br>
>> wrote:<br>
>> <br>
>>> Hello Scott,<br>
>>> <br>
>>> This commit added couple of broken tests to one of our builders:<br>
>>> <br>
>> <a href="http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/12238" rel="noreferrer" target="_blank">http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/12238</a><br>
>>> [9]<br>
>>> <br>
>>> . . .<br>
>>> Failing Tests (3):<br>
>>> Clang :: Driver/print-multi-directory.c<br>
>>> LLVM :: CodeGen/AMDGPU/mubuf-legalize-operands.ll<br>
>>> LLVM :: CodeGen/AMDGPU/mubuf-legalize-operands.mir<br>
>>> <br>
>>> Please have a look?<br>
>>> The builder was already red and did not send notifications on this.<br>
>>> <br>
>>> Thanks<br>
>>> <br>
>>> Galina<br>
>>> <br>
>>> On Tue, Sep 4, 2018 at 2:51 PM Scott Linder via llvm-commits<br>
>>> <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<br>
>>> <br>
>>>> Author: scott.linder<br>
>>>> Date: Tue Sep 4 14:50:47 2018<br>
>>>> New Revision: 341413<br>
>>>> <br>
>>>> URL: <a href="http://llvm.org/viewvc/llvm-project?rev=341413&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=341413&view=rev</a> [1]<br>
>>>> Log:<br>
>>>> [AMDGPU] Legalize VGPR Rsrc operands for MUBUF instructions<br>
>>>> <br>
>>>> Emit a waterfall loop in the general case for a<br>
>>>> potentially-divergent Rsrc<br>
>>>> operand. When practical, avoid this by using Addr64 instructions.<br>
>>>> <br>
>>>> Differential Revision: <a href="https://reviews.llvm.org/D50982" rel="noreferrer" target="_blank">https://reviews.llvm.org/D50982</a> [2]<br>
>>>> <br>
>>>> Added:<br>
>>>> llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll<br>
>>>> llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.mir<br>
>>>> Modified:<br>
>>>> llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.cpp<br>
>>>> llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.h<br>
>>>> llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.td<br>
>>>> <br>
>>>> Modified: llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.cpp<br>
>>>> URL:<br>
>>>> <br>
>>> <br>
>> <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.cpp?rev=341413&r1=341412&r2=341413&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.cpp?rev=341413&r1=341412&r2=341413&view=diff</a><br>
>>>> [3]<br>
>>>> <br>
>>> <br>
>> ==============================================================================<br>
>>>> --- llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.cpp (original)<br>
>>>> +++ llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.cpp Tue Sep 4<br>
>>>> 14:50:47 2018<br>
>>>> @@ -3579,6 +3579,177 @@ void SIInstrInfo::legalizeGenericOperand<br>
>>>> FoldImmediate(*Copy, *Def, OpReg, &MRI);<br>
>>>> }<br>
>>>> <br>
>>>> +// Emit the actual waterfall loop, executing the wrapped<br>
>>>> instruction for each<br>
>>>> +// unique value of \p Rsrc across all lanes. In the best case we<br>
>>>> execute 1<br>
>>>> +// iteration, in the worst case we execute 64 (once per lane).<br>
>>>> +static void emitLoadSRsrcFromVGPRLoop(const SIInstrInfo &TII,<br>
>>>> + MachineRegisterInfo &MRI,<br>
>>>> + MachineBasicBlock &OrigBB,<br>
>>>> + MachineBasicBlock &LoopBB,<br>
>>>> + const DebugLoc &DL,<br>
>>>> + MachineOperand &Rsrc) {<br>
>>>> + MachineBasicBlock::iterator I = LoopBB.begin();<br>
>>>> +<br>
>>>> + unsigned VRsrc = Rsrc.getReg();<br>
>>>> + unsigned VRsrcUndef = getUndefRegState(Rsrc.isUndef());<br>
>>>> +<br>
>>>> + unsigned SaveExec =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SReg_64RegClass);<br>
>>>> + unsigned CondReg0 =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SReg_64RegClass);<br>
>>>> + unsigned CondReg1 =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SReg_64RegClass);<br>
>>>> + unsigned AndCond =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SReg_64RegClass);<br>
>>>> + unsigned SRsrcSub0 =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SGPR_32RegClass);<br>
>>>> + unsigned SRsrcSub1 =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SGPR_32RegClass);<br>
>>>> + unsigned SRsrcSub2 =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SGPR_32RegClass);<br>
>>>> + unsigned SRsrcSub3 =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SGPR_32RegClass);<br>
>>>> + unsigned SRsrc =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SReg_128RegClass);<br>
>>>> +<br>
>>>> + // Beginning of the loop, read the next Rsrc variant.<br>
>>>> + BuildMI(LoopBB, I, DL, TII.get(AMDGPU::V_READFIRSTLANE_B32),<br>
>>>> SRsrcSub0)<br>
>>>> + .addReg(VRsrc, VRsrcUndef, AMDGPU::sub0);<br>
>>>> + BuildMI(LoopBB, I, DL, TII.get(AMDGPU::V_READFIRSTLANE_B32),<br>
>>>> SRsrcSub1)<br>
>>>> + .addReg(VRsrc, VRsrcUndef, AMDGPU::sub1);<br>
>>>> + BuildMI(LoopBB, I, DL, TII.get(AMDGPU::V_READFIRSTLANE_B32),<br>
>>>> SRsrcSub2)<br>
>>>> + .addReg(VRsrc, VRsrcUndef, AMDGPU::sub2);<br>
>>>> + BuildMI(LoopBB, I, DL, TII.get(AMDGPU::V_READFIRSTLANE_B32),<br>
>>>> SRsrcSub3)<br>
>>>> + .addReg(VRsrc, VRsrcUndef, AMDGPU::sub3);<br>
>>>> +<br>
>>>> + BuildMI(LoopBB, I, DL, TII.get(AMDGPU::REG_SEQUENCE), SRsrc)<br>
>>>> + .addReg(SRsrcSub0)<br>
>>>> + .addImm(AMDGPU::sub0)<br>
>>>> + .addReg(SRsrcSub1)<br>
>>>> + .addImm(AMDGPU::sub1)<br>
>>>> + .addReg(SRsrcSub2)<br>
>>>> + .addImm(AMDGPU::sub2)<br>
>>>> + .addReg(SRsrcSub3)<br>
>>>> + .addImm(AMDGPU::sub3);<br>
>>>> +<br>
>>>> + // Update Rsrc operand to use the SGPR Rsrc.<br>
>>>> + Rsrc.setReg(SRsrc);<br>
>>>> + Rsrc.setIsKill(true);<br>
>>>> +<br>
>>>> + // Identify all lanes with identical Rsrc operands in their<br>
>>>> VGPRs.<br>
>>>> + BuildMI(LoopBB, I, DL, TII.get(AMDGPU::V_CMP_EQ_U64_e64),<br>
>>>> CondReg0)<br>
>>>> + .addReg(SRsrc, 0, AMDGPU::sub0_sub1)<br>
>>>> + .addReg(VRsrc, 0, AMDGPU::sub0_sub1);<br>
>>>> + BuildMI(LoopBB, I, DL, TII.get(AMDGPU::V_CMP_EQ_U64_e64),<br>
>>>> CondReg1)<br>
>>>> + .addReg(SRsrc, 0, AMDGPU::sub2_sub3)<br>
>>>> + .addReg(VRsrc, 0, AMDGPU::sub2_sub3);<br>
>>>> + BuildMI(LoopBB, I, DL, TII.get(AMDGPU::S_AND_B64), AndCond)<br>
>>>> + .addReg(CondReg0)<br>
>>>> + .addReg(CondReg1);<br>
>>>> +<br>
>>>> + MRI.setSimpleHint(SaveExec, AndCond);<br>
>>>> +<br>
>>>> + // Update EXEC to matching lanes, saving original to SaveExec.<br>
>>>> + BuildMI(LoopBB, I, DL, TII.get(AMDGPU::S_AND_SAVEEXEC_B64),<br>
>>>> SaveExec)<br>
>>>> + .addReg(AndCond, RegState::Kill);<br>
>>>> +<br>
>>>> + // The original instruction is here; we insert the terminators<br>
>>>> after it.<br>
>>>> + I = LoopBB.end();<br>
>>>> +<br>
>>>> + // Update EXEC, switch all done bits to 0 and all todo bits to<br>
>>>> 1.<br>
>>>> + BuildMI(LoopBB, I, DL, TII.get(AMDGPU::S_XOR_B64_term),<br>
>>>> AMDGPU::EXEC)<br>
>>>> + .addReg(AMDGPU::EXEC)<br>
>>>> + .addReg(SaveExec);<br>
>>>> + BuildMI(LoopBB, I, DL, TII.get(AMDGPU::S_CBRANCH_EXECNZ))<br>
>>>> + .addMBB(&LoopBB);<br>
>>>> +}<br>
>>>> +<br>
>>>> +// Build a waterfall loop around \p MI, replacing the VGPR \p<br>
>>>> Rsrc register<br>
>>>> +// with SGPRs by iterating over all unique values across all<br>
>>>> lanes.<br>
>>>> +static void loadSRsrcFromVGPR(const SIInstrInfo &TII,<br>
>>>> + MachineInstr &MI,<br>
>>>> + MachineOperand &Rsrc) {<br>
>>>> + MachineBasicBlock &MBB = *MI.getParent();<br>
>>>> + MachineFunction &MF = *MBB.getParent();<br>
>>>> + MachineRegisterInfo &MRI = MF.getRegInfo();<br>
>>>> + MachineBasicBlock::iterator I(&MI);<br>
>>>> + const DebugLoc &DL = MI.getDebugLoc();<br>
>>>> +<br>
>>>> + unsigned SaveExec =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SReg_64_XEXECRegClass);<br>
>>>> +<br>
>>>> + // Save the EXEC mask<br>
>>>> + BuildMI(MBB, I, DL, TII.get(AMDGPU::S_MOV_B64), SaveExec)<br>
>>>> + .addReg(AMDGPU::EXEC);<br>
>>>> +<br>
>>>> + // Killed uses in the instruction we are waterfalling around<br>
>>>> will be<br>
>>>> + // incorrect due to the added control-flow.<br>
>>>> + for (auto &MO : MI.uses())<br>
>>>> + if (MO.isReg() && MO.isUse())<br>
>>>> + MRI.clearKillFlags(MO.getReg());<br>
>>>> +<br>
>>>> + // To insert the loop we need to split the block. Move<br>
>>>> everything after this<br>
>>>> + // point to a new block, and insert a new empty block between<br>
>>>> the two.<br>
>>>> + MachineBasicBlock *LoopBB = MF.CreateMachineBasicBlock();<br>
>>>> + MachineBasicBlock *RemainderBB = MF.CreateMachineBasicBlock();<br>
>>>> + MachineFunction::iterator MBBI(MBB);<br>
>>>> + ++MBBI;<br>
>>>> +<br>
>>>> + MF.insert(MBBI, LoopBB);<br>
>>>> + MF.insert(MBBI, RemainderBB);<br>
>>>> +<br>
>>>> + LoopBB->addSuccessor(LoopBB);<br>
>>>> + LoopBB->addSuccessor(RemainderBB);<br>
>>>> +<br>
>>>> + // Move MI to the LoopBB, and the remainder of the block to<br>
>>>> RemainderBB.<br>
>>>> + MachineBasicBlock::iterator J = I++;<br>
>>>> + RemainderBB->transferSuccessorsAndUpdatePHIs(&MBB);<br>
>>>> + RemainderBB->splice(RemainderBB->begin(), &MBB, I, MBB.end());<br>
>>>> + LoopBB->splice(LoopBB->begin(), &MBB, J);<br>
>>>> +<br>
>>>> + MBB.addSuccessor(LoopBB);<br>
>>>> +<br>
>>>> + emitLoadSRsrcFromVGPRLoop(TII, MRI, MBB, *LoopBB, DL, Rsrc);<br>
>>>> +<br>
>>>> + // Restore the EXEC mask<br>
>>>> + MachineBasicBlock::iterator First = RemainderBB->begin();<br>
>>>> + BuildMI(*RemainderBB, First, DL, TII.get(AMDGPU::S_MOV_B64),<br>
>>>> AMDGPU::EXEC)<br>
>>>> + .addReg(SaveExec);<br>
>>>> +}<br>
>>>> +<br>
>>>> +// Extract pointer from Rsrc and return a zero-value Rsrc<br>
>>>> replacement.<br>
>>>> +static std::tuple<unsigned, unsigned> extractRsrcPtr(const<br>
>>>> SIInstrInfo &TII,<br>
>>>> + MachineInstr<br>
>>>> &MI,<br>
>>>> +<br>
>>>> MachineOperand &Rsrc) {<br>
>>>> + MachineBasicBlock &MBB = *MI.getParent();<br>
>>>> + MachineFunction &MF = *MBB.getParent();<br>
>>>> + MachineRegisterInfo &MRI = MF.getRegInfo();<br>
>>>> +<br>
>>>> + // Extract the ptr from the resource descriptor.<br>
>>>> + unsigned RsrcPtr = TII.buildExtractSubReg(MI, MRI, Rsrc,<br>
>>>> + &AMDGPU::VReg_128RegClass, AMDGPU::sub0_sub1,<br>
>>>> &AMDGPU::VReg_64RegClass);<br>
>>>> +<br>
>>>> + // Create an empty resource descriptor<br>
>>>> + unsigned Zero64 =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SReg_64RegClass);<br>
>>>> + unsigned SRsrcFormatLo =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SGPR_32RegClass);<br>
>>>> + unsigned SRsrcFormatHi =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SGPR_32RegClass);<br>
>>>> + unsigned NewSRsrc =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SReg_128RegClass);<br>
>>>> + uint64_t RsrcDataFormat = TII.getDefaultRsrcDataFormat();<br>
>>>> +<br>
>>>> + // Zero64 = 0<br>
>>>> + BuildMI(MBB, MI, MI.getDebugLoc(), TII.get(AMDGPU::S_MOV_B64),<br>
>>>> Zero64)<br>
>>>> + .addImm(0);<br>
>>>> +<br>
>>>> + // SRsrcFormatLo = RSRC_DATA_FORMAT{31-0}<br>
>>>> + BuildMI(MBB, MI, MI.getDebugLoc(), TII.get(AMDGPU::S_MOV_B32),<br>
>>>> SRsrcFormatLo)<br>
>>>> + .addImm(RsrcDataFormat & 0xFFFFFFFF);<br>
>>>> +<br>
>>>> + // SRsrcFormatHi = RSRC_DATA_FORMAT{63-32}<br>
>>>> + BuildMI(MBB, MI, MI.getDebugLoc(), TII.get(AMDGPU::S_MOV_B32),<br>
>>>> SRsrcFormatHi)<br>
>>>> + .addImm(RsrcDataFormat >> 32);<br>
>>>> +<br>
>>>> + // NewSRsrc = {Zero64, SRsrcFormat}<br>
>>>> + BuildMI(MBB, MI, MI.getDebugLoc(),<br>
>>>> TII.get(AMDGPU::REG_SEQUENCE), NewSRsrc)<br>
>>>> + .addReg(Zero64)<br>
>>>> + .addImm(AMDGPU::sub0_sub1)<br>
>>>> + .addReg(SRsrcFormatLo)<br>
>>>> + .addImm(AMDGPU::sub2)<br>
>>>> + .addReg(SRsrcFormatHi)<br>
>>>> + .addImm(AMDGPU::sub3);<br>
>>>> +<br>
>>>> + return std::tie(RsrcPtr, NewSRsrc);<br>
>>>> +}<br>
>>>> +<br>
>>>> void SIInstrInfo::legalizeOperands(MachineInstr &MI) const {<br>
>>>> MachineFunction &MF = *MI.getParent()->getParent();<br>
>>>> MachineRegisterInfo &MRI = MF.getRegInfo();<br>
>>>> @@ -3721,74 +3892,55 @@ void SIInstrInfo::legalizeOperands(Machi<br>
>>>> return;<br>
>>>> }<br>
>>>> <br>
>>>> - // Legalize MUBUF* instructions by converting to addr64 form.<br>
>>>> - // FIXME: If we start using the non-addr64 instructions for<br>
>>>> compute, we<br>
>>>> - // may need to legalize them as above. This especially applies<br>
>>>> to the<br>
>>>> - // buffer_load_format_* variants and variants with idxen (or<br>
>>>> bothen).<br>
>>>> - int SRsrcIdx =<br>
>>>> + // Legalize MUBUF* instructions.<br>
>>>> + int RsrcIdx =<br>
>>>> AMDGPU::getNamedOperandIdx(MI.getOpcode(),<br>
>>>> AMDGPU::OpName::srsrc);<br>
>>>> - if (SRsrcIdx != -1) {<br>
>>>> + if (RsrcIdx != -1) {<br>
>>>> // We have an MUBUF instruction<br>
>>>> - MachineOperand *SRsrc = &MI.getOperand(SRsrcIdx);<br>
>>>> - unsigned SRsrcRC =<br>
>>>> get(MI.getOpcode()).OpInfo[SRsrcIdx].RegClass;<br>
>>>> - if (RI.getCommonSubClass(MRI.getRegClass(SRsrc->getReg()),<br>
>>>> -<br>
>>>> RI.getRegClass(SRsrcRC))) {<br>
>>>> + MachineOperand *Rsrc = &MI.getOperand(RsrcIdx);<br>
>>>> + unsigned RsrcRC =<br>
>>>> get(MI.getOpcode()).OpInfo[RsrcIdx].RegClass;<br>
>>>> + if (RI.getCommonSubClass(MRI.getRegClass(Rsrc->getReg()),<br>
>>>> +<br>
>>>> RI.getRegClass(RsrcRC))) {<br>
>>>> // The operands are legal.<br>
>>>> // FIXME: We may need to legalize operands besided srsrc.<br>
>>>> return;<br>
>>>> }<br>
>>>> <br>
>>>> - MachineBasicBlock &MBB = *MI.getParent();<br>
>>>> + // Legalize a VGPR Rsrc.<br>
>>>> + //<br>
>>>> + // If the instruction is _ADDR64, we can avoid a waterfall by<br>
>>>> extracting<br>
>>>> + // the base pointer from the VGPR Rsrc, adding it to the<br>
>>>> VAddr, then using<br>
>>>> + // a zero-value SRsrc.<br>
>>>> + //<br>
>>>> + // If the instruction is _OFFSET (both idxen and offen<br>
>>>> disabled), and we<br>
>>>> + // support ADDR64 instructions, we can convert to ADDR64 and<br>
>>>> do the same as<br>
>>>> + // above.<br>
>>>> + //<br>
>>>> + // Otherwise we are on non-ADDR64 hardware, and/or we have<br>
>>>> + // idxen/offen/bothen and we fall back to a waterfall loop.<br>
>>>> <br>
>>>> - // Extract the ptr from the resource descriptor.<br>
>>>> - unsigned SRsrcPtr = buildExtractSubReg(MI, MRI, *SRsrc,<br>
>>>> - &AMDGPU::VReg_128RegClass, AMDGPU::sub0_sub1,<br>
>>>> &AMDGPU::VReg_64RegClass);<br>
>>>> -<br>
>>>> - // Create an empty resource descriptor<br>
>>>> - unsigned Zero64 =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SReg_64RegClass);<br>
>>>> - unsigned SRsrcFormatLo =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SGPR_32RegClass);<br>
>>>> - unsigned SRsrcFormatHi =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SGPR_32RegClass);<br>
>>>> - unsigned NewSRsrc =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::SReg_128RegClass);<br>
>>>> - uint64_t RsrcDataFormat = getDefaultRsrcDataFormat();<br>
>>>> -<br>
>>>> - // Zero64 = 0<br>
>>>> - BuildMI(MBB, MI, MI.getDebugLoc(), get(AMDGPU::S_MOV_B64),<br>
>>>> Zero64)<br>
>>>> - .addImm(0);<br>
>>>> -<br>
>>>> - // SRsrcFormatLo = RSRC_DATA_FORMAT{31-0}<br>
>>>> - BuildMI(MBB, MI, MI.getDebugLoc(), get(AMDGPU::S_MOV_B32),<br>
>>>> SRsrcFormatLo)<br>
>>>> - .addImm(RsrcDataFormat & 0xFFFFFFFF);<br>
>>>> -<br>
>>>> - // SRsrcFormatHi = RSRC_DATA_FORMAT{63-32}<br>
>>>> - BuildMI(MBB, MI, MI.getDebugLoc(), get(AMDGPU::S_MOV_B32),<br>
>>>> SRsrcFormatHi)<br>
>>>> - .addImm(RsrcDataFormat >> 32);<br>
>>>> -<br>
>>>> - // NewSRsrc = {Zero64, SRsrcFormat}<br>
>>>> - BuildMI(MBB, MI, MI.getDebugLoc(), get(AMDGPU::REG_SEQUENCE),<br>
>>>> NewSRsrc)<br>
>>>> - .addReg(Zero64)<br>
>>>> - .addImm(AMDGPU::sub0_sub1)<br>
>>>> - .addReg(SRsrcFormatLo)<br>
>>>> - .addImm(AMDGPU::sub2)<br>
>>>> - .addReg(SRsrcFormatHi)<br>
>>>> - .addImm(AMDGPU::sub3);<br>
>>>> + MachineBasicBlock &MBB = *MI.getParent();<br>
>>>> <br>
>>>> MachineOperand *VAddr = getNamedOperand(MI,<br>
>>>> AMDGPU::OpName::vaddr);<br>
>>>> - unsigned NewVAddr =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::VReg_64RegClass);<br>
>>>> - if (VAddr) {<br>
>>>> + if (VAddr && AMDGPU::getIfAddr64Inst(MI.getOpcode()) != -1) {<br>
>>>> // This is already an ADDR64 instruction so we need to add<br>
>>>> the pointer<br>
>>>> // extracted from the resource descriptor to the current<br>
>>>> value of VAddr.<br>
>>>> unsigned NewVAddrLo =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::VGPR_32RegClass);<br>
>>>> unsigned NewVAddrHi =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::VGPR_32RegClass);<br>
>>>> + unsigned NewVAddr =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::VReg_64RegClass);<br>
>>>> <br>
>>>> - // NewVaddrLo = SRsrcPtr:sub0 + VAddr:sub0<br>
>>>> + unsigned RsrcPtr, NewSRsrc;<br>
>>>> + std::tie(RsrcPtr, NewSRsrc) = extractRsrcPtr(*this, MI,<br>
>>>> *Rsrc);<br>
>>>> +<br>
>>>> + // NewVaddrLo = RsrcPtr:sub0 + VAddr:sub0<br>
>>>> DebugLoc DL = MI.getDebugLoc();<br>
>>>> BuildMI(MBB, MI, DL, get(AMDGPU::V_ADD_I32_e32),<br>
>>>> NewVAddrLo)<br>
>>>> - .addReg(SRsrcPtr, 0, AMDGPU::sub0)<br>
>>>> + .addReg(RsrcPtr, 0, AMDGPU::sub0)<br>
>>>> .addReg(VAddr->getReg(), 0, AMDGPU::sub0);<br>
>>>> <br>
>>>> - // NewVaddrHi = SRsrcPtr:sub1 + VAddr:sub1<br>
>>>> + // NewVaddrHi = RsrcPtr:sub1 + VAddr:sub1<br>
>>>> BuildMI(MBB, MI, DL, get(AMDGPU::V_ADDC_U32_e32),<br>
>>>> NewVAddrHi)<br>
>>>> - .addReg(SRsrcPtr, 0, AMDGPU::sub1)<br>
>>>> + .addReg(RsrcPtr, 0, AMDGPU::sub1)<br>
>>>> .addReg(VAddr->getReg(), 0, AMDGPU::sub1);<br>
>>>> <br>
>>>> // NewVaddr = {NewVaddrHi, NewVaddrLo}<br>
>>>> @@ -3797,13 +3949,20 @@ void SIInstrInfo::legalizeOperands(Machi<br>
>>>> .addImm(AMDGPU::sub0)<br>
>>>> .addReg(NewVAddrHi)<br>
>>>> .addImm(AMDGPU::sub1);<br>
>>>> - } else {<br>
>>>> +<br>
>>>> + VAddr->setReg(NewVAddr);<br>
>>>> + Rsrc->setReg(NewSRsrc);<br>
>>>> + } else if (!VAddr && ST.hasAddr64()) {<br>
>>>> // This instructions is the _OFFSET variant, so we need to<br>
>>>> convert it to<br>
>>>> // ADDR64.<br>
>>>> <br>
>>>> <br>
>>> assert(MBB.getParent()->getSubtarget<GCNSubtarget>().getGeneration()<br>
>>>> < AMDGPUSubtarget::VOLCANIC_ISLANDS &&<br>
>>>> "FIXME: Need to emit flat atomics here");<br>
>>>> <br>
>>>> + unsigned RsrcPtr, NewSRsrc;<br>
>>>> + std::tie(RsrcPtr, NewSRsrc) = extractRsrcPtr(*this, MI,<br>
>>>> *Rsrc);<br>
>>>> +<br>
>>>> + unsigned NewVAddr =<br>
>>>> MRI.createVirtualRegister(&AMDGPU::VReg_64RegClass);<br>
>>>> MachineOperand *VData = getNamedOperand(MI,<br>
>>>> AMDGPU::OpName::vdata);<br>
>>>> MachineOperand *Offset = getNamedOperand(MI,<br>
>>>> AMDGPU::OpName::offset);<br>
>>>> MachineOperand *SOffset = getNamedOperand(MI,<br>
>>>> AMDGPU::OpName::soffset);<br>
>>>> @@ -3819,10 +3978,8 @@ void SIInstrInfo::legalizeOperands(Machi<br>
>>>> MachineInstrBuilder MIB =<br>
>>>> BuildMI(MBB, MI, MI.getDebugLoc(), get(Addr64Opcode))<br>
>>>> .add(*VData)<br>
>>>> - .addReg(AMDGPU::NoRegister) // Dummy value for<br>
>>>> vaddr.<br>
>>>> - // This will be replaced later<br>
>>>> - // with the new value of vaddr.<br>
>>>> - .add(*SRsrc)<br>
>>>> + .addReg(NewVAddr)<br>
>>>> + .addReg(NewSRsrc)<br>
>>>> .add(*SOffset)<br>
>>>> .add(*Offset);<br>
>>>> <br>
>>>> @@ -3846,10 +4003,8 @@ void SIInstrInfo::legalizeOperands(Machi<br>
>>>> Addr64 = BuildMI(MBB, MI, MI.getDebugLoc(),<br>
>>>> get(Addr64Opcode))<br>
>>>> .add(*VData)<br>
>>>> .add(*VDataIn)<br>
>>>> - .addReg(AMDGPU::NoRegister) // Dummy value<br>
>>>> for vaddr.<br>
>>>> - // This will be replaced later<br>
>>>> - // with the new value of vaddr.<br>
>>>> - .add(*SRsrc)<br>
>>>> + .addReg(NewVAddr)<br>
>>>> + .addReg(NewSRsrc)<br>
>>>> .add(*SOffset)<br>
>>>> .add(*Offset)<br>
>>>> .addImm(getNamedImmOperand(MI,<br>
>>>> AMDGPU::OpName::slc))<br>
>>>> @@ -3861,19 +4016,15 @@ void SIInstrInfo::legalizeOperands(Machi<br>
>>>> // NewVaddr = {NewVaddrHi, NewVaddrLo}<br>
>>>> BuildMI(MBB, Addr64, Addr64->getDebugLoc(),<br>
>>>> get(AMDGPU::REG_SEQUENCE),<br>
>>>> NewVAddr)<br>
>>>> - .addReg(SRsrcPtr, 0, AMDGPU::sub0)<br>
>>>> + .addReg(RsrcPtr, 0, AMDGPU::sub0)<br>
>>>> .addImm(AMDGPU::sub0)<br>
>>>> - .addReg(SRsrcPtr, 0, AMDGPU::sub1)<br>
>>>> + .addReg(RsrcPtr, 0, AMDGPU::sub1)<br>
>>>> .addImm(AMDGPU::sub1);<br>
>>>> -<br>
>>>> - VAddr = getNamedOperand(*Addr64, AMDGPU::OpName::vaddr);<br>
>>>> - SRsrc = getNamedOperand(*Addr64, AMDGPU::OpName::srsrc);<br>
>>>> + } else {<br>
>>>> + // This is another variant; legalize Rsrc with waterfall<br>
>>>> loop from VGPRs<br>
>>>> + // to SGPRs.<br>
>>>> + loadSRsrcFromVGPR(*this, MI, *Rsrc);<br>
>>>> }<br>
>>>> -<br>
>>>> - // Update the instruction to use NewVaddr<br>
>>>> - VAddr->setReg(NewVAddr);<br>
>>>> - // Update the instruction to use NewSRsrc<br>
>>>> - SRsrc->setReg(NewSRsrc);<br>
>>>> }<br>
>>>> }<br>
>>>> <br>
>>>> Modified: llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.h<br>
>>>> URL:<br>
>>>> <br>
>>> <br>
>> <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.h?rev=341413&r1=341412&r2=341413&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.h?rev=341413&r1=341412&r2=341413&view=diff</a><br>
>>>> [4]<br>
>>>> <br>
>>> <br>
>> ==============================================================================<br>
>>>> --- llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.h (original)<br>
>>>> +++ llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.h Tue Sep 4 14:50:47<br>
>>>> 2018<br>
>>>> @@ -919,6 +919,12 @@ namespace AMDGPU {<br>
>>>> LLVM_READONLY<br>
>>>> int getAddr64Inst(uint16_t Opcode);<br>
>>>> <br>
>>>> + /// Check if \p Opcode is an Addr64 opcode.<br>
>>>> + ///<br>
>>>> + /// \returns \p Opcode if it is an Addr64 opcode, otherwise -1.<br>
>>>> + LLVM_READONLY<br>
>>>> + int getIfAddr64Inst(uint16_t Opcode);<br>
>>>> +<br>
>>>> LLVM_READONLY<br>
>>>> int getMUBUFNoLdsInst(uint16_t Opcode);<br>
>>>> <br>
>>>> Modified: llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.td<br>
>>>> URL:<br>
>>>> <br>
>>> <br>
>> <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.td?rev=341413&r1=341412&r2=341413&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.td?rev=341413&r1=341412&r2=341413&view=diff</a><br>
>>>> [5]<br>
>>>> <br>
>>> <br>
>> ==============================================================================<br>
>>>> --- llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.td (original)<br>
>>>> +++ llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.td Tue Sep 4<br>
>>>> 14:50:47 2018<br>
>>>> @@ -1986,6 +1986,14 @@ def getAddr64Inst : InstrMapping {<br>
>>>> let ValueCols = [["1"]];<br>
>>>> }<br>
>>>> <br>
>>>> +def getIfAddr64Inst : InstrMapping {<br>
>>>> + let FilterClass = "MUBUFAddr64Table";<br>
>>>> + let RowFields = ["OpName"];<br>
>>>> + let ColFields = ["IsAddr64"];<br>
>>>> + let KeyCol = ["1"];<br>
>>>> + let ValueCols = [["1"]];<br>
>>>> +}<br>
>>>> +<br>
>>>> def getMUBUFNoLdsInst : InstrMapping {<br>
>>>> let FilterClass = "MUBUFLdsTable";<br>
>>>> let RowFields = ["OpName"];<br>
>>>> <br>
>>>> Added: llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll<br>
>>>> URL:<br>
>>>> <br>
>>> <br>
>> <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll?rev=341413&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll?rev=341413&view=auto</a><br>
>>>> [6]<br>
>>>> <br>
>>> <br>
>> ==============================================================================<br>
>>>> --- llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll<br>
>>>> (added)<br>
>>>> +++ llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll Tue<br>
>>>> Sep 4 14:50:47 2018<br>
>>>> @@ -0,0 +1,230 @@<br>
>>>> +; RUN: llc -march=amdgcn -mcpu=gfx900 -verify-machineinstrs -o -<br>
>>>> %s | FileCheck %s<br>
>>>> +; RUN: llc -O0 -march=amdgcn -mcpu=gfx900 -verify-machineinstrs<br>
>>>> -o - %s | FileCheck %s --check-prefix=CHECK-O0<br>
>>>> +<br>
>>>> +; Test that we correctly legalize VGPR Rsrc operands in MUBUF<br>
>>>> instructions.<br>
>>>> +<br>
>>>> +; CHECK-LABEL: mubuf_vgpr<br>
>>>> +; CHECK: s_mov_b64 [[SAVEEXEC:s\[[0-9]+:[0-9]+\]]], exec<br>
>>>> +; CHECK: [[LOOPBB:BB[0-9]+_[0-9]+]]:<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC0:[0-9]+]], v0<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC1:[0-9]+]], v1<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC2:[0-9]+]], v2<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC3:[0-9]+]], v3<br>
>>>> +; CHECK: v_cmp_eq_u64_e32 vcc,<br>
>>>> s{{\[}}[[SRSRC0]]:[[SRSRC1]]{{\]}}, v[0:1]<br>
>>>> +; CHECK: v_cmp_eq_u64_e64 [[CMP0:s\[[0-9]+:[0-9]+\]]],<br>
>>>> s{{\[}}[[SRSRC2]]:[[SRSRC3]]{{\]}}, v[2:3]<br>
>>>> +; CHECK: s_and_b64 [[CMP:s\[[0-9]+:[0-9]+\]]], vcc, [[CMP0]]<br>
>>>> +; CHECK: s_and_saveexec_b64 [[CMP]], [[CMP]]<br>
>>>> +; CHECK: s_waitcnt vmcnt(0)<br>
>>>> +; CHECK: buffer_load_format_x [[RES:v[0-9]+]], v4,<br>
>>>> s{{\[}}[[SRSRC0]]:[[SRSRC3]]{{\]}}, 0 idxen<br>
>>>> +; CHECK: s_xor_b64 exec, exec, [[CMP]]<br>
>>>> +; CHECK: s_cbranch_execnz [[LOOPBB]]<br>
>>>> +; CHECK: s_mov_b64 exec, [[SAVEEXEC]]<br>
>>>> +; CHECK: v_mov_b32_e32 v0, [[RES]]<br>
>>>> +define float @mubuf_vgpr(<4 x i32> %i, i32 %c) #0 {<br>
>>>> + %call = call float @llvm.amdgcn.buffer.load.format.f32(<4 x<br>
>>>> i32> %i, i32 %c, i32 0, i1 zeroext false, i1 zeroext false) #1<br>
>>>> + ret float %call<br>
>>>> +}<br>
>>>> +<br>
>>>> +; CHECK-LABEL: mubuf_vgpr_adjacent_in_block<br>
>>>> +<br>
>>>> +; CHECK: s_mov_b64 [[SAVEEXEC:s\[[0-9]+:[0-9]+\]]], exec<br>
>>>> +; CHECK: [[LOOPBB0:BB[0-9]+_[0-9]+]]:<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC0:[0-9]+]], v0<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC1:[0-9]+]], v1<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC2:[0-9]+]], v2<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC3:[0-9]+]], v3<br>
>>>> +; CHECK: v_cmp_eq_u64_e32 vcc,<br>
>>>> s{{\[}}[[SRSRC0]]:[[SRSRC1]]{{\]}}, v[0:1]<br>
>>>> +; CHECK: v_cmp_eq_u64_e64 [[CMP0:s\[[0-9]+:[0-9]+\]]],<br>
>>>> s{{\[}}[[SRSRC2]]:[[SRSRC3]]{{\]}}, v[2:3]<br>
>>>> +; CHECK: s_and_b64 [[CMP:s\[[0-9]+:[0-9]+\]]], vcc, [[CMP0]]<br>
>>>> +; CHECK: s_and_saveexec_b64 [[CMP]], [[CMP]]<br>
>>>> +; CHECK: s_waitcnt vmcnt(0)<br>
>>>> +; CHECK: buffer_load_format_x [[RES0:v[0-9]+]], v8,<br>
>>>> s{{\[}}[[SRSRC0]]:[[SRSRC3]]{{\]}}, 0 idxen<br>
>>>> +; CHECK: s_xor_b64 exec, exec, [[CMP]]<br>
>>>> +; CHECK: s_cbranch_execnz [[LOOPBB0]]<br>
>>>> +<br>
>>>> +; CHECK: s_mov_b64 exec, [[SAVEEXEC]]<br>
>>>> +; FIXME: redundant s_mov<br>
>>>> +; CHECK: s_mov_b64 [[SAVEEXEC:s\[[0-9]+:[0-9]+\]]], exec<br>
>>>> +<br>
>>>> +; CHECK: [[LOOPBB1:BB[0-9]+_[0-9]+]]:<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC0:[0-9]+]], v4<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC1:[0-9]+]], v5<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC2:[0-9]+]], v6<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC3:[0-9]+]], v7<br>
>>>> +; CHECK: v_cmp_eq_u64_e32 vcc,<br>
>>>> s{{\[}}[[SRSRC0]]:[[SRSRC1]]{{\]}}, v[4:5]<br>
>>>> +; CHECK: v_cmp_eq_u64_e64 [[CMP0:s\[[0-9]+:[0-9]+\]]],<br>
>>>> s{{\[}}[[SRSRC2]]:[[SRSRC3]]{{\]}}, v[6:7]<br>
>>>> +; CHECK: s_and_b64 [[CMP:s\[[0-9]+:[0-9]+\]]], vcc, [[CMP0]]<br>
>>>> +; CHECK: s_and_saveexec_b64 [[CMP]], [[CMP]]<br>
>>>> +; CHECK: s_waitcnt vmcnt(0)<br>
>>>> +; CHECK: buffer_load_format_x [[RES1:v[0-9]+]], v8,<br>
>>>> s{{\[}}[[SRSRC0]]:[[SRSRC3]]{{\]}}, 0 idxen<br>
>>>> +; CHECK: s_xor_b64 exec, exec, [[CMP]]<br>
>>>> +; CHECK: s_cbranch_execnz [[LOOPBB1]]<br>
>>>> +<br>
>>>> +; CHECK: s_mov_b64 exec, [[SAVEEXEC]]<br>
>>>> +; CHECK-DAG: global_store_dword v[9:10], [[RES0]], off<br>
>>>> +; CHECK-DAG: global_store_dword v[11:12], [[RES1]], off<br>
>>>> +<br>
>>>> +define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32><br>
>>>> %j, i32 %c, float addrspace(1)* %out0, float addrspace(1)* %out1)<br>
>>>> #0 {<br>
>>>> +entry:<br>
>>>> + %val0 = call float @llvm.amdgcn.buffer.load.format.f32(<4 x<br>
>>>> i32> %i, i32 %c, i32 0, i1 zeroext false, i1 zeroext false) #1<br>
>>>> + %val1 = call float @llvm.amdgcn.buffer.load.format.f32(<4 x<br>
>>>> i32> %j, i32 %c, i32 0, i1 zeroext false, i1 zeroext false) #1<br>
>>>> + store volatile float %val0, float addrspace(1)* %out0<br>
>>>> + store volatile float %val1, float addrspace(1)* %out1<br>
>>>> + ret void<br>
>>>> +}<br>
>>>> +<br>
>>>> +; CHECK-LABEL: mubuf_vgpr_outside_entry<br>
>>>> +<br>
>>>> +; CHECK-DAG: v_mov_b32_e32 [[IDX:v[0-9]+]], s4<br>
>>>> +; CHECK-DAG: s_mov_b64 [[SAVEEXEC:s\[[0-9]+:[0-9]+\]]], exec<br>
>>>> +<br>
>>>> +; CHECK: [[LOOPBB0:BB[0-9]+_[0-9]+]]:<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC0:[0-9]+]], v0<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC1:[0-9]+]], v1<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC2:[0-9]+]], v2<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC3:[0-9]+]], v3<br>
>>>> +; CHECK: v_cmp_eq_u64_e32 vcc,<br>
>>>> s{{\[}}[[SRSRC0]]:[[SRSRC1]]{{\]}}, v[0:1]<br>
>>>> +; CHECK: v_cmp_eq_u64_e64 [[CMP0:s\[[0-9]+:[0-9]+\]]],<br>
>>>> s{{\[}}[[SRSRC2]]:[[SRSRC3]]{{\]}}, v[2:3]<br>
>>>> +; CHECK: s_and_b64 [[CMP:s\[[0-9]+:[0-9]+\]]], vcc, [[CMP0]]<br>
>>>> +; CHECK: s_and_saveexec_b64 [[CMP]], [[CMP]]<br>
>>>> +; CHECK: s_waitcnt vmcnt(0)<br>
>>>> +; CHECK: buffer_load_format_x [[RES:v[0-9]+]], [[IDX]],<br>
>>>> s{{\[}}[[SRSRC0]]:[[SRSRC3]]{{\]}}, 0 idxen<br>
>>>> +; CHECK: s_xor_b64 exec, exec, [[CMP]]<br>
>>>> +; CHECK: s_cbranch_execnz [[LOOPBB0]]<br>
>>>> +<br>
>>>> +; CHECK: s_mov_b64 exec, [[SAVEEXEC]]<br>
>>>> +; CHECK: s_cbranch_execz [[TERMBB:BB[0-9]+_[0-9]+]]<br>
>>>> +<br>
>>>> +; CHECK: BB{{[0-9]+_[0-9]+}}:<br>
>>>> +; CHECK-DAG: v_mov_b32_e32 [[IDX:v[0-9]+]], s4<br>
>>>> +; CHECK-DAG: s_mov_b64 [[SAVEEXEC:s\[[0-9]+:[0-9]+\]]], exec<br>
>>>> +<br>
>>>> +; CHECK: [[LOOPBB1:BB[0-9]+_[0-9]+]]:<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC0:[0-9]+]], v4<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC1:[0-9]+]], v5<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC2:[0-9]+]], v6<br>
>>>> +; CHECK-DAG: v_readfirstlane_b32 s[[SRSRC3:[0-9]+]], v7<br>
>>>> +; CHECK: v_cmp_eq_u64_e32 vcc,<br>
>>>> s{{\[}}[[SRSRC0]]:[[SRSRC1]]{{\]}}, v[4:5]<br>
>>>> +; CHECK: v_cmp_eq_u64_e64 [[CMP0:s\[[0-9]+:[0-9]+\]]],<br>
>>>> s{{\[}}[[SRSRC2]]:[[SRSRC3]]{{\]}}, v[6:7]<br>
>>>> +; CHECK: s_and_b64 [[CMP:s\[[0-9]+:[0-9]+\]]], vcc, [[CMP0]]<br>
>>>> +; CHECK: s_and_saveexec_b64 [[CMP]], [[CMP]]<br>
>>>> +; CHECK: s_waitcnt vmcnt(0)<br>
>>>> +; CHECK: buffer_load_format_x [[RES]], [[IDX]],<br>
>>>> s{{\[}}[[SRSRC0]]:[[SRSRC3]]{{\]}}, 0 idxen<br>
>>>> +; CHECK: s_xor_b64 exec, exec, [[CMP]]<br>
>>>> +; CHECK: s_cbranch_execnz [[LOOPBB1]]<br>
>>>> +<br>
>>>> +; CHECK: s_mov_b64 exec, [[SAVEEXEC]]<br>
>>>> +<br>
>>>> +; CHECK: [[TERMBB]]:<br>
>>>> +; CHECK: global_store_dword v[11:12], [[RES]], off<br>
>>>> +<br>
>>>> +; Confirm spills do not occur between the XOR and branch that<br>
>>>> terminate the<br>
>>>> +; waterfall loop BBs.<br>
>>>> +<br>
>>>> +; CHECK-O0-LABEL: mubuf_vgpr_outside_entry<br>
>>>> +<br>
>>>> +; CHECK-O0-DAG: s_mov_b32 [[IDX_S:s[0-9]+]], s4<br>
>>>> +; CHECK-O0-DAG: v_mov_b32_e32 [[IDX_V:v[0-9]+]], [[IDX_S]]<br>
>>>> +; CHECK-O0-DAG: s_mov_b64 [[SAVEEXEC:s\[[0-9]+:[0-9]+\]]], exec<br>
>>>> +; CHECK-O0-DAG: buffer_store_dword [[IDX_V]], off, s[0:3], s5<br>
>>>> offset:[[IDX_OFF:[0-9]+]] ; 4-byte Folded Spill<br>
>>>> +<br>
>>>> +; CHECK-O0: [[LOOPBB0:BB[0-9]+_[0-9]+]]:<br>
>>>> +; CHECK-O0: buffer_load_dword v[[VRSRC0:[0-9]+]], {{.*}} ; 4-byte<br>
>>>> Folded Reload<br>
>>>> +; CHECK-O0: s_waitcnt vmcnt(0)<br>
>>>> +; CHECK-O0: buffer_load_dword v[[VRSRC1:[0-9]+]], {{.*}} ; 4-byte<br>
>>>> Folded Reload<br>
>>>> +; CHECK-O0: s_waitcnt vmcnt(0)<br>
>>>> +; CHECK-O0: buffer_load_dword v[[VRSRC2:[0-9]+]], {{.*}} ; 4-byte<br>
>>>> Folded Reload<br>
>>>> +; CHECK-O0: s_waitcnt vmcnt(0)<br>
>>>> +; CHECK-O0: buffer_load_dword v[[VRSRC3:[0-9]+]], {{.*}} ; 4-byte<br>
>>>> Folded Reload<br>
>>>> +; CHECK-O0: s_waitcnt vmcnt(0)<br>
>>>> +; CHECK-O0-DAG: v_readfirstlane_b32 s[[SRSRCTMP0:[0-9]+]],<br>
>>>> v[[VRSRC0]]<br>
>>>> +; CHECK-O0-DAG: v_readfirstlane_b32 s[[SRSRCTMP1:[0-9]+]],<br>
>>>> v[[VRSRC1]]<br>
>>>> +; CHECK-O0-DAG: v_readfirstlane_b32 s[[SRSRCTMP2:[0-9]+]],<br>
>>>> v[[VRSRC2]]<br>
>>>> +; CHECK-O0-DAG: v_readfirstlane_b32 s[[SRSRCTMP3:[0-9]+]],<br>
>>>> v[[VRSRC3]]<br>
>>>> +; CHECK-O0-DAG: s_mov_b32 s[[SRSRC0:[0-9]+]], s[[SRSRCTMP0]]<br>
>>>> +; CHECK-O0-DAG: s_mov_b32 s[[SRSRC1:[0-9]+]], s[[SRSRCTMP1]]<br>
>>>> +; CHECK-O0-DAG: s_mov_b32 s[[SRSRC2:[0-9]+]], s[[SRSRCTMP2]]<br>
>>>> +; CHECK-O0-DAG: s_mov_b32 s[[SRSRC3:[0-9]+]], s[[SRSRCTMP3]]<br>
>>>> +; CHECK-O0: v_cmp_eq_u64_e64 [[CMP0:s\[[0-9]+:[0-9]+\]]],<br>
>>>> s{{\[}}[[SRSRC0]]:[[SRSRC1]]{{\]}},<br>
>>>> v{{\[}}[[VRSRC0]]:[[VRSRC1]]{{\]}}<br>
>>>> +; CHECK-O0: v_cmp_eq_u64_e64 [[CMP1:s\[[0-9]+:[0-9]+\]]],<br>
>>>> s{{\[}}[[SRSRC2]]:[[SRSRC3]]{{\]}},<br>
>>>> v{{\[}}[[VRSRC2]]:[[VRSRC3]]{{\]}}<br>
>>>> +; CHECK-O0: s_and_b64 [[CMP:s\[[0-9]+:[0-9]+\]]], [[CMP0]],<br>
>>>> [[CMP1]]<br>
>>>> +; CHECK-O0: s_and_saveexec_b64 [[CMP]], [[CMP]]<br>
>>>> +; CHECK-O0: buffer_load_dword [[IDX:v[0-9]+]], off, s[0:3], s5<br>
>>>> offset:[[IDX_OFF]] ; 4-byte Folded Reload<br>
>>>> +; CHECK-O0: buffer_load_format_x [[RES:v[0-9]+]], [[IDX]],<br>
>>>> s{{\[}}[[SRSRC0]]:[[SRSRC3]]{{\]}}, {{.*}} idxen<br>
>>>> +; CHECK-O0: s_waitcnt vmcnt(0)<br>
>>>> +; CHECK-O0: buffer_store_dword [[RES]], off, s[0:3], s5<br>
>>>> offset:[[RES_OFF_TMP:[0-9]+]] ; 4-byte Folded Spill<br>
>>>> +; CHECK-O0: s_xor_b64 exec, exec, [[CMP]]<br>
>>>> +; CHECK-O0-NEXT: s_cbranch_execnz [[LOOPBB0]]<br>
>>>> +<br>
>>>> +; CHECK-O0: s_mov_b64 exec, [[SAVEEXEC]]<br>
>>>> +; CHECK-O0: buffer_load_dword [[RES:v[0-9]+]], off, s[0:3], s5<br>
>>>> offset:[[RES_OFF_TMP]] ; 4-byte Folded Reload<br>
>>>> +; CHECK-O0: buffer_store_dword [[RES]], off, s[0:3], s5<br>
>>>> offset:[[RES_OFF:[0-9]+]] ; 4-byte Folded Spill<br>
>>>> +; CHECK-O0: s_cbranch_execz [[TERMBB:BB[0-9]+_[0-9]+]]<br>
>>>> +<br>
>>>> +; CHECK-O0: BB{{[0-9]+_[0-9]+}}:<br>
>>>> +; CHECK-O0-DAG: s_mov_b64<br>
>>>> s{{\[}}[[SAVEEXEC0:[0-9]+]]:[[SAVEEXEC1:[0-9]+]]{{\]}}, exec<br>
>>>> +; CHECK-O0-DAG: buffer_store_dword {{v[0-9]+}}, off, s[0:3], s5<br>
>>>> offset:[[IDX_OFF:[0-9]+]] ; 4-byte Folded Spill<br>
>>>> +; CHECK-O0: v_writelane_b32 [[VSAVEEXEC:v[0-9]+]],<br>
>>>> s[[SAVEEXEC0]], [[SAVEEXEC_IDX0:[0-9]+]]<br>
>>>> +; CHECK-O0: v_writelane_b32 [[VSAVEEXEC:v[0-9]+]],<br>
>>>> s[[SAVEEXEC1]], [[SAVEEXEC_IDX1:[0-9]+]]<br>
>>>> +<br>
>>>> +; CHECK-O0: [[LOOPBB1:BB[0-9]+_[0-9]+]]:<br>
>>>> +; CHECK-O0: buffer_load_dword v[[VRSRC0:[0-9]+]], {{.*}} ; 4-byte<br>
>>>> Folded Reload<br>
>>>> +; CHECK-O0: s_waitcnt vmcnt(0)<br>
>>>> +; CHECK-O0: buffer_load_dword v[[VRSRC1:[0-9]+]], {{.*}} ; 4-byte<br>
>>>> Folded Reload<br>
>>>> +; CHECK-O0: s_waitcnt vmcnt(0)<br>
>>>> +; CHECK-O0: buffer_load_dword v[[VRSRC2:[0-9]+]], {{.*}} ; 4-byte<br>
>>>> Folded Reload<br>
>>>> +; CHECK-O0: s_waitcnt vmcnt(0)<br>
>>>> +; CHECK-O0: buffer_load_dword v[[VRSRC3:[0-9]+]], {{.*}} ; 4-byte<br>
>>>> Folded Reload<br>
>>>> +; CHECK-O0: s_waitcnt vmcnt(0)<br>
>>>> +; CHECK-O0-DAG: v_readfirstlane_b32 s[[SRSRCTMP0:[0-9]+]],<br>
>>>> v[[VRSRC0]]<br>
>>>> +; CHECK-O0-DAG: v_readfirstlane_b32 s[[SRSRCTMP1:[0-9]+]],<br>
>>>> v[[VRSRC1]]<br>
>>>> +; CHECK-O0-DAG: v_readfirstlane_b32 s[[SRSRCTMP2:[0-9]+]],<br>
>>>> v[[VRSRC2]]<br>
>>>> +; CHECK-O0-DAG: v_readfirstlane_b32 s[[SRSRCTMP3:[0-9]+]],<br>
>>>> v[[VRSRC3]]<br>
>>>> +; CHECK-O0-DAG: s_mov_b32 s[[SRSRC0:[0-9]+]], s[[SRSRCTMP0]]<br>
>>>> +; CHECK-O0-DAG: s_mov_b32 s[[SRSRC1:[0-9]+]], s[[SRSRCTMP1]]<br>
>>>> +; CHECK-O0-DAG: s_mov_b32 s[[SRSRC2:[0-9]+]], s[[SRSRCTMP2]]<br>
>>>> +; CHECK-O0-DAG: s_mov_b32 s[[SRSRC3:[0-9]+]], s[[SRSRCTMP3]]<br>
>>>> +; CHECK-O0: v_cmp_eq_u64_e64 [[CMP0:s\[[0-9]+:[0-9]+\]]],<br>
>>>> s{{\[}}[[SRSRC0]]:[[SRSRC1]]{{\]}},<br>
>>>> v{{\[}}[[VRSRC0]]:[[VRSRC1]]{{\]}}<br>
>>>> +; CHECK-O0: v_cmp_eq_u64_e64 [[CMP1:s\[[0-9]+:[0-9]+\]]],<br>
>>>> s{{\[}}[[SRSRC2]]:[[SRSRC3]]{{\]}},<br>
>>>> v{{\[}}[[VRSRC2]]:[[VRSRC3]]{{\]}}<br>
>>>> +; CHECK-O0: s_and_b64 [[CMP:s\[[0-9]+:[0-9]+\]]], [[CMP0]],<br>
>>>> [[CMP1]]<br>
>>>> +; CHECK-O0: s_and_saveexec_b64 [[CMP]], [[CMP]]<br>
>>>> +; CHECK-O0: buffer_load_dword [[IDX:v[0-9]+]], off, s[0:3], s5<br>
>>>> offset:[[IDX_OFF]] ; 4-byte Folded Reload<br>
>>>> +; CHECK-O0: buffer_load_format_x [[RES:v[0-9]+]], [[IDX]],<br>
>>>> s{{\[}}[[SRSRC0]]:[[SRSRC3]]{{\]}}, {{.*}} idxen<br>
>>>> +; CHECK-O0: s_waitcnt vmcnt(0)<br>
>>>> +; CHECK-O0: buffer_store_dword [[RES]], off, s[0:3], s5<br>
>>>> offset:[[RES_OFF_TMP:[0-9]+]] ; 4-byte Folded Spill<br>
>>>> +; CHECK-O0: s_xor_b64 exec, exec, [[CMP]]<br>
>>>> +; CHECK-O0-NEXT: s_cbranch_execnz [[LOOPBB1]]<br>
>>>> +<br>
>>>> +; CHECK-O0: v_readlane_b32 s[[SAVEEXEC0:[0-9]+]], [[VSAVEEXEC]],<br>
>>>> [[SAVEEXEC_IDX0]]<br>
>>>> +; CHECK-O0: v_readlane_b32 s[[SAVEEXEC1:[0-9]+]], [[VSAVEEXEC]],<br>
>>>> [[SAVEEXEC_IDX1]]<br>
>>>> +; CHECK-O0: s_mov_b64 exec,<br>
>>>> s{{\[}}[[SAVEEXEC0]]:[[SAVEEXEC1]]{{\]}}<br>
>>>> +; CHECK-O0: buffer_load_dword [[RES:v[0-9]+]], off, s[0:3], s5<br>
>>>> offset:[[RES_OFF_TMP]] ; 4-byte Folded Reload<br>
>>>> +; CHECK-O0: buffer_store_dword [[RES]], off, s[0:3], s5<br>
>>>> offset:[[RES_OFF]] ; 4-byte Folded Spill<br>
>>>> +<br>
>>>> +; CHECK-O0: [[TERMBB]]:<br>
>>>> +; CHECK-O0: buffer_load_dword [[RES:v[0-9]+]], off, s[0:3], s5<br>
>>>> offset:[[RES_OFF]] ; 4-byte Folded Reload<br>
>>>> +; CHECK-O0: global_store_dword v[{{[0-9]+:[0-9]+}}], [[RES]], off<br>
>>>> +<br>
>>>> +define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j,<br>
>>>> i32 %c, float addrspace(1)* %in, float addrspace(1)* %out) #0 {<br>
>>>> +entry:<br>
>>>> + %live.out.reg = call i32 asm sideeffect "s_mov_b32 $0, 17",<br>
>>>> "={s4}" ()<br>
>>>> + %val0 = call float @llvm.amdgcn.buffer.load.format.f32(<4 x<br>
>>>> i32> %i, i32 %live.out.reg, i32 0, i1 zeroext false, i1 zeroext<br>
>>>> false) #1<br>
>>>> + %idx = call i32 @llvm.amdgcn.workitem.id.x() #1<br>
>>>> + %cmp = icmp eq i32 %idx, 0<br>
>>>> + br i1 %cmp, label %bb1, label %bb2<br>
>>>> +<br>
>>>> +bb1:<br>
>>>> + %val1 = call float @llvm.amdgcn.buffer.load.format.f32(<4 x<br>
>>>> i32> %j, i32 %live.out.reg, i32 0, i1 zeroext false, i1 zeroext<br>
>>>> false) #1<br>
>>>> + br label %bb2<br>
>>>> +<br>
>>>> +bb2:<br>
>>>> + %val = phi float [ %val0, %entry ], [ %val1, %bb1 ]<br>
>>>> + store volatile float %val, float addrspace(1)* %out<br>
>>>> + ret void<br>
>>>> +}<br>
>>>> +<br>
>>>> +declare i32 @llvm.amdgcn.workitem.id.x() #1<br>
>>>> +declare float @llvm.amdgcn.buffer.load.format.f32(<4 x i32>, i32,<br>
>>>> i32, i1, i1) #1<br>
>>>> +<br>
>>>> +attributes #0 = { nounwind }<br>
>>>> +attributes #1 = { nounwind readnone }<br>
>>>> <br>
>>>> Added: llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.mir<br>
>>>> URL:<br>
>>>> <br>
>>> <br>
>> <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.mir?rev=341413&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.mir?rev=341413&view=auto</a><br>
>>>> [7]<br>
>>>> <br>
>>> <br>
>> ==============================================================================<br>
>>>> --- llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.mir<br>
>>>> (added)<br>
>>>> +++ llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.mir Tue<br>
>>>> Sep 4 14:50:47 2018<br>
>>>> @@ -0,0 +1,239 @@<br>
>>>> +# RUN: llc -march=amdgcn -mcpu=gfx700 -verify-machineinstrs<br>
>>>> --run-pass=si-fix-sgpr-copies -o - %s | FileCheck %s<br>
>>>> --check-prefixes=COMMON,ADDR64<br>
>>>> +# RUN: llc -march=amdgcn -mcpu=gfx900 -verify-machineinstrs<br>
>>>> --run-pass=si-fix-sgpr-copies -o - %s | FileCheck %s<br>
>>>> --check-prefixes=COMMON,NO-ADDR64<br>
>>>> +<br>
>>>> +# Test that we correctly legalize VGPR Rsrc operands in MUBUF<br>
>>>> instructions.<br>
>>>> +#<br>
>>>> +# On ADDR64 hardware we optimize the _ADDR64 and _OFFSET cases to<br>
>>>> avoid<br>
>>>> +# needing a waterfall. For all other instruction variants, and<br>
>>>> when we are<br>
>>>> +# on non-ADDR64 hardware, we emit a waterfall loop.<br>
>>>> +<br>
>>>> +# COMMON-LABEL: name: idxen<br>
>>>> +# COMMON-LABEL: bb.0:<br>
>>>> +# COMMON-NEXT: successors: %bb.1({{.*}})<br>
>>>> +# COMMON: [[VRSRC:%[0-9]+]]:vreg_128 = REG_SEQUENCE %0,<br>
>>>> %subreg.sub0, %1, %subreg.sub1, %2, %subreg.sub2, %3, %subreg.sub3<br>
>>>> +# COMMON: [[SAVEEXEC:%[0-9]+]]:sreg_64_xexec = S_MOV_B64 $exec<br>
>>>> +# COMMON-LABEL: bb.1:<br>
>>>> +# COMMON-NEXT: successors: %bb.1({{.*}}), %bb.2({{.*}})<br>
>>>> +# COMMON: [[SRSRC0:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub0, implicit $exec<br>
>>>> +# COMMON: [[SRSRC1:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub1, implicit $exec<br>
>>>> +# COMMON: [[SRSRC2:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub2, implicit $exec<br>
>>>> +# COMMON: [[SRSRC3:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub3, implicit $exec<br>
>>>> +# COMMON: [[SRSRC:%[0-9]+]]:sreg_128 = REG_SEQUENCE [[SRSRC0]],<br>
>>>> %subreg.sub0, [[SRSRC1]], %subreg.sub1, [[SRSRC2]], %subreg.sub2,<br>
>>>> [[SRSRC3]], %subreg.sub3<br>
>>>> +# COMMON: [[CMP0:%[0-9]+]]:sreg_64 = V_CMP_EQ_U64_e64<br>
>>>> [[SRSRC]].sub0_sub1, [[VRSRC]].sub0_sub1, implicit $exec<br>
>>>> +# COMMON: [[CMP1:%[0-9]+]]:sreg_64 = V_CMP_EQ_U64_e64<br>
>>>> [[SRSRC]].sub2_sub3, [[VRSRC]].sub2_sub3, implicit $exec<br>
>>>> +# COMMON: [[CMP:%[0-9]+]]:sreg_64 = S_AND_B64 [[CMP0]], [[CMP1]],<br>
>>>> implicit-def $scc<br>
>>>> +# COMMON: [[TMPEXEC:%[0-9]+]]:sreg_64 = S_AND_SAVEEXEC_B64 killed<br>
>>>> [[CMP]], implicit-def $exec, implicit-def $scc, implicit $exec<br>
>>>> +# COMMON: {{[0-9]+}}:vgpr_32 = BUFFER_LOAD_FORMAT_X_IDXEN %4,<br>
>>>> killed [[SRSRC]], 0, 0, 0, 0, 0, implicit $exec<br>
>>>> +# COMMON: $exec = S_XOR_B64_term $exec, [[TMPEXEC]], implicit-def<br>
>>>> $scc<br>
>>>> +# COMMON: S_CBRANCH_EXECNZ %bb.1, implicit $exec<br>
>>>> +# COMMON-LABEL bb.2:<br>
>>>> +# COMMON: $exec = S_MOV_B64 [[SAVEEXEC]]<br>
>>>> +---<br>
>>>> +name: idxen<br>
>>>> +liveins:<br>
>>>> + - { reg: '$vgpr0', virtual-reg: '%0' }<br>
>>>> + - { reg: '$vgpr1', virtual-reg: '%1' }<br>
>>>> + - { reg: '$vgpr2', virtual-reg: '%2' }<br>
>>>> + - { reg: '$vgpr3', virtual-reg: '%3' }<br>
>>>> + - { reg: '$vgpr4', virtual-reg: '%4' }<br>
>>>> + - { reg: '$sgpr30_sgpr31', virtual-reg: '%5' }<br>
>>>> +body: |<br>
>>>> + bb.0:<br>
>>>> + liveins: $vgpr0, $vgpr1, $vgpr2, $vgpr3, $vgpr4,<br>
>>>> $sgpr30_sgpr31<br>
>>>> + %5:sreg_64 = COPY $sgpr30_sgpr31<br>
>>>> + %4:vgpr_32 = COPY $vgpr4<br>
>>>> + %3:vgpr_32 = COPY $vgpr3<br>
>>>> + %2:vgpr_32 = COPY $vgpr2<br>
>>>> + %1:vgpr_32 = COPY $vgpr1<br>
>>>> + %0:vgpr_32 = COPY $vgpr0<br>
>>>> + %6:sreg_128 = REG_SEQUENCE %0, %subreg.sub0, %1,<br>
>>>> %subreg.sub1, %2, %subreg.sub2, %3, %subreg.sub3<br>
>>>> + %7:vgpr_32 = BUFFER_LOAD_FORMAT_X_IDXEN %4, killed %6, 0, 0,<br>
>>>> 0, 0, 0, implicit $exec<br>
>>>> + $sgpr30_sgpr31 = COPY %5<br>
>>>> + $vgpr0 = COPY %7<br>
>>>> + S_SETPC_B64_return $sgpr30_sgpr31, implicit $vgpr0<br>
>>>> +...<br>
>>>> +<br>
>>>> +# COMMON-LABEL: name: offen<br>
>>>> +# COMMON-LABEL: bb.0:<br>
>>>> +# COMMON-NEXT: successors: %bb.1({{.*}})<br>
>>>> +# COMMON: [[VRSRC:%[0-9]+]]:vreg_128 = REG_SEQUENCE %0,<br>
>>>> %subreg.sub0, %1, %subreg.sub1, %2, %subreg.sub2, %3, %subreg.sub3<br>
>>>> +# COMMON: [[SAVEEXEC:%[0-9]+]]:sreg_64_xexec = S_MOV_B64 $exec<br>
>>>> +# COMMON-LABEL: bb.1:<br>
>>>> +# COMMON-NEXT: successors: %bb.1({{.*}}), %bb.2({{.*}})<br>
>>>> +# COMMON: [[SRSRC0:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub0, implicit $exec<br>
>>>> +# COMMON: [[SRSRC1:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub1, implicit $exec<br>
>>>> +# COMMON: [[SRSRC2:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub2, implicit $exec<br>
>>>> +# COMMON: [[SRSRC3:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub3, implicit $exec<br>
>>>> +# COMMON: [[SRSRC:%[0-9]+]]:sreg_128 = REG_SEQUENCE [[SRSRC0]],<br>
>>>> %subreg.sub0, [[SRSRC1]], %subreg.sub1, [[SRSRC2]], %subreg.sub2,<br>
>>>> [[SRSRC3]], %subreg.sub3<br>
>>>> +# COMMON: [[CMP0:%[0-9]+]]:sreg_64 = V_CMP_EQ_U64_e64<br>
>>>> [[SRSRC]].sub0_sub1, [[VRSRC]].sub0_sub1, implicit $exec<br>
>>>> +# COMMON: [[CMP1:%[0-9]+]]:sreg_64 = V_CMP_EQ_U64_e64<br>
>>>> [[SRSRC]].sub2_sub3, [[VRSRC]].sub2_sub3, implicit $exec<br>
>>>> +# COMMON: [[CMP:%[0-9]+]]:sreg_64 = S_AND_B64 [[CMP0]], [[CMP1]],<br>
>>>> implicit-def $scc<br>
>>>> +# COMMON: [[TMPEXEC:%[0-9]+]]:sreg_64 = S_AND_SAVEEXEC_B64 killed<br>
>>>> [[CMP]], implicit-def $exec, implicit-def $scc, implicit $exec<br>
>>>> +# COMMON: {{[0-9]+}}:vgpr_32 = BUFFER_LOAD_FORMAT_X_OFFEN %4,<br>
>>>> killed [[SRSRC]], 0, 0, 0, 0, 0, implicit $exec<br>
>>>> +# COMMON: $exec = S_XOR_B64_term $exec, [[TMPEXEC]], implicit-def<br>
>>>> $scc<br>
>>>> +# COMMON: S_CBRANCH_EXECNZ %bb.1, implicit $exec<br>
>>>> +# COMMON-LABEL bb.2:<br>
>>>> +# COMMON: $exec = S_MOV_B64 [[SAVEEXEC]]<br>
>>>> +---<br>
>>>> +name: offen<br>
>>>> +liveins:<br>
>>>> + - { reg: '$vgpr0', virtual-reg: '%0' }<br>
>>>> + - { reg: '$vgpr1', virtual-reg: '%1' }<br>
>>>> + - { reg: '$vgpr2', virtual-reg: '%2' }<br>
>>>> + - { reg: '$vgpr3', virtual-reg: '%3' }<br>
>>>> + - { reg: '$vgpr4', virtual-reg: '%4' }<br>
>>>> + - { reg: '$sgpr30_sgpr31', virtual-reg: '%5' }<br>
>>>> +body: |<br>
>>>> + bb.0:<br>
>>>> + liveins: $vgpr0, $vgpr1, $vgpr2, $vgpr3, $vgpr4,<br>
>>>> $sgpr30_sgpr31<br>
>>>> + %5:sreg_64 = COPY $sgpr30_sgpr31<br>
>>>> + %4:vgpr_32 = COPY $vgpr4<br>
>>>> + %3:vgpr_32 = COPY $vgpr3<br>
>>>> + %2:vgpr_32 = COPY $vgpr2<br>
>>>> + %1:vgpr_32 = COPY $vgpr1<br>
>>>> + %0:vgpr_32 = COPY $vgpr0<br>
>>>> + %6:sreg_128 = REG_SEQUENCE %0, %subreg.sub0, %1,<br>
>>>> %subreg.sub1, %2, %subreg.sub2, %3, %subreg.sub3<br>
>>>> + %7:vgpr_32 = BUFFER_LOAD_FORMAT_X_OFFEN %4, killed %6, 0, 0,<br>
>>>> 0, 0, 0, implicit $exec<br>
>>>> + $sgpr30_sgpr31 = COPY %5<br>
>>>> + $vgpr0 = COPY %7<br>
>>>> + S_SETPC_B64_return $sgpr30_sgpr31, implicit $vgpr0<br>
>>>> +...<br>
>>>> +<br>
>>>> +# COMMON-LABEL: name: bothen<br>
>>>> +# COMMON-LABEL: bb.0:<br>
>>>> +# COMMON-NEXT: successors: %bb.1({{.*}})<br>
>>>> +# COMMON: [[VRSRC:%[0-9]+]]:vreg_128 = REG_SEQUENCE %0,<br>
>>>> %subreg.sub0, %1, %subreg.sub1, %2, %subreg.sub2, %3, %subreg.sub3<br>
>>>> +# COMMON: [[SAVEEXEC:%[0-9]+]]:sreg_64_xexec = S_MOV_B64 $exec<br>
>>>> +# COMMON-LABEL: bb.1:<br>
>>>> +# COMMON-NEXT: successors: %bb.1({{.*}}), %bb.2({{.*}})<br>
>>>> +# COMMON: [[SRSRC0:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub0, implicit $exec<br>
>>>> +# COMMON: [[SRSRC1:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub1, implicit $exec<br>
>>>> +# COMMON: [[SRSRC2:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub2, implicit $exec<br>
>>>> +# COMMON: [[SRSRC3:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub3, implicit $exec<br>
>>>> +# COMMON: [[SRSRC:%[0-9]+]]:sreg_128 = REG_SEQUENCE [[SRSRC0]],<br>
>>>> %subreg.sub0, [[SRSRC1]], %subreg.sub1, [[SRSRC2]], %subreg.sub2,<br>
>>>> [[SRSRC3]], %subreg.sub3<br>
>>>> +# COMMON: [[CMP0:%[0-9]+]]:sreg_64 = V_CMP_EQ_U64_e64<br>
>>>> [[SRSRC]].sub0_sub1, [[VRSRC]].sub0_sub1, implicit $exec<br>
>>>> +# COMMON: [[CMP1:%[0-9]+]]:sreg_64 = V_CMP_EQ_U64_e64<br>
>>>> [[SRSRC]].sub2_sub3, [[VRSRC]].sub2_sub3, implicit $exec<br>
>>>> +# COMMON: [[CMP:%[0-9]+]]:sreg_64 = S_AND_B64 [[CMP0]], [[CMP1]],<br>
>>>> implicit-def $scc<br>
>>>> +# COMMON: [[TMPEXEC:%[0-9]+]]:sreg_64 = S_AND_SAVEEXEC_B64 killed<br>
>>>> [[CMP]], implicit-def $exec, implicit-def $scc, implicit $exec<br>
>>>> +# COMMON: {{[0-9]+}}:vgpr_32 = BUFFER_LOAD_FORMAT_X_BOTHEN %4,<br>
>>>> killed [[SRSRC]], 0, 0, 0, 0, 0, implicit $exec<br>
>>>> +# COMMON: $exec = S_XOR_B64_term $exec, [[TMPEXEC]], implicit-def<br>
>>>> $scc<br>
>>>> +# COMMON: S_CBRANCH_EXECNZ %bb.1, implicit $exec<br>
>>>> +# COMMON-LABEL bb.2:<br>
>>>> +# COMMON: $exec = S_MOV_B64 [[SAVEEXEC]]<br>
>>>> +---<br>
>>>> +name: bothen<br>
>>>> +liveins:<br>
>>>> + - { reg: '$vgpr0', virtual-reg: '%0' }<br>
>>>> + - { reg: '$vgpr1', virtual-reg: '%1' }<br>
>>>> + - { reg: '$vgpr2', virtual-reg: '%2' }<br>
>>>> + - { reg: '$vgpr3', virtual-reg: '%3' }<br>
>>>> + - { reg: '$vgpr4_vgpr5', virtual-reg: '%4' }<br>
>>>> + - { reg: '$sgpr30_sgpr31', virtual-reg: '%5' }<br>
>>>> +body: |<br>
>>>> + bb.0:<br>
>>>> + liveins: $vgpr0, $vgpr1, $vgpr2, $vgpr3, $vgpr4,<br>
>>>> $sgpr30_sgpr31<br>
>>>> + %5:sreg_64 = COPY $sgpr30_sgpr31<br>
>>>> + %4:vreg_64 = COPY $vgpr4_vgpr5<br>
>>>> + %3:vgpr_32 = COPY $vgpr3<br>
>>>> + %2:vgpr_32 = COPY $vgpr2<br>
>>>> + %1:vgpr_32 = COPY $vgpr1<br>
>>>> + %0:vgpr_32 = COPY $vgpr0<br>
>>>> + %6:sreg_128 = REG_SEQUENCE %0, %subreg.sub0, %1,<br>
>>>> %subreg.sub1, %2, %subreg.sub2, %3, %subreg.sub3<br>
>>>> + %7:vgpr_32 = BUFFER_LOAD_FORMAT_X_BOTHEN %4, killed %6, 0, 0,<br>
>>>> 0, 0, 0, implicit $exec<br>
>>>> + $sgpr30_sgpr31 = COPY %5<br>
>>>> + $vgpr0 = COPY %7<br>
>>>> + S_SETPC_B64_return $sgpr30_sgpr31, implicit $vgpr0<br>
>>>> +...<br>
>>>> +<br>
>>>> +# COMMON-LABEL: name: addr64<br>
>>>> +# COMMON-LABEL: bb.0:<br>
>>>> +# COMMON: %12:vreg_64 = COPY %8.sub0_sub1<br>
>>>> +# COMMON: %13:sreg_64 = S_MOV_B64 0<br>
>>>> +# COMMON: %14:sgpr_32 = S_MOV_B32 0<br>
>>>> +# COMMON: %15:sgpr_32 = S_MOV_B32 61440<br>
>>>> +# COMMON: %16:sreg_128 = REG_SEQUENCE %13, %subreg.sub0_sub1,<br>
>>>> %14, %subreg.sub2, %15, %subreg.sub3<br>
>>>> +# COMMON: %9:vgpr_32 = V_ADD_I32_e32 %12.sub0, %4.sub0,<br>
>>>> implicit-def $vcc, implicit $exec<br>
>>>> +# COMMON: %10:vgpr_32 = V_ADDC_U32_e32 %12.sub1, %4.sub1,<br>
>>>> implicit-def $vcc, implicit $vcc, implicit $exec<br>
>>>> +# COMMON: %11:vreg_64 = REG_SEQUENCE %9, %subreg.sub0, %10,<br>
>>>> %subreg.sub1<br>
>>>> +# COMMON: {{[0-9]+}}:vgpr_32 = BUFFER_LOAD_FORMAT_X_ADDR64 %11,<br>
>>>> killed %16, 0, 0, 0, 0, 0, implicit $exec<br>
>>>> +---<br>
>>>> +name: addr64<br>
>>>> +liveins:<br>
>>>> + - { reg: '$vgpr0', virtual-reg: '%0' }<br>
>>>> + - { reg: '$vgpr1', virtual-reg: '%1' }<br>
>>>> + - { reg: '$vgpr2', virtual-reg: '%2' }<br>
>>>> + - { reg: '$vgpr3', virtual-reg: '%3' }<br>
>>>> + - { reg: '$vgpr4_vgpr5', virtual-reg: '%4' }<br>
>>>> + - { reg: '$sgpr30_sgpr31', virtual-reg: '%5' }<br>
>>>> +body: |<br>
>>>> + bb.0:<br>
>>>> + liveins: $vgpr0, $vgpr1, $vgpr2, $vgpr3, $vgpr4,<br>
>>>> $sgpr30_sgpr31<br>
>>>> + %5:sreg_64 = COPY $sgpr30_sgpr31<br>
>>>> + %4:vreg_64 = COPY $vgpr4_vgpr5<br>
>>>> + %3:vgpr_32 = COPY $vgpr3<br>
>>>> + %2:vgpr_32 = COPY $vgpr2<br>
>>>> + %1:vgpr_32 = COPY $vgpr1<br>
>>>> + %0:vgpr_32 = COPY $vgpr0<br>
>>>> + %6:sreg_128 = REG_SEQUENCE %0, %subreg.sub0, %1,<br>
>>>> %subreg.sub1, %2, %subreg.sub2, %3, %subreg.sub3<br>
>>>> + %7:vgpr_32 = BUFFER_LOAD_FORMAT_X_ADDR64 %4, killed %6, 0, 0,<br>
>>>> 0, 0, 0, implicit $exec<br>
>>>> + $sgpr30_sgpr31 = COPY %5<br>
>>>> + $vgpr0 = COPY %7<br>
>>>> + S_SETPC_B64_return $sgpr30_sgpr31, implicit $vgpr0<br>
>>>> +...<br>
>>>> +<br>
>>>> +# COMMON-LABEL: name: offset<br>
>>>> +# COMMON-LABEL: bb.0:<br>
>>>> +<br>
>>>> +# NO-ADDR64-NEXT: successors: %bb.1({{.*}})<br>
>>>> +# NO-ADDR64: [[VRSRC:%[0-9]+]]:vreg_128 = REG_SEQUENCE %0,<br>
>>>> %subreg.sub0, %1, %subreg.sub1, %2, %subreg.sub2, %3, %subreg.sub3<br>
>>>> +# NO-ADDR64: [[SAVEEXEC:%[0-9]+]]:sreg_64_xexec = S_MOV_B64 $exec<br>
>>>> +# NO-ADDR64-LABEL: bb.1:<br>
>>>> +# NO-ADDR64-NEXT: successors: %bb.1({{.*}}), %bb.2({{.*}})<br>
>>>> +# NO-ADDR64: [[SRSRC0:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub0, implicit $exec<br>
>>>> +# NO-ADDR64: [[SRSRC1:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub1, implicit $exec<br>
>>>> +# NO-ADDR64: [[SRSRC2:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub2, implicit $exec<br>
>>>> +# NO-ADDR64: [[SRSRC3:%[0-9]+]]:sgpr_32 = V_READFIRSTLANE_B32<br>
>>>> [[VRSRC]].sub3, implicit $exec<br>
>>>> +# NO-ADDR64: [[SRSRC:%[0-9]+]]:sreg_128 = REG_SEQUENCE<br>
>>>> [[SRSRC0]], %subreg.sub0, [[SRSRC1]], %subreg.sub1, [[SRSRC2]],<br>
>>>> %subreg.sub2, [[SRSRC3]], %subreg.sub3<br>
>>>> +# NO-ADDR64: [[CMP0:%[0-9]+]]:sreg_64 = V_CMP_EQ_U64_e64<br>
>>>> [[SRSRC]].sub0_sub1, [[VRSRC]].sub0_sub1, implicit $exec<br>
>>>> +# NO-ADDR64: [[CMP1:%[0-9]+]]:sreg_64 = V_CMP_EQ_U64_e64<br>
>>>> [[SRSRC]].sub2_sub3, [[VRSRC]].sub2_sub3, implicit $exec<br>
>>>> +# NO-ADDR64: [[CMP:%[0-9]+]]:sreg_64 = S_AND_B64 [[CMP0]],<br>
>>>> [[CMP1]], implicit-def $scc<br>
>>>> +# NO-ADDR64: [[TMPEXEC:%[0-9]+]]:sreg_64 = S_AND_SAVEEXEC_B64<br>
>>>> killed [[CMP]], implicit-def $exec, implicit-def $scc, implicit<br>
>>>> $exec<br>
>>>> +# NO-ADDR64: {{[0-9]+}}:vgpr_32 = BUFFER_LOAD_FORMAT_X_OFFSET<br>
>>>> killed [[SRSRC]], 0, 0, 0, 0, 0, implicit $exec<br>
>>>> +# NO-ADDR64: $exec = S_XOR_B64_term $exec, [[TMPEXEC]],<br>
>>>> implicit-def $scc<br>
>>>> +# NO-ADDR64: S_CBRANCH_EXECNZ %bb.1, implicit $exec<br>
>>>> +# NO-ADDR64-LABEL bb.2:<br>
>>>> +# NO-ADDR64: $exec = S_MOV_B64 [[SAVEEXEC]]<br>
>>>> +<br>
>>>> +# ADDR64: [[VRSRC:%[0-9]+]]:vreg_128 = REG_SEQUENCE %0,<br>
>>>> %subreg.sub0, %1, %subreg.sub1, %2, %subreg.sub2, %3, %subreg.sub3<br>
>>>> +# ADDR64: [[RSRCPTR:%[0-9]+]]:vreg_64 = COPY [[VRSRC]].sub0_sub1<br>
>>>> +# ADDR64: [[ZERO64:%[0-9]+]]:sreg_64 = S_MOV_B64 0<br>
>>>> +# ADDR64: [[RSRCFMTLO:%[0-9]+]]:sgpr_32 = S_MOV_B32 0<br>
>>>> +# ADDR64: [[RSRCFMTHI:%[0-9]+]]:sgpr_32 = S_MOV_B32 61440<br>
>>>> +# ADDR64: [[ZERORSRC:%[0-9]+]]:sreg_128 = REG_SEQUENCE<br>
>>>> [[ZERO64]], %subreg.sub0_sub1, [[RSRCFMTLO]], %subreg.sub2,<br>
>>>> [[RSRCFMTHI]], %subreg.sub3<br>
>>>> +# ADDR64: [[VADDR64:%[0-9]+]]:vreg_64 = REG_SEQUENCE<br>
>>>> [[RSRCPTR]].sub0, %subreg.sub0, [[RSRCPTR]].sub1, %subreg.sub1<br>
>>>> +# ADDR64: {{[0-9]+}}:vgpr_32 = BUFFER_LOAD_FORMAT_X_ADDR64<br>
>>>> [[VADDR64]], [[ZERORSRC]], 0, 0, 0, 0, 0, implicit $exec<br>
>>>> +<br>
>>>> +---<br>
>>>> +name: offset<br>
>>>> +liveins:<br>
>>>> + - { reg: '$vgpr0', virtual-reg: '%0' }<br>
>>>> + - { reg: '$vgpr1', virtual-reg: '%1' }<br>
>>>> + - { reg: '$vgpr2', virtual-reg: '%2' }<br>
>>>> + - { reg: '$vgpr3', virtual-reg: '%3' }<br>
>>>> + - { reg: '$vgpr4_vgpr5', virtual-reg: '%4' }<br>
>>>> + - { reg: '$sgpr30_sgpr31', virtual-reg: '%5' }<br>
>>>> +body: |<br>
>>>> + bb.0:<br>
>>>> + liveins: $vgpr0, $vgpr1, $vgpr2, $vgpr3, $vgpr4,<br>
>>>> $sgpr30_sgpr31<br>
>>>> + %5:sreg_64 = COPY $sgpr30_sgpr31<br>
>>>> + %4:vreg_64 = COPY $vgpr4_vgpr5<br>
>>>> + %3:vgpr_32 = COPY $vgpr3<br>
>>>> + %2:vgpr_32 = COPY $vgpr2<br>
>>>> + %1:vgpr_32 = COPY $vgpr1<br>
>>>> + %0:vgpr_32 = COPY $vgpr0<br>
>>>> + %6:sreg_128 = REG_SEQUENCE %0, %subreg.sub0, %1,<br>
>>>> %subreg.sub1, %2, %subreg.sub2, %3, %subreg.sub3<br>
>>>> + %7:vgpr_32 = BUFFER_LOAD_FORMAT_X_OFFSET killed %6, 0, 0, 0,<br>
>>>> 0, 0, implicit $exec<br>
>>>> + $sgpr30_sgpr31 = COPY %5<br>
>>>> + $vgpr0 = COPY %7<br>
>>>> + S_SETPC_B64_return $sgpr30_sgpr31, implicit $vgpr0<br>
>>>> +...<br>
>>>> <br>
>>>> _______________________________________________<br>
>>>> llvm-commits mailing list<br>
>>>> <a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
>>>> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a> [8]<br>
>> <br>
>> <br>
>> Links:<br>
>> ------<br>
>> [1] <a href="http://llvm.org/viewvc/llvm-project?rev=341413&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=341413&view=rev</a><br>
>> [2] <a href="https://reviews.llvm.org/D50982" rel="noreferrer" target="_blank">https://reviews.llvm.org/D50982</a><br>
>> [3]<br>
>> <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.cpp?rev=341413&r1=341412&r2=341413&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.cpp?rev=341413&r1=341412&r2=341413&view=diff</a><br>
>> [4]<br>
>> <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.h?rev=341413&r1=341412&r2=341413&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.h?rev=341413&r1=341412&r2=341413&view=diff</a><br>
>> [5]<br>
>> <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.td?rev=341413&r1=341412&r2=341413&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.td?rev=341413&r1=341412&r2=341413&view=diff</a><br>
>> [6]<br>
>> <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll?rev=341413&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll?rev=341413&view=auto</a><br>
>> [7]<br>
>> <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.mir?rev=341413&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AMDGPU/mubuf-legalize-operands.mir?rev=341413&view=auto</a><br>
>> [8] <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
>> [9]<br>
>> <a href="http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/12238" rel="noreferrer" target="_blank">http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/12238</a><br>
</blockquote></div>