[llvm-dev] InlineSpiller - hoists leave virtual registers without live intervals

Tue Nov 5 15:44:47 PST 2019

> On Nov 5, 2019, at 11:41 AM, Wei Mi <wmi at google.com> wrote:
> 
> 
> 
> On Mon, Nov 4, 2019 at 6:41 PM <alex.davies at iinet.net.au <mailto:alex.davies at iinet.net.au>> wrote:
> From: Wei Mi <wmi at google.com <mailto:wmi at google.com>> 
> > I havn't noticed before that TII.storeRegToStackSlot is allowed to create new virtual registers. I am surprised to know it is allowed to do that. I am mostly working on x86 and TII.storeRegToStackSlot for x86 doesn't create new register. As you said, if TII.storeRegToStackSlot is allowed to create new virtual registers, that could be a problem. It is not exposed maybe because of some later pass cover the problem -- maybe RegScavenger? 
> 
> > Are you aware of any other architecture other than AMDGPU on which TII.storeRegToStackSlot creates new virtual registers? Could you explain in which case new virtual registers will be created? 
> 
> It is quite possible it's a largely forgotten feature from before RegScavenger was fully featured, however being able to storeRegToStackSlot did solve a major problem for me.
> 
> 
> I see. You expect that TII.storeRegToStackSlot could allow new temporary to be generated to handle the irregular spill easily and the patch http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20130812/184331.html <http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20130812/184331.html> said it did some preparation for that, but looks like in reality no one actually let TII.storeRegToStackSlot generate new temporary yet. I find RegScavenger calls TII.storeRegToStackSlot as well when spill is needed to scavenge a register. If TII.storeRegToStackSlot allows to generate new temporary, that will be a problem for RegScavenger as well (Looks like RegScavenger doesn't work in an iterative way)
> 
> So I guess currently most implementation in llvm assumes TII.storeRegToStackSlot doesn't generate new virtual register usage?
>  
> On the architecture I'm on (and many DSPs like it, I suspect), there's many special function registers that control various aspects of the CPU. These registers can be pushed, popped, and moved to/from GPRs. The issue is that the architecture has no "move from SFR to stack + offset" command, which LLVM requires for stack-frame store/loads. They must go through a GPR intermediate. Even solving the frame address would not avoid this issue, as there is no "move from SFR to [address]" either, they must either be directly pushed/popped from stack, or go through an intermediate.
> 
> Implementing "getLargestLegalSuperClass" to return a RegisterClass of (GPRs + SFRs) goes a long way towards avoiding these difficult spills, as they'll often be assigned a GPR over call-clobbering etc, but unfortunately AFAIK there's no way to tell the register allocator that spills must be from the GPR side of that union. Sometimes, it will request that the SFR gets spilled, and so in those instances you ultimately need a temporary in storeRegToStackSlot. One option would be a pseudo, followed by a pass using RegScavenger to provide the intermediate - but I would expect the codegen to be worse than generating the intermediate in storeRegToStackSlot, if supported.
> 
> Really though, this is all just a workaround to a problem. What's really desired/wanted, is just a way to say that SFRs must be spilled to GPRs, a "getSpillRegClass" if you will. This seems a missing piece/problem in general, whilst trying to find out how to address the problem I came across at least one question to llvm-dev on how to do it, unanswered iirc.
> 
> AMDGPU also seemingly has an extensive/complex workaround in "EnableSpillVGPRToAGPR", implemented in SILowerSGPRSpills, which if I understand it correctly, attempts to simplify similar two-stage spills post-regalloc by eliminating unnecessary frame indices. I'm not familiar with the architecture and may be wrong on the constraints they're working around, but on glancing it over, it seems to me that if a hypothetical "spillRegClass()" could be expressed, AMDGPU too would not need a temporary to be allocated in storeRegToStackSlot, and could also do away with much of SILowerSGPRSpills as well.
> 
>         /// Special case of eliminateFrameIndex. Returns true if the SGPR was spilled to
>         /// a VGPR and the stack slot can be safely eliminated when all other users are
>         /// handled.
>         bool SIRegisterInfo::eliminateSGPRToVGPRSpillFrameIndex
> 
> Please do let me know if I've missed a better way at addressing this, perhaps there is just the function I'm looking for already.
> 
> Yes, EnableSpillVGPRToAGPR seems to work from your description. But I am not familiar with GPU register allocation and have no experience about how this kind of irregular spill is handled in llvm now generally. 
> 
> Quentin, do you have suggestion about how such irregular spill should be handled? 

I think the way it is handled is by having staged register allocation:
1.  Allocate the most constrained class with spill to another but less constrained class
2. Repeat until all the classes are allocated

Hopefully the last class that is spilled is one that can load/store directly on the stack with offsets.

In Alex’ case, that would be, allocate control registers then allocate gprs.

This kind of staged approach is I think what Matt has been working on recently:
commit 5b0922fe1f9dcecfc1f92bec21f1c8f3849daf31
Author: Matt Arsenault <Matthew.Arsenault at amd.com>
Date:   Wed Jul 3 23:32:29 2019 +0000

    AMDGPU: Add pass to lower SGPR spills

    This is split out from my patches to split register allocation into a
    separate SGPR and VGPR phase, and has some parts that aren't yet used
    (like maintaining LiveIntervals).

    This simplifies making the frame pointer register callee saved. As it
    is now, the code to determine callee saves needs to predict all the
    possible SGPR spills and how many callee saved VGPRs are needed. By
    handling this before PrologEpilogInserter, it's possible to just check
    the spill objects that already exist.

    Change-Id: I29e6df4034afcf949e06f8ef44206acb94696f04
    llvm-svn: 365095

>  
> 
> I added Mattias and Matthew to the thread as well because they are very experienced on GPU register allocation.
> 
> Thanks,
> Wei.
>  
> 
> Thank you,
> Alex
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191105/a966abf0/attachment-0001.html>