<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Nov 5, 2019, at 11:41 AM, Wei Mi <<a href="mailto:wmi@google.com" class="">wmi@google.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div dir="ltr" class=""><br class=""></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Nov 4, 2019 at 6:41 PM <<a href="mailto:alex.davies@iinet.net.au" class="">alex.davies@iinet.net.au</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">From: Wei Mi <<a href="mailto:wmi@google.com" target="_blank" class="">wmi@google.com</a>> <br class="">
> I havn't noticed before that TII.storeRegToStackSlot is allowed to create new virtual registers. I am surprised to know it is allowed to do that. I am mostly working on x86 and TII.storeRegToStackSlot for x86 doesn't create new register. As you said, if TII.storeRegToStackSlot is allowed to create new virtual registers, that could be a problem. It is not exposed maybe because of some later pass cover the problem -- maybe RegScavenger? <br class="">
<br class="">
> Are you aware of any other architecture other than AMDGPU on which TII.storeRegToStackSlot creates new virtual registers? Could you explain in which case new virtual registers will be created? <br class="">
<br class="">
It is quite possible it's a largely forgotten feature from before RegScavenger was fully featured, however being able to storeRegToStackSlot did solve a major problem for me.<br class="">
<br class=""></blockquote><div class=""><br class=""></div><div class="">I see. You expect that TII.storeRegToStackSlot could allow new temporary to be generated to handle the irregular spill easily and the patch <a href="http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20130812/184331.html" class="">http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20130812/184331.html</a> said it did some preparation for that, but looks like in reality no one actually let TII.storeRegToStackSlot generate new temporary yet. I find RegScavenger calls TII.storeRegToStackSlot as well when spill is needed to scavenge a register. If TII.storeRegToStackSlot allows to generate new temporary, that will be a problem for RegScavenger as well (Looks like RegScavenger doesn't work in an iterative way)</div><div class=""><br class=""></div><div class="">So I guess currently most implementation in llvm assumes TII.storeRegToStackSlot doesn't generate new virtual register usage?</div>
<div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
On the architecture I'm on (and many DSPs like it, I suspect), there's many special function registers that control various aspects of the CPU. These registers can be pushed, popped, and moved to/from GPRs. The issue is that the architecture has no "move from SFR to stack + offset" command, which LLVM requires for stack-frame store/loads. They must go through a GPR intermediate. Even solving the frame address would not avoid this issue, as there is no "move from SFR to [address]" either, they must either be directly pushed/popped from stack, or go through an intermediate.<br class="">
<br class="">
Implementing "getLargestLegalSuperClass" to return a RegisterClass of (GPRs + SFRs) goes a long way towards avoiding these difficult spills, as they'll often be assigned a GPR over call-clobbering etc, but unfortunately AFAIK there's no way to tell the register allocator that spills must be from the GPR side of that union. Sometimes, it will request that the SFR gets spilled, and so in those instances you ultimately need a temporary in storeRegToStackSlot. One option would be a pseudo, followed by a pass using RegScavenger to provide the intermediate - but I would expect the codegen to be worse than generating the intermediate in storeRegToStackSlot, if supported.<br class="">
<br class="">
Really though, this is all just a workaround to a problem. What's really desired/wanted, is just a way to say that SFRs must be spilled to GPRs, a "getSpillRegClass" if you will. This seems a missing piece/problem in general, whilst trying to find out how to address the problem I came across at least one question to llvm-dev on how to do it, unanswered iirc.<br class="">
<br class="">
AMDGPU also seemingly has an extensive/complex workaround in "EnableSpillVGPRToAGPR", implemented in SILowerSGPRSpills, which if I understand it correctly, attempts to simplify similar two-stage spills post-regalloc by eliminating unnecessary frame indices. I'm not familiar with the architecture and may be wrong on the constraints they're working around, but on glancing it over, it seems to me that if a hypothetical "spillRegClass()" could be expressed, AMDGPU too would not need a temporary to be allocated in storeRegToStackSlot, and could also do away with much of SILowerSGPRSpills as well.<br class="">
<br class="">
/// Special case of eliminateFrameIndex. Returns true if the SGPR was spilled to<br class="">
/// a VGPR and the stack slot can be safely eliminated when all other users are<br class="">
/// handled.<br class="">
bool SIRegisterInfo::eliminateSGPRToVGPRSpillFrameIndex<br class="">
<br class="">
Please do let me know if I've missed a better way at addressing this, perhaps there is just the function I'm looking for already.<br class=""></blockquote><div class=""><br class=""></div><div class="">Yes, EnableSpillVGPRToAGPR seems to work from your description. But I am not familiar with GPU register allocation and have no experience about how this kind of irregular spill is handled in llvm now generally. </div><div class=""><br class=""></div><div class="">Quentin, do you have suggestion about how such irregular spill should be handled? </div></div></div></div></blockquote><div><br class=""></div><div>I think the way it is handled is by having staged register allocation:</div><div>1. Allocate the most constrained class with spill to another but less constrained class</div><div>2. Repeat until all the classes are allocated</div><div><br class=""></div><div>Hopefully the last class that is spilled is one that can load/store directly on the stack with offsets.</div><div><br class=""></div><div>In Alex’ case, that would be, allocate control registers then allocate gprs.</div><div><br class=""></div><div>This kind of staged approach is I think what Matt has been working on recently:</div><div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; color: rgb(159, 160, 28);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">commit 5b0922fe1f9dcecfc1f92bec21f1c8f3849daf31</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Author: Matt Arsenault <<a href="mailto:Matthew.Arsenault@amd.com" class="">Matthew.Arsenault@amd.com</a>></span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Date: Wed Jul 3 23:32:29 2019 +0000</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; min-height: 13px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""></span><br class=""></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> AMDGPU: Add pass to lower SGPR spills</span></div><p style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; min-height: 13px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> </span><br class="webkit-block-placeholder"></p><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> This is </span><span style="font-variant-ligatures: no-common-ligatures; color: #ffffff; background-color: #000000" class="">split</span><span style="font-variant-ligatures: no-common-ligatures" class=""> out from my patches to </span><span style="font-variant-ligatures: no-common-ligatures; color: #ffffff; background-color: #000000" class="">split</span><span style="font-variant-ligatures: no-common-ligatures" class=""> register allocation into a</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> separate SGPR and VGPR phase, and has some parts that aren't yet used</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> (like maintaining LiveIntervals).</span></div><p style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; min-height: 13px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> </span><br class="webkit-block-placeholder"></p><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> This simplifies making the frame pointer register callee saved. As it</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> is now, the code to determine callee saves needs to predict all the</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> possible SGPR spills and how many callee saved VGPRs are needed. By</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> handling this before PrologEpilogInserter, it's possible to just check</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> the spill objects that already exist.</span></div><p style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; min-height: 13px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> </span><br class="webkit-block-placeholder"></p><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Change-Id: I29e6df4034afcf949e06f8ef44206acb94696f04</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> llvm-svn: 365095</span></div></div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_quote"><div class=""> </div><div class=""><br class=""></div><div class="">I added Mattias and Matthew to the thread as well because they are very experienced on GPU register allocation.</div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">Wei.</div><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br class="">
Thank you,<br class="">
Alex<br class="">
<br class="">
</blockquote></div></div>
</div></blockquote></div><br class=""></body></html>