<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Sep 15, 2020, at 5:01 AM, Evgeny Leviant <<a href="mailto:eleviant@accesssoftek.com" class="">eleviant@accesssoftek.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="WordSection1" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">Sorry, it seems I have figured out the answer myself:</span></div><p class="MsoNormal" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class=""> </span></p><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">Instruction</span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">$x0, $x2 = LDRXpre $x0, 1</span></div><p class="MsoNormal" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class=""> </span></p><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">will have 4 arguments, so it seems possible to assign both SchedRead and SchedWrite for $x0 and the result sched</span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">list for LDRXpre would be:</span></div><p class="MsoNormal" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class=""> </span></p><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">[WriteAdr, WriteLD, ReadAdr]</span></div><p class="MsoNormal" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class=""> </span></p><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">Strange that AArch64InstrFormats.td doesn’t implement this</span></div></div></div></blockquote><div><br class=""></div><div>Looking at AArch64InstrFormats.td, it has the writeback operand in a different order. </div><div>Sched<[WriteLD, WriteAdr]>;</div><div><br class=""></div><div>If that’s incorrect, it might be worth fixing.</div><div><br class=""></div><div>Otherwise, that looks like the right answer. The address writeback is not really load forwarding. It's a totally separate scheduling resource with its own latency.</div><div><br class=""></div><div>-Andy</div><br class=""><blockquote type="cite" class=""><div class="WordSection1" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0cm 0cm;" class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif; border: none; padding: 0cm;" class=""><b class="">From:<span class="Apple-converted-space"> </span></b><a href="mailto:eleviant@accesssoftek.com" style="color: blue; text-decoration: underline;" class="">Evgeny Leviant</a><br class=""><b class="">Sent:<span class="Apple-converted-space"> </span></b>15 сентября 2020 г. 14:24<br class=""><b class="">To:<span class="Apple-converted-space"> </span></b><a href="mailto:atrick@apple.com" style="color: blue; text-decoration: underline;" class="">Andrew Trick</a><br class=""><b class="">Cc:<span class="Apple-converted-space"> </span></b><a href="mailto:llvm-dev@lists.llvm.org" style="color: blue; text-decoration: underline;" class="">llvm-dev@lists.llvm.org</a><br class=""><b class="">Subject:<span class="Apple-converted-space"> </span></b>RE: [EXTERNAL] Re: [llvm-dev] Simulation of load-store forwarding with MI scheduler on AArch64</div></div><p class="MsoNormal" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> </p></div><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><div class="WordSection1"><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">Thanks for prompt response, Andy</span></div><p class="MsoNormal" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> </p><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">This will work for cases when address is not modified. However this doesn’t seem to work for pre/post increment load stores.</span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">Consider data to address forwarding:</span></div><p class="MsoNormal" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class=""> </span></p><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">$x0 = ldr x0, [x1]</span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">$x0, $x2 = ldr x2, [x0, 16]!</span></div><p class="MsoNormal" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class=""> </span></p><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">The second instruction will have it’s own latency for address modification ($x0 register). So I don’t see how we can use ReadAdr stuff</span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">here. May be forwarding is not supposed to work in such cases for ARM cpus? Cortex-A55 software optimization guide says this:</span></div><p class="MsoNormal" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class=""> </span></p><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">“</span>load data from a limited set of load instructions can be forwarded from the beginning of the wr pipeline stage to either the load or store AGU base operand<span lang="EN-US" class="">”</span></div><p class="MsoNormal" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class=""> </span></p><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">However nothing is said about pre/post indexed forms.</span></div><p class="MsoNormal" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> </p><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0cm 0cm;" class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif; border: none; padding: 0cm;" class=""><b class="">From:<span class="Apple-converted-space"> </span></b><a href="mailto:atrick@apple.com" style="color: blue; text-decoration: underline;" class="">Andrew Trick</a><br class=""><b class="">Sent:<span class="Apple-converted-space"> </span></b>15 сентября 2020 г. 7:04<br class=""><b class="">To:<span class="Apple-converted-space"> </span></b><a href="mailto:eleviant@accesssoftek.com" style="color: blue; text-decoration: underline;" class="">Evgeny Leviant</a><br class=""><b class="">Cc:<span class="Apple-converted-space"> </span></b><a href="mailto:llvm-dev@lists.llvm.org" style="color: blue; text-decoration: underline;" class="">llvm-dev@lists.llvm.org</a><br class=""><b class="">Subject:<span class="Apple-converted-space"> </span></b>[EXTERNAL] Re: [llvm-dev] Simulation of load-store forwarding with MI scheduler on AArch64</div></div><p class="MsoNormal" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> </p></div><div class=""><div style="font-size: 9pt; font-family: Calibri, sans-serif;" class=""><h3 style="background-color: rgb(255, 255, 255); font-size: 10pt; border: 1px dotted rgb(0, 51, 51); padding: 0.8em;" class=""><span style="color: rgb(255, 102, 0);" class="">CAUTION:<strong class=""> </strong></span>This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If you suspect potential phishing or spam email, report it to <a href="mailto:ReportSpam@accesssoftek.com" class="">ReportSpam@accesssoftek.com</a></h3></div><div class=""><br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Sep 14, 2020, at 9:40 AM, Evgeny Leviant via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="" style="color: blue; text-decoration: underline;">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="WordSection1" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;"><div class="" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class="">Hi list,</span></div><div class="" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class=""> </span></div><div class="" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class="">Is it possible to simulate load to store forwarding on aarch64 with MI scheduling model on AArch64?</span></div><div class="" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class="">For instance $x0 data latency in the example below should be 1 cycle</span></div><div class="" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class=""> </span></div><div class="" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class="">ldr $x0, [$x1]</span></div><div class="" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class="">str $x0, [$x2]</span></div><div class="" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class=""> </span></div><div class="" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class="">But it should be 4 cycles if we have another instruction:</span></div><div class="" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class=""> </span></div><div class="" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class="">ldr $x0, [$x1]</span></div><div class="" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class="">add $x0, $x0, 4</span></div><div class="" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class=""> </span></div><div class="" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class="">For ALU instructions it’s possible to use either ReadAdvance or SchedReadAdvance, but I don’t see how</span></div><div class="" style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" class="">to do this with WriteLD or WriteST. Is there some workaround?</span></div></div></div></blockquote><br class=""></div><div class=""><div class="">The main purpose of ReadAdvance is pipeline forwarding.</div><div class=""><br class=""></div><div class="">I think you can just want a read resource in your subtarget like this:</div><div class=""><br class=""></div><div class=""> def ReadAdr : SchedReadAdvance<3, [WriteLD]></div><div class=""><br class=""></div><div class="">Briefly glancing at the AArch64 target I see this for stores:</div><div class=""><br class=""></div><div class=""> Sched<[WriteST]>;</div><div class=""><br class=""></div><div class="">So it doesn't look like there's any existing name for the store’s address operand. You could add a general ReadAdr SchedRead resource</div><div class="">in AArch64Schedule.td. Then you would need to change the ReadAdr line in your subtarget to an override:</div><div class=""><br class=""></div><div class=""> def : ReadAdvance<ReadAdr, 3, [WriteLD]></div><div class=""><br class=""></div><div class="">Or instead you can just add a rule in your subtarget listing the opcodes or using a regex, and using the ReadAdr resource that you defined in the same file.</div><div class=""><br class=""></div><div class=""> def : InstRW<[WriteST, ReadAdr], (instregex "ST(someregex)$")>;</div><div class=""><br class=""></div><div class="">Being careful about store-pair and vector stores.</div><div class=""><br class=""></div><div class="">Then you always want to debug your target’s llvm-tblgen command by adding a flag</div><div class="">-debug-only=subtarget-emitter</div><div class=""><br class=""></div><div class="">And even trace the schedule for some simple cases with -debug-only=machine-scheduler</div><div class=""><br class=""></div><div class="">I haven't actually done any of this in several years, someone with more recent experience may have better tips.</div><div class=""><br class=""></div></div>-Andy</div></div></div></blockquote></div><br class=""></body></html>