<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div></div><div><br></div><div>* Phabricator </div><div><div style="margin: 0px; font-family: Menlo;"><br></div></div><div style="margin: 0px; font-family: Menlo;"><a href="http://reviews.llvm.org/D4367">http://reviews.llvm.org/D4367</a></div><div style="margin: 0px; font-family: Menlo;"><br></div><div>Perhaps I looked at the wrong place, but I didn’t find people I thought I could send the review. Please add yourself if you are interested.</div><div><br></div><div>* Motivation + Example</div><div><br></div><div>The opportunity for this optimization is across the llvm test suite and benchmarks. </div><div><br></div><div>Specific example: SingleSource/Benchmarks/Shootout/matrix (compiled with O3 flto for AArch64 gives a >20% gain):</div><div><br></div><div>Current assembly snippet:</div><div><span style="font-family: Menlo;"><br></span></div><div><span style="font-family: Menlo;">0000000100007d24</span><span class="Apple-tab-span" style="font-family: Menlo; white-space: pre;"> </span><span style="font-family: Menlo;"><font color="#4f7a28">mul</font> </span><span style="font-family: Menlo;">w6, w23, w6</span><span class="Apple-tab-span" style="font-family: Menlo; white-space: pre;"> </span><span style="font-family: Menlo;">// Chain of madds</span></div><div><div style="margin: 0px; font-family: Menlo;">0000000100007d28 <font color="#ff6a00">madd</font> w5, w7, w5, w6<span class="Apple-tab-span" style="white-space:pre"> </span>// All multiplies on critical path!</div><div style="margin: 0px; font-family: Menlo;">0000000100007d2c ldp w6, w7, [x4, #8]<span class="Apple-tab-span" style="white-space:pre"> </span></div><div style="margin: 0px; font-family: Menlo;">0000000100007d30 ldr w23, [x11, x2]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d34 <font color="#11053b">madd</font> w5, w23, w6, w5</div><div style="margin: 0px; font-family: Menlo;">0000000100007d38 ldr w6, [x12, x2]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d3c madd w5, w6, w7, w5</div><div style="margin: 0px; font-family: Menlo;">0000000100007d40 ldr w6, [x13, x2]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d44 ldp w7, w23, [x4, #16]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d48 madd w5, w6, w7, w5</div><div style="margin: 0px; font-family: Menlo;"><div style="margin: 0px;">0000000100007d4c ldr w6, [x14, x2]</div><div style="margin: 0px;">0000000100007d50 madd w5, w6, w23, w5</div><div style="margin: 0px;">0000000100007d54 ldr w6, [x15, x2]</div><div style="margin: 0px;">0000000100007d58 ldp w7, w23, [x4, #24]</div><div style="margin: 0px;">0000000100007d5c madd w5, w6, w7, w5</div><div style="margin: 0px;">0000000100007d60 ldr w6, [x16, x2]</div><div style="margin: 0px;">0000000100007d64 madd w5, w6, w23, w5</div><div style="margin: 0px;">0000000100007d68 ldr w6, [x17, x2]</div><div style="margin: 0px;">0000000100007d6c ldp w7, w23, [x4, #32]</div><div style="margin: 0px;">0000000100007d70 ldr w24, [x0, x2]</div><div style="margin: 0px;">0000000100007d74 madd w5, w6, w7, w5</div><div><br></div></div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>…</div></div><div>With machine combiner the multiplies can execute in parallel shortening the critical path (>20% gain):</div><div><br></div><div><br></div><div style="margin: 0px; font-family: Menlo;">0000000100007cf4 <font color="#4f7a28">mul</font> w5, w7, w5<span class="Apple-tab-span" style="white-space:pre"> </span>// Multiplies can execute in parallel</div><div style="margin: 0px; font-family: Menlo;">0000000100007cf8 ldp w7, w23, [x4, #8]<span class="Apple-tab-span" style="white-space:pre"> </span>// off critical path</div><div style="margin: 0px; font-family: Menlo;">0000000100007cfc ldr w24, [x10, x2]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d00 <font color="#ff6a00">mul</font> w6, w24, w6</div><div style="margin: 0px; font-family: Menlo;">0000000100007d04 ldr w24, [x11, x2]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d08 mul w7, w24, w7</div><div style="margin: 0px; font-family: Menlo;">0000000100007d0c ldr w24, [x12, x2]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d10 mul w23, w24, w23</div><div style="margin: 0px; font-family: Menlo;">0000000100007d14 ldr w24, [x13, x2]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d18 <font color="#d95000">add </font> w5, w6, w5</div><div style="margin: 0px; font-family: Menlo;">0000000100007d1c ldp w6, w25, [x4, #16]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d20 mul w6, w24, w6</div><div style="margin: 0px; font-family: Menlo;">0000000100007d24 ldr w24, [x14, x2]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d28 mul w24, w24, w25</div><div style="margin: 0px; font-family: Menlo;">0000000100007d2c ldr w25, [x15, x2]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d30 add w5, w7, w5</div><div style="margin: 0px; font-family: Menlo;">0000000100007d34 ldp w7, w26, [x4, #24]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d38 mul w7, w25, w7</div><div style="margin: 0px; font-family: Menlo;">0000000100007d3c ldr w25, [x16, x2]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d40 mul w25, w25, w26</div><div style="margin: 0px; font-family: Menlo;">0000000100007d44 ldr w26, [x17, x2]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d48 add w5, w23, w5</div><div style="margin: 0px; font-family: Menlo;">0000000100007d4c ldp w23, w27, [x4, #32]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d50 mul w23, w26, w23</div><div style="margin: 0px; font-family: Menlo;">0000000100007d54 ldr w26, [x0, x2]</div><div style="margin: 0px; font-family: Menlo;">0000000100007d58 mul w26, w26, w27</div><div style="margin: 0px; font-family: Menlo;">0000000100007d5c add w5, w6, w5</div><div><span class="Apple-tab-span" style="white-space:pre"> </span> ….</div><div><br></div><div>* Algorithm in more detail</div><div><br></div><div>0. Looks for patterns within a basic block</div><div>1. For each instruction check if it can be combined with other instructions. For each combination provide a (opaque) pattern in a list.</div><div>2. For each pattern in the list generate the alternative instruction sequence. This sequence is owned by the machine function, but not hooked up to the basic block etc. This way of creating and evaluating alternative machine instructions results in negligible extra memory use.</div><div>3. Evaluate if a new pattern is more efficient. It is more efficient</div><div>a) when the new pattern has fewer instructions in Os</div><div>b) when neither critical path nor resource length increases</div><div>4. Replace the old instructions when the new sequence is more efficient. the</div><div><br></div><div><br></div><div>* Side-effects and risks</div><div><br></div><div>- Patterns that are evaluated in the machine combiner are no longer combined by the DAG combiner. The logic changes from “always combine” to combine only when beneficial at the machine level. Specifically for any pattern supported by the machine combiner is blocked for the DAG combiner.</div><div>- The new code sequence can increase register pressure and results in additional spill/fill code, eg. at function entry. So it can make difference whether instruction combining happens at the DAG or the machine level. The reason is that different optimizations kick in at the DAG level and the code can look different at later machine level. One interesting case I spotted is the unit test DivRem where the madd substitution results in two live ranges for the constant -3. It increases register pressure by one and gives an extra stp/ldp at entry/exit. A possible future remedy is the fusion of equivalent live ranges in special cases. </div><div><br></div><div>* Shortcomings</div><div><br></div><div>- Pattern are limited to simple combinations like mul+add. But it is possible enhance the implementation to support combining for more complex sequences mul, add , add, add,… .</div><div>- Local scope: The local basic block scope could be extended to trace scope. </div><div>- Modeling accuracy: Dynamic latencies could be significantly larger than modeled (eg. for loads) so a madd could be generated that should not be. This is a general risk with any machine scheduling technique.</div><div><br></div><div>* Acknowledgements</div><div> </div><div>Arnold provided an early inspiring prototype. Yi’s excellent analysis outlined the potential benefits. Andy convinced me of a more general critical path analysis. </div><div><br></div><div><br></div><div>Thanks</div><div>Gerolf</div><div><br></div><div><div>On Jun 2, 2014, at 3:41 PM, Gerolf Hoflehner <<a href="mailto:ghoflehner@apple.com">ghoflehner@apple.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">Hi,<br><br>we noticed that combining instructions at the ISEL level can result in sub-optimal schedules, since an expensive instruction could be folded and lengthen the critical path. A simple illustrative example for this is folding mul-mul-add into mul-madd (fused multiply add), which serializes two multiplications instead of allowing them to execute in parallel. Unfortunately ISEL does not have the information necessary to avoid sub-optimial cases in all instances. The alternative is to post-pone instruction combining to a later pass and decide on whether the combined code sequence performs better. This combiner pass would run at the machine IR level, use the machine trace model information, and roughly implement the following architecture independent algorithm initially targeting mul-add/sub pattern:<br><br>* Initial Instruction Combiner Algorithm <br>0. Recognize instruction patterns in a single basic block<br>1. For each add/sub check if it can be combined to madd/msub<br>2. For each pattern found, generate the alternative instruction sequence outside the basic block that contains mul - add/sub<br>3. Evaluate if the new pattern is more efficient:<br>a) In Os substitute when the new pattern has fewer instructions<br>b) Otherwise substitute when neither critical path nor resource length increases based on the machine trace model<br><br>Do you have any concerns? Or is this the pass you always wanted? :-) <br><br>Cheers<br>Gerolf<br><br><br><br><br><br><br><br></blockquote></div><br></body></html>