<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Oct 3, 2019 at 7:01 AM Finkel, Hal J. <<a href="mailto:hfinkel@anl.gov" target="_blank">hfinkel@anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>On 10/2/19 5:12 PM, Hal Finkel wrote:<br></p>
<blockquote type="cite">
<div>On 10/1/19 12:35 AM, Serge Pavlov via llvm-dev wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr"><div>The main concern about such approach is performance drop. Using constrained FP operations means that optimizations on FP operations are turned off, this is the main reason of using them. Even if non-default FP environment is used in a small piece of a function,
optimizations are turned off in entire function. For many practical application this is unacceptable.<br>
</div>
</div>
</blockquote>
<p>The reason, as you're likely aware, that the constrained FP operations must be used within the entire function is that, if you mix the constrained FP operations with the normal ones, there's no way to prevent code motion from intermixing them.</p></blockquote></div></blockquote><div>This proposal presents a way to prevent such intermixing. In some basic block we use normal FP operations, in others - constrained, BB attributes allows to check validity of instruction moves.</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF"><blockquote type="cite"><p> The solution
I recall being discussed to this problem of a function which requires constrained operations only in part is outlining in Clang - this does introduce function-call overhead (although perhaps some MI-level inlining pass could mitigate that in part), but otherwise
permits normal optimization of the normal FP operations.<br></p>
</blockquote>
<p>Johannes and I discussed the outlining here offline, and two notes:<br></p>
<p> 1. The outlining itself will prevent the undesired code motion today, but in the future we'll have IPO transformations that will need to be specifically taught to avoid moving FP operations into these outlined functions.</p>
<p> 2. The outlined functions will need to be marked with noinline and also noimplicitfloat. In fact, all functions using the constrained intrinsics might need to be marked with noimplicitfloat. The above-mentioned restrictions on IPO passes might be conditioned
on the noimplicitfloat attribute.</p></div></blockquote><div><br></div><div>Outlining is an interesting solution but unfortunately it is not an option for processors for machine learning. Branching is expensive on them and some processors do not have call instruction, all function calls must be eventually inlined. On the other hand rounding control is especially important in such processors, as they usually operate short data types and using proper rounding mode can gain precision. They often allow encoding rounding mode in instruction and making a call just to execute a couple of instructions is not acceptable.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF">
<blockquote type="cite"><blockquote type="cite"><div dir="ltr"><div>Although this approach prevents from moving instructions, it does not prevent from moving basic blocks. The code that uses non-default FP environment at some point must set appropriate state registers, do necessary operations and then restore the original mode.
If this activity is scattered by several basic blocks, block-level optimizations can break these arrangement, for instance a basic block with default FP operations can be moved after the block that sets non-default FP environment.<br>
</div>
</div>
</blockquote>
<p>Can you please provide some pseudocode to illustrate this problem? Moving basic blocks moves the instructions within them, and I don't see how our current semantics would prevent illegal reorderings of the instructions but not prevent illegal reorderings
of groups of those same instructions. At the LLVM level, we currently model the FP-environment state as a kind of memory, and so the operations which adjust the FP-environment state must also be marked as writing to memory, but that's true with essentially
all external program state, and that should prevent all illegal reordering.</p></blockquote></div></blockquote><div> </div><div>Let' consider a transformation like LoopUnswitch. The source:</div><div><br></div><div>for (int i = 0; i < N, ++i) {</div><div> #pragma STDC FENV_ACCESS</div><div> set_fp_environment(X);</div><div> if (i > K)</div><div> some_func();</div><div> // Basic block that calculates condition starts here.</div><div> bool f = float_a < float_b;</div><div> if (f)</div><div> do_1(i);</div><div> else</div><div> do_2(i);</div><div>}</div><div><br></div><div>As basic block that calculates condition `f` does not depend on values calculated in the loop, it can be hoisted:</div><div><br></div><div><div>bool f = float_a < float_b;</div><div>if (f) {</div><div> for (int i = 0; i < N, ++i) {</div><div> #pragma STDC FENV_ACCESS</div><div> set_fp_environment(X);</div><div></div><div> if (i > K)</div><div> some_func();</div><div> do_1(i);<br></div><div> }<br></div><div></div><div>} else {</div><div> for (int i = 0; i < N, ++i) {</div><div> #pragma STDC FENV_ACCESS</div><div> set_fp_environment(X);</div><div></div><div> if (i > K)</div><div> some_func();</div><div> do_2(i);<br></div><div> }<br></div><div></div><div></div><div>}</div><div><br></div><div>Nothing prevents from moving the BB that calculates condition. The BB being moved does not have data dependencies that prohibit such relocation. The code does not adjust the FP-environment state so may be moved ahead of `set_fp_environment`. But the transformed code has different semantics, as `f` is calculated in different FP environment. To prevent from such transformations we would need to consider all FP operations as accessing FP state modeled as memory. It would prevent from any code reordering and result in performance drop.</div><div></div></div></div></div>