[llvm-dev] [RFC] Using basic block attributes to implement non-default floating point environment

[Serge Pavlov via llvm-dev]

On Thu, Oct 3, 2019 at 7:01 AM Finkel, Hal J. <hfinkel at anl.gov> wrote:

> On 10/2/19 5:12 PM, Hal Finkel wrote:
>
> On 10/1/19 12:35 AM, Serge Pavlov via llvm-dev wrote:
>
> The main concern about such approach is performance drop. Using
> constrained FP operations means that optimizations on FP operations are
> turned off, this is the main reason of using them. Even if non-default FP
> environment is used in a small piece of a function, optimizations are
> turned off in entire function. For many practical application this is
> unacceptable.
>
> The reason, as you're likely aware, that the constrained FP operations
> must be used within the entire function is that, if you mix the constrained
> FP operations with the normal ones, there's no way to prevent code motion
> from intermixing them.
>
> This proposal presents a way to prevent such intermixing. In some basic
block we use normal FP operations, in others - constrained, BB attributes
allows to check validity of instruction moves.

> The solution I recall being discussed to this problem of a function which
> requires constrained operations only in part is outlining in Clang - this
> does introduce function-call overhead (although perhaps some MI-level
> inlining pass could mitigate that in part), but otherwise permits normal
> optimization of the normal FP operations.
>
> Johannes and I discussed the outlining here offline, and two notes:
>
>  1. The outlining itself will prevent the undesired code motion today, but
> in the future we'll have IPO transformations that will need to be
> specifically taught to avoid moving FP operations into these outlined
> functions.
>
>  2. The outlined functions will need to be marked with noinline and also
> noimplicitfloat. In fact, all functions using the constrained intrinsics
> might need to be marked with noimplicitfloat. The above-mentioned
> restrictions on IPO passes might be conditioned on the noimplicitfloat
> attribute.
>

Outlining is an interesting solution but unfortunately it is not an option
for processors for machine learning. Branching is expensive on them and
some processors do not have call instruction, all function calls must be
eventually inlined. On the other hand rounding control is especially
important in such processors, as they usually operate short data types and
using proper rounding mode can gain precision. They often allow encoding
rounding mode in instruction and making a call just to execute a couple of
instructions is not acceptable.

Although this approach prevents from moving instructions, it does not
> prevent from moving basic blocks. The code that uses non-default FP
> environment at some point must set appropriate state registers, do
> necessary operations and then restore the original mode. If this activity
> is scattered by several basic blocks, block-level optimizations can break
> these arrangement, for instance a basic block with default FP operations
> can be moved after the block that sets non-default FP environment.
>
> Can you please provide some pseudocode to illustrate this problem? Moving
> basic blocks moves the instructions within them, and I don't see how our
> current semantics would prevent illegal reorderings of the instructions but
> not prevent illegal reorderings of groups of those same instructions. At
> the LLVM level, we currently model the FP-environment state as a kind of
> memory, and so the operations which adjust the FP-environment state must
> also be marked as writing to memory, but that's true with essentially all
> external program state, and that should prevent all illegal reordering.
>
>
Let' consider a transformation like LoopUnswitch. The source:

for (int i = 0; i < N, ++i) {
    #pragma STDC FENV_ACCESS
    set_fp_environment(X);
    if (i > K)
        some_func();
    // Basic block that calculates condition starts here.
    bool f = float_a < float_b;
    if (f)
        do_1(i);
    else
        do_2(i);
}

As basic block that calculates condition `f` does not depend on values
calculated in the loop, it can be hoisted:

bool f = float_a < float_b;
if (f) {
    for (int i = 0; i < N, ++i) {
        #pragma STDC FENV_ACCESS
        set_fp_environment(X);
        if (i > K)
            some_func();
        do_1(i);
    }
} else {
    for (int i = 0; i < N, ++i) {
        #pragma STDC FENV_ACCESS
        set_fp_environment(X);
        if (i > K)
            some_func();
        do_2(i);
    }
}

Nothing prevents from moving the BB that calculates condition. The BB being
moved does not have data dependencies that prohibit such relocation. The
code does not adjust the FP-environment state so may be moved ahead of
`set_fp_environment`. But the transformed code has different semantics, as
`f` is calculated in different FP environment. To prevent from such
transformations we would need to consider all FP operations as accessing FP
state modeled as memory. It would prevent from any code reordering and
result in performance drop.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191003/ffbf79ea/attachment.html>