<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle21
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">(Speaking only for myself here, and mostly as someone who doesn’t typically write floating-point-heavy code).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The root issue we have here is that, as with many compiler extensions, fast-math flags ends up creating a vaguely-defined variant of the C specification governed by the “obvious” semantics, and is the case with “obvious” semantics, there
are several different “obvious” results.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Given the standard C taste for undefined behavior, it would seem to me that the most
<i>natural</i> definition of -ffinite-math-only would be to say that any operation that produces NaN or infinity results is undefined behavior, or produces a poison value using LLVM’s somewhat tighter definition here [1]. This notably doesn’t give a clear answer
on what to do with floating-point operations that don’t produce floating-point results (e.g., casts, comparison operators), and the volume of discussions on this point is I think indicative that there are multiple reasonable options here. Personally, I find
the extension of the UB to cases that consume but do not produce floating-point values to be the most natural option.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">It’s also the case that many users don’t like undefined behavior as a concept, in large part because it can be very difficult to work around in a few cases where it is desired to explicitly override the undefined behavior. For some of the
more basic integer UB, clang already provides builtin overflow checking macros to handle the I-want-to-check-if-it-overflowed-without-UB case, for example. And if fast math flags are to create UB, then similar functionality to override the floating-point UB
ought to be provided. Already, C provides a mechanism to twiddle floating-point behavior on a per-scope basis (e.g., #pragma STDC FENV_ACCESS, CX_LIMITED_RANGE, FP_CONTRACT). LLVM already supports these flags on a per-instruction basis, so it really shouldn’t
be very difficult to have Clang support pragmas to twiddle fast-math flags like the existing C pragmas. And in this model, the -ffast-math and related flags are doing nothing more than setting the default values of these pragmas.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">In that vein, I can imagine a user writing a program that would look something like this:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:Consolas">int some_hard_math_kernel(float *inputs, float *outputs, int N) {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas"> {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas"> #pragma clang fast_math off<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas"> for (int i = 0; i < N; i++) {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas"> if (isinf(inputs[i]) || isnan(inputs[i]))<br>
return ILLEGAL_ARGUMENT;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas"> }<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas"> }<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas"> #pragma clang fast_math on<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas"> // Do fancy math here…<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas"> // and if we see isnan(x) here, even if it’s in a library routine [compiled with -ffast-math],<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas"> // or maybe implied by some operation the compiler understands [say, complex multiplication]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas"> // it is optimized to false.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas"> return SUCCESS;<br>
}<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I can clearly see use cases where the programmer might wish to have the optimizer eliminate any isnan calls that are generated when -ffast-math is used, but like other UB, I think it is extremely beneficial to provide
<i>some</i> way to explicitly opt-out of UB on a case-by-case basis.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I would even go so far as to suggest that maybe the C standards committee should discuss how to handle at least the nsz/nnan/ninf parts of fast-math flags, given that very similar concepts seem to exist in all of the major C/C++ compilers.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">[1] I fully expect any user who is knowledgeable about poison in LLVM—which admittedly is a fairly expert user—would expect poison to kick in most of the time C or C++ provides for undefined behavior, and potentially to rely on that expectation.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> llvm-dev <llvm-dev-bounces@lists.llvm.org> <b>On Behalf Of
</b>Mehdi AMINI via llvm-dev<br>
<b>Sent:</b> Thursday, September 9, 2021 14:29<br>
<b>To:</b> Serge Pavlov <sepavloff@gmail.com><br>
<b>Cc:</b> llvm-dev@lists.llvm.org; cfe-dev@lists.llvm.org<br>
<b>Subject:</b> Re: [llvm-dev] [cfe-dev] Should isnan be optimized out in fast-math mode?<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Thu, Sep 9, 2021 at 10:34 AM Serge Pavlov via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org">cfe-dev@lists.llvm.org</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<p class="MsoNormal">Let me describe a real life example.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">There is a realtime program that processes float values from a huge array. Calculations do not produce NaNs and do not expect them. Using -ffinite-math-only substantially speeds up the program, so it is highly desirable to use it. The problem
is that the array contains NaNs, they mark elements that should not be processed.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">An obvious solution is to check an element for NaN, and if it is not, process it. Now there is no clean way to do so. Only workarounds, like using integer arithmetics. The function 'isnan' became useless. And there are many cases when users
complain of this optimization.<o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I personally would separate the "pre-processing" of the input in a compilation unit that isn't compiled with -ffinite-math-only and isolate the perf-critical routines to be compiled with this flag if needed (I'd also like a sanitizer to
have a build mode that validate that no NaNs are ever seen in this routines).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">In general, Krzysztof's reasoning in this thread makes sense to me, in particular in terms of being consistent with how we treat isnan(x) vs isnan(x+0) for example.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">-- <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Mehdi<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal">--Serge<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Fri, Sep 10, 2021 at 12:09 AM Chris Tetreault <<a href="mailto:ctetreau@quicinc.com" target="_blank">ctetreau@quicinc.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">If the issue is that users want their asserts to fire, then they should be encouraged to only enable fast math in release builds.
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>From:</b> llvm-dev <<a href="mailto:llvm-dev-bounces@lists.llvm.org" target="_blank">llvm-dev-bounces@lists.llvm.org</a>>
<b>On Behalf Of </b>Serge Pavlov via llvm-dev<br>
<b>Sent:</b> Thursday, September 9, 2021 9:53 AM<br>
<b>To:</b> Krzysztof Parzyszek <<a href="mailto:kparzysz@quicinc.com" target="_blank">kparzysz@quicinc.com</a>><br>
<b>Cc:</b> LLVM Developers <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>>;
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<b>Subject:</b> Re: [llvm-dev] [cfe-dev] Should isnan be optimized out in fast-math mode?<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p align="center" style="text-align:center"><strong><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black;background:yellow">WARNING:</span></strong><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black;background:yellow">
This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.</span><o:p></o:p></p>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Thu, Sep 9, 2021 at 11:29 PM Krzysztof Parzyszek <<a href="mailto:kparzysz@quicinc.com" target="_blank">kparzysz@quicinc.com</a>> wrote:<o:p></o:p></p>
</div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">This goes back to what these options actually imply. The interpretation that I favor is “this code will never see a NaN”, or “the program can assume that no floating point expression
will evaluate to a NaN”. The benefit of that is that it’s intuitively clear. In that case “isnan(x)” is false, because x cannot be a NaN. There is no distinction between “isnan(x+x)” and “isnan(x)”. If the user wants to preserve “isnan(x)”, they can apply
some pragma (which clang may actually have already).<o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">It is apparent simplicity. As the discussion in gcc mail list demonstrated (<a href="https://gcc.gnu.org/pipermail/gcc-patches/2020-April/544641.html" target="_blank">https://gcc.gnu.org/pipermail/gcc-patches/2020-April/544641.html</a>)
this is actually an impromissing way. From a practical viewpoint it is also a bad solution as users cannot even check the assertions.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">To be honest, I’m not sure that I understand your argument. Are you saying that under your interpretation we could optimize “isnan(x+x) -> false”, but not “isnan(x) -> false”?<o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Argument of `isnan(x+x)` is a result of arithmetic operation. According to the meaning of -ffinite-math-only it cannot produce NaN. So this call can be optimized out. In the general
case `isnan(x)` value may be, say, loaded from memory. Load is not an arithmetic operation, so nothing prevents from loading NaN. Optimizing the call out is dangerous in this case.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:9.0pt;font-family:Consolas">--
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:8.0pt;font-family:Consolas">Krzysztof Parzyszek
<a href="mailto:kparzysz@quicinc.com" target="_blank"><span style="color:#0563C1">kparzysz@quicinc.com</span></a> AI tools development</span><o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>From:</b> Serge Pavlov <<a href="mailto:sepavloff@gmail.com" target="_blank">sepavloff@gmail.com</a>>
<br>
<b>Sent:</b> Thursday, September 9, 2021 11:10 AM<br>
<b>To:</b> Krzysztof Parzyszek <<a href="mailto:kparzysz@quicinc.com" target="_blank">kparzysz@quicinc.com</a>><br>
<b>Cc:</b> Chris Lattner <<a href="mailto:clattner@nondot.org" target="_blank">clattner@nondot.org</a>>; James Y Knight <<a href="mailto:jyknight@google.com" target="_blank">jyknight@google.com</a>>; LLVM Developers <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>>;
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<b>Subject:</b> Re: [cfe-dev] [llvm-dev] Should isnan be optimized out in fast-math mode?<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p align="center" style="text-align:center"><strong><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black;background:yellow">WARNING:</span></strong><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black;background:yellow">
This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.</span><o:p></o:p></p>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Thu, Sep 9, 2021 at 8:30 PM Krzysztof Parzyszek via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>> wrote:<o:p></o:p></p>
</div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">If we say that the fast-math flags are “enabling optimizations that the presence of nans otherwise prohibits”, then there is no reason for clang to keep calls to “isnan” around,
or to keep checks like “fpclassify(x) == it’s_a_nan” unfolded. These are exactly the types of optimizations that the presence of NaNs would prohibit.<o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Transformation 'x * 0 -> 0' is an optimization allowed in the absence of nans as arguments, because it produces a program that behaves identically under the given restrictions.
Replacement of `isnan(x + x)` is also an optimization under the same restrictions. Replacement of `isnan(x)` in general case is not, because we cannot assume that x cannot be a NaN.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I understand the need for having some NaN-handling preserved in an otherwise finite-math code. We already have fast-math-related attributes attached to each function in the LLVM
IR, so we could introduce a source-level attribute for enabling/disabling these flags per function.<o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">GCC allows using `#pragma GCC optimize ("finite-math-only")` or `#pragma GCC optimize ("no-finite-math-only")` to enable/disable optimization per function basis. Clang could support
this pragmf or maybe `#pragma clang fp` can be extended to support similar functionality.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:9.0pt;font-family:Consolas">--
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:8.0pt;font-family:Consolas">Krzysztof Parzyszek
<a href="mailto:kparzysz@quicinc.com" target="_blank"><span style="color:#0563C1">kparzysz@quicinc.com</span></a> AI tools development</span><o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>From:</b> cfe-dev <<a href="mailto:cfe-dev-bounces@lists.llvm.org" target="_blank">cfe-dev-bounces@lists.llvm.org</a>>
<b>On Behalf Of </b>Chris Lattner via cfe-dev<br>
<b>Sent:</b> Wednesday, September 8, 2021 5:51 PM<br>
<b>To:</b> James Y Knight <<a href="mailto:jyknight@google.com" target="_blank">jyknight@google.com</a>><br>
<b>Cc:</b> LLVM Developers <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>>; Clang Dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [cfe-dev] [llvm-dev] Should isnan be optimized out in fast-math mode?<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p align="center" style="text-align:center"><strong><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black;background:yellow">WARNING:</span></strong><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black;background:yellow">
This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.</span><o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Sep 8, 2021, at 3:27 PM, James Y Knight via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<o:p></o:p></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I expressed my strong support for this on the previous thread, but I'll just repost the most important piece...
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I believe the proposed semantics from the Clang level ought to be:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> The -ffinite-math-only and -fno-signed-zeros options do not impact the ability to accurately load, store, copy, or pass or return such values from general function calls. They
also do not impact any of the "non-computational" and "quiet-computational" IEEE-754 operations, which includes classification functions (fpclassify, signbit, isinf/isnan/etc), sign-modification (copysign, fabs, and negation `-(x)`), as well as the totalorder and
totalordermag functions. Those correctly handle NaN, Inf, and signed zeros even when the flags are in effect. These flags <i>do</i> affect the behavior of other expressions and math standard-library calls, as well as comparison operations.<o:p></o:p></p>
</div>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">FWIW, I completely agree - these flags are about enabling optimizations that the presence of nans otherwise prohibits. We shouldn’t take a literal interpretation of an old GCC
manual, as that would not be useful.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">If we converge on this definition, I think it should be documented. This is a source of confusion that comes up periodically.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">-Chris<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"> <o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I would not expect this to have an actual negative impact on the performance benefit of those flags, since the optimization benefits mainly arise from comparisons and the general
computation instructions which are unchanged.<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">In further support of this position, I note that the previous thread uncovered at least one vendor -- Apple (<a href="https://opensource.apple.com/source/Libm/Libm-2026/Source/Intel/math.h.auto.html" target="_blank">https://opensource.apple.com/source/Libm/Libm-2026/Source/Intel/math.h.auto.html</a>)
-- going out of their way to cause isnan and friends to function properly with -ffast-math enabled.<o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"> <o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Wed, Sep 8, 2021 at 1:02 PM Serge Pavlov via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi all,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">One of the purposes of `llvm::isnan` was to help preserve the check made by `isnan` if fast-math mode is<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">specified (<a href="https://reviews.llvm.org/D104854" target="_blank">https://reviews.llvm.org/D104854</a>). I'd like to describe reason for that and propose to use the behavior<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">implemented in that patch.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The option `-ffast-math` is often used when performance is important, as it allows a compiler to generate faster code.
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">This option itself is a collection of different optimization techniques, each having its own option. For this topic only the<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">option `-ffinite-math-only` is of interest. With it the compiler treats floating point numbers as mathematical real numbers,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">so transformations like `0 * x -> 0` become valid.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">In clang documentation (<a href="https://clang.llvm.org/docs/UsersManual.html#cmdoption-ffast-math" target="_blank">https://clang.llvm.org/docs/UsersManual.html#cmdoption-ffast-math</a>)
this option is described as:<br>
<br>
"Allow floating-point optimizations that assume arguments and results are not NaNs or +-Inf."<br>
<br>
GCC documentation (<a href="https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html" target="_blank">https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html</a>) is a bit more concrete:<br>
<br>
"Allow optimizations for floating-point arithmetic that assume that arguments and results are not NaNs or +-Infs."<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">**What is the issue?**<br>
<br>
C standard defines a macro `isnan`, which can be mapped to an intrinsic function provided by the compiler. For both<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">clang and gcc it is `__builtin_isnan`. How should this function behave if `-ffinite-math-only` is specified? Should it make a<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">real check or the compiler can assume that it always returns false?<br>
<br>
GCC optimizes out `isnan`. It follows from the viewpoint that (<a href="https://gcc.gnu.org/bugzilla/show_bug.cgi?id=50724#c1" target="_blank">https://gcc.gnu.org/bugzilla/show_bug.cgi?id=50724#c1</a>):<br>
<br>
"With -ffinite-math-only you are telling that there are no NaNs and thus GCC optimizes isnan (x) to 0."<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Such treatment of `-ffinite-math-only` has sufficient drawbacks. In particular it makes it impossible to check validity of<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">data: a user cannot write<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">assert(!isnan(x));<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">because the compiler replaces the actual function call with its expected value. There are many complaints in GCC bug<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">tracker (for instance
<a href="https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84949" target="_blank">https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84949</a> or
<a href="https://gcc.gnu.org/bugzilla/show_bug.cgi?id=50724" target="_blank">https://gcc.gnu.org/bugzilla/show_bug.cgi?id=50724</a>)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">as well as in forums (<a href="https://stackoverflow.com/questions/47703436/isnan-does-not-work-correctly-with-ofast-flags" target="_blank">https://stackoverflow.com/questions/47703436/isnan-does-not-work-correctly-with-ofast-flags</a>
or <br>
<a href="https://stackoverflow.com/questions/22931147/stdisinf-does-not-work-with-ffast-math-how-to-check-for-infinity" target="_blank">https://stackoverflow.com/questions/22931147/stdisinf-does-not-work-with-ffast-math-how-to-check-for-infinity</a>). Proposed<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">solutions are using integer operations to make the check, to turn off `-ffinite-math-only` in some parts of the code or to<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">ensure that libc function is called. It clearly demonstrates that `isnan` in this case is useless, but users need its functionality<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">and do not have a proper tool to make required checks. The similar direction was criticized in llvm as well (<a href="https://reviews.llvm.org/D18513#387418" target="_blank">https://reviews.llvm.org/D18513#387418</a>).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">**Why imposing restrictions on floating types is bad?**<br>
<br>
If `-ffinite-math-only` modifies properties of `double` type, several issues arise, for instance:<br>
- What should return `std::numeric_limits<double>::has_quiet_NaN()`?<br>
- What body should have this function if it is used in a program where some functions are compiled with `fast-math` and some without?<br>
- Should inlining of a function compiled with `fast-math` to a function compiled without it be prohibited in inliner?<br>
- Should `std::isnan(std::numeric_limits<float>::quiet_NaN())` be true?<br>
<br>
If the type `double` cannot have NaN value, it means that `double` and `double` under `-ffinite-math-only` are different types<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">(<a href="https://gcc.gnu.org/pipermail/gcc-patches/2020-April/544641.html" target="_blank">https://gcc.gnu.org/pipermail/gcc-patches/2020-April/544641.html</a>). Such a way can
solve these problems but it is so expensive<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">that hardly it has a chance to be realized.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">**The solution**<br>
<br>
Instead of modifying properties of floating point types, the effect of `-ffinite-math-only` can be expressed as a restriction on<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">operation usage. Actually clang and gcc documentation already follows this way. Fast-math flags in llvm IR also are attributes<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">of instructions. The only question is whether `isnan` and similar functions are floating-point arithmetic.<br>
<br>
From a practical viewpoint, treating non-computational functions as arithmetic does not add any advantage. If a code extensively<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">uses `isnan` (so could profit by their removal), it is likely it is not suitable for -ffinite-math-only. This interpretation however creates<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">the problems described above. So it is profitable to consider `isnan` and similar functions as non-arithmetical.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">**Why is it safe to leave `isnan`?**<br>
<br>
The probable concern of this solution is deviation from gcc behavior. There are several reasons why this is not an issue.<br>
<br>
1. -ffinite-math-only is an optimization option. A correct program compiled with -ffinite-math-only and without it should behave<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> identically, if conditions for using -ffinite-math-only are fulfilled. So making the check cannot break functionality.<br>
2. `isnan` is implemented by libc, which can map it to a compiler builtin or use its own implementation, depending on<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> configuration options. `isnan` implemented in libc obviously always does the real check.<br>
3. ICC and MSVC preserve `isnan` in fast-math mode.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The proposal is to not consider `isnan` and other such functions as arithmetic operations and do not optimize them out<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">just because -ffinite-math-only is specified. Of course, there are cases when `isnan` may be optimized out, for instance,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt">`isnan(a + b)` may be optimized if -ffinite-math-only is in effect due to the assumption (result of arithmetic operation is not NaN).<br>
<br>
What are your opinions?<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks,<br>
--Serge<o:p></o:p></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal">_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</div>
</div>
</body>
</html>