<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal" style="margin-left:.5in">If `has_nan` returns "true", it means that the explanation "there are no NaNs" does not work anymore and something more complex is needed to explain the effect of the option. In this case it is difficult to say
that this approach is "intuitively clear".<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">If your program has “x = *p”, it means that at this point p is never a null pointer. Does this imply that the type of p can no longer represent a null pointer?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Consolas">-- </span>
<span style="font-size:9.0pt;font-family:Consolas"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:Consolas">Krzysztof Parzyszek
<a href="mailto:kparzysz@quicinc.com"><span style="color:#0563C1">kparzysz@quicinc.com</span></a> AI tools development<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> cfe-dev <cfe-dev-bounces@lists.llvm.org> <b>On Behalf Of
</b>Serge Pavlov via cfe-dev<br>
<b>Sent:</b> Tuesday, September 14, 2021 7:04 AM<br>
<b>To:</b> Chris Tetreault <ctetreau@quicinc.com><br>
<b>Cc:</b> llvm-dev@lists.llvm.org; cfe-dev@lists.llvm.org<br>
<b>Subject:</b> Re: [cfe-dev] [llvm-dev] Should isnan be optimized out in fast-math mode?<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p align="center" style="text-align:center"><strong><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black;background:yellow">WARNING:</span></strong><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black;background:yellow">
This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.</span><o:p></o:p></p>
<div>
<div>
<div>
<p class="MsoNormal">On Mon, Sep 13, 2021 at 9:03 PM Krzysztof Parzyszek <<a href="mailto:kparzysz@quicinc.com" target="_blank">kparzysz@quicinc.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">If the compiler provides “isnan”, the user can’t redefine it. Redefining/undefining any function or a macro provided by a compiler is UB.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal">Actually it does not matter. This is needed only to emulate the "old" behavior, which itself breaks the standard.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The “old” behavior can be tuned with #pragmas to restore the functionality of NaNs where needed.<o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Did you mean `#pragma GCC optimize("ffinite-math-only")`? Clang does not support it.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The “old” behavior doesn’t have a problem with “has_nan”---it returns “true”. What other issues are there?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"> If `has_nan` returns "true", it means that the explanation "there are no NaNs" does not work anymore and something more complex is needed to explain the effect of the option. In this case it is difficult to say that this approach is "intuitively
clear".<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">On Mon, Sep 13, 2021 at 10:28 PM Arthur O'Dwyer <<a href="mailto:arthur.j.odwyer@gmail.com" target="_blank">arthur.j.odwyer@gmail.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">Btw, I don't think this thread has paid enough attention to Richard Smith's suggestion:<o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I can only subscribe to James Y Knight's opinion. Indeed, it can be a good criterion of which operations should work in finite-math-only mode and which can not work. The only thing which I worry about is the possibility of checking the
operation result for infinity (and nan for symmetry). But the suggested criterion is formulated in terms of arguments, not results, so it must allow such checks.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Thanks,<br>
--Serge<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Tue, Sep 14, 2021 at 12:50 AM Serge Pavlov <<a href="mailto:sepavloff@gmail.com" target="_blank">sepavloff@gmail.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal">What I'd like to emphasize is that this option was introduced not for logical consistency, but for practical needs. It allows users to get faster code and this is why it is an important option. We are discussing two ways, which are not
equivalent. If `isnan` is unconditionally optimized out, users that need it have to use workarounds, which leads to loss of portability and performance. If `isnan` is preserved, no workarounds are required, simple redefinition results in the "old" behavior.
It seems to me that implementation of this option should pursue practical needs and should enable most use cases. The current implementation does not fit user needs, as it follows from the complaints in gcc bug tracker and forums. We could make clang more
user-friendly if this option would be implemented slightly differently than now.
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">On Mon, Sep 13, 2021 at 11:46 PM Chris Tetreault <<a href="mailto:ctetreau@quicinc.com" target="_blank">ctetreau@quicinc.com</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">… is guaranteed to work, and I read that fast-math enables the compiler to reason about constructs like `x + 0` being equal to `x`, then I’m going to be very confused when:<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">You are right, this was a bad idea. Compiler may optimize out `isnan` but only when it deduces that the value cannot be NaN, but not due to the user's promise. It is especially important for `isinf`. Addition of two finite values may produce
infinity and there is no universal way to predict it. It is probably not an issue for types like float or double, but ML cores use halfs or even minifloats, where overflow is much more probable. If in the code:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">```<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">float r = a + b;<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">if (isinf(r)) {...<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">```<o:p></o:p></p>
</div>
<p class="MsoNormal">`isinf` were optimized out just because -ffinite-math-only is in effect, the user cannot check if overflow did not occur. This contrasts with the definition of `ninf` in LLVM IR:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">"No Infs - Allow optimizations to assume the arguments and result are not +/-Inf."<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">It is possible to ensure that arguments are not Infs but for the result it is much more difficult to guarantee.<o:p></o:p></p>
<div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">Thanks,<br>
--Serge<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Mon, Sep 13, 2021 at 11:46 PM Chris Tetreault <<a href="mailto:ctetreau@quicinc.com" target="_blank">ctetreau@quicinc.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Honestly, we can do this until the end of time. I think we both agree, that for either scheme, there exists workarounds. The question is which workarounds are more palatable, which
is a matter of opinion. I think we’ve come to an impasse, so let me just state that my opinion on the question “Should isnan be optimized out in fast-math mode?” is “Yes”, which is what you asked to get in your original message. I think that the implementation
of fast-math will be cleaner if we don’t special case a bunch of random constructs in order to do what the user meant instead of what they said. I think fast-math is a notorious footgun, and any attempts to mitigate this will only reduce the effectiveness
of the tool, while not really improving the user experience.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">As a user, if I read that:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">```<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">if (isnan(x)) {<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">```<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">… is guaranteed to work, and I read that fast-math enables the compiler to reason about constructs like `x + 0` being equal to `x`, then I’m going to be very confused when:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">```<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">if (isnan(x + 0)) {<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">```<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">… does not also work. I’m going to open a bug and complain, and the slide down the slippery slope will continue. You and I understand the difference, and the technical reason why
`isnan(x)` is supported but `isnan(x + 0)` isn’t, but Joe Coder just trying to figure out why he’s got NaN in his matrices despite his careful NaN handling code. Joe is not a compiler expert, and on the face of it, it seems like a silly limitation. This will
never end until fast-math is gutted.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> Chris Tetreault<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>From:</b> Serge Pavlov <<a href="mailto:sepavloff@gmail.com" target="_blank">sepavloff@gmail.com</a>>
<br>
<b>Sent:</b> Friday, September 10, 2021 9:21 PM<br>
<b>To:</b> Chris Tetreault <<a href="mailto:ctetreau@quicinc.com" target="_blank">ctetreau@quicinc.com</a>><br>
<b>Cc:</b> Richard Smith <<a href="mailto:richard@metafoo.co.uk" target="_blank">richard@metafoo.co.uk</a>>;
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>;
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<b>Subject:</b> Re: [llvm-dev] [cfe-dev] Should isnan be optimized out in fast-math mode?<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p align="center" style="text-align:center"><strong><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black;background:yellow">WARNING:</span></strong><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black;background:yellow">
This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.</span><o:p></o:p></p>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Sat, Sep 11, 2021 at 2:39 AM Chris Tetreault <<a href="mailto:ctetreau@quicinc.com" target="_blank">ctetreau@quicinc.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The problem is that math code is often templated, so `template <typename T> MyMatrixT<T> safeMul(const MyMatrixT<T> & lhs …` is going to be in a header.<o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">No problem, the user can write:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">```<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">#ifdef <span style="font-size:13.5pt;font-family:"Courier New";color:black">__FAST_MATH__</span><o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">#undef isnan<br>
#define isnan(x) false <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">#endif<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">```<br>
and put it somewhere in the headers.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Sat, Sep 11, 2021 at 2:39 AM Chris Tetreault <<a href="mailto:ctetreau@quicinc.com" target="_blank">ctetreau@quicinc.com</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Regardless, my position isn’t “there is no NaN”. My position is “you cannot count on operations on NaN working”. <o:p></o:p></p>
</blockquote>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Exactly. Attempts to express the condition of -ffast-math as restrictions on types are not fruitful. I think it is the reason why GCC documentation does not use simple and clear
"there is no NaN" but prefers more complicated wording about arithmetic.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Sat, Sep 11, 2021 at 2:39 AM Chris Tetreault <<a href="mailto:ctetreau@quicinc.com" target="_blank">ctetreau@quicinc.com</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I think working around these sorts of issues is something that C and C++ developers are used to. These sorts of “inconsistent” between compilers behaviors is something we accept
because we know it comes with improved performance. In this case, the fix is easy, so I don’t think this corner case is worth supporting. Especially when the fix is also just one line:<br>
```<br>
#define myIsNan(x) (reinterpret_cast<uint32_t>(x) == THE_BIT_PATTERN_OF_MY_SENTINEL_NAN)<br>
```<o:p></o:p></p>
</blockquote>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">It won't work in this way. If `x == 5.0`, then `reinterpret_cast<uint32_t>(x) == 5`. What you need there is a bitcast. Standard C does not have such. To emulate it a reinterpret_cast
of memory can be used: `<span style="font-size:10.0pt">*<span style="border:none windowtext 1.0pt;padding:0in">reinterpret_cast</span><<span style="border:none windowtext 1.0pt;padding:0in">int</span> *>(&x)`. Another way is to use a union. Both these solutions
require operations with memory, which is not good for performance, especially on GPU and ML cores. Of course, a smart compiler can eliminate memory operation, but it does not have to do it always, as it is only optimization. Moving a value between float and
integer pipelines also may incur a performance penalty. At the same time this check often may be done with a single instruction.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks,<br>
--Serge<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</body>
</html>