<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Those are all good points. Your reassociation point in the context of inlining is particularly interesting.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">FWIW, we also have a case where a customer wants '-fno-associative-math' to suppress reassociation under '-ffastmath'. It would take me a while to find the specifics
of the issue, but it was (if my memory is right) more of a real use-case. (That is to say, the code that was "failing" due to reassociation didn't have an obvious fix like the reciprocal situation, here, other than to turn off fast-math.) In fact, the request
to suppress reassociation was the motivation for creating PR27372 in the first place (which eventually fed into this thread). I have to say that on the reassociation point, my concern is that to really suppress that, we will have to suppress so much, that
there will hardly be any point in using -ffast-math.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I'd say your comments here are very similar to what Nicolai said in another subthread of this discussion:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">>> I'd be really curious to know if there is anybody who really needs arcp
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">>> without fp-contract=fast or vice versa, or who needs both of these but
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">>> not the X*log2(0.5*Y) transform you mentioned, and so on.[1]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">>> ...<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">>> [1] One case I _can_ think of (and which may have been a reason for the
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">>> proliferation of flags in the first place) is somebody who enables fast
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">>> math, but then doesn't want their results to change when they update the
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">>> compiler and get a new set of optimizations. But IMO that's a use case
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">>> that should be explicitly rejected.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I think those are all really good points, and an argument can be made that when -ffast-math gives you results you don't want, then you just have to turn it off.
Essentially, the user can't "have his cake and eat it too".<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">All that said, I think we (the company I work for, Sony) will have to implement support for these switches. It comes down to GCC has these switches (e.g., -fno-reciprocal-math
and -fno-associative-math), and they do suppress the transformations for our customers. They switch to Clang/LLVM, they use the same switches, and it doesn't "work". So as a practical matter, I think we will support them. Whether the LLVM community in general
feels that that's required, is another question. Until for your recent comments here, and Nicolai's comments above, I would have thought the answer was clearly yes. But maybe that's not the case.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">In summary, irrespective of any (subjective?) assessment of how legitimate a particular use-case is, do we want switches like:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> -ffast-math -fno-reciprocal-math<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> -ffast-math -fno-associative-math<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">to work?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">For me, the answer is yes, because I have multiple customers that tell me they really want to leave -ffast-math on, but they want to be able to disable these
sub-categories. I've been approaching this under the assumption that the answer is yes for the Clang/LLVM community in general.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">-Warren</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> mehdi.amini@apple.com [mailto:mehdi.amini@apple.com]
<br>
<b>Sent:</b> Wednesday, November 16, 2016 10:46 PM<br>
<b>To:</b> Ristow, Warren <warren.ristow@sony.com><br>
<b>Cc:</b> Kaylor, Andrew <andrew.kaylor@intel.com>; llvm-dev@lists.llvm.org; Nicolai Hähnle <nhaehnle@gmail.com><br>
<b>Subject:</b> Re: [llvm-dev] RFC: Consider changing the semantics of 'fast' flag implying all fast-math-flags<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Nov 16, 2016, at 10:04 PM, Ristow, Warren <<a href="mailto:warren.ristow@sony.com">warren.ristow@sony.com</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">> Can you elaborate what kind of runtime failure is the reciprocal transformation triggering?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Yes. It was along the lines of:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> {</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> float x = a / c;</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> float y = b / c;</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> if (y == 1.0f) {</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> // do some processing for when 'b' and 'c' are equal</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> } else {</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> // do other processing</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> }</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> use(x, y);</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> }</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Of course they understood they could easily change this code once they understood the issue.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">But the fact that it "failed" for non-edge-case values of 'c', they were worried. As an example of the non-edge-case aspect, when 'c' is 41.0f (and so of course
'b' is 41.0f), intuitively they felt that this “would work precisely”, even with fast-math. Once they understood more, they agreed this was reasonable with fast-math, but they had the underlying concern that if they encountered one case where 'num' and 'den'
were equal (and non-edge-case), yet 'num / den' wasn't precisely 1.0f, then even if they fixed this situation where they encountered it, it might be lurking elsewhere in their code, and so they wanted to disable that transformation.</span><o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks for elaborating.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I’d be reluctant to call this situation a real use-case though.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Is the the distinction on reciprocal really make sense here? This user can have the same “surprising" anywhere in their code-base with reassociation as well: <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">void foo (float a, float b) {<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> float x = a - b; <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> if (x == 0) <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> … // only if a == b<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">}<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">That would sound totally reasonable, unless foo is inlined and reassociation would lead to a non-zero value for x even when a and b passed in to foo "if it wasn’t inlined" would be identical!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">(Reminds me somehow of a client that was bitten by nnan: their assumption was that as long as they didn’t introduce NaN in the program everything was fine. However with fast-math some transformations were introducing NaN where there wasn’t
before and propagating to other computation that were transformed under the assumption that no NaN would show up, it also turns out that making the code safe against NaN and efficient at the same time is hard, especially when the code itself it compiled with
fast-math).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">— <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Mehdi<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><br>
<br>
</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<div>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span class="apple-converted-space"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> </span></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><a href="mailto:mehdi.amini@apple.com">mehdi.amini@apple.com</a>
[<a href="mailto:mehdi.amini@apple.com">mailto:mehdi.amini@apple.com</a>]<span class="apple-converted-space"> </span><br>
<b>Sent:</b><span class="apple-converted-space"> </span>Wednesday, November 16, 2016 7:11 PM<br>
<b>To:</b><span class="apple-converted-space"> </span>Ristow, Warren <<a href="mailto:warren.ristow@sony.com">warren.ristow@sony.com</a>><br>
<b>Cc:</b><span class="apple-converted-space"> </span>Kaylor, Andrew <<a href="mailto:andrew.kaylor@intel.com">andrew.kaylor@intel.com</a>>;
<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>; Nicolai Hähnle <<a href="mailto:nhaehnle@gmail.com">nhaehnle@gmail.com</a>><br>
<b>Subject:</b><span class="apple-converted-space"> </span>Re: [llvm-dev] RFC: Consider changing the semantics of 'fast' flag implying all fast-math-flags</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal">On Nov 16, 2016, at 6:22 PM, Ristow, Warren <<a href="mailto:warren.ristow@sony.com"><span style="color:purple">warren.ristow@sony.com</span></a>> wrote:<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">> ... except that Warren’s proposal that started this discussion seems to imply that he</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">> has a use case that requires reciprocals to be turned off separately.</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Just to close this loose end, yes I have a use case.</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Specifically, we have a customer that turns on '‑ffast‑math', but was getting a runtime failure due to the reciprocal-transformation being done. </span><o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Can you elaborate what kind of runtime failure is the reciprocal transformation triggering? <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">— <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Mehdi<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><br>
<br>
<br>
<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">They don't want turn off fast‑math because they like the performance improvement, and can live with the imprecision in most cases. So they wanted to suppress
just the reciprocal-transformation. I intended to tell them the solution was simple: use '‑ffast‑math ‑fno‑reciprocal‑math'. But on trying it myself, I ran into the issue here.</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Thanks,</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">-Warren</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<div>
<div>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span class="apple-converted-space"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> </span></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Kaylor,
Andrew [<a href="mailto:andrew.kaylor@intel.com"><span style="color:purple">mailto:andrew.kaylor@intel.com</span></a>]<span class="apple-converted-space"> </span><br>
<b>Sent:</b><span class="apple-converted-space"> </span>Wednesday, November 16, 2016 4:13 PM<br>
<b>To:</b><span class="apple-converted-space"> </span>Mehdi Amini <<a href="mailto:mehdi.amini@apple.com"><span style="color:purple">mehdi.amini@apple.com</span></a>>; Ristow, Warren <<a href="mailto:warren.ristow@sony.com"><span style="color:purple">warren.ristow@sony.com</span></a>>;<span class="apple-converted-space"> </span><a href="mailto:llvm-dev@lists.llvm.org"><span style="color:purple">llvm-dev@lists.llvm.org</span></a>;
Nicolai Hähnle <<a href="mailto:nhaehnle@gmail.com"><span style="color:purple">nhaehnle@gmail.com</span></a>><br>
<b>Subject:</b><span class="apple-converted-space"> </span>RE: [llvm-dev] RFC: Consider changing the semantics of 'fast' flag implying all fast-math-flags</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I don’t really like the idea of updating checks of UnsafeAlgebra() to depend on all of the other flags. It seems like it would be preferable to look at each
optimization and figure out which flags it actually requires. I suspect that in many cases the “new” flag (i.e. allowing reassociation, etc.) will be what is actually needed anyway.</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I would be inclined to agree with Niolai’s suggestion of combining all the flags related to value safety, except that Warren’s proposal that started this discussion
seems to imply that he has a use case that requires reciprocals to be turned off separately.</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">-Andy</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><a name="_MailEndCompose"></a><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<div>
<div>
<p class="MsoNormal"><a name="_____replyseparator"></a><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span class="apple-converted-space"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> </span></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">llvm-dev
[<a href="mailto:llvm-dev-bounces@lists.llvm.org"><span style="color:purple">mailto:llvm-dev-bounces@lists.llvm.org</span></a>]<span class="apple-converted-space"> </span><b>On Behalf Of<span class="apple-converted-space"> </span></b>Mehdi Amini via llvm-dev<br>
<b>Sent:</b><span class="apple-converted-space"> </span>Wednesday, November 16, 2016 8:55 AM<br>
<b>To:</b><span class="apple-converted-space"> </span>Ristow, Warren <<a href="mailto:warren.ristow@sony.com"><span style="color:purple">warren.ristow@sony.com</span></a>><br>
<b>Cc:</b><span class="apple-converted-space"> </span><a href="mailto:llvm-dev@lists.llvm.org"><span style="color:purple">llvm-dev@lists.llvm.org</span></a><br>
<b>Subject:</b><span class="apple-converted-space"> </span>Re: [llvm-dev] RFC: Consider changing the semantics of 'fast' flag implying all fast-math-flags</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal">On Nov 15, 2016, at 11:59 PM, Ristow, Warren <<a href="mailto:warren.ristow@sony.com"><span style="color:purple">warren.ristow@sony.com</span></a>> wrote:<o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">Hi,</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">Thanks for the quick feedback. I see your points, but I have a few questions/comments. I'll start at the end of the previous post:</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">> ...</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">> I think these are valuable problems to solve, but you should tackle them piece by piece:</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">> 1) the clang part of overriding the individual FMF and emitting the right IR is the first thing to fix.</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">> 2) the backend is still using the global UnsafeFPMath and it should be killed.</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">I addressed this point (2) for the reciprocal aspect in the patch, but of course that wasn't useful without doing something about (1).</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">Regarding (1), over at<span class="apple-converted-space"> </span><a href="https://reviews.llvm.org/D26708#596610"><span style="color:purple">https://reviews.llvm.org/D26708#596610</span></a>,
David made the same point that it should be done in Clang. I can understand that, but I wonder whether having the concept of the 'fast' flag in the IR that implies all the other FMF makes sense? I'm not seeing a good reason for it, but since this is very
new to me, I can easily imagine I'm missing the big picture.</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">For example, in the LLVM IR (<a href="http://llvm.org/docs/LangRef.html#fast-math-flags"><span style="color:purple">http://llvm.org/docs/LangRef.html#fast-math-flags</span></a>)
the fast-math flags 'nnan', 'ninf', 'nsz', 'arcp' and 'fast’ are defined. Except for 'fast', each of these has a fairly specific definition of what they mean. For example, for 'arcp':</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> arcp => "Allow optimizations to use the reciprocal of an argument rather</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> than perform division."</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">'fast' is unusual, in that it describes a fairly generic set of aggressive floating-point optimizations:</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> fast => "Allow algebraically equivalent transformations that may dramatically</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> change results in floating point (e.g. reassociate). This flag implies</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> all the others."</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">Very loosely, 'fast' means "all the aggressive FP-transformations that are not controlled by one of the other 4, plus it implies all the other 4". If for terminology,
we call those additional aggressive optimizations 'aggr', then we have:</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> 'fast' == 'aggr' + 'nnan' + 'ninf' + 'nsz' + 'arcp'</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">So as I see it, if we want to disable only one of the other ones (like 'arcp', in my case), there isn't any way to express that with these IR flags defined this way.
In short, we cannot turn on all the flags besides 'arcp'. To do that, what we want is that somehow for the Clang switches:</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> '-ffast-math -fno-reciprocal-math'</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">to ultimately result in LLVM IR that has the following flags on in appropriate FP ops:</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> 'aggr' + 'nnan' + 'ninf' + ‘nsz'</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</blockquote>
<div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal">Make sense, I missed that we can’t *subtract* from fast at the IR level.<o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal">I wouldn’t be opposed to have something along the line of “aggr”, but there is a tradeoff: some transformation may be harder to guard with this model.<o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal">Maybe that could be a starting point: changing the “UnsafeAlgebra” bit in the FMF to be “aggr” you mention and replace all the query to FastMathFlags::UnsafeAlgebra() to return true if all the bits are set in the Flags. This alone should
be nothing more than a mechanical change I believe.<o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal">The important part is then auditing all the users of UnsafeAlgebra() in the middle end and check if they can be “downgraded” to aggr safely: i.e. if they don’t need aggr *and* another flag.<o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal">— <o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal">Mehdi<o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"> <o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">But I don't see a way to express 'aggr' in the IR. We could do this, if we change the definition of the IR 'fast' flag to remove that sentence about implying all the
others:</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> fast => "Allow algebraically equivalent transformations that may dramatically</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> change results in floating point (e.g. reassociate).</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">(If we do something like that, we may want to change the name from 'fast' to something else (like 'aggr'), to avoid tying it too closely to the concept of the '-ffast-math'
switch.)</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">As an aside, I don't know if the "reassociate" example is the only other transformation that's allowed by 'fast' (I presume it isn't), but I think reassociation would
be better expressed by a separate flag, which could then be controlled independently via '-f[no]-associative-math' switch. Not having that flag exist separately in the FMF is the origin of PR27372. But creating that flag and using it in the appropriate places
would still run into these problems of 'fast' implying all the others, which would make it impossible to disable reassociation while leaving all the other FMF transformations enabled.</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">To ask a concrete question using the current definition of 'fast' (which includes enabling reassociation, as the LLVM IR documentation of FMF says), how can we express
in the IR that reciprocal-transformations are not allowed, but reassociation is allowed?</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">So the bottom line is that I do see there are issues in Clang that are relevant. But as long as 'fast' means "'aggr' plus all the other FMF transformations", I don't
see how we can effectively disable a subset of those other FMF transformations (while leaving 'aggr' transformations, such as reassociation, enabled). With that in mind, my patch took one step in having 'fast' no longer imply all the others.</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">Thanks,</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D">-Warren</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>