<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:998651977;
mso-list-type:hybrid;
mso-list-template-ids:-1051139306 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hi Renato,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks for the reply.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’m not entirely clear what course of action you are suggesting. I’ll start with the two statements I didn’t understand.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">> Finally, I think we need to separate the IR from DAG/MIR behaviour. It seems to me that the target option is what overrides the behaviour, not function/module options, so we should worry about the targets' behaviour, not at which level
the flag is<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I don’t know what you’re saying here. What do you mean by “we should worry about the targets' behaviour”? If the IR says fp-contract is not allowed for a given instruction, why should the target options be allowed to overrule that?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">> There's a perfectly valid solution that has module/function/instruction flags controlling behaviour, with the most specific overriding the least specific, and none of that overridden by the target. This means we can still use the same
IR flags in the same way (thus not forcing all front-ends to change) and still correct the behaviour by not making the target ignore all that.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I have a bit of an idea of what you’re getting at here, but I’d like to ask for a clarification. I think you’re saying that an instruction flag would take precedence over a function attribute, and a function attribute would take precedence
over module metadata, right? The question I would have is whether we are going to view the absence of a flag/attribute/metadata as meaningful.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">For example, given the rule you propose, if an instruction has the ‘reassoc’ fast-math flag set, but the function has an attribute "unsafe-fp-math"="false", then we would permit that instruction to be reassociated. But if the instruction
does not have any fast-math flags set and the function has an attribute "unsafe-fp-math"="true", does the flag not being set at the instruction level take precedence over the function attribute? I suppose to maintain the current behavior it would have to.
I was a bit surprised to discover that clang handles the “unsafe-fp-math” math function attribute correctly for this. If I use -ffast-math on the command line, but disable one of the fast-math flags for part of a function, it will set “unsafe-fp-math” to “false”.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><a href="https://godbolt.org/z/Mj88nbMq1">https://godbolt.org/z/Mj88nbMq1</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">In this case, clang will set TargetOptions::UnsafeFPMath to true, but SelectionDAGISel::runOnMachineFunction() calls TargetMachine::resetTargetOptions() which resets it based on the function attribute. So for this case, it all works out.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Two notes, however:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<ol style="margin-top:0in" start="1" type="1">
<li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">resetTargetOptions doesn’t handle the AllowFPOpFusion setting (as far as I can tell, there is no corresponding function attribute)<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">There is a comment for resetTargetOptions which says this:<o:p></o:p></li></ol>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">// FIXME: This function needs to go away for a number of reasons:<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">// a) global state on the TargetMachine is terrible in general,<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">// b) these target options should be passed only on the function<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">// and not on the TargetMachine (via TargetOptions) at all.<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">As you may guess, I completely agree with the comment. FWIW, this comment was added by Eric Christopher in 2015 (<a href="https://github.com/llvm/llvm-project/commit/35a8a62125ccfa0ef804584738a295ed5c17750f">https://github.com/llvm/llvm-project/commit/35a8a62125ccfa0ef804584738a295ed5c17750f</a>).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The core of my complaint about the current state of things is that the backend is treating the fast-math flags as if they are “on or undecided” rather than “on or off.” IMO, the absence of a fast-math flag in the IR means that the behavior
controlled by that flag is not permitted. That’s the way the IR is treated in IR-level optimization passes, but the backend codegen (at least in places) behaves as though the absence of a fast-math flag means “not permitted unless enabled by TargetOptions.”
That’s bad as a starting point, but it’s particularly bad when you start linking together IR from multiple modules that may have been created from different compilation units with different options.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’d also like to make one thing explicit. While the change I’m proposing would change the behavior of the backend for all front ends, it wouldn’t “break” them in the sense of producing incorrect results. When TargetOptions::AllowFPOpFusion==FPOpFusionMode::Fast
the backend is *<b>allowed</b>* to fuse operations, but it isn’t required or guaranteed to fuse them. With the change I am proposing, the backend wouldn’t form fuse FP options unless the ‘contract’ flag were set, but that wouldn’t lead to incorrect results.
It would lead to lost performance in some cases, but I think that can be recovered by generating IR with the semantics that were intended. Similarly, for the other fast-math flags. However, I will admit that it could be an uncomfortable transition for some
front ends.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Regarding bloating the IR, I’m not sure if this is a theoretical problem or real one. The bitcode writer will emit a byte for the fast-math flags if any are set for each call or FP operation, and if none are set it will not emit that byte.
For clang, this is a non-issue because clang always sets the fast-math flags on the individual operations. It’s at least theoretically possible that there is some front end that never sets the fast-math flags and relies on the global setting. In that case,
adding the fast-math flags would bloat the IR considerably. For most of the FMF, omitting the flags would cause considerable missed optimization opportunities in the middle end, but the ‘contract’ flag is a bit of a special case because it isn’t typically
used at all before codegen.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">-Andy<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Renato Golin <rengolin@gmail.com> <br>
<b>Sent:</b> Friday, October 29, 2021 3:00 AM<br>
<b>To:</b> Kaylor, Andrew <andrew.kaylor@intel.com><br>
<b>Cc:</b> llvm-dev@lists.llvm.org; Ulrich Weigand <Ulrich.Weigand@de.ibm.com>; Yaxun Liu <yaxun.liu@amd.com>; Ballman, Aaron <aaron.ballman@intel.com>; Ammarguellat, Zahira <zahira.ammarguellat@intel.com>; Sebastian Pop <sebpop@gmail.com><br>
<b>Subject:</b> Re: [llvm-dev] [RFC] Eliminating non-IR floating-point controls in the selection DAG<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Hi Andrew,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">IIRC, the fp-contract change was to fix some broken code but not break more. Copying Sebastian who was working on that at that time.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I agree we shouldn't have overriding behaviour flags in the back-end if the IR explicitly says what it wants. But I'd be cautious as to move all such flags to instructions.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">First, this would be a behavioural change that needs the IR to change, so would affect every LLVM IR front-end, which makes it a pervasive change throughout the downstream users. So, if we decide we want to do this, we need to replace the
current mess with a consistent implementation that wont break everybody else's.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Second, module/function flags can control fine-grain behaviour without bloating the IR. I don't know how the instruction flag changes the binary representation, it's probably very small anyway, but so are the module/function ones, so overall,
a definite increase in size.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Finally, I think we need to separate the IR from DAG/MIR behaviour. It seems to me that the target option is what overrides the behaviour, not function/module options, so we should worry about the targets' behaviour, not at which level
the flag is set.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">There's a perfectly valid solution that has module/function/instruction flags controlling behaviour, with the most specific overriding the least specific, and none of that overridden by the target. This means we can still use the same IR
flags in the same way (thus not forcing all front-ends to change) and still correct the behaviour by not making the target ignore all that.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Does any of that make any sense?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">cheers,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">--renato<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Fri, 29 Oct 2021 at 01:22, Kaylor, Andrew via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi everyone,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">This is related to the recent thread about fp-contract and front end pragma controls, but I want to generalize the discussion in terms of how the target-independent codegen in the
backend is implemented.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Until sometime in 2017 (I think) the fast-math flags were not passed through to the Selection DAG, and so the only ways to control floating-point behavior were through settings
in the TargetOptions or by settings function attributes. Since 2017, however, the fast-math flags have been attached to floating-point nodes in the selection DAG. This leads to some ambiguous situations where the TargetOptions or function attributes can override
the absence of fast-math flags on individual nodes. An example of this is the fp-contract setting. If a source file is compiled with clang using the ‘-ffp-contract=fast’ setting but the file contains either “#pragma STDC FP_CONTRACT OFF” or “#pragma clang
fp contract(off)” the front end will generate IR without the ‘contract’ fast-math flag set, but the X86 backend will generate an FMA instruction anyway.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a href="https://godbolt.org/z/dov6EcE8G" target="_blank">https://godbolt.org/z/dov6EcE8G</a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">This is particularly bad in the case of CUDA, because CUDA uses fp-contract=fast by default. So, the user’s code can explicitly say “don’t generate fma here” and the compiler will
respond, “meh, I think I will anyway.”<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a href="https://godbolt.org/z/c4h1nK9M3" target="_blank">https://godbolt.org/z/c4h1nK9M3</a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">There are other cases where the backend code will check for TargetOption::UnsafeFPMath for things like reassociation that can be represented using fast-math flags.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">That brings me to the RFC part of my message. I’d like to start updating the backend so that it doesn’t do things like this. As a general principle, I would say, “All semantics
must be represented in the IR and the backend must respect the IR semantics.” And a corollary: “Anything which can be represented at the instruction level must be represented at the instruction level.” This corollary would eliminate potential conflicts between
function attributes (like "unsafe-fp-math") and individual IR instructions.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">As a first step toward this goal, I’ve prepared a patch which closes the back door for fp-contract control.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a href="https://reviews.llvm.org/D112760" target="_blank">https://reviews.llvm.org/D112760</a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">This patch is currently incomplete, in as much as I didn’t update failing tests for several target architectures. I did update the X86 and AMDGPU tests to provide examples of how
they can be made to work. I will fix the rest if we decide this is the correct direction. There is a failing CUDA test in the clang front end that I think will require a different approach involving some driver changes to get clang to generate IR for the semantics
it intends rather than setting an option and counting on the backend to disregard the IR.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks in advance for any feedback!<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">-Andy<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</body>
</html>