<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<blockquote style="border-left: 3px solid rgb(200, 200, 200); border-top-color: rgb(200, 200, 200); border-right-color: rgb(200, 200, 200); border-bottom-color: rgb(200, 200, 200); padding-left: 1ex; margin-left: 0.8ex; color: rgb(102, 102, 102);">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> JF Bastien <jfbastien@apple.com><br>
<b>Sent:</b> Wednesday, April 29, 2020 4:24 PM<br>
<b>To:</b> Finkel, Hal J. <hfinkel@anl.gov><br>
<b>Cc:</b> Cranmer, Joshua <joshua.cranmer@intel.com>; llvm-dev@lists.llvm.org <llvm-dev@lists.llvm.org><br>
<b>Subject:</b> Re: [llvm-dev] Nontemporal memory accesses and fences</font>
<div> </div>
</div>
<div class="" style="word-wrap:break-word; line-break:after-white-space"><br class="">
<div><br class="">
<div class="">On Apr 29, 2020, at 2:13 PM, Finkel, Hal J. <<a href="mailto:hfinkel@anl.gov" class="">hfinkel@anl.gov</a>> wrote:</div>
<br class="x_Apple-interchange-newline">
<div class="">
<div class="">
<hr tabindex="-1" class="" style="display:inline-block; width:916.015625px">
<div id="x_divRplyFwdMsg" dir="ltr" class=""><font face="Calibri, sans-serif" class="" style="font-size:11pt"><b class="">From:</b><span class="x_Apple-converted-space"> </span>llvm-dev <<a href="mailto:llvm-dev-bounces@lists.llvm.org" class="">llvm-dev-bounces@lists.llvm.org</a>>
on behalf of JF Bastien via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>><br class="">
<b class="">Sent:</b><span class="x_Apple-converted-space"> </span>Tuesday, April 28, 2020 4:54 PM<br class="">
<b class="">To:</b><span class="x_Apple-converted-space"> </span>Cranmer, Joshua <<a href="mailto:joshua.cranmer@intel.com" class="">joshua.cranmer@intel.com</a>><br class="">
<b class="">Cc:</b><span class="x_Apple-converted-space"> </span><a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a> <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>><br class="">
<b class="">Subject:</b><span class="x_Apple-converted-space"> </span>Re: [llvm-dev] Nontemporal memory accesses and fences</font>
<div class=""> </div>
</div>
<div class="" style="word-wrap:break-word; line-break:after-white-space">I see nontemporals as a nice builtin so you don’t have to hand-write the assembly. I don’t think existing hardware is consistent enough to allow us to expose nice semantics at a language
or IR level, in the sense that we probably don’t want to define what happens if you step out of the comfortable uses for nontemporals (i.e. what happens when you start having temporality). A builtin pre / post fence seems fine to me, but that still leaves
a gaping semantic hole… which I’m also fine with.
<div class=""><br class="">
</div>
<div class="">FWIW you have the same problem, somewhat worse, with some vector load / store instructions on some ISAs.</div>
</div>
</div>
<div class="" style="font-family:Helvetica; font-size:12px; font-style:normal; font-variant-caps:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<br class="">
</div>
<div class="" style="font-family:Helvetica; font-size:12px; font-style:normal; font-variant-caps:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
I was under the impression that we had decided to move forward with the fence, and it continues to make sense to me.</div>
</div>
<div><br class="">
</div>
<div>I probably missed a prior discussion, so a link would be great! That being said, fences sound fine to me as long as they’re “nice builtins instead of inline asm” and don’t try to offer more than “whatever the hardware instruction, if any, means”.</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I was referring to your RFC linked below (from 2016). We might also have talked about this offline. The fences seemed like a sensible solution. Reviewing the RFC thread, it seemed like: 1) We needed separate load and store fences and 2) There may have
been some uncertainty how to map these on ARM.</div>
<div><br>
</div>
<div> -Hal</div>
<div><br>
</div>
<blockquote style="border-left: 3px solid rgb(200, 200, 200); border-top-color: rgb(200, 200, 200); border-right-color: rgb(200, 200, 200); border-bottom-color: rgb(200, 200, 200); padding-left: 1ex; margin-left: 0.8ex; color: rgb(102, 102, 102);">
<div>
<div class="" style="word-wrap:break-word; line-break:after-white-space">
<div>
<div><br class="">
</div>
<div><br class="">
</div>
<div class="">
<div class="" style="font-family:Helvetica; font-size:12px; font-style:normal; font-variant-caps:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
-Hal</div>
<div class="" style="font-family:Helvetica; font-size:12px; font-style:normal; font-variant-caps:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<br class="">
</div>
<div class="">
<div class="" style="word-wrap:break-word; line-break:after-white-space">
<div class=""><br class="">
<div class=""><br class="">
<div class="">On Apr 28, 2020, at 2:42 PM, Cranmer, Joshua <<a href="mailto:joshua.cranmer@intel.com" class="">joshua.cranmer@intel.com</a>> wrote:</div>
<br class="x_x_Apple-interchange-newline">
<div class="">
<div class="x_x_WordSection1" style="font-family:Helvetica; font-size:12px; font-style:normal; font-variant-caps:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
The current specification of the behavior of the !nontemporal attribute in LLVM, and the __builtin_nontemporal_* functions in Clang, is rather spartan and underspecified. In effect, it says the following things:</div>
<ul type="disc" class="" style="margin-bottom:0in; margin-top:0in">
<li class="x_x_MsoListParagraph" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
Atomic !nontemporal has no defined semantics</li><li class="x_x_MsoListParagraph" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
!nontemporal may use special instructions to save cache bandwidth, such as “MOVNT” on x86.</li></ul>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
</div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
What is crucially lacking from this specification is its effects in relation to other memory ordering constructs, namely fences. In lieu of text to the contrary, one could reasonably come to the conclusion that this code should be sufficient to generate correctly
working code:</div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
</div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
store i64 %x, i64* %addr, !nontemporal !100</div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
fence release</div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
</div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
But on x86 today, it does not: the first store is lowered to a MOVNT, while the fence release is lowered to a compiler-only memory barrier instead of an SFENCE (as regular load/store/atomicrmw operations already have these operations built into their execution
semantics). So the fence instructions appear to be lowered with the expectation that nontemporal instructions do not exist, which is not implied by the language reference.</div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
</div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
I’ve been pointed to an earlier thread (<a href="https://lists.llvm.org/pipermail/llvm-dev/2016-January/093912.html" class="" style="color:rgb(149,79,114); text-decoration:underline">https://lists.llvm.org/pipermail/llvm-dev/2016-January/093912.html</a>) that
suggests an introduction of nontemporal-specific fences, but there appears to have been no effort to actually push this into LLVM. What’s the current status of this work, and what is the right way to move forward and clarifying the semantics of !nontemporal
with respect to threads.</div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
<span class="" style="font-size:9pt"> </span></div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
<span class="" style="font-size:9pt">--<span class="x_x_Apple-converted-space"> </span></span></div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
<i class=""><span class="" style="font-size:9pt">Joshua Cranmer</span></i></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br class="">
</div>
</div>
</blockquote>
<div>
<div class="" style="word-wrap:break-word; line-break:after-white-space"></div>
</div>
</body>
</html>