<div dir="auto">SGTM.</div><div class="gmail_extra"><br><div class="gmail_quote">On Dec 15, 2016 08:09, "Matthew Simpson" <<a href="mailto:mssimpso@codeaurora.org">mssimpso@codeaurora.org</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>If there are no objections, I'll submit a patch for review that sets the default value of "-enable-cond-stores-vec" to "true". Thanks!</div><div><br></div><div>-- Matt</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Dec 14, 2016 at 12:55 PM, Michael Kuperstein via llvm-dev <span dir="ltr"><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I haven't verified what Matt described is what actually happens, but assuming it is - that is a known issue in the x86 cost model.<div><br><div>Vectorizing interleaved memory accesses on x86 was, until recently, disabled by default. It's been enabled since r284779, but the cost model is very conservative, and basically assumes we're going to scalarize interleaved ops.</div><div><br></div><div>I believe Farhana is working on improving that.</div><span class="m_7981918144634236779HOEnZb"><font color="#888888"><div><br></div><div>Michael<br><div><br></div></div></font></span><div><div class="m_7981918144634236779h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Dec 14, 2016 at 8:44 AM, Das, Dibyendu <span dir="ltr"><<a href="mailto:Dibyendu.Das@amd.com" target="_blank">Dibyendu.Das@amd.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div class="m_7981918144634236779m_1965636533696774162m_641736246953467876WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Hi Matt-<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Yeah I used a pretty recent llvm (post 3.9) on an x86-64 ( both AMD and Intel ).
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">-dibyendu<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Matthew Simpson [mailto:<a href="mailto:mssimpso@codeaurora.org" target="_blank">mssimpso@codeaurora.or<wbr>g</a>]
<br>
<b>Sent:</b> Wednesday, December 14, 2016 10:03 PM<br>
<b>To:</b> Das, Dibyendu <<a href="mailto:Dibyendu.Das@amd.com" target="_blank">Dibyendu.Das@amd.com</a>><br>
<b>Cc:</b> Michael Kuperstein <<a href="mailto:mkuper@google.com" target="_blank">mkuper@google.com</a>>; <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a></span></p><div><div class="m_7981918144634236779m_1965636533696774162h5"><br>
<b>Subject:</b> Re: [llvm-dev] Enabling scalarized conditional stores in the loop vectorizer<u></u><u></u></div></div><p></p><div><div class="m_7981918144634236779m_1965636533696774162h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">Hi Dibyendu,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Are you using a recent compiler? What architecture are you targeting? The target will determine whether the vectorizer thinks vectorization is profitable without having to manually force the vector width.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">For example, top-of-trunk vectorizes your snippet with "clang -O2 -mllvm -enable-cond-stores-vec" and "--target=aarch64-unknown-linu<wbr>x-gnu". However, with "--target=x86_64-unknown-linux<wbr>-gnu" the vectorizer doesn't find the snippet profitable
to vectorize.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">This is probably due to the interleaved load in the loop. When targeting AArch64, the cost model reports the interleaved load as inexpensive (AArch64 has dedicated instructions for interleaved memory accesses), but when targeting X86 it
doesn't. You can take a look at the costs with "-mllvm -debug-only=loop-vectorize"<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Hope that helps.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">-- Matt<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Wed, Dec 14, 2016 at 12:59 AM, Das, Dibyendu via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Hi Michael-</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Since you bring up libquantum performance can you let me know what the IR will look like for this small
code snippet (libquantum-like) with –enable-cond-stores-vec ? I ask because I don’t see vectorization kicking in unless -force-vector-width=<> is specified. Let me know if I am missing something.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<div style="border:none;border-bottom:solid windowtext 1.0pt;padding:0in 0in 1.0pt 0in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">-Thx</span><u></u><u></u></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d">struct nodeTy</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d">{</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> unsigned int c1;</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> unsigned int c2;</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> unsigned int state;</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d">};</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d">struct quantum_reg</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d">{</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> struct nodeTy node[32];</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> unsigned int size;</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d">};</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d">void</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d">quantum_toffoli(int control1, int control2, int target, struct quantum_reg *reg, int n)</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d">{</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> int i;</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> int N = reg->size;</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> for(i=0; i < N; i++)</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> {</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> if(reg->node[i].state & ((unsigned int)1 << control1))</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> if(reg->node[i].state & ((unsigned int)1 << control2))</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> reg->node[i].state ^= ((unsigned int)1 << target);</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> }</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Calibri",sans-serif;color:#1f497d">}</span><u></u><u></u></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> llvm-dev [mailto:<a href="mailto:llvm-dev-bounces@lists.llvm.org" target="_blank">llvm-dev-bounces@lists<wbr>.llvm.org</a>]
<b>On Behalf Of </b>Matthew Simpson via llvm-dev<br>
<b>Sent:</b> Tuesday, December 13, 2016 7:12 PM<br>
<b>To:</b> Michael Kuperstein <<a href="mailto:mkuper@google.com" target="_blank">mkuper@google.com</a>><br>
<b>Cc:</b> llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm-dev] Enabling scalarized conditional stores in the loop vectorizer</span><u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Hi Michael,<u></u><u></u></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks for testing this on your benchmarks and target. I think the results will help guide the direction we go. I tested the feature with spec2k/2k6 on AArch64/Kryo and saw minor
performance swings, aside from a large (30%) improvement in spec2k6/libquantum. The primary loop in that benchmark has a conditional store, so I expected it to benefit.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Regarding the cost model, I think the vectorizer's modeling of the conditional stores is good. We could potentially improve it by using profile information if available. But I'm
not sure of the quality of the individual TTI implementations other than AArch64. I assume they are adequate.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Since the conditional stores remain scalar in the vector loop, their cost is essentially the same as it is in the scalar loop (aside from scalarization overhead, which we account
for). So when we compare the cost of the scalar and vector loops when deciding to vectorize, we're basically comparing the cost of everything else.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">-- Matt<u></u><u></u></p>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal">On Mon, Dec 12, 2016 at 7:03 PM, Michael Kuperstein via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">Conceptually speaking, I think we really ought to enable this.<u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Practically, I'm going to test it on our benchmarks (on x86), and see if we have any regressions - this seems like a fairly major change.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Re targets - let's see where we stand w.r.t regressions first. What kind of performance testing have you already run on this? Do you know of specific targets where the cost model
is known to be good enough, so it's clearly beneficial?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">(+Arnold, who probably knows why this is disabled by default. :-) )<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> Michael<u></u><u></u></p>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal">On Mon, Dec 12, 2016 at 2:52 PM, Matthew Simpson <<a href="mailto:mssimpso@codeaurora.org" target="_blank">mssimpso@codeaurora.org</a>> wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal">Hi,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">I'd like to enable the scalarized conditional stores feature in the loop vectorizer (-enable-cond-stores-vec=true)<wbr>. The feature allows us to vectorize loops containing conditional
stores that must be scalarized and predicated in the vectorized loop.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Note that this flag does not affect the decision to generate masked vector stores. That is a separate feature and is guarded by a TTI hook. Currently, we give up on loops containing
conditional stores that must be scalarized (i.e., conditional stores that can't be represented with masked vector stores). If the feature is enabled, we attempt to vectorize those loops if profitable, while scalarizing and predicating the conditional stores.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">I think these stores are fairly well modeled in the cost model at this point using the static estimates. They're modeled similar to the way we model other non-store conditional
instructions that must be scalarized and predicated (e.g., instructions that may divide by zero); however, only the conditional stores are currently disabled by default.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">I'd appreciate any opinions on how/if we can enable this feature. For example, can we enable it for all targets or would a target-by-target opt-in mechanism using a TTI hook be
preferable? If you'd like to test the feature on your target, please report any significant regressions and improvements you find.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks!<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">-- Matt<u></u><u></u></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
______________________________<wbr>_________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-dev</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
______________________________<wbr>_________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-dev</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div></div></div>
</div>
</blockquote></div><br></div></div></div></div></div>
<br>______________________________<wbr>_________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-dev</a><br>
<br></blockquote></div><br></div>
</blockquote></div></div>