<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;
color:black;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
tt
{mso-style-priority:99;
font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;
color:black;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
color:black;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="white" lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">> and we need to end up with this:<br>
> <br>
<tt><span style="font-size:10.0pt">> int start = 0;</span></tt><span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt>> if (n >= vf2) {</tt><br>
<tt>> if (check) {</tt><br>
<tt>> if (n >= vf) {</tt><br>
<tt>> for (...; start += vf)</tt><br>
<tt>> ...</tt><br>
<tt>> }</tt><br>
> <br>
<tt>> for (...; start += vf2)</tt><br>
<tt>> ...</tt><br>
<tt>> }</tt><br>
<tt>> }</tt><br>
> <br>
<tt>> for (i = start; i < n; ++i) {</tt><br>
<tt>> ...</tt><br>
<tt>> }</tt><br>
<br>
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I really don’t think this is the right approach because when “</span><tt><span style="font-size:10.0pt">n < vf</span></tt><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">”
it executes alias check to prove aliasing for epilog vector loop.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Epilog loop has very limited iterations and it may be expensive to run alias check for it.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Ashutosh<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext"> Hal Finkel [mailto:hfinkel@anl.gov]
<br>
<b>Sent:</b> Wednesday, March 15, 2017 12:04 AM<br>
<b>To:</b> Adam Nemet <anemet@apple.com><br>
<b>Cc:</b> Nema, Ashutosh <Ashutosh.Nema@amd.com>; Zaks, Ayal <ayal.zaks@intel.com>; Renato Golin <renato.golin@linaro.org>; mkuper@google.com; Mehdi Amini <mehdi.amini@apple.com>; Daniel Berlin <dberlin@dberlin.org>; llvm-dev <llvm-dev@lists.llvm.org><br>
<b>Subject:</b> Re: [llvm-dev] [Proposal][RFC] Epilog loop vectorization<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<p style="margin-left:.5in"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in">On 03/14/2017 12:11 PM, Adam Nemet wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-left:.5in">On Mar 14, 2017, at 9:49 AM, Hal Finkel <<a href="mailto:hfinkel@anl.gov">hfinkel@anl.gov</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<o:p> </o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in">On 03/14/2017 11:21 AM, Adam Nemet wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-left:.5in">On Mar 14, 2017, at 6:00 AM, Nema, Ashutosh <<a href="mailto:Ashutosh.Nema@amd.com">Ashutosh.Nema@amd.com</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Summarizing the discussion on the implementation approaches.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Discussed about two approaches, first running ‘InnerLoopVectorizer’ again on the epilog loop immediately after vectorizing the original loop within the same vectorization pass, the second approach where re-running
vectorization pass and limiting vectorization factor of epilog loop by metadata.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><Approach-2><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Challenges with re-running the vectorizer pass:<o:p></o:p></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">1)</span><span style="font-size:7.0pt"> <span class="apple-converted-space"> </span></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Reusing
alias check result:<span class="apple-converted-space"> </span><o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">When vectorizer pass runs again it finds the epilog loop as a new loop and it may generates alias check, this new alias check may overkill the gains
of epilog vectorization.<o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">We should use the already computed alias check result instead of re computing again.<o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">2)</span><span style="font-size:7.0pt"> <span class="apple-converted-space"> </span></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Rerun
the vectorizer and hoist the new alias check:<o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">It’s not possible to hoist alias checks as its not fully redundant (not dominated by other checks), it’s not getting execute in all paths.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><Mail Attachment.jpeg><o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in">I don’t understand. Looks like you have the same alias checks for the epilog loop too. How is this CFG different from the re-vectorization of the scalar loop?
<o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<p class="MsoNormal" style="margin-left:.5in"><br>
You're looking at the wrong thing. This *is* the image from re-vectorization. The other image (linked below in step (3)) shows the other option.<o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in">Ah ok, the numbering confused me here.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> Would be good to have both CFGs here and highlighting the difference.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">I thought that the whole point was that *if* you reached the epilog vector loop via the first vector loop, you want to bypass the alias checks before the epilog vector.<o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<p class="MsoNormal" style="margin-left:.5in"><br>
Yes, but, that's not quite true now. You can also reach the epilogue loop if you fail the min-trip-count check, and so you don't know anything about the aliasing checks.<o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">OK, so we want this loops to be handled specially. We effectively say that we only vectorize this loop if it does not require any alias checks or if the alias checks can be predicate-forwarded to this loop from
existing checks.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">This still seems like an orthogonal issue that may be interesting to solve independently. In other words this could be a nice feature in the vectorizer anyway: the loop is estimated to be low-trip count so feel
free to predicate the new vector loop so that the alias check result could be reused from some other block. We obviously don’t have this capability today but it’s something that could be nice aside from the vectorizer.<o:p></o:p></p>
</div>
</div>
</blockquote>
<p class="MsoNormal" style="margin-left:.5in"><br>
That sounds great. I'm not sure, however, exactly what this means. This "predicate forwarding" sounds like a non-local restructuring of the surrounding code (because the predicates aren't known to be true at that point, we need to search different predecessors
to find one in which the conditions might be true and insert the vector loop across that predecessor edge instead). Maybe we could do this, for example, by calling SE->isKnownPredicate enhanced with some additional context sensitivity because we currently
check dominating conditional branches for things that are AddRecs in some loop? Moreover, we then have the problem of restructuring the order of the trip-count checks (because we need to fast fail to the scalar loop for the smallest trip counts). Maybe we
can do this the same way? This means finding a dominating check on the trip count that implies the check we're about to insert, change that check to the check we want, keeping the original only on the true side of the trip-count check (i.e. if the trip count
is larger than the small threshold, then check the large threshold).<br>
<br>
If we're going to do all of that, then I'd lean toward saying that this does not belong in the vectorizer at all. Rather, this seems like something we'd want in some general transformation (this seems somewhat akin to what JumpThreading does). The general transformation
seems something like this; the basic vectorized loop looks like this:<br>
<br>
<tt><span style="font-size:10.0pt">int start = 0;</span></tt><br>
<tt><span style="font-size:10.0pt">if (n >= vf) {</span></tt><span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt> if (check) {</tt><br>
<tt> for (...; start += vf)</tt><br>
<tt> ...</tt><br>
<tt> }</tt><br>
<tt>}</tt><br>
</span><br>
<tt><span style="font-size:10.0pt">for (i = start; i < n; ++i) {</span></tt><span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt> ...</tt><br>
<tt>}</tt><br>
</span><br>
and after we vectorize the epilogue loop we end up with this:<br>
<br>
<tt><span style="font-size:10.0pt">int start = 0;</span></tt><br>
<tt><span style="font-size:10.0pt">if (n >= vf) {</span></tt><span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt> if (check) {</tt><br>
<tt> for (...; start += vf)</tt><br>
<tt> ...</tt><br>
<tt> }</tt><br>
<tt>}</tt><br>
</span><br>
<tt><span style="font-size:10.0pt">if (n >= vf2) {</span></tt><span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt> if (check) {</tt><br>
<tt> for (...; start += vf2)</tt><br>
<tt> ...</tt><br>
<tt> }</tt><br>
<tt>}</tt><br>
</span><br>
<tt><span style="font-size:10.0pt">for (i = start; i < n; ++i) {</span></tt><span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt> ...</tt><br>
<tt>}</tt></span><br>
<br>
and we need to end up with this:<br>
<br>
<tt><span style="font-size:10.0pt">int start = 0;</span></tt><span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt>if (n >= vf2) {</tt><br>
<tt> if (check) {</tt><br>
<tt> if (n >= vf) {</tt><br>
<tt> for (...; start += vf)</tt><br>
<tt> ...</tt><br>
<tt> }</tt><br>
<br>
<tt> for (...; start += vf2)</tt><br>
<tt> ...</tt><br>
<tt> }</tt><br>
<tt>}</tt><br>
<br>
<tt>for (i = start; i < n; ++i) {</tt><br>
<tt> ...</tt><br>
<tt>}</tt><br>
</span><br>
where we've recognized here that 'check' is the same in both cases, and that because vf2 < vf, the one trip-count check implies the other. This latter part seems like the part that our existing passes might not know what to do with currently. Thoughts?<br>
<br>
-Hal<br>
<br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Adam<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><br>
-Hal<br>
<br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">I still don’t understand why that’s not possible with some sophisticated predicate propagation independent from the vectorizer. I am not saying it’s already possible but it should be.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Adam<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:.5in"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">NOTE: We cannot prepone alias check as its expensive compared to other checks.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><Approach-1><o:p></o:p></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">1)</span><span style="font-size:7.0pt"> <span class="apple-converted-space"> </span></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Current
patch depends on the existing functionality of LoopVectorizer, it uses ‘InnerLoopVectorizer’ again to vectorize the epilog loop, as it happens in the same vectorization pass we have flexibility to reuse already computed alias result check & limit vectorization
factor for the epilog loop.<span class="apple-converted-space"> </span><o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">2)</span><span style="font-size:7.0pt"> <span class="apple-converted-space"> </span></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">It
does not generate the blocks for new block layout explicitly, rather it depends on ‘InnerLoopVectorizer::createEmptyLoop’ to generate new block layout. The new block layout get automatically generated by calling the ‘InnerLoopVectorizer:: vectorize’ again.<o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">3)</span><span style="font-size:7.0pt"> <span class="apple-converted-space"> </span></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Block
layout description with epilog loop vectorization is available at<o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><a href="https://reviews.llvm.org/file/data/fxg5vx3capyj257rrn5j/PHID-FILE-x6thnbf6ub55ep5yhalu/LayoutDescription.png"><span style="color:purple">https://reviews.llvm.org/file/data/fxg5vx3capyj257rrn5j/PHID-FILE-x6thnbf6ub55ep5yhalu/LayoutDescription.png</span></a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Approach-1 looks feasible, please comment if any objections.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Regards,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Ashutosh<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:7.5pt;font-family:"Helvetica",sans-serif">...<o:p></o:p></span></p>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</blockquote>
<p class="MsoNormal" style="margin-left:.5in"><br>
<br>
<o:p></o:p></p>
<pre style="margin-left:.5in">-- <o:p></o:p></pre>
<pre style="margin-left:.5in">Hal Finkel<o:p></o:p></pre>
<pre style="margin-left:.5in">Lead, Compiler Technology and Programming Languages<o:p></o:p></pre>
<pre style="margin-left:.5in">Leadership Computing Facility<o:p></o:p></pre>
<pre style="margin-left:.5in">Argonne National Laboratory<o:p></o:p></pre>
</div>
</body>
</html>