<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Thank you, thank you, thank you!</p>
<p>This should be enough for me to find and address the issue. <br>
</p>
<p>Thank you for the help!</p>
<p>Philip<br>
</p>
<div class="moz-cite-prefix">On 5/27/21 11:29 AM, Stefan Pintilie
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:OF19D50B3B.B1905522-ON002586E2.00654CFA-002586E2.00659820@notes.na.collabserv.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div class="socmaildefaultfont" dir="ltr"
style="font-family:Arial, Helvetica, sans-serif;font-size:10pt">
<div dir="ltr"> </div>
<div dir="ltr">Hi Philip,</div>
<div dir="ltr"> </div>
<div dir="ltr">We have managed to track down more information on
the issue causing the failure with the LV patch.<br>
It seems that Cost->requiresScalarEpilogue() sometimes
returns true when the scalar epilogue is actually not
required.<br>
One of the failing tests is:<br>
test-suite/SingleSource/Benchmarks/Misc/oourafft.c<br>
That test has a loop that runs a fixed 1024 times. If the max
vector interleave is a factor of 1024 then
Cost->requiresScalarEpilogue() still returns true even
though the scalar epilogue is not actually required.<br>
The following examples were run on a Little Endian machine:<br>
FAIL - clang -DNDEBUG -fuse-ld=ld -O3 -DNDEBUG -w
-Werror=date-time -mllvm -force-vector-interleave=4
${TESTSUITE}/SingleSource/Benchmarks/Misc/oourafft.c -lm<br>
FAIL - clang -DNDEBUG -fuse-ld=ld -O3 -DNDEBUG -w
-Werror=date-time -mllvm -force-vector-interleave=8
${TESTSUITE}/SingleSource/Benchmarks/Misc/oourafft.c -lm<br>
PASS - clang -DNDEBUG -fuse-ld=ld -O3 -DNDEBUG -w
-Werror=date-time -mllvm -force-vector-interleave=12
${TESTSUITE}/SingleSource/Benchmarks/Misc/oourafft.c -lm<br>
Looking at the IR immediately following LV we see this.</div>
<div dir="ltr"> </div>
<div dir="ltr">Working IR:<br>
middle.block:<br>
%cmp.n = icmp eq i64 1024, 1024<br>
br i1 %cmp.n, label %for.end, label %scalar.ph </div>
<div dir="ltr"> </div>
<div dir="ltr">Failing IR:<br>
middle.block:<br>
br label %scalar.ph</div>
<div dir="ltr"><br>
We traced the problem to
InterleavedAccessInfo::analyzeInterleaving where we set
RequiresScalarEpilogue to true.<br>
We probably shouldn't be setting that to TRUE for this test
case. Perhaps it matters that the loop wasn't actually
vectorized in this case (VF == 1). That may impact the
computation of RequiresScalarEpilogue.</div>
<div dir="ltr"> </div>
<div dir="ltr">Hope this helps,<br>
Stefan</div>
<div dir="ltr"> </div>
<blockquote data-history-content-modified="1" dir="ltr"
style="border-left:solid #aaaaaa 2px; margin-left:5px;
padding-left:5px; direction:ltr; margin-right:0px">-----
Original message -----<br>
From: Stefan Pintilie via llvm-commits
<a class="moz-txt-link-rfc2396E" href="mailto:llvm-commits@lists.llvm.org"><llvm-commits@lists.llvm.org></a><br>
Sent by: "llvm-commits"
<a class="moz-txt-link-rfc2396E" href="mailto:llvm-commits-bounces@lists.llvm.org"><llvm-commits-bounces@lists.llvm.org></a><br>
To: <a class="moz-txt-link-abbreviated" href="mailto:listmail@philipreames.com">listmail@philipreames.com</a><br>
Cc: <a class="moz-txt-link-abbreviated" href="mailto:llvmlistbot@llvm.org">llvmlistbot@llvm.org</a>, <a class="moz-txt-link-abbreviated" href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a>,
<a class="moz-txt-link-abbreviated" href="mailto:benny.kra@gmail.com">benny.kra@gmail.com</a>, Nemanja Ivanovic
<a class="moz-txt-link-rfc2396E" href="mailto:nemanjai@ca.ibm.com"><nemanjai@ca.ibm.com></a>, <a class="moz-txt-link-abbreviated" href="mailto:akuegel@google.com">akuegel@google.com</a>, LLVM on Power
<a class="moz-txt-link-rfc2396E" href="mailto:powerllvm@ca.ibm.com"><powerllvm@ca.ibm.com></a><br>
Subject: [EXTERNAL] RE: [llvm] 7fe41ac - Revert "[LV]
Unconditionally branch from middle to scalar preheader if the
scalar loop must execute"<br>
Date: Tue, May 25, 2021 7:53 PM<br>
<div dir="ltr" style="font-family:Arial, Helvetica,
sans-serif;font-size:10pt">
<div dir="ltr"> </div>
<div dir="ltr">Hi Philip,<br>
<br>
I have run a -print-after-all and I'm going to send you
the IR before and after the Loop Vectorize pass. Hopefully
this will help.<br>
Let me know if you need more information or if you want me
to run something on my end.<br>
</div>
<div dir="ltr">Best,</div>
<div dir="ltr">Stefan</div>
<div dir="ltr"> </div>
<blockquote data-history-content-modified="1" dir="ltr"
style="border-left:solid #aaaaaa 2px; margin-left:5px;
padding-left:5px; direction:ltr; margin-right:0px">-----
Original message -----<br>
From: Philip Reames <a class="moz-txt-link-rfc2396E" href="mailto:listmail@philipreames.com"><listmail@philipreames.com></a><br>
To: Nemanja Ivanovic <a class="moz-txt-link-rfc2396E" href="mailto:nemanjai@ca.ibm.com"><nemanjai@ca.ibm.com></a><br>
Cc: <a class="moz-txt-link-abbreviated" href="mailto:akuegel@google.com">akuegel@google.com</a>, <a class="moz-txt-link-abbreviated" href="mailto:benny.kra@gmail.com">benny.kra@gmail.com</a>,
<a class="moz-txt-link-abbreviated" href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a>, <a class="moz-txt-link-abbreviated" href="mailto:llvmlistbot@llvm.org">llvmlistbot@llvm.org</a>, LLVM on
Power <a class="moz-txt-link-rfc2396E" href="mailto:powerllvm@ca.ibm.com"><powerllvm@ca.ibm.com></a>, Stefan Pintilie
<a class="moz-txt-link-rfc2396E" href="mailto:stefanp@ca.ibm.com"><stefanp@ca.ibm.com></a><br>
Subject: [EXTERNAL] Re: [llvm] 7fe41ac - Revert "[LV]
Unconditionally branch from middle to scalar preheader if
the scalar loop must execute"<br>
Date: Tue, May 25, 2021 5:38 PM<br>
<br>
<!-- BaNnErBlUrFlE-HeAdEr-start -->
<!-- BaNnErBlUrFlE-HeAdEr-end -->
<!--Notes ACF
<meta http-equiv="Content-Type" content="text/html; charset=utf8" >-->
<!-- BaNnErBlUrFlE-BoDy-start -->
<!-- Preheader Text : BEGIN --> <span style="display:none
!important;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;">
If you have the before and after IR, that's really all I
probably need. If you share those, I can take the
investigation from there. Philip On 5/25/21 2:12 PM,
Nemanja Ivanovic wrote:
</span>
<!-- Preheader Text : END -->
<!-- Email Banner : BEGIN --> <span style="display:none
!important;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerStart</span>
<!--[if ((ie)|(mso))]>
<table border="0" cellspacing="0" cellpadding="0" width="100%" style="padding: 3px 0px 16px 0px;" ><tr><td>
<table class="pfptBannerTableMSO" border="0" cellspacing="0" cellpadding="0" style="padding: 0px 10px 5px 6px; width: 100%;border-radius:4px;border-top:4px solid #90a4ae;background-color:#d0d8dc;" ><tr><td valign="top" >
<table align="left" border="0" cellspacing="0" cellpadding="0" style="padding: 4px 8px 4px 8px" >
<tr><td><span class="pfptTitleMSO" style="color:#000000 !important;font-family: 'Arial', sans-serif;font-weight:bold !important;font-size:14px !important;" >
This Message Is From an External Sender
</span></td></tr>
<tr><td><span class="pfptSubtitleMSO" style="color:#000000 !important;font-weight:normal !important;font-family: 'Arial', sans-serif; font-size:12px !important;" >
This message came from outside your organization.
</span></td></tr>
</table>
</td></tr></table>
</td></tr></table>
<![endif]-->
<!--[if !((ie)|(mso))]-->
<div style="margin:16px 0px 16px 0px; padding:8px 16px 8px
16px; border-radius: 4px; min-width:
200px;background-color: #d0d8dc; border-top: 4px solid
#90a4ae;">
<div style="display: inline-block; margin: 0px 0px 1px
0px; max-width: 600px;">
<div style="color:#000000 !important;font-family:
'Arial', sans-serif !important;font-weight:bold
!important;font-size:14px
!important;line-height:18px;display:block;">This
Message Is From an External Sender</div>
<div style="color:#000000
!important;font-weight:normal
!important;font-family: 'Arial', sans-serif
!important;font-size:12px
!important;line-height:18px;margin-top:2px;display:block">This
message came from outside your organization.</div>
</div>
</div>
<!--[endif]--> <span style="display:none
!important;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerEnd</span><br>
<!-- Email Banner : END -->
<!-- BaNnErBlUrFlE-BoDy-end -->
<p>If you have the before and after IR, that's really all
I probably need. If you share those, I can take the
investigation from there.</p>
<p>Philip</p>
<div>On 5/25/21 2:12 PM, Nemanja Ivanovic wrote:</div>
<blockquote
cite="mid:OFC99B9E23.4F6BE4A3-ON002586E0.00745021-002586E0.00748300@notes.na.collabserv.com"
type="cite"><!--Notes ACF
<meta http-equiv="content-type" content="text/html; charset=utf8" >-->
<div dir="ltr" style="font-family:Arial, Helvetica,
sans-serif;font-size:10pt">
<div dir="ltr">Hi Philip,</div>
<div dir="ltr">I am sorry about the late reply - this
was a 4 day weekend here in Toronto. We kind of
started with looking at the difference in the
post-LV IR with and without your patch for the
SingleSource test case. Stefan is currently looking
at it and we're hoping to figure out what's going on
soon. Would it help you if we shared the two IR
files with you to get your opinion on what might be
happening as well?</div>
<div dir="ltr"> </div>
<div dir="ltr">Nemanja Ivanovic<br>
LLVM PPC Backend Development<br>
IBM Toronto Lab<br>
Email: <a href="mailto:nemanjai@ca.ibm.com"
target="_blank" moz-do-not-send="true">nemanjai@ca.ibm.com</a><br>
Phone: 905-413-3388</div>
<div dir="ltr"> </div>
<div dir="ltr"> </div>
<blockquote data-history-content-modified="1"
dir="ltr" style="border-left:solid #aaaaaa 2px;
margin-left:5px; padding-left:5px; direction:ltr;
margin-right:0px">----- Original message -----<br>
From: Philip Reames <a
href="mailto:listmail@philipreames.com"
target="_blank" moz-do-not-send="true"><listmail@philipreames.com></a><br>
To: Nemanja Ivanovic <a
href="mailto:nemanjai@ca.ibm.com" target="_blank"
moz-do-not-send="true"><nemanjai@ca.ibm.com></a><br>
Cc: <a href="mailto:akuegel@google.com"
target="_blank" moz-do-not-send="true">akuegel@google.com</a>,
<a href="mailto:benny.kra@gmail.com" target="_blank"
moz-do-not-send="true">benny.kra@gmail.com</a>, <a
href="mailto:llvm-commits@lists.llvm.org"
target="_blank" moz-do-not-send="true">llvm-commits@lists.llvm.org</a>,
<a href="mailto:llvmlistbot@llvm.org"
target="_blank" moz-do-not-send="true">llvmlistbot@llvm.org</a>,
LLVM on Power <a href="mailto:powerllvm@ca.ibm.com"
target="_blank" moz-do-not-send="true"><powerllvm@ca.ibm.com></a><br>
Subject: [EXTERNAL] Re: [llvm] 7fe41ac - Revert
"[LV] Unconditionally branch from middle to scalar
preheader if the scalar loop must execute"<br>
Date: Wed, May 19, 2021 2:15 PM<br>
<br>
<!--Notes ACF
<meta http-equiv="Content-Type" content="text/html; charset=utf8" >-->
<p>Looking at the various failures, I see both LE
and BE bots failing. The BE bot shows a
miscompile in one of the test suite benchmarks.
The LE bots appear to be showing a crash while
using a stage1 build clang to build stage2 clang. </p>
<p>LE example: <a
href="https://lab.llvm.org/buildbot#builders/19/builds/4236"
moz-do-not-send="true" target="_blank">https://lab.llvm.org/buildbot#builders/19/builds/4236</a><br>
BE example: <a
href="https://lab.llvm.org/buildbot#builders/100/builds/5762"
moz-do-not-send="true" target="_blank">https://lab.llvm.org/buildbot#builders/100/builds/5762</a></p>
<p>So both are showing miscompiles, just with
different symptoms. Frankly, the BE looks easier
to debug (much code making it into the miscompiled
binary.)</p>
<p>Oddly, pretty much only PPC bots are failing.
The one exception is a stage2 failure on AArch64 (<a
href="https://lab.llvm.org/buildbot/#/builders/111/builds/2027"
moz-do-not-send="true" target="_blank">https://lab.llvm.org/buildbot/#/builders/111/builds/2027</a>)
which looks similar to the LE failure above. </p>
<p>Given this appears to be target specific, I am
*guessing* there's some vectorizer hook which is
causing a different codepath to be executed.
Before we start trying to get me access to
hardware, do you have any guesses on what that
hook might be? If you can give me a good hint on
where to look, I suspect I can probably find the
issue that way. </p>
<p>Philip</p>
<div>On 5/18/21 3:07 AM, Nemanja Ivanovic wrote:</div>
<blockquote
cite="mid:OF443062AC.F5998286-ON002586D9.00373B50-002586D9.0037A584@notes.na.collabserv.com"
type="cite"><!--Notes ACF
<meta http-equiv="content-type" content="text/html; charset=utf8" >-->
<div dir="ltr" style="font-family:Arial,
Helvetica, sans-serif;font-size:10pt">
<div dir="ltr">Hi Philip,</div>
<div dir="ltr">I am not sure what happened with
your first attempt to contact us and how we
missed it. We would be more than happy to help
you debug this issue. Do you know if this only
affects the big endian bot or if it also fails
on little endian bots? In the latter case, we
can certainly provide access to a little
endian machine hosted at OSU/OSL. In the
former case, we don't have a machine available
and we'll have to do the debugging and report
to you (which might take a bit longer).</div>
<div dir="ltr"> </div>
<div dir="ltr">Nemanja Ivanovic<br>
LLVM PPC Backend Development<br>
IBM Toronto Lab<br>
Email: <a href="mailto:nemanjai@ca.ibm.com"
moz-do-not-send="true" target="_blank">nemanjai@ca.ibm.com</a><br>
Phone: 905-413-3388</div>
<div dir="ltr"> </div>
<div dir="ltr"> </div>
<blockquote data-history-content-modified="1"
data-history-expanded="1" dir="ltr"
style="border-left:solid #aaaaaa 2px;
margin-left:5px; padding-left:5px;
direction:ltr; margin-right:0px">-----
Original message -----<br>
From: Philip Reames <a
href="mailto:listmail@philipreames.com"
moz-do-not-send="true" target="_blank"><listmail@philipreames.com></a><br>
To: Adrian Kuegel <a
href="mailto:akuegel@google.com"
moz-do-not-send="true" target="_blank"><akuegel@google.com></a><br>
Cc: Benjamin Kramer <a
href="mailto:benny.kra@gmail.com"
moz-do-not-send="true" target="_blank"><benny.kra@gmail.com></a>,
Adrian Kuegel <a
href="mailto:llvmlistbot@llvm.org"
moz-do-not-send="true" target="_blank"><llvmlistbot@llvm.org></a>,
llvm-commits <a
href="mailto:llvm-commits@lists.llvm.org"
moz-do-not-send="true" target="_blank"><llvm-commits@lists.llvm.org></a>,
<a href="mailto:powerllvm@ca.ibm.com"
moz-do-not-send="true" target="_blank">powerllvm@ca.ibm.com</a><br>
Subject: [EXTERNAL] Re: [llvm] 7fe41ac -
Revert "[LV] Unconditionally branch from
middle to scalar preheader if the scalar loop
must execute"<br>
Date: Mon, May 17, 2021 11:59 PM<br>
<br>
<!-- BaNnErBlUrFlE-HeAdEr-start -->
<!-- BaNnErBlUrFlE-HeAdEr-end -->
<!--Notes ACF
<meta http-equiv="Content-Type" content="text/html; charset=utf8" >-->
<!-- BaNnErBlUrFlE-BoDy-start -->
<!-- Preheader Text : BEGIN --> <span
style="display:none!important;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;">
I tried another cycle to see if this had
been resolved, but am still seeing build bot
failures, nearly exclusively on PPC. (And
one arm self host bot.) @PPC Bot Owner - I
need help reducing a test case for the
failure seen here: </span>
<!-- Preheader Text : END -->
<!-- Email Banner : BEGIN --> <span
style="display:none!important;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerStart</span>
<!--[if ((ie)|(mso))]>
<table border="0" cellspacing="0" cellpadding="0" width="100%" style="padding: 3px 0px 16px 0px;" ><tr><td>
<table class="pfptBannerTableMSO" border="0" cellspacing="0" cellpadding="0" style="padding: 0px 10px 5px 6px; width: 100%;border-radius:4px;border-top:4px solid #90a4ae;background-color:#d0d8dc;" ><tr><td valign="top" >
<table align="left" border="0" cellspacing="0" cellpadding="0" style="padding: 4px 8px 4px 8px" >
<tr><td><span class="pfptTitleMSO" style="color:#000000 !important;font-family: 'Arial', sans-serif;font-weight:bold !important;font-size:14px !important;" >
This Message Is From an External Sender
</span></td></tr>
<tr><td><span class="pfptSubtitleMSO" style="color:#000000 !important;font-weight:normal !important;font-family: 'Arial', sans-serif; font-size:12px !important;" >
This message came from outside your organization.
</span></td></tr>
</table>
</td></tr></table>
</td></tr></table>
<![endif]-->
<!--[if !((ie)|(mso))]-->
<div style="margin:16px 0px 16px 0px;
padding:8px 16px 8px 16px; border-radius:
4px; min-width: 200px;background-color:
#d0d8dc; border-top: 4px solid #90a4ae;">
<div style="display: inline-block; margin:
0px 0px 1px 0px; max-width: 600px;">
<div style="color:#000000
!important;font-family: 'Arial',
sans-serif !important;font-weight:bold
!important;font-size:14px
!important;line-height:18px;display:block;">This
Message Is From an External Sender</div>
<div style="color:#000000
!important;font-weight:normal
!important;font-family: 'Arial',
sans-serif !important;font-size:12px
!important;line-height:18px;margin-top:2px;display:block">This
message came from outside your
organization.</div>
</div>
</div>
<!--[endif]--> <span
style="display:none!important;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerEnd</span><br>
<!-- Email Banner : END -->
<!-- BaNnErBlUrFlE-BoDy-end -->
<p>I tried another cycle to see if this had
been resolved, but am still seeing build bot
failures, nearly exclusively on PPC. (And
one arm self host bot.) </p>
<p>@PPC Bot Owner - I need help reducing a
test case for the failure seen here: <a
href="https://lab.llvm.org/buildbot#builders/100/builds/5762"
moz-do-not-send="true" target="_blank">https://lab.llvm.org/buildbot#builders/100/builds/5762</a>.
I am stuck, and unable to make progress for
nearly 3 months now. I would greatly
appreciate help. </p>
<p>Philip</p>
<div>On 5/5/21 11:59 PM, Adrian Kuegel wrote:</div>
<blockquote
cite="mid:CAHhV=OX3NbAExy4YsQWLH0fC-zXiG_VpToAtB72NhkUwwYekRQ@mail.gmail.com"
type="cite"><!--Notes ACF
<meta http-equiv="content-type" content="text/html; charset=utf8" >-->
<div dir="ltr">
<div dir="ltr">Sounds good to me, and
thanks for sending the heads up :)</div>
<div>
<div dir="ltr">On Wed, May 5, 2021 at
10:01 PM Philip Reames <<a
href="mailto:listmail@philipreames.com"
moz-do-not-send="true"
target="_blank">listmail@philipreames.com</a>>
wrote:</div>
<blockquote style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">FYI,
I'm going to try recommitting this
without changes in a day or so.<br>
<br>
I never heard back from a PPC bot
owner, and I don't have enough<br>
information to really debug anything
from the builtbot log. I did run<br>
across a latent issue which this patch
may very well have exposed at<br>
much higher frequency; the previous
patch in this series (which is much<br>
more restrictive) does appear to have
increased frequency. That was<br>
worked around in 80e80250. My
educated guess is that same issue<br>
triggered the miscompile seen on the
ppc bot, but that is more of a<br>
guess than I'd really prefer.<br>
<br>
I'm going to submit this during off
hours, and watch the bots fairly<br>
closely after submit. Hopefully this
either cycles clean or I get a<br>
better clue as to what the root issue
is.<br>
<br>
Philip<br>
<br>
On 2/8/21 9:09 PM, Philip Reames via
llvm-commits wrote:<br>
> Ben,<br>
><br>
> Thanks for the clarification.
The log does not make the fact this is<br>
> an execution failure obvious.<br>
><br>
> No, I don't have access to a PPC
machine.<br>
><br>
> I am going to need some
assistance from the bot owner on this.
At a<br>
> minimum, IR for the test in
question (before optimization, but on<br>
> target platform) seems like a
reasonable ask.<br>
><br>
> I strongly suspect this change is
simply exposing another latent<br>
> issue. Or at least, I've
reviewed the change and don't see
anything<br>
> likely to cause runtime crashes
w/o also tripping compiler asserts.<br>
><br>
> Philip<br>
><br>
><br>
> On 2/8/21 5:21 AM, Benjamin
Kramer wrote:<br>
>> `execution_time` failures
mean that the bot succeeded building a
test<br>
>> but it failed when running
it. I'm relatively certain that this
is the<br>
>> same issue Adrian is seeing
-- binaries segfaulting early on PPC.<br>
>><br>
>> The bot log output isn't
helpful at all for investigating why
this is<br>
>> happening. Do you happen to
have access to a PPC machine?<br>
>><br>
>> On Fri, Feb 5, 2021 at 6:05
PM Philip Reames via llvm-commits<br>
>> <<a
href="mailto:llvm-commits@lists.llvm.org"
moz-do-not-send="true"
target="_blank">llvm-commits@lists.llvm.org</a>>
wrote:<br>
>>> Adrian,<br>
>>><br>
>>> I'm going to need you to
provide a bit more information here.
The test<br>
>>> failure in stage1 was
fixed at the time you reverted this
patch. The<br>
>>> remaining failure in the
bot is very unclear. What is a
execution_time<br>
>>> failure? From the log
output, the "failing" run finished in
0.5<br>
>>> seconds,<br>
>>> whereas the previous
"succeeding" run finished in 11
seconds. Without<br>
>>> further context, I'd say
that's no failure.<br>
>>><br>
>>> I'll also note that I did
not receive email from this bot. I
received<br>
>>> notice from the various
other bots and fixed the ARM test
issue, but<br>
>>> unless I missed it in
with the others, this bot is not
notifying.<br>
>>><br>
>>> In general, I'm a fan of
fast reverts, but I have to admit,
this one<br>
>>> appears borderline at the
moment.<br>
>>><br>
>>> Philip<br>
>>><br>
>>> On 2/5/21 3:53 AM, Adrian
Kuegel via llvm-commits wrote:<br>
>>>> Author: Adrian Kuegel<br>
>>>> Date:
2021-02-05T12:51:03+01:00<br>
>>>> New Revision:
7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c<br>
>>>><br>
>>>> URL:<br>
>>>> <a
href="https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c"
moz-do-not-send="true"
rel="noreferrer" target="_blank">https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c</a><br>
>>>> DIFF:<br>
>>>> <a
href="https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c.diff"
moz-do-not-send="true"
rel="noreferrer" target="_blank">https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c.diff</a><br>
>>>><br>
>>>> LOG: Revert "[LV]
Unconditionally branch from middle to
scalar<br>
>>>> preheader if the
scalar loop must execute"<br>
>>>><br>
>>>> This reverts commit
3e5ce49e5371ce4feadbf97dd5c2b652d9db3d1d.<br>
>>>><br>
>>>> Tests started failing
on PPC, for example:<br>
>>>> <a
href="http://lab.llvm.org:8011/#/builders/105/builds/5569"
moz-do-not-send="true"
rel="noreferrer" target="_blank">http://lab.llvm.org:8011/#/builders/105/builds/5569</a><br>
>>>><br>
>>>> Added:<br>
>>>><br>
>>>><br>
>>>> Modified:<br>
>>>>
llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
>>>>
llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
>>>>
llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
>>>>
llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
>>>>
llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
>>>><br>
>>>> Removed:<br>
>>>><br>
>>>><br>
>>>><br>
>>>>
################################################################################<br>
>>>><br>
>>>> diff --git
a/llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
>>>>
b/llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
>>>> index
8a89158788cf..de4fb446fdf2 100644<br>
>>>> ---
a/llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
>>>> +++
b/llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
>>>> @@ -44,11 +44,11 @@
LoopVersioning::LoopVersioning(const<br>
>>>> LoopAccessInfo
&LAI,<br>
>>>>
AliasChecks(Checks.begin(),
Checks.end()),<br>
>>>>
Preds(LAI.getPSE().getUnionPredicate()),
LAI(LAI), LI(LI),<br>
>>>> DT(DT),<br>
>>>> SE(SE) {<br>
>>>> +
assert(L->getUniqueExitBlock()
&& "No single exit block");<br>
>>>> }<br>
>>>><br>
>>>> void
LoopVersioning::versionLoop(<br>
>>>> const
SmallVectorImpl<Instruction *>
&DefsUsedOutside) {<br>
>>>> -
assert(VersionedLoop->getUniqueExitBlock()
&& "No single exit<br>
>>>> block");<br>
>>>>
assert(VersionedLoop->isLoopSimplifyForm()
&&<br>
>>>> "Loop is
not in loop-simplify form");<br>
>>>><br>
>>>><br>
>>>> diff --git
a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
>>>>
b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
>>>> index
3277842edbfe..6bce0caeb36f 100644<br>
>>>> ---
a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
>>>> +++
b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
>>>> @@ -852,7 +852,7 @@
class InnerLoopVectorizer {<br>
>>>> /// Middle Block
between the vector and the scalar.<br>
>>>> BasicBlock
*LoopMiddleBlock;<br>
>>>><br>
>>>> - /// The unique
ExitBlock of the scalar loop if one
exists. Note<br>
>>>> that<br>
>>>> + /// The (unique)
ExitBlock of the scalar loop. Note
that<br>
>>>> /// there can be
multiple exiting edges reaching this
block.<br>
>>>> BasicBlock
*LoopExitBlock;<br>
>>>><br>
>>>> @@ -3147,13 +3147,9
@@ void<br>
>>>>
InnerLoopVectorizer::emitMinimumIterationCountCheck(Loop
*L,<br>
>>>>
DT->getNode(Bypass)->getIDom())
&&<br>
>>>> "TC check
is expected to dominate Bypass");<br>
>>>><br>
>>>> - // Update
dominator for Bypass & LoopExit
(if needed).<br>
>>>> + // Update
dominator for Bypass & LoopExit.<br>
>>>>
DT->changeImmediateDominator(Bypass,
TCCheckBlock);<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - // If there is
an epilogue which must run, there's no
edge<br>
>>>> from the<br>
>>>> - // middle block
to exit blocks and thus no need to
update the<br>
>>>> immediate<br>
>>>> - // dominator of
the exit blocks.<br>
>>>> -
DT->changeImmediateDominator(LoopExitBlock,
TCCheckBlock);<br>
>>>> +
DT->changeImmediateDominator(LoopExitBlock,
TCCheckBlock);<br>
>>>><br>
>>>>
ReplaceInstWithInst(<br>
>>>>
TCCheckBlock->getTerminator(),<br>
>>>> @@ -3192,11 +3188,7
@@ void<br>
>>>>
InnerLoopVectorizer::emitSCEVChecks(Loop
*L, BasicBlock *Bypass) {<br>
>>>> // Update
dominator only if this is first RT
check.<br>
>>>> if
(LoopBypassBlocks.empty()) {<br>
>>>>
DT->changeImmediateDominator(Bypass,
SCEVCheckBlock);<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - // If there is
an epilogue which must run, there's no
edge<br>
>>>> from the<br>
>>>> - // middle
block to exit blocks and thus no need
to update<br>
>>>> the immediate<br>
>>>> - // dominator
of the exit blocks.<br>
>>>> -
DT->changeImmediateDominator(LoopExitBlock,
SCEVCheckBlock);<br>
>>>> +
DT->changeImmediateDominator(LoopExitBlock,
SCEVCheckBlock);<br>
>>>> }<br>
>>>><br>
>>>>
ReplaceInstWithInst(<br>
>>>> @@ -3252,11 +3244,7
@@ void<br>
>>>>
InnerLoopVectorizer::emitMemRuntimeChecks(Loop
*L, BasicBlock<br>
>>>> *Bypass) {<br>
>>>> // Update
dominator only if this is first RT
check.<br>
>>>> if
(LoopBypassBlocks.empty()) {<br>
>>>>
DT->changeImmediateDominator(Bypass,
MemCheckBlock);<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - // If there is
an epilogue which must run, there's no
edge<br>
>>>> from the<br>
>>>> - // middle
block to exit blocks and thus no need
to update<br>
>>>> the immediate<br>
>>>> - // dominator
of the exit blocks.<br>
>>>> -
DT->changeImmediateDominator(LoopExitBlock,
MemCheckBlock);<br>
>>>> +
DT->changeImmediateDominator(LoopExitBlock,
MemCheckBlock);<br>
>>>> }<br>
>>>><br>
>>>> Instruction
*FirstCheckInst;<br>
>>>> @@ -3381,10 +3369,9
@@ Value<br>
>>>>
*InnerLoopVectorizer::emitTransformedIndex(<br>
>>>> Loop
*InnerLoopVectorizer::createVectorLoopSkeleton(StringRef<br>
>>>> Prefix) {<br>
>>>> LoopScalarBody =
OrigLoop->getHeader();<br>
>>>>
LoopVectorPreHeader =
OrigLoop->getLoopPreheader();<br>
>>>> + LoopExitBlock =
OrigLoop->getUniqueExitBlock();<br>
>>>> +
assert(LoopExitBlock && "Must
have an exit block");<br>
>>>>
assert(LoopVectorPreHeader &&
"Invalid loop structure");<br>
>>>> - LoopExitBlock =
OrigLoop->getUniqueExitBlock(); //
may be nullptr<br>
>>>> -
assert((LoopExitBlock ||
Cost->requiresScalarEpilogue())
&&<br>
>>>> - "multiple
exit loop without required
epilogue?");<br>
>>>><br>
>>>> LoopMiddleBlock
=<br>
>>>>
SplitBlock(LoopVectorPreHeader,<br>
>>>>
LoopVectorPreHeader->getTerminator(),
DT,<br>
>>>> @@ -3393,20 +3380,12
@@ Loop<br>
>>>>
*InnerLoopVectorizer::createVectorLoopSkeleton(StringRef
Prefix) {<br>
>>>>
SplitBlock(LoopMiddleBlock,<br>
>>>>
LoopMiddleBlock->getTerminator(),
DT, LI,<br>
>>>>
nullptr, Twine(Prefix) + "<a
href="http://scalar.ph"
moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>");<br>
>>>><br>
>>>> + // Set up branch
from middle block to the exit and
scalar<br>
>>>> preheader blocks.<br>
>>>> + //
completeLoopSkeleton will update the
condition to use an<br>
>>>> iteration check,<br>
>>>> + // if required to
decide whether to execute the
remainder.<br>
>>>> + BranchInst *BrInst
=<br>
>>>> +
BranchInst::Create(LoopExitBlock,
LoopScalarPreHeader,<br>
>>>> Builder.getTrue());<br>
>>>> auto
*ScalarLatchTerm =<br>
>>>>
OrigLoop->getLoopLatch()->getTerminator();<br>
>>>> -<br>
>>>> - // Set up the
middle block terminator. Two cases:<br>
>>>> - // 1) If we know
that we must execute the scalar
epilogue, emit an<br>
>>>> - //
unconditional branch.<br>
>>>> - // 2) Otherwise,
we must have a single unique exit
block (due to<br>
>>>> how we<br>
>>>> - // implement
the multiple exit case). In this
case, set up a<br>
>>>> conditonal<br>
>>>> - // branch from
the middle block to the loop scalar
preheader,<br>
>>>> and the<br>
>>>> - // exit block.
completeLoopSkeleton will update the<br>
>>>> condition to use an<br>
>>>> - // iteration
check, if required to decide whether
to execute<br>
>>>> the remainder.<br>
>>>> - BranchInst *BrInst
= Cost->requiresScalarEpilogue() ?<br>
>>>> -
BranchInst::Create(LoopScalarPreHeader)
:<br>
>>>> -
BranchInst::Create(LoopExitBlock,
LoopScalarPreHeader,<br>
>>>>
-
Builder.getTrue());<br>
>>>>
BrInst->setDebugLoc(ScalarLatchTerm->getDebugLoc());<br>
>>>>
ReplaceInstWithInst(LoopMiddleBlock->getTerminator(),
BrInst);<br>
>>>><br>
>>>> @@ -3418,11 +3397,7
@@ Loop<br>
>>>>
*InnerLoopVectorizer::createVectorLoopSkeleton(StringRef
Prefix) {<br>
>>>>
nullptr, nullptr, Twine(Prefix) +
"vector.body");<br>
>>>><br>
>>>> // Update
dominator for loop exit.<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - // If there is
an epilogue which must run, there's no
edge<br>
>>>> from the<br>
>>>> - // middle block
to exit blocks and thus no need to
update the<br>
>>>> immediate<br>
>>>> - // dominator of
the exit blocks.<br>
>>>> -
DT->changeImmediateDominator(LoopExitBlock,
LoopMiddleBlock);<br>
>>>> +
DT->changeImmediateDominator(LoopExitBlock,
LoopMiddleBlock);<br>
>>>><br>
>>>> // Create and
register the new vector loop.<br>
>>>> Loop *Lp =
LI->AllocateLoop();<br>
>>>> @@ -3519,14 +3494,10
@@ BasicBlock<br>
>>>>
*InnerLoopVectorizer::completeLoopSkeleton(Loop
*L,<br>
>>>> auto
*ScalarLatchTerm =<br>
>>>>
OrigLoop->getLoopLatch()->getTerminator();<br>
>>>><br>
>>>> // Add a check
in the middle block to see if we have
completed<br>
>>>> - // all of the
iterations in the first vector loop.
Three cases:<br>
>>>> - // 1) If we
require a scalar epilogue, there is no
conditional<br>
>>>> branch as<br>
>>>> - // we
unconditionally branch to the scalar
preheader. Do<br>
>>>> nothing.<br>
>>>> - // 2) If (N -
N%VF) == N, then we *don't* need to
run the<br>
>>>> remainder.<br>
>>>> - // Thus if tail
is to be folded, we know we don't need
to run<br>
>>>> the<br>
>>>> - // remainder
and we can use the previous value for
the<br>
>>>> condition (true).<br>
>>>> - // 3) Otherwise,
construct a runtime check.<br>
>>>> - if
(!Cost->requiresScalarEpilogue()
&&<br>
>>>>
!Cost->foldTailByMasking()) {<br>
>>>> + // all of the
iterations in the first vector loop.<br>
>>>> + // If (N - N%VF)
== N, then we *don't* need to run the
remainder.<br>
>>>> + // If tail is to
be folded, we know we don't need to
run the<br>
>>>> remainder.<br>
>>>> + if
(!Cost->foldTailByMasking()) {<br>
>>>> Instruction
*CmpN =
CmpInst::Create(Instruction::ICmp,<br>
>>>> CmpInst::ICMP_EQ,<br>
>>>>
Count, VectorTripCount,<br>
>>>> "cmp.n",<br>
>>>>
LoopMiddleBlock->getTerminator());<br>
>>>> @@ -3590,17 +3561,17
@@ BasicBlock<br>
>>>>
*InnerLoopVectorizer::createVectorizedLoopSkeleton()
{<br>
>>>> | [ ]_|
<-- vector loop.<br>
>>>> | |<br>
>>>> | v<br>
>>>> - \ -[ ] <---
middle-block.<br>
>>>> - \/ |<br>
>>>> - /\ v<br>
>>>> - | ->[ ]
<--- new preheader.<br>
>>>> + | -[ ] <---
middle-block.<br>
>>>> + | / |<br>
>>>> + | / v<br>
>>>> + -|- >[ ]
<--- new preheader.<br>
>>>> | |<br>
>>>> - (opt) v
<-- edge from middle to exit iff
epilogue is not<br>
>>>> required.<br>
>>>> + | v<br>
>>>> | [ ] \<br>
>>>> - | [ ]_|
<-- old scalar loop to handle
remainder (scalar<br>
>>>> epilogue).<br>
>>>> + | [ ]_|
<-- old scalar loop to handle
remainder.<br>
>>>> \ |<br>
>>>> \ v<br>
>>>> - >[ ]
<-- exit block(s).<br>
>>>> + >[ ]
<-- exit block.<br>
>>>> ...<br>
>>>> */<br>
>>>><br>
>>>> @@ -4021,18 +3992,13
@@ void<br>
>>>>
InnerLoopVectorizer::fixVectorizedLoop()
{<br>
>>>> // Forget the
original basic block.<br>
>>>>
PSE.getSE()->forgetLoop(OrigLoop);<br>
>>>><br>
>>>> - // If we inserted
an edge from the middle block to the
unique<br>
>>>> exit block,<br>
>>>> - // update uses
outside the loop (phis) to account for
the newly<br>
>>>> inserted<br>
>>>> - // edge.<br>
>>>> - if
(!Cost->requiresScalarEpilogue()) {<br>
>>>> - // Fix-up
external users of the induction
variables.<br>
>>>> - for (auto
&Entry :
Legal->getInductionVars())<br>
>>>> -
fixupIVUsers(Entry.first,
Entry.second,<br>
>>>> -
getOrCreateVectorTripCount(LI->getLoopFor(LoopVectorBody)),<br>
>>>> -
IVEndValues[Entry.first],
LoopMiddleBlock);<br>
>>>> + // Fix-up external
users of the induction variables.<br>
>>>> + for (auto
&Entry :
Legal->getInductionVars())<br>
>>>> +
fixupIVUsers(Entry.first,
Entry.second,<br>
>>>> +
getOrCreateVectorTripCount(LI->getLoopFor(LoopVectorBody)),<br>
>>>> +
IVEndValues[Entry.first],
LoopMiddleBlock);<br>
>>>><br>
>>>> - fixLCSSAPHIs();<br>
>>>> - }<br>
>>>> + fixLCSSAPHIs();<br>
>>>> for (Instruction
*PI : PredicatedInstructions)<br>
>>>>
sinkScalarOperands(&*PI);<br>
>>>><br>
>>>> @@ -4250,13 +4216,12
@@ void<br>
>>>>
InnerLoopVectorizer::fixFirstOrderRecurrence(PHINode
*Phi) {<br>
>>>> // recurrence in
the exit block, and then add an edge
for the<br>
>>>> middle block.<br>
>>>> // Note that
LCSSA does not imply single entry when
the<br>
>>>> original scalar loop<br>
>>>> // had multiple
exiting edges (as we always run the
last<br>
>>>> iteration in the<br>
>>>> - // scalar
epilogue); in that case, there is no
edge from middle<br>
>>>> to exit and<br>
>>>> - // and thus no
phis which needed updated.<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - for (PHINode
&LCSSAPhi :
LoopExitBlock->phis())<br>
>>>> - if
(any_of(LCSSAPhi.incoming_values(),<br>
>>>> -
[Phi](Value *V) { return V == Phi; }))<br>
>>>> -
LCSSAPhi.addIncoming(ExtractForPhiUsedOutsideLoop,<br>
>>>> LoopMiddleBlock);<br>
>>>> + // scalar
epilogue); in that case, the exiting
path through<br>
>>>> middle will be<br>
>>>> + // dynamically
dead and the value picked for the phi
doesn't<br>
>>>> matter.<br>
>>>> + for (PHINode
&LCSSAPhi :
LoopExitBlock->phis())<br>
>>>> + if
(any_of(LCSSAPhi.incoming_values(),<br>
>>>> +
[Phi](Value *V) { return V == Phi; }))<br>
>>>> +
LCSSAPhi.addIncoming(ExtractForPhiUsedOutsideLoop,<br>
>>>> LoopMiddleBlock);<br>
>>>> }<br>
>>>><br>
>>>> void
InnerLoopVectorizer::fixReduction(PHINode
*Phi) {<br>
>>>> @@ -4421,11 +4386,10
@@ void<br>
>>>>
InnerLoopVectorizer::fixReduction(PHINode
*Phi) {<br>
>>>> // We know that
the loop is in LCSSA form. We need to
update<br>
>>>> the PHI nodes<br>
>>>> // in the exit
blocks. See comment on analogous loop
in<br>
>>>> //
fixFirstOrderRecurrence for a more
complete explaination of<br>
>>>> the logic.<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - for (PHINode
&LCSSAPhi :
LoopExitBlock->phis())<br>
>>>> - if
(any_of(LCSSAPhi.incoming_values(),<br>
>>>> -
[LoopExitInst](Value *V) { return V ==<br>
>>>> LoopExitInst; }))<br>
>>>> -
LCSSAPhi.addIncoming(ReducedPartRdx,
LoopMiddleBlock);<br>
>>>> + for (PHINode
&LCSSAPhi :
LoopExitBlock->phis())<br>
>>>> + if
(any_of(LCSSAPhi.incoming_values(),<br>
>>>> +
[LoopExitInst](Value *V) { return V ==<br>
>>>> LoopExitInst; }))<br>
>>>> +
LCSSAPhi.addIncoming(ReducedPartRdx,
LoopMiddleBlock);<br>
>>>><br>
>>>> // Fix the
scalar loop reduction variable with
the incoming<br>
>>>> reduction sum<br>
>>>> // from the
vector body and from the backedge
value.<br>
>>>> @@ -8074,11 +8038,7
@@ BasicBlock<br>
>>>>
*EpilogueVectorizerMainLoop::emitMinimumIterationCountCheck(<br>
>>>><br>
>>>> // Update
dominator for Bypass & LoopExit.<br>
>>>>
DT->changeImmediateDominator(Bypass,
TCCheckBlock);<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - // For loops
with multiple exits, there's no edge
from the<br>
>>>> middle block<br>
>>>> - // to exit
blocks (as the epilogue must run) and
thus no<br>
>>>> need to update<br>
>>>> - // the
immediate dominator of the exit
blocks.<br>
>>>> -
DT->changeImmediateDominator(LoopExitBlock,
TCCheckBlock);<br>
>>>> +
DT->changeImmediateDominator(LoopExitBlock,
TCCheckBlock);<br>
>>>><br>
>>>>
LoopBypassBlocks.push_back(TCCheckBlock);<br>
>>>><br>
>>>> @@ -8142,12 +8102,7
@@<br>
>>>>
EpilogueVectorizerEpilogueLoop::createEpilogueVectorizedLoopSkeleton()<br>
>>>> {<br>
>>>><br>
>>>>
DT->changeImmediateDominator(LoopScalarPreHeader,<br>
>>>>
EPI.EpilogueIterationCountCheck);<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - // If there is
an epilogue which must run, there's no
edge<br>
>>>> from the<br>
>>>> - // middle block
to exit blocks and thus no need to
update the<br>
>>>> immediate<br>
>>>> - // dominator of
the exit blocks.<br>
>>>> -
DT->changeImmediateDominator(LoopExitBlock,<br>
>>>> -
EPI.EpilogueIterationCountCheck);<br>
>>>> +
DT->changeImmediateDominator(LoopExitBlock,<br>
>>>>
EPI.EpilogueIterationCountCheck);<br>
>>>><br>
>>>> // Keep track of
bypass blocks, as they feed start
values to<br>
>>>> the induction<br>
>>>> // phis in the
scalar loop preheader.<br>
>>>><br>
>>>> diff --git<br>
>>>>
a/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
>>>>
b/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
>>>> index
ec280bf5d5e4..7d4a3c5c9935 100644<br>
>>>> ---<br>
>>>>
a/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
>>>> +++<br>
>>>>
b/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
>>>> @@ -471,9 +471,10 @@
define i16 @multiple_exit(i16* %p, i32
%n) {<br>
>>>> ; CHECK-NEXT:
[[TMP15:%.*]] = icmp eq i32
[[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT:
br i1 [[TMP15]], label
[[MIDDLE_BLOCK:%.*]],<br>
>>>> label
[[VECTOR_BODY]], [[LOOP6:!llvm.loop
!.*]]<br>
>>>> ; CHECK:
middle.block:<br>
>>>> +; CHECK-NEXT:
[[CMP_N:%.*]] = icmp eq i32 [[TMP2]],
[[N_VEC]]<br>
>>>> ; CHECK-NEXT:
[[VECTOR_RECUR_EXTRACT:%.*]] =
extractelement<br>
>>>> <4 x i16>
[[WIDE_LOAD]], i32 3<br>
>>>> ; CHECK-NEXT:
[[VECTOR_RECUR_EXTRACT_FOR_PHI:%.*]] =<br>
>>>> extractelement <4
x i16> [[WIDE_LOAD]], i32 2<br>
>>>> -; CHECK-NEXT: br
label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP_N]], label [[IF_END:%.*]],
label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph"
moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[SCALAR_RECUR_INIT:%.*]] = phi i16 [
0,<br>
>>>> [[ENTRY:%.*]] ], [
[[VECTOR_RECUR_EXTRACT]],
[[MIDDLE_BLOCK]] ]<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i32 [
[[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY]] ]<br>
>>>> @@ -485,14 +486,14 @@
define i16 @multiple_exit(i16* %p, i32
%n) {<br>
>>>> ; CHECK-NEXT:
[[B:%.*]] = getelementptr inbounds
i16, i16*<br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> ; CHECK-NEXT:
[[REC_NEXT]] = load i16, i16* [[B]],
align 2<br>
>>>> ; CHECK-NEXT:
[[CMP:%.*]] = icmp slt i32 [[I]],
[[N]]<br>
>>>> -; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label<br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label
[[IF_END]]<br>
>>>> ; CHECK:
for.body:<br>
>>>> ; CHECK-NEXT:
store i16 [[SCALAR_RECUR]], i16*
[[B]], align 4<br>
>>>> ; CHECK-NEXT:
[[INC]] = add nsw i32 [[I]], 1<br>
>>>> ; CHECK-NEXT:
[[CMP2:%.*]] = icmp slt i32 [[I]],
2096<br>
>>>> ; CHECK-NEXT:
br i1 [[CMP2]], label [[FOR_COND]],
label<br>
>>>> [[IF_END]],
[[LOOP7:!llvm.loop !.*]]<br>
>>>> ; CHECK:
if.end:<br>
>>>> -; CHECK-NEXT:
[[REC_LCSSA:%.*]] = phi i16 [
[[SCALAR_RECUR]],<br>
>>>> [[FOR_BODY]] ], [
[[SCALAR_RECUR]], [[FOR_COND]] ]<br>
>>>> +; CHECK-NEXT:
[[REC_LCSSA:%.*]] = phi i16 [
[[SCALAR_RECUR]],<br>
>>>> [[FOR_BODY]] ], [
[[SCALAR_RECUR]], [[FOR_COND]] ], [<br>
>>>>
[[VECTOR_RECUR_EXTRACT_FOR_PHI]],
[[MIDDLE_BLOCK]] ]<br>
>>>> ; CHECK-NEXT:
ret i16 [[REC_LCSSA]]<br>
>>>> ;<br>
>>>> entry:<br>
>>>> @@ -557,9 +558,10 @@
define i16 @multiple_exit2(i16* %p,
i32 %n) {<br>
>>>> ; CHECK-NEXT:
[[TMP15:%.*]] = icmp eq i32
[[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT:
br i1 [[TMP15]], label
[[MIDDLE_BLOCK:%.*]],<br>
>>>> label
[[VECTOR_BODY]], [[LOOP8:!llvm.loop
!.*]]<br>
>>>> ; CHECK:
middle.block:<br>
>>>> +; CHECK-NEXT:
[[CMP_N:%.*]] = icmp eq i32 [[TMP2]],
[[N_VEC]]<br>
>>>> ; CHECK-NEXT:
[[VECTOR_RECUR_EXTRACT:%.*]] =
extractelement<br>
>>>> <4 x i16>
[[WIDE_LOAD]], i32 3<br>
>>>> ; CHECK-NEXT:
[[VECTOR_RECUR_EXTRACT_FOR_PHI:%.*]] =<br>
>>>> extractelement <4
x i16> [[WIDE_LOAD]], i32 2<br>
>>>> -; CHECK-NEXT: br
label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP_N]], label [[IF_END:%.*]],
label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph"
moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[SCALAR_RECUR_INIT:%.*]] = phi i16 [
0,<br>
>>>> [[ENTRY:%.*]] ], [
[[VECTOR_RECUR_EXTRACT]],
[[MIDDLE_BLOCK]] ]<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i32 [
[[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY]] ]<br>
>>>> @@ -571,14 +573,14 @@
define i16 @multiple_exit2(i16* %p,
i32 %n) {<br>
>>>> ; CHECK-NEXT:
[[B:%.*]] = getelementptr inbounds
i16, i16*<br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> ; CHECK-NEXT:
[[REC_NEXT]] = load i16, i16* [[B]],
align 2<br>
>>>> ; CHECK-NEXT:
[[CMP:%.*]] = icmp slt i32 [[I]],
[[N]]<br>
>>>> -; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label<br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label
[[IF_END]]<br>
>>>> ; CHECK:
for.body:<br>
>>>> ; CHECK-NEXT:
store i16 [[SCALAR_RECUR]], i16*
[[B]], align 4<br>
>>>> ; CHECK-NEXT:
[[INC]] = add nsw i32 [[I]], 1<br>
>>>> ; CHECK-NEXT:
[[CMP2:%.*]] = icmp slt i32 [[I]],
2096<br>
>>>> ; CHECK-NEXT:
br i1 [[CMP2]], label [[FOR_COND]],
label<br>
>>>> [[IF_END]],
[[LOOP9:!llvm.loop !.*]]<br>
>>>> ; CHECK:
if.end:<br>
>>>> -; CHECK-NEXT:
[[REC_LCSSA:%.*]] = phi i16 [
[[SCALAR_RECUR]],<br>
>>>> [[FOR_COND]] ], [ 10,
[[FOR_BODY]] ]<br>
>>>> +; CHECK-NEXT:
[[REC_LCSSA:%.*]] = phi i16 [
[[SCALAR_RECUR]],<br>
>>>> [[FOR_COND]] ], [ 10,
[[FOR_BODY]] ], [<br>
>>>>
[[VECTOR_RECUR_EXTRACT_FOR_PHI]],
[[MIDDLE_BLOCK]] ]<br>
>>>> ; CHECK-NEXT:
ret i16 [[REC_LCSSA]]<br>
>>>> ;<br>
>>>> entry:<br>
>>>><br>
>>>> diff --git<br>
>>>>
a/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
>>>>
b/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
>>>> index
f0ba677348ab..0d4bdf0ecac3 100644<br>
>>>> ---
a/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
>>>> +++
b/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
>>>> @@ -447,7 +447,7 @@
define void @even_load_static_tc(i32*
noalias<br>
>>>> nocapture readonly
%A, i32* noalia<br>
>>>> ; CHECK-NEXT:
[[TMP6:%.*]] = icmp eq i64
[[INDEX_NEXT]], 508<br>
>>>> ; CHECK-NEXT:
br i1 [[TMP6]], label
[[MIDDLE_BLOCK:%.*]],<br>
>>>> label
[[VECTOR_BODY]], [[LOOP12:!llvm.loop
!.*]]<br>
>>>> ; CHECK:
middle.block:<br>
>>>> -; CHECK-NEXT: br
label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br
i1 false, label
[[FOR_COND_CLEANUP:%.*]],<br>
>>>> label [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph"
moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [
1016,<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
br label [[FOR_BODY:%.*]]<br>
>>>> @@ -463,7 +463,7 @@
define void @even_load_static_tc(i32*
noalias<br>
>>>> nocapture readonly
%A, i32* noalia<br>
>>>> ; CHECK-NEXT:
store i32 [[MUL]], i32* [[ARRAYIDX2]],
align 4<br>
>>>> ; CHECK-NEXT:
[[INDVARS_IV_NEXT]] = add nuw nsw i64<br>
>>>> [[INDVARS_IV]], 2<br>
>>>> ; CHECK-NEXT:
[[CMP:%.*]] = icmp ult i64
[[INDVARS_IV]], 1022<br>
>>>> -; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label<br>
>>>>
[[FOR_COND_CLEANUP:%.*]],
[[LOOP13:!llvm.loop !.*]]<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label<br>
>>>> [[FOR_COND_CLEANUP]],
[[LOOP13:!llvm.loop !.*]]<br>
>>>> ;<br>
>>>> entry:<br>
>>>> br label
%for.body<br>
>>>> @@ -528,7 +528,7 @@
define void @even_load_dynamic_tc(i32*
noalias<br>
>>>> nocapture readonly
%A, i32* noali<br>
>>>> ; CHECK-NEXT:
[[TMP12:%.*]] = icmp eq i64
[[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT:
br i1 [[TMP12]], label
[[MIDDLE_BLOCK:%.*]],<br>
>>>> label
[[VECTOR_BODY]], [[LOOP14:!llvm.loop
!.*]]<br>
>>>> ; CHECK:
middle.block:<br>
>>>> -; CHECK-NEXT: br
label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br
i1 false, label
[[FOR_COND_CLEANUP:%.*]],<br>
>>>> label [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph"
moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [
[[IND_END]],<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
br label [[FOR_BODY:%.*]]<br>
>>>> @@ -544,7 +544,7 @@
define void @even_load_dynamic_tc(i32*
noalias<br>
>>>> nocapture readonly
%A, i32* noali<br>
>>>> ; CHECK-NEXT:
store i32 [[MUL]], i32* [[ARRAYIDX2]],
align 4<br>
>>>> ; CHECK-NEXT:
[[INDVARS_IV_NEXT]] = add nuw nsw i64<br>
>>>> [[INDVARS_IV]], 2<br>
>>>> ; CHECK-NEXT:
[[CMP:%.*]] = icmp ult i64
[[INDVARS_IV_NEXT]],<br>
>>>> [[N]]<br>
>>>> -; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label<br>
>>>>
[[FOR_COND_CLEANUP:%.*]],
[[LOOP15:!llvm.loop !.*]]<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label<br>
>>>> [[FOR_COND_CLEANUP]],
[[LOOP15:!llvm.loop !.*]]<br>
>>>> ;<br>
>>>> entry:<br>
>>>> br label
%for.body<br>
>>>> @@ -973,7 +973,7 @@
define void @PR27626_0(%pair.i32 *%p,
i32 %z,<br>
>>>> i64 %n) {<br>
>>>> ; CHECK-NEXT:
[[TMP19:%.*]] = icmp eq i64
[[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT:
br i1 [[TMP19]], label
[[MIDDLE_BLOCK:%.*]],<br>
>>>> label
[[VECTOR_BODY]], [[LOOP24:!llvm.loop
!.*]]<br>
>>>> ; CHECK:
middle.block:<br>
>>>> -; CHECK-NEXT: br
label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br
i1 false, label [[FOR_END:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph"
moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [
[[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
br label [[FOR_BODY:%.*]]<br>
>>>> @@ -985,7 +985,7 @@
define void @PR27626_0(%pair.i32 *%p,
i32 %z,<br>
>>>> i64 %n) {<br>
>>>> ; CHECK-NEXT:
store i32 [[Z]], i32* [[P_I_Y]], align
4<br>
>>>> ; CHECK-NEXT:
[[I_NEXT]] = add nuw nsw i64 [[I]], 1<br>
>>>> ; CHECK-NEXT:
[[COND:%.*]] = icmp slt i64
[[I_NEXT]], [[N]]<br>
>>>> -; CHECK-NEXT: br
i1 [[COND]], label [[FOR_BODY]], label<br>
>>>> [[FOR_END:%.*]],
[[LOOP25:!llvm.loop !.*]]<br>
>>>> +; CHECK-NEXT: br
i1 [[COND]], label [[FOR_BODY]], label<br>
>>>> [[FOR_END]],
[[LOOP25:!llvm.loop !.*]]<br>
>>>> ; CHECK:
for.end:<br>
>>>> ; CHECK-NEXT:
ret void<br>
>>>> ;<br>
>>>> @@ -1066,7 +1066,7 @@
define i32 @PR27626_1(%pair.i32 *%p,
i64 %n) {<br>
>>>> ; CHECK-NEXT:
[[RDX_SHUF3:%.*]] = shufflevector
<4 x i32><br>
>>>> [[BIN_RDX]], <4 x
i32> poison, <4 x i32>
<i32 1, i32 undef, i32<br>
>>>> undef, i32 undef><br>
>>>> ; CHECK-NEXT:
[[BIN_RDX4:%.*]] = add <4 x i32>
[[BIN_RDX]],<br>
>>>> [[RDX_SHUF3]]<br>
>>>> ; CHECK-NEXT:
[[TMP19:%.*]] = extractelement <4 x
i32><br>
>>>> [[BIN_RDX4]], i32 0<br>
>>>> -; CHECK-NEXT: br
label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br
i1 false, label [[FOR_END:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph"
moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [
[[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
[[BC_MERGE_RDX:%.*]] = phi i32 [
[[TMP19]],<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY]] ]<br>
>>>> @@ -1081,9 +1081,10
@@ define i32 @PR27626_1(%pair.i32
*%p, i64 %n) {<br>
>>>> ; CHECK-NEXT:
[[TMP21]] = add nsw i32 [[TMP20]],
[[S]]<br>
>>>> ; CHECK-NEXT:
[[I_NEXT]] = add nuw nsw i64 [[I]], 1<br>
>>>> ; CHECK-NEXT:
[[COND:%.*]] = icmp slt i64
[[I_NEXT]], [[N]]<br>
>>>> -; CHECK-NEXT: br
i1 [[COND]], label [[FOR_BODY]], label<br>
>>>> [[FOR_END:%.*]],
[[LOOP27:!llvm.loop !.*]]<br>
>>>> +; CHECK-NEXT: br
i1 [[COND]], label [[FOR_BODY]], label<br>
>>>> [[FOR_END]],
[[LOOP27:!llvm.loop !.*]]<br>
>>>> ; CHECK:
for.end:<br>
>>>> -; CHECK-NEXT: ret
i32 [[TMP21]]<br>
>>>> +; CHECK-NEXT:
[[TMP22:%.*]] = phi i32 [ [[TMP21]],
[[FOR_BODY]]<br>
>>>> ], [ [[TMP19]],
[[MIDDLE_BLOCK]] ]<br>
>>>> +; CHECK-NEXT: ret
i32 [[TMP22]]<br>
>>>> ;<br>
>>>> entry:<br>
>>>> br label
%for.body<br>
>>>> @@ -1162,7 +1163,7 @@
define void @PR27626_2(%pair.i32 *%p,
i64 %n,<br>
>>>> i32 %z) {<br>
>>>> ; CHECK-NEXT:
[[TMP20:%.*]] = icmp eq i64
[[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT:
br i1 [[TMP20]], label
[[MIDDLE_BLOCK:%.*]],<br>
>>>> label
[[VECTOR_BODY]], [[LOOP28:!llvm.loop
!.*]]<br>
>>>> ; CHECK:
middle.block:<br>
>>>> -; CHECK-NEXT: br
label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br
i1 false, label [[FOR_END:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph"
moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [
[[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
br label [[FOR_BODY:%.*]]<br>
>>>> @@ -1176,7 +1177,7 @@
define void @PR27626_2(%pair.i32 *%p,
i64 %n,<br>
>>>> i32 %z) {<br>
>>>> ; CHECK-NEXT:
store i32 [[TMP21]], i32* [[P_I_Y]],
align 4<br>
>>>> ; CHECK-NEXT:
[[I_NEXT]] = add nuw nsw i64 [[I]], 1<br>
>>>> ; CHECK-NEXT:
[[COND:%.*]] = icmp slt i64
[[I_NEXT]], [[N]]<br>
>>>> -; CHECK-NEXT: br
i1 [[COND]], label [[FOR_BODY]], label<br>
>>>> [[FOR_END:%.*]],
[[LOOP29:!llvm.loop !.*]]<br>
>>>> +; CHECK-NEXT: br
i1 [[COND]], label [[FOR_BODY]], label<br>
>>>> [[FOR_END]],
[[LOOP29:!llvm.loop !.*]]<br>
>>>> ; CHECK:
for.end:<br>
>>>> ; CHECK-NEXT:
ret void<br>
>>>> ;<br>
>>>> @@ -1263,7 +1264,7 @@
define i32 @PR27626_3(%pair.i32 *%p,
i64 %n,<br>
>>>> i32 %z) {<br>
>>>> ; CHECK-NEXT:
[[RDX_SHUF3:%.*]] = shufflevector
<4 x i32><br>
>>>> [[BIN_RDX]], <4 x
i32> poison, <4 x i32>
<i32 1, i32 undef, i32<br>
>>>> undef, i32 undef><br>
>>>> ; CHECK-NEXT:
[[BIN_RDX4:%.*]] = add <4 x i32>
[[BIN_RDX]],<br>
>>>> [[RDX_SHUF3]]<br>
>>>> ; CHECK-NEXT:
[[TMP22:%.*]] = extractelement <4 x
i32><br>
>>>> [[BIN_RDX4]], i32 0<br>
>>>> -; CHECK-NEXT: br
label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br
i1 false, label [[FOR_END:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph"
moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [
[[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
[[BC_MERGE_RDX:%.*]] = phi i32 [
[[TMP22]],<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY]] ]<br>
>>>> @@ -1281,9 +1282,10
@@ define i32 @PR27626_3(%pair.i32
*%p, i64 %n,<br>
>>>> i32 %z) {<br>
>>>> ; CHECK-NEXT:
[[TMP25]] = add nsw i32 [[TMP24]],
[[S]]<br>
>>>> ; CHECK-NEXT:
[[I_NEXT]] = add nuw nsw i64 [[I]], 1<br>
>>>> ; CHECK-NEXT:
[[COND:%.*]] = icmp slt i64
[[I_NEXT]], [[N]]<br>
>>>> -; CHECK-NEXT: br
i1 [[COND]], label [[FOR_BODY]], label<br>
>>>> [[FOR_END:%.*]],
[[LOOP31:!llvm.loop !.*]]<br>
>>>> +; CHECK-NEXT: br
i1 [[COND]], label [[FOR_BODY]], label<br>
>>>> [[FOR_END]],
[[LOOP31:!llvm.loop !.*]]<br>
>>>> ; CHECK:
for.end:<br>
>>>> -; CHECK-NEXT: ret
i32 [[TMP25]]<br>
>>>> +; CHECK-NEXT:
[[TMP26:%.*]] = phi i32 [ [[TMP25]],
[[FOR_BODY]]<br>
>>>> ], [ [[TMP22]],
[[MIDDLE_BLOCK]] ]<br>
>>>> +; CHECK-NEXT: ret
i32 [[TMP26]]<br>
>>>> ;<br>
>>>> entry:<br>
>>>> br label
%for.body<br>
>>>><br>
>>>> diff --git
a/llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
>>>>
b/llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
>>>> index
f32002fae2b6..91780789088b 100644<br>
>>>> ---
a/llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
>>>> +++
b/llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
>>>> @@ -146,14 +146,15 @@
define void @early_exit(i16* %p, i32
%n) {<br>
>>>> ; CHECK-NEXT:
[[TMP10:%.*]] = icmp eq i32
[[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT:
br i1 [[TMP10]], label
[[MIDDLE_BLOCK:%.*]],<br>
>>>> label
[[VECTOR_BODY]], [[LOOP4:!llvm.loop
!.*]]<br>
>>>> ; CHECK:
middle.block:<br>
>>>> -; CHECK-NEXT: br
label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:
[[CMP_N:%.*]] = icmp eq i32 [[TMP1]],
[[N_VEC]]<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP_N]], label [[IF_END:%.*]],
label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph"
moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i32 [
[[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
br label [[FOR_COND:%.*]]<br>
>>>> ; CHECK:
for.cond:<br>
>>>> ; CHECK-NEXT:
[[I:%.*]] = phi i32 [
[[BC_RESUME_VAL]],<br>
>>>> [[SCALAR_PH]] ], [
[[INC:%.*]], [[FOR_BODY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
[[CMP:%.*]] = icmp slt i32 [[I]],
[[N]]<br>
>>>> -; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label<br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label
[[IF_END]]<br>
>>>> ; CHECK:
for.body:<br>
>>>> ; CHECK-NEXT:
[[IPROM:%.*]] = sext i32 [[I]] to i64<br>
>>>> ; CHECK-NEXT:
[[B:%.*]] = getelementptr inbounds
i16, i16*<br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> @@ -285,14 +286,15 @@
define void @multiple_unique_exit(i16*
%p,<br>
>>>> i32 %n) {<br>
>>>> ; CHECK-NEXT:
[[TMP11:%.*]] = icmp eq i32
[[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT:
br i1 [[TMP11]], label
[[MIDDLE_BLOCK:%.*]],<br>
>>>> label
[[VECTOR_BODY]], [[LOOP6:!llvm.loop
!.*]]<br>
>>>> ; CHECK:
middle.block:<br>
>>>> -; CHECK-NEXT: br
label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:
[[CMP_N:%.*]] = icmp eq i32 [[TMP2]],
[[N_VEC]]<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP_N]], label [[IF_END:%.*]],
label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph"
moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i32 [
[[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
br label [[FOR_COND:%.*]]<br>
>>>> ; CHECK:
for.cond:<br>
>>>> ; CHECK-NEXT:
[[I:%.*]] = phi i32 [
[[BC_RESUME_VAL]],<br>
>>>> [[SCALAR_PH]] ], [
[[INC:%.*]], [[FOR_BODY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
[[CMP:%.*]] = icmp slt i32 [[I]],
[[N]]<br>
>>>> -; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label<br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label
[[IF_END]]<br>
>>>> ; CHECK:
for.body:<br>
>>>> ; CHECK-NEXT:
[[IPROM:%.*]] = sext i32 [[I]] to i64<br>
>>>> ; CHECK-NEXT:
[[B:%.*]] = getelementptr inbounds
i16, i16*<br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> @@ -372,14 +374,17 @@
define i32 @multiple_unique_exit2(i16*
%p,<br>
>>>> i32 %n) {<br>
>>>> ; CHECK-NEXT:
[[TMP11:%.*]] = icmp eq i32
[[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT:
br i1 [[TMP11]], label
[[MIDDLE_BLOCK:%.*]],<br>
>>>> label
[[VECTOR_BODY]], [[LOOP8:!llvm.loop
!.*]]<br>
>>>> ; CHECK:
middle.block:<br>
>>>> -; CHECK-NEXT: br
label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:
[[CMP_N:%.*]] = icmp eq i32 [[TMP2]],
[[N_VEC]]<br>
>>>> +; CHECK-NEXT:
[[IND_ESCAPE:%.*]] = sub i32
[[N_VEC]], 1<br>
>>>> +; CHECK-NEXT:
[[IND_ESCAPE1:%.*]] = sub i32
[[N_VEC]], 1<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP_N]], label [[IF_END:%.*]],
label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph"
moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i32 [
[[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
br label [[FOR_COND:%.*]]<br>
>>>> ; CHECK:
for.cond:<br>
>>>> ; CHECK-NEXT:
[[I:%.*]] = phi i32 [
[[BC_RESUME_VAL]],<br>
>>>> [[SCALAR_PH]] ], [
[[INC:%.*]], [[FOR_BODY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
[[CMP:%.*]] = icmp slt i32 [[I]],
[[N]]<br>
>>>> -; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label<br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label
[[IF_END]]<br>
>>>> ; CHECK:
for.body:<br>
>>>> ; CHECK-NEXT:
[[IPROM:%.*]] = sext i32 [[I]] to i64<br>
>>>> ; CHECK-NEXT:
[[B:%.*]] = getelementptr inbounds
i16, i16*<br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> @@ -388,7 +393,7 @@
define i32 @multiple_unique_exit2(i16*
%p, i32<br>
>>>> %n) {<br>
>>>> ; CHECK-NEXT:
[[CMP2:%.*]] = icmp slt i32 [[I]],
2096<br>
>>>> ; CHECK-NEXT:
br i1 [[CMP2]], label [[FOR_COND]],
label<br>
>>>> [[IF_END]],
[[LOOP9:!llvm.loop !.*]]<br>
>>>> ; CHECK:
if.end:<br>
>>>> -; CHECK-NEXT:
[[I_LCSSA:%.*]] = phi i32 [ [[I]],
[[FOR_BODY]]<br>
>>>> ], [ [[I]],
[[FOR_COND]] ]<br>
>>>> +; CHECK-NEXT:
[[I_LCSSA:%.*]] = phi i32 [ [[I]],
[[FOR_BODY]]<br>
>>>> ], [ [[I]],
[[FOR_COND]] ], [ [[IND_ESCAPE1]],
[[MIDDLE_BLOCK]] ]<br>
>>>> ; CHECK-NEXT:
ret i32 [[I_LCSSA]]<br>
>>>> ;<br>
>>>> ; TAILFOLD-LABEL:
@multiple_unique_exit2(<br>
>>>> @@ -461,14 +466,15 @@
define i32 @multiple_unique_exit3(i16*
%p,<br>
>>>> i32 %n) {<br>
>>>> ; CHECK-NEXT:
[[TMP11:%.*]] = icmp eq i32
[[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT:
br i1 [[TMP11]], label
[[MIDDLE_BLOCK:%.*]],<br>
>>>> label
[[VECTOR_BODY]], [[LOOP10:!llvm.loop
!.*]]<br>
>>>> ; CHECK:
middle.block:<br>
>>>> -; CHECK-NEXT: br
label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:
[[CMP_N:%.*]] = icmp eq i32 [[TMP2]],
[[N_VEC]]<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP_N]], label [[IF_END:%.*]],
label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph"
moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i32 [
[[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
br label [[FOR_COND:%.*]]<br>
>>>> ; CHECK:
for.cond:<br>
>>>> ; CHECK-NEXT:
[[I:%.*]] = phi i32 [
[[BC_RESUME_VAL]],<br>
>>>> [[SCALAR_PH]] ], [
[[INC:%.*]], [[FOR_BODY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
[[CMP:%.*]] = icmp slt i32 [[I]],
[[N]]<br>
>>>> -; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label<br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP]], label [[FOR_BODY]], label
[[IF_END]]<br>
>>>> ; CHECK:
for.body:<br>
>>>> ; CHECK-NEXT:
[[IPROM:%.*]] = sext i32 [[I]] to i64<br>
>>>> ; CHECK-NEXT:
[[B:%.*]] = getelementptr inbounds
i16, i16*<br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> @@ -477,7 +483,7 @@
define i32 @multiple_unique_exit3(i16*
%p, i32<br>
>>>> %n) {<br>
>>>> ; CHECK-NEXT:
[[CMP2:%.*]] = icmp slt i32 [[I]],
2096<br>
>>>> ; CHECK-NEXT:
br i1 [[CMP2]], label [[FOR_COND]],
label<br>
>>>> [[IF_END]],
[[LOOP11:!llvm.loop !.*]]<br>
>>>> ; CHECK:
if.end:<br>
>>>> -; CHECK-NEXT:
[[EXIT:%.*]] = phi i32 [ 0,
[[FOR_COND]] ], [ 1,<br>
>>>> [[FOR_BODY]] ]<br>
>>>> +; CHECK-NEXT:
[[EXIT:%.*]] = phi i32 [ 0,
[[FOR_COND]] ], [ 1,<br>
>>>> [[FOR_BODY]] ], [ 0,
[[MIDDLE_BLOCK]] ]<br>
>>>> ; CHECK-NEXT:
ret i32 [[EXIT]]<br>
>>>> ;<br>
>>>> ; TAILFOLD-LABEL:
@multiple_unique_exit3(<br>
>>>> @@ -994,7 +1000,8 @@
define void @scalar_predication(float*
%addr) {<br>
>>>> ; CHECK-NEXT:
[[TMP10:%.*]] = icmp eq i64
[[INDEX_NEXT]], 200<br>
>>>> ; CHECK-NEXT:
br i1 [[TMP10]], label
[[MIDDLE_BLOCK:%.*]],<br>
>>>> label
[[VECTOR_BODY]], [[LOOP12:!llvm.loop
!.*]]<br>
>>>> ; CHECK:
middle.block:<br>
>>>> -; CHECK-NEXT: br
label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:
[[CMP_N:%.*]] = icmp eq i64 201, 200<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP_N]], label [[EXIT:%.*]],
label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph"
moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [ 200,<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
br label [[LOOP_HEADER:%.*]]<br>
>>>> @@ -1002,7 +1009,7 @@
define void @scalar_predication(float*
%addr) {<br>
>>>> ; CHECK-NEXT:
[[IV:%.*]] = phi i64 [
[[BC_RESUME_VAL]],<br>
>>>> [[SCALAR_PH]] ], [
[[IV_NEXT:%.*]], [[LOOP_LATCH:%.*]] ]<br>
>>>> ; CHECK-NEXT:
[[GEP:%.*]] = getelementptr float,
float*<br>
>>>> [[ADDR]], i64 [[IV]]<br>
>>>> ; CHECK-NEXT:
[[EXITCOND_NOT:%.*]] = icmp eq i64
[[IV]], 200<br>
>>>> -; CHECK-NEXT: br
i1 [[EXITCOND_NOT]], label
[[EXIT:%.*]], label<br>
>>>> [[LOOP_BODY:%.*]]<br>
>>>> +; CHECK-NEXT: br
i1 [[EXITCOND_NOT]], label [[EXIT]],
label<br>
>>>> [[LOOP_BODY:%.*]]<br>
>>>> ; CHECK:
loop.body:<br>
>>>> ; CHECK-NEXT:
[[TMP11:%.*]] = load float, float*
[[GEP]],<br>
>>>> align 4<br>
>>>> ; CHECK-NEXT:
[[PRED:%.*]] = fcmp oeq float
[[TMP11]],<br>
>>>> 0.000000e+00<br>
>>>> @@ -1088,7 +1095,8 @@
define i32 @me_reduction(i32* %addr) {<br>
>>>> ; CHECK-NEXT:
[[RDX_SHUF:%.*]] = shufflevector <2
x i32><br>
>>>> [[TMP5]], <2 x
i32> poison, <2 x i32>
<i32 1, i32 undef><br>
>>>> ; CHECK-NEXT:
[[BIN_RDX:%.*]] = add <2 x i32>
[[TMP5]],<br>
>>>> [[RDX_SHUF]]<br>
>>>> ; CHECK-NEXT:
[[TMP7:%.*]] = extractelement <2 x
i32><br>
>>>> [[BIN_RDX]], i32 0<br>
>>>> -; CHECK-NEXT: br
label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:
[[CMP_N:%.*]] = icmp eq i64 201, 200<br>
>>>> +; CHECK-NEXT: br
i1 [[CMP_N]], label [[EXIT:%.*]],
label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph"
moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [ 200,<br>
>>>> [[MIDDLE_BLOCK]] ], [
0, [[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
[[BC_MERGE_RDX:%.*]] = phi i32 [ 0,
[[ENTRY]]<br>
>>>> ], [ [[TMP7]],
[[MIDDLE_BLOCK]] ]<br>
>>>> @@ -1098,7 +1106,7 @@
define i32 @me_reduction(i32* %addr) {<br>
>>>> ; CHECK-NEXT:
[[ACCUM:%.*]] = phi i32 [
[[BC_MERGE_RDX]],<br>
>>>> [[SCALAR_PH]] ], [
[[ACCUM_NEXT:%.*]], [[LOOP_LATCH]] ]<br>
>>>> ; CHECK-NEXT:
[[GEP:%.*]] = getelementptr i32, i32*
[[ADDR]],<br>
>>>> i64 [[IV]]<br>
>>>> ; CHECK-NEXT:
[[EXITCOND_NOT:%.*]] = icmp eq i64
[[IV]], 200<br>
>>>> -; CHECK-NEXT: br
i1 [[EXITCOND_NOT]], label
[[EXIT:%.*]], label<br>
>>>> [[LOOP_LATCH]]<br>
>>>> +; CHECK-NEXT: br
i1 [[EXITCOND_NOT]], label [[EXIT]],
label<br>
>>>> [[LOOP_LATCH]]<br>
>>>> ; CHECK:
loop.latch:<br>
>>>> ; CHECK-NEXT:
[[TMP8:%.*]] = load i32, i32* [[GEP]],
align 4<br>
>>>> ; CHECK-NEXT:
[[ACCUM_NEXT]] = add i32 [[ACCUM]],
[[TMP8]]<br>
>>>> @@ -1106,7 +1114,7 @@
define i32 @me_reduction(i32* %addr) {<br>
>>>> ; CHECK-NEXT:
[[EXITCOND2_NOT:%.*]] = icmp eq i64
[[IV]], 400<br>
>>>> ; CHECK-NEXT:
br i1 [[EXITCOND2_NOT]], label
[[EXIT]], label<br>
>>>> [[LOOP_HEADER]],
[[LOOP15:!llvm.loop !.*]]<br>
>>>> ; CHECK:
exit:<br>
>>>> -; CHECK-NEXT:
[[LCSSA:%.*]] = phi i32 [ 0,
[[LOOP_HEADER]] ], [<br>
>>>> [[ACCUM_NEXT]],
[[LOOP_LATCH]] ]<br>
>>>> +; CHECK-NEXT:
[[LCSSA:%.*]] = phi i32 [ 0,
[[LOOP_HEADER]] ], [<br>
>>>> [[ACCUM_NEXT]],
[[LOOP_LATCH]] ], [ [[TMP7]],
[[MIDDLE_BLOCK]] ]<br>
>>>> ; CHECK-NEXT:
ret i32 [[LCSSA]]<br>
>>>> ;<br>
>>>> ; TAILFOLD-LABEL:
@me_reduction(<br>
>>>><br>
>>>><br>
>>>><br>
>>>>
_______________________________________________<br>
>>>> llvm-commits mailing
list<br>
>>>> <a
href="mailto:llvm-commits@lists.llvm.org"
moz-do-not-send="true"
target="_blank">llvm-commits@lists.llvm.org</a><br>
>>>> <a
href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits"
moz-do-not-send="true"
rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
>>>
_______________________________________________<br>
>>> llvm-commits mailing list<br>
>>> <a
href="mailto:llvm-commits@lists.llvm.org"
moz-do-not-send="true"
target="_blank">llvm-commits@lists.llvm.org</a><br>
>>> <a
href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits"
moz-do-not-send="true"
rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
>
_______________________________________________<br>
> llvm-commits mailing list<br>
> <a
href="mailto:llvm-commits@lists.llvm.org"
moz-do-not-send="true"
target="_blank">llvm-commits@lists.llvm.org</a><br>
> <a
href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits"
moz-do-not-send="true"
rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a></blockquote>
</div>
</div>
</blockquote>
</blockquote>
<div dir="ltr"> </div>
</div>
</blockquote>
</blockquote>
<div dir="ltr"> </div>
</div>
</blockquote>
</blockquote>
<div dir="ltr"> </div>
</div>
<div id="MIMEAttachInfoDiv" style="display:none"
title="octet-stream|AfterLoopVec.out"> </div>
<div id="MIMEAttachInfoDiv" style="display:none"
title="octet-stream|BeforeLoopVec.out"> </div>
<div><font size="2" face="Default Monospace,Courier
New,Courier,monospace">_______________________________________________<br>
llvm-commits mailing list<br>
<a class="moz-txt-link-abbreviated" href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a><br>
<a
href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits"
target="_blank" moz-do-not-send="true">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a> </font></div>
</blockquote>
<div dir="ltr"> </div>
</div>
<br>
</blockquote>
</body>
</html>