<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Looking at the various failures, I see both LE and BE bots
failing. The BE bot shows a miscompile in one of the test suite
benchmarks. The LE bots appear to be showing a crash while using
a stage1 build clang to build stage2 clang. <br>
</p>
<p>LE example: <a moz-do-not-send="true"
href="https://lab.llvm.org/buildbot#builders/19/builds/4236">https://lab.llvm.org/buildbot#builders/19/builds/4236</a><br>
BE example: <a class="moz-txt-link-freetext"
href="https://lab.llvm.org/buildbot#builders/100/builds/5762">https://lab.llvm.org/buildbot#builders/100/builds/5762</a></p>
<p>So both are showing miscompiles, just with different symptoms.
Frankly, the BE looks easier to debug (much code making it into
the miscompiled binary.)<br>
</p>
<p>Oddly, pretty much only PPC bots are failing. The one exception
is a stage2 failure on AArch64
(<a class="moz-txt-link-freetext" href="https://lab.llvm.org/buildbot/#/builders/111/builds/2027">https://lab.llvm.org/buildbot/#/builders/111/builds/2027</a>) which
looks similar to the LE failure above. </p>
<p>Given this appears to be target specific, I am *guessing* there's
some vectorizer hook which is causing a different codepath to be
executed. Before we start trying to get me access to hardware, do
you have any guesses on what that hook might be? If you can give
me a good hint on where to look, I suspect I can probably find the
issue that way. <br>
</p>
<p>Philip<br>
</p>
<div class="moz-cite-prefix">On 5/18/21 3:07 AM, Nemanja Ivanovic
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:OF443062AC.F5998286-ON002586D9.00373B50-002586D9.0037A584@notes.na.collabserv.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div class="socmaildefaultfont" dir="ltr"
style="font-family:Arial, Helvetica, sans-serif;font-size:10pt">
<div dir="ltr">Hi Philip,</div>
<div dir="ltr">I am not sure what happened with your first
attempt to contact us and how we missed it. We would be more
than happy to help you debug this issue. Do you know if this
only affects the big endian bot or if it also fails on little
endian bots? In the latter case, we can certainly provide
access to a little endian machine hosted at OSU/OSL. In the
former case, we don't have a machine available and we'll have
to do the debugging and report to you (which might take a bit
longer).</div>
<div dir="ltr"> </div>
<div dir="ltr">Nemanja Ivanovic<br>
LLVM PPC Backend Development<br>
IBM Toronto Lab<br>
Email: <a class="moz-txt-link-abbreviated" href="mailto:nemanjai@ca.ibm.com">nemanjai@ca.ibm.com</a><br>
Phone: 905-413-3388</div>
<div dir="ltr"> </div>
<div dir="ltr"> </div>
<blockquote data-history-content-modified="1"
data-history-expanded="1" dir="ltr" style="border-left:solid
#aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr;
margin-right:0px">----- Original message -----<br>
From: Philip Reames <a class="moz-txt-link-rfc2396E" href="mailto:listmail@philipreames.com"><listmail@philipreames.com></a><br>
To: Adrian Kuegel <a class="moz-txt-link-rfc2396E" href="mailto:akuegel@google.com"><akuegel@google.com></a><br>
Cc: Benjamin Kramer <a class="moz-txt-link-rfc2396E" href="mailto:benny.kra@gmail.com"><benny.kra@gmail.com></a>, Adrian Kuegel
<a class="moz-txt-link-rfc2396E" href="mailto:llvmlistbot@llvm.org"><llvmlistbot@llvm.org></a>, llvm-commits
<a class="moz-txt-link-rfc2396E" href="mailto:llvm-commits@lists.llvm.org"><llvm-commits@lists.llvm.org></a>, <a class="moz-txt-link-abbreviated" href="mailto:powerllvm@ca.ibm.com">powerllvm@ca.ibm.com</a><br>
Subject: [EXTERNAL] Re: [llvm] 7fe41ac - Revert "[LV]
Unconditionally branch from middle to scalar preheader if the
scalar loop must execute"<br>
Date: Mon, May 17, 2021 11:59 PM<br>
<br>
<!-- BaNnErBlUrFlE-HeAdEr-start -->
<!-- BaNnErBlUrFlE-HeAdEr-end -->
<!--Notes ACF
<meta http-equiv="Content-Type" content="text/html; charset=utf8" >-->
<!-- BaNnErBlUrFlE-BoDy-start -->
<!-- Preheader Text : BEGIN --> <span style="display:none
!important;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;">
I tried another cycle to see if this had been resolved, but
am still seeing build bot failures, nearly exclusively on
PPC. (And one arm self host bot.) @PPC Bot Owner - I need
help reducing a test case for the failure seen here:
</span>
<!-- Preheader Text : END -->
<!-- Email Banner : BEGIN --> <span style="display:none
!important;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerStart</span>
<!--[if ((ie)|(mso))]>
<table border="0" cellspacing="0" cellpadding="0" width="100%" style="padding: 3px 0px 16px 0px;" ><tr><td>
<table class="pfptBannerTableMSO" border="0" cellspacing="0" cellpadding="0" style="padding: 0px 10px 5px 6px; width: 100%;border-radius:4px;border-top:4px solid #90a4ae;background-color:#d0d8dc;" ><tr><td valign="top" >
<table align="left" border="0" cellspacing="0" cellpadding="0" style="padding: 4px 8px 4px 8px" >
<tr><td><span class="pfptTitleMSO" style="color:#000000 !important;font-family: 'Arial', sans-serif;font-weight:bold !important;font-size:14px !important;" >
This Message Is From an External Sender
</span></td></tr>
<tr><td><span class="pfptSubtitleMSO" style="color:#000000 !important;font-weight:normal !important;font-family: 'Arial', sans-serif; font-size:12px !important;" >
This message came from outside your organization.
</span></td></tr>
</table>
</td></tr></table>
</td></tr></table>
<![endif]-->
<!--[if !((ie)|(mso))]-->
<div style="margin:16px 0px 16px 0px; padding:8px 16px 8px
16px; border-radius: 4px; min-width: 200px;background-color:
#d0d8dc; border-top: 4px solid #90a4ae;">
<div style="display: inline-block; margin: 0px 0px 1px 0px;
max-width: 600px;">
<div style="color:#000000 !important;font-family: 'Arial',
sans-serif !important;font-weight:bold
!important;font-size:14px
!important;line-height:18px;display:block;">This Message
Is From an External Sender</div>
<div style="color:#000000 !important;font-weight:normal
!important;font-family: 'Arial', sans-serif
!important;font-size:12px
!important;line-height:18px;margin-top:2px;display:block">This
message came from outside your organization.</div>
</div>
</div>
<!--[endif]--> <span style="display:none
!important;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerEnd</span><br>
<!-- Email Banner : END -->
<!-- BaNnErBlUrFlE-BoDy-end -->
<p>I tried another cycle to see if this had been resolved, but
am still seeing build bot failures, nearly exclusively on
PPC. (And one arm self host bot.) </p>
<p>@PPC Bot Owner - I need help reducing a test case for the
failure seen here: <a
href="https://lab.llvm.org/buildbot#builders/100/builds/5762"
target="_blank" moz-do-not-send="true">https://lab.llvm.org/buildbot#builders/100/builds/5762</a>.
I am stuck, and unable to make progress for nearly 3 months
now. I would greatly appreciate help. </p>
<p>Philip</p>
<div>On 5/5/21 11:59 PM, Adrian Kuegel wrote:</div>
<blockquote
cite="mid:CAHhV=OX3NbAExy4YsQWLH0fC-zXiG_VpToAtB72NhkUwwYekRQ@mail.gmail.com"
type="cite"><!--Notes ACF
<meta http-equiv="content-type" content="text/html; charset=utf8" >-->
<div dir="ltr">
<div dir="ltr">Sounds good to me, and thanks for sending
the heads up :)</div>
<div>
<div dir="ltr">On Wed, May 5, 2021 at 10:01 PM Philip
Reames <<a href="mailto:listmail@philipreames.com"
moz-do-not-send="true" target="_blank">listmail@philipreames.com</a>>
wrote:</div>
<blockquote style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">FYI, I'm going to
try recommitting this without changes in a day or so.<br>
<br>
I never heard back from a PPC bot owner, and I don't
have enough<br>
information to really debug anything from the builtbot
log. I did run<br>
across a latent issue which this patch may very well
have exposed at<br>
much higher frequency; the previous patch in this
series (which is much<br>
more restrictive) does appear to have increased
frequency. That was<br>
worked around in 80e80250. My educated guess is that
same issue<br>
triggered the miscompile seen on the ppc bot, but that
is more of a<br>
guess than I'd really prefer.<br>
<br>
I'm going to submit this during off hours, and watch
the bots fairly<br>
closely after submit. Hopefully this either cycles
clean or I get a<br>
better clue as to what the root issue is.<br>
<br>
Philip<br>
<br>
On 2/8/21 9:09 PM, Philip Reames via llvm-commits
wrote:<br>
> Ben,<br>
><br>
> Thanks for the clarification. The log does not
make the fact this is<br>
> an execution failure obvious.<br>
><br>
> No, I don't have access to a PPC machine.<br>
><br>
> I am going to need some assistance from the bot
owner on this. At a<br>
> minimum, IR for the test in question (before
optimization, but on<br>
> target platform) seems like a reasonable ask.<br>
><br>
> I strongly suspect this change is simply exposing
another latent<br>
> issue. Or at least, I've reviewed the change and
don't see anything<br>
> likely to cause runtime crashes w/o also tripping
compiler asserts.<br>
><br>
> Philip<br>
><br>
><br>
> On 2/8/21 5:21 AM, Benjamin Kramer wrote:<br>
>> `execution_time` failures mean that the bot
succeeded building a test<br>
>> but it failed when running it. I'm relatively
certain that this is the<br>
>> same issue Adrian is seeing -- binaries
segfaulting early on PPC.<br>
>><br>
>> The bot log output isn't helpful at all for
investigating why this is<br>
>> happening. Do you happen to have access to a
PPC machine?<br>
>><br>
>> On Fri, Feb 5, 2021 at 6:05 PM Philip Reames
via llvm-commits<br>
>> <<a
href="mailto:llvm-commits@lists.llvm.org"
moz-do-not-send="true" target="_blank">llvm-commits@lists.llvm.org</a>>
wrote:<br>
>>> Adrian,<br>
>>><br>
>>> I'm going to need you to provide a bit
more information here. The test<br>
>>> failure in stage1 was fixed at the time
you reverted this patch. The<br>
>>> remaining failure in the bot is very
unclear. What is a execution_time<br>
>>> failure? From the log output, the
"failing" run finished in 0.5<br>
>>> seconds,<br>
>>> whereas the previous "succeeding" run
finished in 11 seconds. Without<br>
>>> further context, I'd say that's no
failure.<br>
>>><br>
>>> I'll also note that I did not receive
email from this bot. I received<br>
>>> notice from the various other bots and
fixed the ARM test issue, but<br>
>>> unless I missed it in with the others,
this bot is not notifying.<br>
>>><br>
>>> In general, I'm a fan of fast reverts,
but I have to admit, this one<br>
>>> appears borderline at the moment.<br>
>>><br>
>>> Philip<br>
>>><br>
>>> On 2/5/21 3:53 AM, Adrian Kuegel via
llvm-commits wrote:<br>
>>>> Author: Adrian Kuegel<br>
>>>> Date: 2021-02-05T12:51:03+01:00<br>
>>>> New Revision:
7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c<br>
>>>><br>
>>>> URL:<br>
>>>> <a
href="https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c"
moz-do-not-send="true" rel="noreferrer"
target="_blank">https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c</a><br>
>>>> DIFF:<br>
>>>> <a
href="https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c.diff"
moz-do-not-send="true" rel="noreferrer"
target="_blank">https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c.diff</a><br>
>>>><br>
>>>> LOG: Revert "[LV] Unconditionally
branch from middle to scalar<br>
>>>> preheader if the scalar loop must
execute"<br>
>>>><br>
>>>> This reverts commit
3e5ce49e5371ce4feadbf97dd5c2b652d9db3d1d.<br>
>>>><br>
>>>> Tests started failing on PPC, for
example:<br>
>>>> <a
href="http://lab.llvm.org:8011/#/builders/105/builds/5569"
moz-do-not-send="true" rel="noreferrer"
target="_blank">http://lab.llvm.org:8011/#/builders/105/builds/5569</a><br>
>>>><br>
>>>> Added:<br>
>>>><br>
>>>><br>
>>>> Modified:<br>
>>>>
llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
>>>>
llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
>>>>
llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
>>>>
llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
>>>>
llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
>>>><br>
>>>> Removed:<br>
>>>><br>
>>>><br>
>>>><br>
>>>>
################################################################################<br>
>>>><br>
>>>> diff --git
a/llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
>>>>
b/llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
>>>> index 8a89158788cf..de4fb446fdf2
100644<br>
>>>> ---
a/llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
>>>> +++
b/llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
>>>> @@ -44,11 +44,11 @@
LoopVersioning::LoopVersioning(const<br>
>>>> LoopAccessInfo &LAI,<br>
>>>> AliasChecks(Checks.begin(),
Checks.end()),<br>
>>>>
Preds(LAI.getPSE().getUnionPredicate()), LAI(LAI),
LI(LI),<br>
>>>> DT(DT),<br>
>>>> SE(SE) {<br>
>>>> + assert(L->getUniqueExitBlock()
&& "No single exit block");<br>
>>>> }<br>
>>>><br>
>>>> void LoopVersioning::versionLoop(<br>
>>>> const
SmallVectorImpl<Instruction *>
&DefsUsedOutside) {<br>
>>>> -
assert(VersionedLoop->getUniqueExitBlock()
&& "No single exit<br>
>>>> block");<br>
>>>>
assert(VersionedLoop->isLoopSimplifyForm()
&&<br>
>>>> "Loop is not in
loop-simplify form");<br>
>>>><br>
>>>><br>
>>>> diff --git
a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
>>>>
b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
>>>> index 3277842edbfe..6bce0caeb36f
100644<br>
>>>> ---
a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
>>>> +++
b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
>>>> @@ -852,7 +852,7 @@ class
InnerLoopVectorizer {<br>
>>>> /// Middle Block between the
vector and the scalar.<br>
>>>> BasicBlock *LoopMiddleBlock;<br>
>>>><br>
>>>> - /// The unique ExitBlock of the
scalar loop if one exists. Note<br>
>>>> that<br>
>>>> + /// The (unique) ExitBlock of the
scalar loop. Note that<br>
>>>> /// there can be multiple
exiting edges reaching this block.<br>
>>>> BasicBlock *LoopExitBlock;<br>
>>>><br>
>>>> @@ -3147,13 +3147,9 @@ void<br>
>>>>
InnerLoopVectorizer::emitMinimumIterationCountCheck(Loop
*L,<br>
>>>> DT->getNode(Bypass)->getIDom())
&&<br>
>>>> "TC check is expected to
dominate Bypass");<br>
>>>><br>
>>>> - // Update dominator for Bypass
& LoopExit (if needed).<br>
>>>> + // Update dominator for Bypass
& LoopExit.<br>
>>>>
DT->changeImmediateDominator(Bypass, TCCheckBlock);<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - // If there is an epilogue which
must run, there's no edge<br>
>>>> from the<br>
>>>> - // middle block to exit blocks
and thus no need to update the<br>
>>>> immediate<br>
>>>> - // dominator of the exit blocks.<br>
>>>> -
DT->changeImmediateDominator(LoopExitBlock,
TCCheckBlock);<br>
>>>> +
DT->changeImmediateDominator(LoopExitBlock,
TCCheckBlock);<br>
>>>><br>
>>>> ReplaceInstWithInst(<br>
>>>>
TCCheckBlock->getTerminator(),<br>
>>>> @@ -3192,11 +3188,7 @@ void<br>
>>>>
InnerLoopVectorizer::emitSCEVChecks(Loop *L,
BasicBlock *Bypass) {<br>
>>>> // Update dominator only if this
is first RT check.<br>
>>>> if (LoopBypassBlocks.empty()) {<br>
>>>>
DT->changeImmediateDominator(Bypass,
SCEVCheckBlock);<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - // If there is an epilogue
which must run, there's no edge<br>
>>>> from the<br>
>>>> - // middle block to exit
blocks and thus no need to update<br>
>>>> the immediate<br>
>>>> - // dominator of the exit
blocks.<br>
>>>> -
DT->changeImmediateDominator(LoopExitBlock,
SCEVCheckBlock);<br>
>>>> +
DT->changeImmediateDominator(LoopExitBlock,
SCEVCheckBlock);<br>
>>>> }<br>
>>>><br>
>>>> ReplaceInstWithInst(<br>
>>>> @@ -3252,11 +3244,7 @@ void<br>
>>>>
InnerLoopVectorizer::emitMemRuntimeChecks(Loop *L,
BasicBlock<br>
>>>> *Bypass) {<br>
>>>> // Update dominator only if this
is first RT check.<br>
>>>> if (LoopBypassBlocks.empty()) {<br>
>>>>
DT->changeImmediateDominator(Bypass,
MemCheckBlock);<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - // If there is an epilogue
which must run, there's no edge<br>
>>>> from the<br>
>>>> - // middle block to exit
blocks and thus no need to update<br>
>>>> the immediate<br>
>>>> - // dominator of the exit
blocks.<br>
>>>> -
DT->changeImmediateDominator(LoopExitBlock,
MemCheckBlock);<br>
>>>> +
DT->changeImmediateDominator(LoopExitBlock,
MemCheckBlock);<br>
>>>> }<br>
>>>><br>
>>>> Instruction *FirstCheckInst;<br>
>>>> @@ -3381,10 +3369,9 @@ Value<br>
>>>>
*InnerLoopVectorizer::emitTransformedIndex(<br>
>>>> Loop
*InnerLoopVectorizer::createVectorLoopSkeleton(StringRef<br>
>>>> Prefix) {<br>
>>>> LoopScalarBody =
OrigLoop->getHeader();<br>
>>>> LoopVectorPreHeader =
OrigLoop->getLoopPreheader();<br>
>>>> + LoopExitBlock =
OrigLoop->getUniqueExitBlock();<br>
>>>> + assert(LoopExitBlock &&
"Must have an exit block");<br>
>>>> assert(LoopVectorPreHeader
&& "Invalid loop structure");<br>
>>>> - LoopExitBlock =
OrigLoop->getUniqueExitBlock(); // may be nullptr<br>
>>>> - assert((LoopExitBlock ||
Cost->requiresScalarEpilogue()) &&<br>
>>>> - "multiple exit loop without
required epilogue?");<br>
>>>><br>
>>>> LoopMiddleBlock =<br>
>>>>
SplitBlock(LoopVectorPreHeader,<br>
>>>>
LoopVectorPreHeader->getTerminator(), DT,<br>
>>>> @@ -3393,20 +3380,12 @@ Loop<br>
>>>>
*InnerLoopVectorizer::createVectorLoopSkeleton(StringRef
Prefix) {<br>
>>>> SplitBlock(LoopMiddleBlock,<br>
>>>> LoopMiddleBlock->getTerminator(),
DT, LI,<br>
>>>> nullptr,
Twine(Prefix) + "<a href="http://scalar.ph"
moz-do-not-send="true" rel="noreferrer"
target="_blank">scalar.ph</a>");<br>
>>>><br>
>>>> + // Set up branch from middle block
to the exit and scalar<br>
>>>> preheader blocks.<br>
>>>> + // completeLoopSkeleton will
update the condition to use an<br>
>>>> iteration check,<br>
>>>> + // if required to decide whether
to execute the remainder.<br>
>>>> + BranchInst *BrInst =<br>
>>>> +
BranchInst::Create(LoopExitBlock, LoopScalarPreHeader,<br>
>>>> Builder.getTrue());<br>
>>>> auto *ScalarLatchTerm =<br>
>>>>
OrigLoop->getLoopLatch()->getTerminator();<br>
>>>> -<br>
>>>> - // Set up the middle block
terminator. Two cases:<br>
>>>> - // 1) If we know that we must
execute the scalar epilogue, emit an<br>
>>>> - // unconditional branch.<br>
>>>> - // 2) Otherwise, we must have a
single unique exit block (due to<br>
>>>> how we<br>
>>>> - // implement the multiple exit
case). In this case, set up a<br>
>>>> conditonal<br>
>>>> - // branch from the middle block
to the loop scalar preheader,<br>
>>>> and the<br>
>>>> - // exit block.
completeLoopSkeleton will update the<br>
>>>> condition to use an<br>
>>>> - // iteration check, if required
to decide whether to execute<br>
>>>> the remainder.<br>
>>>> - BranchInst *BrInst =
Cost->requiresScalarEpilogue() ?<br>
>>>> -
BranchInst::Create(LoopScalarPreHeader) :<br>
>>>> -
BranchInst::Create(LoopExitBlock, LoopScalarPreHeader,<br>
>>>> -
Builder.getTrue());<br>
>>>>
BrInst->setDebugLoc(ScalarLatchTerm->getDebugLoc());<br>
>>>>
ReplaceInstWithInst(LoopMiddleBlock->getTerminator(),
BrInst);<br>
>>>><br>
>>>> @@ -3418,11 +3397,7 @@ Loop<br>
>>>>
*InnerLoopVectorizer::createVectorLoopSkeleton(StringRef
Prefix) {<br>
>>>> nullptr, nullptr,
Twine(Prefix) + "vector.body");<br>
>>>><br>
>>>> // Update dominator for loop
exit.<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - // If there is an epilogue which
must run, there's no edge<br>
>>>> from the<br>
>>>> - // middle block to exit blocks
and thus no need to update the<br>
>>>> immediate<br>
>>>> - // dominator of the exit blocks.<br>
>>>> -
DT->changeImmediateDominator(LoopExitBlock,
LoopMiddleBlock);<br>
>>>> +
DT->changeImmediateDominator(LoopExitBlock,
LoopMiddleBlock);<br>
>>>><br>
>>>> // Create and register the new
vector loop.<br>
>>>> Loop *Lp =
LI->AllocateLoop();<br>
>>>> @@ -3519,14 +3494,10 @@ BasicBlock<br>
>>>>
*InnerLoopVectorizer::completeLoopSkeleton(Loop *L,<br>
>>>> auto *ScalarLatchTerm =<br>
>>>>
OrigLoop->getLoopLatch()->getTerminator();<br>
>>>><br>
>>>> // Add a check in the middle
block to see if we have completed<br>
>>>> - // all of the iterations in the
first vector loop. Three cases:<br>
>>>> - // 1) If we require a scalar
epilogue, there is no conditional<br>
>>>> branch as<br>
>>>> - // we unconditionally branch to
the scalar preheader. Do<br>
>>>> nothing.<br>
>>>> - // 2) If (N - N%VF) == N, then we
*don't* need to run the<br>
>>>> remainder.<br>
>>>> - // Thus if tail is to be
folded, we know we don't need to run<br>
>>>> the<br>
>>>> - // remainder and we can use the
previous value for the<br>
>>>> condition (true).<br>
>>>> - // 3) Otherwise, construct a
runtime check.<br>
>>>> - if
(!Cost->requiresScalarEpilogue() &&<br>
>>>> !Cost->foldTailByMasking()) {<br>
>>>> + // all of the iterations in the
first vector loop.<br>
>>>> + // If (N - N%VF) == N, then we
*don't* need to run the remainder.<br>
>>>> + // If tail is to be folded, we
know we don't need to run the<br>
>>>> remainder.<br>
>>>> + if (!Cost->foldTailByMasking())
{<br>
>>>> Instruction *CmpN =
CmpInst::Create(Instruction::ICmp,<br>
>>>> CmpInst::ICMP_EQ,<br>
>>>>
Count,
VectorTripCount,<br>
>>>> "cmp.n",<br>
>>>> LoopMiddleBlock->getTerminator());<br>
>>>> @@ -3590,17 +3561,17 @@ BasicBlock<br>
>>>>
*InnerLoopVectorizer::createVectorizedLoopSkeleton() {<br>
>>>> | [ ]_| <-- vector
loop.<br>
>>>> | |<br>
>>>> | v<br>
>>>> - \ -[ ] <--- middle-block.<br>
>>>> - \/ |<br>
>>>> - /\ v<br>
>>>> - | ->[ ] <--- new
preheader.<br>
>>>> + | -[ ] <--- middle-block.<br>
>>>> + | / |<br>
>>>> + | / v<br>
>>>> + -|- >[ ] <--- new
preheader.<br>
>>>> | |<br>
>>>> - (opt) v <-- edge from
middle to exit iff epilogue is not<br>
>>>> required.<br>
>>>> + | v<br>
>>>> | [ ] \<br>
>>>> - | [ ]_| <-- old scalar
loop to handle remainder (scalar<br>
>>>> epilogue).<br>
>>>> + | [ ]_| <-- old scalar
loop to handle remainder.<br>
>>>> \ |<br>
>>>> \ v<br>
>>>> - >[ ] <-- exit
block(s).<br>
>>>> + >[ ] <-- exit block.<br>
>>>> ...<br>
>>>> */<br>
>>>><br>
>>>> @@ -4021,18 +3992,13 @@ void<br>
>>>>
InnerLoopVectorizer::fixVectorizedLoop() {<br>
>>>> // Forget the original basic
block.<br>
>>>>
PSE.getSE()->forgetLoop(OrigLoop);<br>
>>>><br>
>>>> - // If we inserted an edge from the
middle block to the unique<br>
>>>> exit block,<br>
>>>> - // update uses outside the loop
(phis) to account for the newly<br>
>>>> inserted<br>
>>>> - // edge.<br>
>>>> - if
(!Cost->requiresScalarEpilogue()) {<br>
>>>> - // Fix-up external users of the
induction variables.<br>
>>>> - for (auto &Entry :
Legal->getInductionVars())<br>
>>>> - fixupIVUsers(Entry.first,
Entry.second,<br>
>>>> -
getOrCreateVectorTripCount(LI->getLoopFor(LoopVectorBody)),<br>
>>>> -
IVEndValues[Entry.first], LoopMiddleBlock);<br>
>>>> + // Fix-up external users of the
induction variables.<br>
>>>> + for (auto &Entry :
Legal->getInductionVars())<br>
>>>> + fixupIVUsers(Entry.first,
Entry.second,<br>
>>>> +
getOrCreateVectorTripCount(LI->getLoopFor(LoopVectorBody)),<br>
>>>> +
IVEndValues[Entry.first], LoopMiddleBlock);<br>
>>>><br>
>>>> - fixLCSSAPHIs();<br>
>>>> - }<br>
>>>> + fixLCSSAPHIs();<br>
>>>> for (Instruction *PI :
PredicatedInstructions)<br>
>>>> sinkScalarOperands(&*PI);<br>
>>>><br>
>>>> @@ -4250,13 +4216,12 @@ void<br>
>>>>
InnerLoopVectorizer::fixFirstOrderRecurrence(PHINode
*Phi) {<br>
>>>> // recurrence in the exit block,
and then add an edge for the<br>
>>>> middle block.<br>
>>>> // Note that LCSSA does not
imply single entry when the<br>
>>>> original scalar loop<br>
>>>> // had multiple exiting edges
(as we always run the last<br>
>>>> iteration in the<br>
>>>> - // scalar epilogue); in that case,
there is no edge from middle<br>
>>>> to exit and<br>
>>>> - // and thus no phis which needed
updated.<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - for (PHINode &LCSSAPhi :
LoopExitBlock->phis())<br>
>>>> - if
(any_of(LCSSAPhi.incoming_values(),<br>
>>>> - [Phi](Value *V) {
return V == Phi; }))<br>
>>>> -
LCSSAPhi.addIncoming(ExtractForPhiUsedOutsideLoop,<br>
>>>> LoopMiddleBlock);<br>
>>>> + // scalar epilogue); in that case,
the exiting path through<br>
>>>> middle will be<br>
>>>> + // dynamically dead and the value
picked for the phi doesn't<br>
>>>> matter.<br>
>>>> + for (PHINode &LCSSAPhi :
LoopExitBlock->phis())<br>
>>>> + if
(any_of(LCSSAPhi.incoming_values(),<br>
>>>> + [Phi](Value *V) {
return V == Phi; }))<br>
>>>> +
LCSSAPhi.addIncoming(ExtractForPhiUsedOutsideLoop,<br>
>>>> LoopMiddleBlock);<br>
>>>> }<br>
>>>><br>
>>>> void
InnerLoopVectorizer::fixReduction(PHINode *Phi) {<br>
>>>> @@ -4421,11 +4386,10 @@ void<br>
>>>>
InnerLoopVectorizer::fixReduction(PHINode *Phi) {<br>
>>>> // We know that the loop is in
LCSSA form. We need to update<br>
>>>> the PHI nodes<br>
>>>> // in the exit blocks. See
comment on analogous loop in<br>
>>>> // fixFirstOrderRecurrence for a
more complete explaination of<br>
>>>> the logic.<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - for (PHINode &LCSSAPhi :
LoopExitBlock->phis())<br>
>>>> - if
(any_of(LCSSAPhi.incoming_values(),<br>
>>>> -
[LoopExitInst](Value *V) { return V ==<br>
>>>> LoopExitInst; }))<br>
>>>> -
LCSSAPhi.addIncoming(ReducedPartRdx, LoopMiddleBlock);<br>
>>>> + for (PHINode &LCSSAPhi :
LoopExitBlock->phis())<br>
>>>> + if
(any_of(LCSSAPhi.incoming_values(),<br>
>>>> + [LoopExitInst](Value
*V) { return V ==<br>
>>>> LoopExitInst; }))<br>
>>>> +
LCSSAPhi.addIncoming(ReducedPartRdx, LoopMiddleBlock);<br>
>>>><br>
>>>> // Fix the scalar loop reduction
variable with the incoming<br>
>>>> reduction sum<br>
>>>> // from the vector body and from
the backedge value.<br>
>>>> @@ -8074,11 +8038,7 @@ BasicBlock<br>
>>>>
*EpilogueVectorizerMainLoop::emitMinimumIterationCountCheck(<br>
>>>><br>
>>>> // Update dominator for Bypass
& LoopExit.<br>
>>>>
DT->changeImmediateDominator(Bypass, TCCheckBlock);<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - // For loops with multiple
exits, there's no edge from the<br>
>>>> middle block<br>
>>>> - // to exit blocks (as the
epilogue must run) and thus no<br>
>>>> need to update<br>
>>>> - // the immediate dominator of
the exit blocks.<br>
>>>> -
DT->changeImmediateDominator(LoopExitBlock,
TCCheckBlock);<br>
>>>> +
DT->changeImmediateDominator(LoopExitBlock,
TCCheckBlock);<br>
>>>><br>
>>>>
LoopBypassBlocks.push_back(TCCheckBlock);<br>
>>>><br>
>>>> @@ -8142,12 +8102,7 @@<br>
>>>>
EpilogueVectorizerEpilogueLoop::createEpilogueVectorizedLoopSkeleton()<br>
>>>> {<br>
>>>><br>
>>>>
DT->changeImmediateDominator(LoopScalarPreHeader,<br>
>>>> EPI.EpilogueIterationCountCheck);<br>
>>>> - if
(!Cost->requiresScalarEpilogue())<br>
>>>> - // If there is an epilogue which
must run, there's no edge<br>
>>>> from the<br>
>>>> - // middle block to exit blocks
and thus no need to update the<br>
>>>> immediate<br>
>>>> - // dominator of the exit blocks.<br>
>>>> -
DT->changeImmediateDominator(LoopExitBlock,<br>
>>>> - EPI.EpilogueIterationCountCheck);<br>
>>>> +
DT->changeImmediateDominator(LoopExitBlock,<br>
>>>> EPI.EpilogueIterationCountCheck);<br>
>>>><br>
>>>> // Keep track of bypass blocks,
as they feed start values to<br>
>>>> the induction<br>
>>>> // phis in the scalar loop
preheader.<br>
>>>><br>
>>>> diff --git<br>
>>>>
a/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
>>>>
b/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
>>>> index ec280bf5d5e4..7d4a3c5c9935
100644<br>
>>>> ---<br>
>>>>
a/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
>>>> +++<br>
>>>>
b/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
>>>> @@ -471,9 +471,10 @@ define i16
@multiple_exit(i16* %p, i32 %n) {<br>
>>>> ; CHECK-NEXT: [[TMP15:%.*]] =
icmp eq i32 [[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT: br i1 [[TMP15]],
label [[MIDDLE_BLOCK:%.*]],<br>
>>>> label [[VECTOR_BODY]],
[[LOOP6:!llvm.loop !.*]]<br>
>>>> ; CHECK: middle.block:<br>
>>>> +; CHECK-NEXT: [[CMP_N:%.*]] =
icmp eq i32 [[TMP2]], [[N_VEC]]<br>
>>>> ; CHECK-NEXT:
[[VECTOR_RECUR_EXTRACT:%.*]] = extractelement<br>
>>>> <4 x i16> [[WIDE_LOAD]], i32 3<br>
>>>> ; CHECK-NEXT:
[[VECTOR_RECUR_EXTRACT_FOR_PHI:%.*]] =<br>
>>>> extractelement <4 x i16>
[[WIDE_LOAD]], i32 2<br>
>>>> -; CHECK-NEXT: br label
[[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br i1 [[CMP_N]],
label [[IF_END:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph" moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[SCALAR_RECUR_INIT:%.*]] = phi i16 [ 0,<br>
>>>> [[ENTRY:%.*]] ], [
[[VECTOR_RECUR_EXTRACT]], [[MIDDLE_BLOCK]] ]<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i32 [ [[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY]] ]<br>
>>>> @@ -485,14 +486,14 @@ define i16
@multiple_exit(i16* %p, i32 %n) {<br>
>>>> ; CHECK-NEXT: [[B:%.*]] =
getelementptr inbounds i16, i16*<br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> ; CHECK-NEXT: [[REC_NEXT]] =
load i16, i16* [[B]], align 2<br>
>>>> ; CHECK-NEXT: [[CMP:%.*]] =
icmp slt i32 [[I]], [[N]]<br>
>>>> -; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label<br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label [[IF_END]]<br>
>>>> ; CHECK: for.body:<br>
>>>> ; CHECK-NEXT: store i16
[[SCALAR_RECUR]], i16* [[B]], align 4<br>
>>>> ; CHECK-NEXT: [[INC]] = add nsw
i32 [[I]], 1<br>
>>>> ; CHECK-NEXT: [[CMP2:%.*]] =
icmp slt i32 [[I]], 2096<br>
>>>> ; CHECK-NEXT: br i1 [[CMP2]],
label [[FOR_COND]], label<br>
>>>> [[IF_END]], [[LOOP7:!llvm.loop !.*]]<br>
>>>> ; CHECK: if.end:<br>
>>>> -; CHECK-NEXT: [[REC_LCSSA:%.*]] =
phi i16 [ [[SCALAR_RECUR]],<br>
>>>> [[FOR_BODY]] ], [ [[SCALAR_RECUR]],
[[FOR_COND]] ]<br>
>>>> +; CHECK-NEXT: [[REC_LCSSA:%.*]] =
phi i16 [ [[SCALAR_RECUR]],<br>
>>>> [[FOR_BODY]] ], [ [[SCALAR_RECUR]],
[[FOR_COND]] ], [<br>
>>>> [[VECTOR_RECUR_EXTRACT_FOR_PHI]],
[[MIDDLE_BLOCK]] ]<br>
>>>> ; CHECK-NEXT: ret i16
[[REC_LCSSA]]<br>
>>>> ;<br>
>>>> entry:<br>
>>>> @@ -557,9 +558,10 @@ define i16
@multiple_exit2(i16* %p, i32 %n) {<br>
>>>> ; CHECK-NEXT: [[TMP15:%.*]] =
icmp eq i32 [[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT: br i1 [[TMP15]],
label [[MIDDLE_BLOCK:%.*]],<br>
>>>> label [[VECTOR_BODY]],
[[LOOP8:!llvm.loop !.*]]<br>
>>>> ; CHECK: middle.block:<br>
>>>> +; CHECK-NEXT: [[CMP_N:%.*]] =
icmp eq i32 [[TMP2]], [[N_VEC]]<br>
>>>> ; CHECK-NEXT:
[[VECTOR_RECUR_EXTRACT:%.*]] = extractelement<br>
>>>> <4 x i16> [[WIDE_LOAD]], i32 3<br>
>>>> ; CHECK-NEXT:
[[VECTOR_RECUR_EXTRACT_FOR_PHI:%.*]] =<br>
>>>> extractelement <4 x i16>
[[WIDE_LOAD]], i32 2<br>
>>>> -; CHECK-NEXT: br label
[[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br i1 [[CMP_N]],
label [[IF_END:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph" moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[SCALAR_RECUR_INIT:%.*]] = phi i16 [ 0,<br>
>>>> [[ENTRY:%.*]] ], [
[[VECTOR_RECUR_EXTRACT]], [[MIDDLE_BLOCK]] ]<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i32 [ [[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY]] ]<br>
>>>> @@ -571,14 +573,14 @@ define i16
@multiple_exit2(i16* %p, i32 %n) {<br>
>>>> ; CHECK-NEXT: [[B:%.*]] =
getelementptr inbounds i16, i16*<br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> ; CHECK-NEXT: [[REC_NEXT]] =
load i16, i16* [[B]], align 2<br>
>>>> ; CHECK-NEXT: [[CMP:%.*]] =
icmp slt i32 [[I]], [[N]]<br>
>>>> -; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label<br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label [[IF_END]]<br>
>>>> ; CHECK: for.body:<br>
>>>> ; CHECK-NEXT: store i16
[[SCALAR_RECUR]], i16* [[B]], align 4<br>
>>>> ; CHECK-NEXT: [[INC]] = add nsw
i32 [[I]], 1<br>
>>>> ; CHECK-NEXT: [[CMP2:%.*]] =
icmp slt i32 [[I]], 2096<br>
>>>> ; CHECK-NEXT: br i1 [[CMP2]],
label [[FOR_COND]], label<br>
>>>> [[IF_END]], [[LOOP9:!llvm.loop !.*]]<br>
>>>> ; CHECK: if.end:<br>
>>>> -; CHECK-NEXT: [[REC_LCSSA:%.*]] =
phi i16 [ [[SCALAR_RECUR]],<br>
>>>> [[FOR_COND]] ], [ 10, [[FOR_BODY]] ]<br>
>>>> +; CHECK-NEXT: [[REC_LCSSA:%.*]] =
phi i16 [ [[SCALAR_RECUR]],<br>
>>>> [[FOR_COND]] ], [ 10, [[FOR_BODY]] ],
[<br>
>>>> [[VECTOR_RECUR_EXTRACT_FOR_PHI]],
[[MIDDLE_BLOCK]] ]<br>
>>>> ; CHECK-NEXT: ret i16
[[REC_LCSSA]]<br>
>>>> ;<br>
>>>> entry:<br>
>>>><br>
>>>> diff --git<br>
>>>>
a/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
>>>>
b/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
>>>> index f0ba677348ab..0d4bdf0ecac3
100644<br>
>>>> ---
a/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
>>>> +++
b/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
>>>> @@ -447,7 +447,7 @@ define void
@even_load_static_tc(i32* noalias<br>
>>>> nocapture readonly %A, i32* noalia<br>
>>>> ; CHECK-NEXT: [[TMP6:%.*]] =
icmp eq i64 [[INDEX_NEXT]], 508<br>
>>>> ; CHECK-NEXT: br i1 [[TMP6]],
label [[MIDDLE_BLOCK:%.*]],<br>
>>>> label [[VECTOR_BODY]],
[[LOOP12:!llvm.loop !.*]]<br>
>>>> ; CHECK: middle.block:<br>
>>>> -; CHECK-NEXT: br label
[[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br i1 false, label
[[FOR_COND_CLEANUP:%.*]],<br>
>>>> label [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph" moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [ 1016,<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0,
[[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT: br label
[[FOR_BODY:%.*]]<br>
>>>> @@ -463,7 +463,7 @@ define void
@even_load_static_tc(i32* noalias<br>
>>>> nocapture readonly %A, i32* noalia<br>
>>>> ; CHECK-NEXT: store i32
[[MUL]], i32* [[ARRAYIDX2]], align 4<br>
>>>> ; CHECK-NEXT:
[[INDVARS_IV_NEXT]] = add nuw nsw i64<br>
>>>> [[INDVARS_IV]], 2<br>
>>>> ; CHECK-NEXT: [[CMP:%.*]] =
icmp ult i64 [[INDVARS_IV]], 1022<br>
>>>> -; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label<br>
>>>> [[FOR_COND_CLEANUP:%.*]],
[[LOOP13:!llvm.loop !.*]]<br>
>>>> +; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label<br>
>>>> [[FOR_COND_CLEANUP]],
[[LOOP13:!llvm.loop !.*]]<br>
>>>> ;<br>
>>>> entry:<br>
>>>> br label %for.body<br>
>>>> @@ -528,7 +528,7 @@ define void
@even_load_dynamic_tc(i32* noalias<br>
>>>> nocapture readonly %A, i32* noali<br>
>>>> ; CHECK-NEXT: [[TMP12:%.*]] =
icmp eq i64 [[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT: br i1 [[TMP12]],
label [[MIDDLE_BLOCK:%.*]],<br>
>>>> label [[VECTOR_BODY]],
[[LOOP14:!llvm.loop !.*]]<br>
>>>> ; CHECK: middle.block:<br>
>>>> -; CHECK-NEXT: br label
[[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br i1 false, label
[[FOR_COND_CLEANUP:%.*]],<br>
>>>> label [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph" moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [ [[IND_END]],<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0,
[[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT: br label
[[FOR_BODY:%.*]]<br>
>>>> @@ -544,7 +544,7 @@ define void
@even_load_dynamic_tc(i32* noalias<br>
>>>> nocapture readonly %A, i32* noali<br>
>>>> ; CHECK-NEXT: store i32
[[MUL]], i32* [[ARRAYIDX2]], align 4<br>
>>>> ; CHECK-NEXT:
[[INDVARS_IV_NEXT]] = add nuw nsw i64<br>
>>>> [[INDVARS_IV]], 2<br>
>>>> ; CHECK-NEXT: [[CMP:%.*]] =
icmp ult i64 [[INDVARS_IV_NEXT]],<br>
>>>> [[N]]<br>
>>>> -; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label<br>
>>>> [[FOR_COND_CLEANUP:%.*]],
[[LOOP15:!llvm.loop !.*]]<br>
>>>> +; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label<br>
>>>> [[FOR_COND_CLEANUP]],
[[LOOP15:!llvm.loop !.*]]<br>
>>>> ;<br>
>>>> entry:<br>
>>>> br label %for.body<br>
>>>> @@ -973,7 +973,7 @@ define void
@PR27626_0(%pair.i32 *%p, i32 %z,<br>
>>>> i64 %n) {<br>
>>>> ; CHECK-NEXT: [[TMP19:%.*]] =
icmp eq i64 [[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT: br i1 [[TMP19]],
label [[MIDDLE_BLOCK:%.*]],<br>
>>>> label [[VECTOR_BODY]],
[[LOOP24:!llvm.loop !.*]]<br>
>>>> ; CHECK: middle.block:<br>
>>>> -; CHECK-NEXT: br label
[[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br i1 false, label
[[FOR_END:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph" moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [ [[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0,
[[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT: br label
[[FOR_BODY:%.*]]<br>
>>>> @@ -985,7 +985,7 @@ define void
@PR27626_0(%pair.i32 *%p, i32 %z,<br>
>>>> i64 %n) {<br>
>>>> ; CHECK-NEXT: store i32 [[Z]],
i32* [[P_I_Y]], align 4<br>
>>>> ; CHECK-NEXT: [[I_NEXT]] = add
nuw nsw i64 [[I]], 1<br>
>>>> ; CHECK-NEXT: [[COND:%.*]] =
icmp slt i64 [[I_NEXT]], [[N]]<br>
>>>> -; CHECK-NEXT: br i1 [[COND]],
label [[FOR_BODY]], label<br>
>>>> [[FOR_END:%.*]], [[LOOP25:!llvm.loop
!.*]]<br>
>>>> +; CHECK-NEXT: br i1 [[COND]],
label [[FOR_BODY]], label<br>
>>>> [[FOR_END]], [[LOOP25:!llvm.loop
!.*]]<br>
>>>> ; CHECK: for.end:<br>
>>>> ; CHECK-NEXT: ret void<br>
>>>> ;<br>
>>>> @@ -1066,7 +1066,7 @@ define i32
@PR27626_1(%pair.i32 *%p, i64 %n) {<br>
>>>> ; CHECK-NEXT: [[RDX_SHUF3:%.*]]
= shufflevector <4 x i32><br>
>>>> [[BIN_RDX]], <4 x i32> poison,
<4 x i32> <i32 1, i32 undef, i32<br>
>>>> undef, i32 undef><br>
>>>> ; CHECK-NEXT: [[BIN_RDX4:%.*]]
= add <4 x i32> [[BIN_RDX]],<br>
>>>> [[RDX_SHUF3]]<br>
>>>> ; CHECK-NEXT: [[TMP19:%.*]] =
extractelement <4 x i32><br>
>>>> [[BIN_RDX4]], i32 0<br>
>>>> -; CHECK-NEXT: br label
[[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br i1 false, label
[[FOR_END:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph" moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [ [[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0,
[[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
[[BC_MERGE_RDX:%.*]] = phi i32 [ [[TMP19]],<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY]] ]<br>
>>>> @@ -1081,9 +1081,10 @@ define i32
@PR27626_1(%pair.i32 *%p, i64 %n) {<br>
>>>> ; CHECK-NEXT: [[TMP21]] = add
nsw i32 [[TMP20]], [[S]]<br>
>>>> ; CHECK-NEXT: [[I_NEXT]] = add
nuw nsw i64 [[I]], 1<br>
>>>> ; CHECK-NEXT: [[COND:%.*]] =
icmp slt i64 [[I_NEXT]], [[N]]<br>
>>>> -; CHECK-NEXT: br i1 [[COND]],
label [[FOR_BODY]], label<br>
>>>> [[FOR_END:%.*]], [[LOOP27:!llvm.loop
!.*]]<br>
>>>> +; CHECK-NEXT: br i1 [[COND]],
label [[FOR_BODY]], label<br>
>>>> [[FOR_END]], [[LOOP27:!llvm.loop
!.*]]<br>
>>>> ; CHECK: for.end:<br>
>>>> -; CHECK-NEXT: ret i32 [[TMP21]]<br>
>>>> +; CHECK-NEXT: [[TMP22:%.*]] = phi
i32 [ [[TMP21]], [[FOR_BODY]]<br>
>>>> ], [ [[TMP19]], [[MIDDLE_BLOCK]] ]<br>
>>>> +; CHECK-NEXT: ret i32 [[TMP22]]<br>
>>>> ;<br>
>>>> entry:<br>
>>>> br label %for.body<br>
>>>> @@ -1162,7 +1163,7 @@ define void
@PR27626_2(%pair.i32 *%p, i64 %n,<br>
>>>> i32 %z) {<br>
>>>> ; CHECK-NEXT: [[TMP20:%.*]] =
icmp eq i64 [[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT: br i1 [[TMP20]],
label [[MIDDLE_BLOCK:%.*]],<br>
>>>> label [[VECTOR_BODY]],
[[LOOP28:!llvm.loop !.*]]<br>
>>>> ; CHECK: middle.block:<br>
>>>> -; CHECK-NEXT: br label
[[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br i1 false, label
[[FOR_END:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph" moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [ [[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0,
[[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT: br label
[[FOR_BODY:%.*]]<br>
>>>> @@ -1176,7 +1177,7 @@ define void
@PR27626_2(%pair.i32 *%p, i64 %n,<br>
>>>> i32 %z) {<br>
>>>> ; CHECK-NEXT: store i32
[[TMP21]], i32* [[P_I_Y]], align 4<br>
>>>> ; CHECK-NEXT: [[I_NEXT]] = add
nuw nsw i64 [[I]], 1<br>
>>>> ; CHECK-NEXT: [[COND:%.*]] =
icmp slt i64 [[I_NEXT]], [[N]]<br>
>>>> -; CHECK-NEXT: br i1 [[COND]],
label [[FOR_BODY]], label<br>
>>>> [[FOR_END:%.*]], [[LOOP29:!llvm.loop
!.*]]<br>
>>>> +; CHECK-NEXT: br i1 [[COND]],
label [[FOR_BODY]], label<br>
>>>> [[FOR_END]], [[LOOP29:!llvm.loop
!.*]]<br>
>>>> ; CHECK: for.end:<br>
>>>> ; CHECK-NEXT: ret void<br>
>>>> ;<br>
>>>> @@ -1263,7 +1264,7 @@ define i32
@PR27626_3(%pair.i32 *%p, i64 %n,<br>
>>>> i32 %z) {<br>
>>>> ; CHECK-NEXT: [[RDX_SHUF3:%.*]]
= shufflevector <4 x i32><br>
>>>> [[BIN_RDX]], <4 x i32> poison,
<4 x i32> <i32 1, i32 undef, i32<br>
>>>> undef, i32 undef><br>
>>>> ; CHECK-NEXT: [[BIN_RDX4:%.*]]
= add <4 x i32> [[BIN_RDX]],<br>
>>>> [[RDX_SHUF3]]<br>
>>>> ; CHECK-NEXT: [[TMP22:%.*]] =
extractelement <4 x i32><br>
>>>> [[BIN_RDX4]], i32 0<br>
>>>> -; CHECK-NEXT: br label
[[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: br i1 false, label
[[FOR_END:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph" moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [ [[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0,
[[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
[[BC_MERGE_RDX:%.*]] = phi i32 [ [[TMP22]],<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY]] ]<br>
>>>> @@ -1281,9 +1282,10 @@ define i32
@PR27626_3(%pair.i32 *%p, i64 %n,<br>
>>>> i32 %z) {<br>
>>>> ; CHECK-NEXT: [[TMP25]] = add
nsw i32 [[TMP24]], [[S]]<br>
>>>> ; CHECK-NEXT: [[I_NEXT]] = add
nuw nsw i64 [[I]], 1<br>
>>>> ; CHECK-NEXT: [[COND:%.*]] =
icmp slt i64 [[I_NEXT]], [[N]]<br>
>>>> -; CHECK-NEXT: br i1 [[COND]],
label [[FOR_BODY]], label<br>
>>>> [[FOR_END:%.*]], [[LOOP31:!llvm.loop
!.*]]<br>
>>>> +; CHECK-NEXT: br i1 [[COND]],
label [[FOR_BODY]], label<br>
>>>> [[FOR_END]], [[LOOP31:!llvm.loop
!.*]]<br>
>>>> ; CHECK: for.end:<br>
>>>> -; CHECK-NEXT: ret i32 [[TMP25]]<br>
>>>> +; CHECK-NEXT: [[TMP26:%.*]] = phi
i32 [ [[TMP25]], [[FOR_BODY]]<br>
>>>> ], [ [[TMP22]], [[MIDDLE_BLOCK]] ]<br>
>>>> +; CHECK-NEXT: ret i32 [[TMP26]]<br>
>>>> ;<br>
>>>> entry:<br>
>>>> br label %for.body<br>
>>>><br>
>>>> diff --git
a/llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
>>>>
b/llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
>>>> index f32002fae2b6..91780789088b
100644<br>
>>>> ---
a/llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
>>>> +++
b/llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
>>>> @@ -146,14 +146,15 @@ define void
@early_exit(i16* %p, i32 %n) {<br>
>>>> ; CHECK-NEXT: [[TMP10:%.*]] =
icmp eq i32 [[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT: br i1 [[TMP10]],
label [[MIDDLE_BLOCK:%.*]],<br>
>>>> label [[VECTOR_BODY]],
[[LOOP4:!llvm.loop !.*]]<br>
>>>> ; CHECK: middle.block:<br>
>>>> -; CHECK-NEXT: br label
[[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: [[CMP_N:%.*]] =
icmp eq i32 [[TMP1]], [[N_VEC]]<br>
>>>> +; CHECK-NEXT: br i1 [[CMP_N]],
label [[IF_END:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph" moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i32 [ [[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0,
[[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT: br label
[[FOR_COND:%.*]]<br>
>>>> ; CHECK: for.cond:<br>
>>>> ; CHECK-NEXT: [[I:%.*]] = phi
i32 [ [[BC_RESUME_VAL]],<br>
>>>> [[SCALAR_PH]] ], [ [[INC:%.*]],
[[FOR_BODY:%.*]] ]<br>
>>>> ; CHECK-NEXT: [[CMP:%.*]] =
icmp slt i32 [[I]], [[N]]<br>
>>>> -; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label<br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label [[IF_END]]<br>
>>>> ; CHECK: for.body:<br>
>>>> ; CHECK-NEXT: [[IPROM:%.*]] =
sext i32 [[I]] to i64<br>
>>>> ; CHECK-NEXT: [[B:%.*]] =
getelementptr inbounds i16, i16*<br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> @@ -285,14 +286,15 @@ define void
@multiple_unique_exit(i16* %p,<br>
>>>> i32 %n) {<br>
>>>> ; CHECK-NEXT: [[TMP11:%.*]] =
icmp eq i32 [[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT: br i1 [[TMP11]],
label [[MIDDLE_BLOCK:%.*]],<br>
>>>> label [[VECTOR_BODY]],
[[LOOP6:!llvm.loop !.*]]<br>
>>>> ; CHECK: middle.block:<br>
>>>> -; CHECK-NEXT: br label
[[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: [[CMP_N:%.*]] =
icmp eq i32 [[TMP2]], [[N_VEC]]<br>
>>>> +; CHECK-NEXT: br i1 [[CMP_N]],
label [[IF_END:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph" moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i32 [ [[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0,
[[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT: br label
[[FOR_COND:%.*]]<br>
>>>> ; CHECK: for.cond:<br>
>>>> ; CHECK-NEXT: [[I:%.*]] = phi
i32 [ [[BC_RESUME_VAL]],<br>
>>>> [[SCALAR_PH]] ], [ [[INC:%.*]],
[[FOR_BODY:%.*]] ]<br>
>>>> ; CHECK-NEXT: [[CMP:%.*]] =
icmp slt i32 [[I]], [[N]]<br>
>>>> -; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label<br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label [[IF_END]]<br>
>>>> ; CHECK: for.body:<br>
>>>> ; CHECK-NEXT: [[IPROM:%.*]] =
sext i32 [[I]] to i64<br>
>>>> ; CHECK-NEXT: [[B:%.*]] =
getelementptr inbounds i16, i16*<br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> @@ -372,14 +374,17 @@ define i32
@multiple_unique_exit2(i16* %p,<br>
>>>> i32 %n) {<br>
>>>> ; CHECK-NEXT: [[TMP11:%.*]] =
icmp eq i32 [[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT: br i1 [[TMP11]],
label [[MIDDLE_BLOCK:%.*]],<br>
>>>> label [[VECTOR_BODY]],
[[LOOP8:!llvm.loop !.*]]<br>
>>>> ; CHECK: middle.block:<br>
>>>> -; CHECK-NEXT: br label
[[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: [[CMP_N:%.*]] =
icmp eq i32 [[TMP2]], [[N_VEC]]<br>
>>>> +; CHECK-NEXT: [[IND_ESCAPE:%.*]]
= sub i32 [[N_VEC]], 1<br>
>>>> +; CHECK-NEXT: [[IND_ESCAPE1:%.*]]
= sub i32 [[N_VEC]], 1<br>
>>>> +; CHECK-NEXT: br i1 [[CMP_N]],
label [[IF_END:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph" moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i32 [ [[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0,
[[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT: br label
[[FOR_COND:%.*]]<br>
>>>> ; CHECK: for.cond:<br>
>>>> ; CHECK-NEXT: [[I:%.*]] = phi
i32 [ [[BC_RESUME_VAL]],<br>
>>>> [[SCALAR_PH]] ], [ [[INC:%.*]],
[[FOR_BODY:%.*]] ]<br>
>>>> ; CHECK-NEXT: [[CMP:%.*]] =
icmp slt i32 [[I]], [[N]]<br>
>>>> -; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label<br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label [[IF_END]]<br>
>>>> ; CHECK: for.body:<br>
>>>> ; CHECK-NEXT: [[IPROM:%.*]] =
sext i32 [[I]] to i64<br>
>>>> ; CHECK-NEXT: [[B:%.*]] =
getelementptr inbounds i16, i16*<br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> @@ -388,7 +393,7 @@ define i32
@multiple_unique_exit2(i16* %p, i32<br>
>>>> %n) {<br>
>>>> ; CHECK-NEXT: [[CMP2:%.*]] =
icmp slt i32 [[I]], 2096<br>
>>>> ; CHECK-NEXT: br i1 [[CMP2]],
label [[FOR_COND]], label<br>
>>>> [[IF_END]], [[LOOP9:!llvm.loop !.*]]<br>
>>>> ; CHECK: if.end:<br>
>>>> -; CHECK-NEXT: [[I_LCSSA:%.*]] =
phi i32 [ [[I]], [[FOR_BODY]]<br>
>>>> ], [ [[I]], [[FOR_COND]] ]<br>
>>>> +; CHECK-NEXT: [[I_LCSSA:%.*]] =
phi i32 [ [[I]], [[FOR_BODY]]<br>
>>>> ], [ [[I]], [[FOR_COND]] ], [
[[IND_ESCAPE1]], [[MIDDLE_BLOCK]] ]<br>
>>>> ; CHECK-NEXT: ret i32
[[I_LCSSA]]<br>
>>>> ;<br>
>>>> ; TAILFOLD-LABEL:
@multiple_unique_exit2(<br>
>>>> @@ -461,14 +466,15 @@ define i32
@multiple_unique_exit3(i16* %p,<br>
>>>> i32 %n) {<br>
>>>> ; CHECK-NEXT: [[TMP11:%.*]] =
icmp eq i32 [[INDEX_NEXT]],<br>
>>>> [[N_VEC]]<br>
>>>> ; CHECK-NEXT: br i1 [[TMP11]],
label [[MIDDLE_BLOCK:%.*]],<br>
>>>> label [[VECTOR_BODY]],
[[LOOP10:!llvm.loop !.*]]<br>
>>>> ; CHECK: middle.block:<br>
>>>> -; CHECK-NEXT: br label
[[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: [[CMP_N:%.*]] =
icmp eq i32 [[TMP2]], [[N_VEC]]<br>
>>>> +; CHECK-NEXT: br i1 [[CMP_N]],
label [[IF_END:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph" moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i32 [ [[N_VEC]],<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0,
[[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT: br label
[[FOR_COND:%.*]]<br>
>>>> ; CHECK: for.cond:<br>
>>>> ; CHECK-NEXT: [[I:%.*]] = phi
i32 [ [[BC_RESUME_VAL]],<br>
>>>> [[SCALAR_PH]] ], [ [[INC:%.*]],
[[FOR_BODY:%.*]] ]<br>
>>>> ; CHECK-NEXT: [[CMP:%.*]] =
icmp slt i32 [[I]], [[N]]<br>
>>>> -; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label<br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT: br i1 [[CMP]],
label [[FOR_BODY]], label [[IF_END]]<br>
>>>> ; CHECK: for.body:<br>
>>>> ; CHECK-NEXT: [[IPROM:%.*]] =
sext i32 [[I]] to i64<br>
>>>> ; CHECK-NEXT: [[B:%.*]] =
getelementptr inbounds i16, i16*<br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> @@ -477,7 +483,7 @@ define i32
@multiple_unique_exit3(i16* %p, i32<br>
>>>> %n) {<br>
>>>> ; CHECK-NEXT: [[CMP2:%.*]] =
icmp slt i32 [[I]], 2096<br>
>>>> ; CHECK-NEXT: br i1 [[CMP2]],
label [[FOR_COND]], label<br>
>>>> [[IF_END]], [[LOOP11:!llvm.loop !.*]]<br>
>>>> ; CHECK: if.end:<br>
>>>> -; CHECK-NEXT: [[EXIT:%.*]] = phi
i32 [ 0, [[FOR_COND]] ], [ 1,<br>
>>>> [[FOR_BODY]] ]<br>
>>>> +; CHECK-NEXT: [[EXIT:%.*]] = phi
i32 [ 0, [[FOR_COND]] ], [ 1,<br>
>>>> [[FOR_BODY]] ], [ 0, [[MIDDLE_BLOCK]]
]<br>
>>>> ; CHECK-NEXT: ret i32 [[EXIT]]<br>
>>>> ;<br>
>>>> ; TAILFOLD-LABEL:
@multiple_unique_exit3(<br>
>>>> @@ -994,7 +1000,8 @@ define void
@scalar_predication(float* %addr) {<br>
>>>> ; CHECK-NEXT: [[TMP10:%.*]] =
icmp eq i64 [[INDEX_NEXT]], 200<br>
>>>> ; CHECK-NEXT: br i1 [[TMP10]],
label [[MIDDLE_BLOCK:%.*]],<br>
>>>> label [[VECTOR_BODY]],
[[LOOP12:!llvm.loop !.*]]<br>
>>>> ; CHECK: middle.block:<br>
>>>> -; CHECK-NEXT: br label
[[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: [[CMP_N:%.*]] =
icmp eq i64 201, 200<br>
>>>> +; CHECK-NEXT: br i1 [[CMP_N]],
label [[EXIT:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph" moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [ 200,<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0,
[[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT: br label
[[LOOP_HEADER:%.*]]<br>
>>>> @@ -1002,7 +1009,7 @@ define void
@scalar_predication(float* %addr) {<br>
>>>> ; CHECK-NEXT: [[IV:%.*]] = phi
i64 [ [[BC_RESUME_VAL]],<br>
>>>> [[SCALAR_PH]] ], [ [[IV_NEXT:%.*]],
[[LOOP_LATCH:%.*]] ]<br>
>>>> ; CHECK-NEXT: [[GEP:%.*]] =
getelementptr float, float*<br>
>>>> [[ADDR]], i64 [[IV]]<br>
>>>> ; CHECK-NEXT:
[[EXITCOND_NOT:%.*]] = icmp eq i64 [[IV]], 200<br>
>>>> -; CHECK-NEXT: br i1
[[EXITCOND_NOT]], label [[EXIT:%.*]], label<br>
>>>> [[LOOP_BODY:%.*]]<br>
>>>> +; CHECK-NEXT: br i1
[[EXITCOND_NOT]], label [[EXIT]], label<br>
>>>> [[LOOP_BODY:%.*]]<br>
>>>> ; CHECK: loop.body:<br>
>>>> ; CHECK-NEXT: [[TMP11:%.*]] =
load float, float* [[GEP]],<br>
>>>> align 4<br>
>>>> ; CHECK-NEXT: [[PRED:%.*]] =
fcmp oeq float [[TMP11]],<br>
>>>> 0.000000e+00<br>
>>>> @@ -1088,7 +1095,8 @@ define i32
@me_reduction(i32* %addr) {<br>
>>>> ; CHECK-NEXT: [[RDX_SHUF:%.*]]
= shufflevector <2 x i32><br>
>>>> [[TMP5]], <2 x i32> poison,
<2 x i32> <i32 1, i32 undef><br>
>>>> ; CHECK-NEXT: [[BIN_RDX:%.*]] =
add <2 x i32> [[TMP5]],<br>
>>>> [[RDX_SHUF]]<br>
>>>> ; CHECK-NEXT: [[TMP7:%.*]] =
extractelement <2 x i32><br>
>>>> [[BIN_RDX]], i32 0<br>
>>>> -; CHECK-NEXT: br label
[[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT: [[CMP_N:%.*]] =
icmp eq i64 201, 200<br>
>>>> +; CHECK-NEXT: br i1 [[CMP_N]],
label [[EXIT:%.*]], label<br>
>>>> [[SCALAR_PH]]<br>
>>>> ; CHECK: <a
href="http://scalar.ph" moz-do-not-send="true"
rel="noreferrer" target="_blank">scalar.ph</a>:<br>
>>>> ; CHECK-NEXT:
[[BC_RESUME_VAL:%.*]] = phi i64 [ 200,<br>
>>>> [[MIDDLE_BLOCK]] ], [ 0,
[[ENTRY:%.*]] ]<br>
>>>> ; CHECK-NEXT:
[[BC_MERGE_RDX:%.*]] = phi i32 [ 0, [[ENTRY]]<br>
>>>> ], [ [[TMP7]], [[MIDDLE_BLOCK]] ]<br>
>>>> @@ -1098,7 +1106,7 @@ define i32
@me_reduction(i32* %addr) {<br>
>>>> ; CHECK-NEXT: [[ACCUM:%.*]] =
phi i32 [ [[BC_MERGE_RDX]],<br>
>>>> [[SCALAR_PH]] ], [
[[ACCUM_NEXT:%.*]], [[LOOP_LATCH]] ]<br>
>>>> ; CHECK-NEXT: [[GEP:%.*]] =
getelementptr i32, i32* [[ADDR]],<br>
>>>> i64 [[IV]]<br>
>>>> ; CHECK-NEXT:
[[EXITCOND_NOT:%.*]] = icmp eq i64 [[IV]], 200<br>
>>>> -; CHECK-NEXT: br i1
[[EXITCOND_NOT]], label [[EXIT:%.*]], label<br>
>>>> [[LOOP_LATCH]]<br>
>>>> +; CHECK-NEXT: br i1
[[EXITCOND_NOT]], label [[EXIT]], label<br>
>>>> [[LOOP_LATCH]]<br>
>>>> ; CHECK: loop.latch:<br>
>>>> ; CHECK-NEXT: [[TMP8:%.*]] =
load i32, i32* [[GEP]], align 4<br>
>>>> ; CHECK-NEXT: [[ACCUM_NEXT]] =
add i32 [[ACCUM]], [[TMP8]]<br>
>>>> @@ -1106,7 +1114,7 @@ define i32
@me_reduction(i32* %addr) {<br>
>>>> ; CHECK-NEXT:
[[EXITCOND2_NOT:%.*]] = icmp eq i64 [[IV]], 400<br>
>>>> ; CHECK-NEXT: br i1
[[EXITCOND2_NOT]], label [[EXIT]], label<br>
>>>> [[LOOP_HEADER]], [[LOOP15:!llvm.loop
!.*]]<br>
>>>> ; CHECK: exit:<br>
>>>> -; CHECK-NEXT: [[LCSSA:%.*]] = phi
i32 [ 0, [[LOOP_HEADER]] ], [<br>
>>>> [[ACCUM_NEXT]], [[LOOP_LATCH]] ]<br>
>>>> +; CHECK-NEXT: [[LCSSA:%.*]] = phi
i32 [ 0, [[LOOP_HEADER]] ], [<br>
>>>> [[ACCUM_NEXT]], [[LOOP_LATCH]] ], [
[[TMP7]], [[MIDDLE_BLOCK]] ]<br>
>>>> ; CHECK-NEXT: ret i32 [[LCSSA]]<br>
>>>> ;<br>
>>>> ; TAILFOLD-LABEL: @me_reduction(<br>
>>>><br>
>>>><br>
>>>><br>
>>>>
_______________________________________________<br>
>>>> llvm-commits mailing list<br>
>>>> <a
href="mailto:llvm-commits@lists.llvm.org"
moz-do-not-send="true" target="_blank">llvm-commits@lists.llvm.org</a><br>
>>>> <a
href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits"
moz-do-not-send="true" rel="noreferrer"
target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
>>>
_______________________________________________<br>
>>> llvm-commits mailing list<br>
>>> <a
href="mailto:llvm-commits@lists.llvm.org"
moz-do-not-send="true" target="_blank">llvm-commits@lists.llvm.org</a><br>
>>> <a
href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits"
moz-do-not-send="true" rel="noreferrer"
target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
> _______________________________________________<br>
> llvm-commits mailing list<br>
> <a href="mailto:llvm-commits@lists.llvm.org"
moz-do-not-send="true" target="_blank">llvm-commits@lists.llvm.org</a><br>
> <a
href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits"
moz-do-not-send="true" rel="noreferrer"
target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a></blockquote>
</div>
</div>
</blockquote>
</blockquote>
<div dir="ltr"> </div>
</div>
<br>
</blockquote>
</body>
</html>