<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>I tried another cycle to see if this had been resolved, but am
still seeing build bot failures, nearly exclusively on PPC. (And
one arm self host bot.)Â <br>
</p>
<p>@PPC Bot Owner - I need help reducing a test case for the failure
seen here:
<a class="moz-txt-link-freetext" href="https://lab.llvm.org/buildbot#builders/100/builds/5762">https://lab.llvm.org/buildbot#builders/100/builds/5762</a>. I am
stuck, and unable to make progress for nearly 3 months now. I
would greatly appreciate help. <br>
</p>
<p>Philip<br>
</p>
<div class="moz-cite-prefix">On 5/5/21 11:59 PM, Adrian Kuegel
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAHhV=OX3NbAExy4YsQWLH0fC-zXiG_VpToAtB72NhkUwwYekRQ@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div dir="ltr">Sounds good to me, and thanks for sending the
heads up :)</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, May 5, 2021 at 10:01
PM Philip Reames <<a
href="mailto:listmail@philipreames.com"
moz-do-not-send="true">listmail@philipreames.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">FYI, I'm going to try
recommitting this without changes in a day or so.<br>
<br>
I never heard back from a PPC bot owner, and I don't have
enough <br>
information to really debug anything from the builtbot log.Â
I did run <br>
across a latent issue which this patch may very well have
exposed at <br>
much higher frequency; the previous patch in this series
(which is much <br>
more restrictive) does appear to have increased frequency.Â
That was <br>
worked around in 80e80250. My educated guess is that same
issue <br>
triggered the miscompile seen on the ppc bot, but that is
more of a <br>
guess than I'd really prefer.<br>
<br>
I'm going to submit this during off hours, and watch the
bots fairly <br>
closely after submit. Hopefully this either cycles clean or
I get a <br>
better clue as to what the root issue is.<br>
<br>
Philip<br>
<br>
On 2/8/21 9:09 PM, Philip Reames via llvm-commits wrote:<br>
> Ben,<br>
><br>
> Thanks for the clarification. The log does not make
the fact this is <br>
> an execution failure obvious.<br>
><br>
> No, I don't have access to a PPC machine.<br>
><br>
> I am going to need some assistance from the bot owner
on this. At a <br>
> minimum, IR for the test in question (before
optimization, but on <br>
> target platform) seems like a reasonable ask.<br>
><br>
> I strongly suspect this change is simply exposing
another latent <br>
> issue. Or at least, I've reviewed the change and don't
see anything <br>
> likely to cause runtime crashes w/o also tripping
compiler asserts.<br>
><br>
> Philip<br>
><br>
><br>
> On 2/8/21 5:21 AM, Benjamin Kramer wrote:<br>
>> `execution_time` failures mean that the bot
succeeded building a test<br>
>> but it failed when running it. I'm relatively
certain that this is the<br>
>> same issue Adrian is seeing -- binaries segfaulting
early on PPC.<br>
>><br>
>> The bot log output isn't helpful at all for
investigating why this is<br>
>> happening. Do you happen to have access to a PPC
machine?<br>
>><br>
>> On Fri, Feb 5, 2021 at 6:05 PM Philip Reames via
llvm-commits<br>
>> <<a href="mailto:llvm-commits@lists.llvm.org"
target="_blank" moz-do-not-send="true">llvm-commits@lists.llvm.org</a>>
wrote:<br>
>>> Adrian,<br>
>>><br>
>>> I'm going to need you to provide a bit more
information here. The test<br>
>>> failure in stage1 was fixed at the time you
reverted this patch. The<br>
>>> remaining failure in the bot is very unclear.Â
What is a execution_time<br>
>>> failure? From the log output, the "failing" run
finished in 0.5 <br>
>>> seconds,<br>
>>> whereas the previous "succeeding" run finished
in 11 seconds. Without<br>
>>> further context, I'd say that's no failure.<br>
>>><br>
>>> I'll also note that I did not receive email
from this bot. I received<br>
>>> notice from the various other bots and fixed
the ARM test issue, but<br>
>>> unless I missed it in with the others, this bot
is not notifying.<br>
>>><br>
>>> In general, I'm a fan of fast reverts, but I
have to admit, this one<br>
>>> appears borderline at the moment.<br>
>>><br>
>>> Philip<br>
>>><br>
>>> On 2/5/21 3:53 AM, Adrian Kuegel via
llvm-commits wrote:<br>
>>>> Author: Adrian Kuegel<br>
>>>> Date: 2021-02-05T12:51:03+01:00<br>
>>>> New Revision:
7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c<br>
>>>><br>
>>>> URL: <br>
>>>> <a
href="https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c</a><br>
>>>> DIFF: <br>
>>>> <a
href="https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c.diff"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c.diff</a><br>
>>>><br>
>>>> LOG: Revert "[LV] Unconditionally branch
from middle to scalar <br>
>>>> preheader if the scalar loop must execute"<br>
>>>><br>
>>>> This reverts commit
3e5ce49e5371ce4feadbf97dd5c2b652d9db3d1d.<br>
>>>><br>
>>>> Tests started failing on PPC, for example:<br>
>>>> <a
href="http://lab.llvm.org:8011/#/builders/105/builds/5569"
rel="noreferrer" target="_blank" moz-do-not-send="true">http://lab.llvm.org:8011/#/builders/105/builds/5569</a><br>
>>>><br>
>>>> Added:<br>
>>>><br>
>>>><br>
>>>> Modified:<br>
>>>> Â Â Â Â Â
llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
>>>> Â Â Â Â Â
llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
>>>>
llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
>>>>
llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
>>>> Â Â Â Â Â
llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
>>>><br>
>>>> Removed:<br>
>>>><br>
>>>><br>
>>>><br>
>>>>
################################################################################
<br>
>>>><br>
>>>> diff --git
a/llvm/lib/Transforms/Utils/LoopVersioning.cpp <br>
>>>>
b/llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
>>>> index 8a89158788cf..de4fb446fdf2 100644<br>
>>>> ---
a/llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
>>>> +++
b/llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
>>>> @@ -44,11 +44,11 @@
LoopVersioning::LoopVersioning(const <br>
>>>> LoopAccessInfo &LAI,<br>
>>>> Â Â Â Â Â Â Â Â AliasChecks(Checks.begin(),
Checks.end()),<br>
>>>> Â Â Â Â Â Â Â Â
Preds(LAI.getPSE().getUnionPredicate()), LAI(LAI), LI(LI), <br>
>>>> DT(DT),<br>
>>>> Â Â Â Â Â Â Â Â SE(SE) {<br>
>>>> +Â assert(L->getUniqueExitBlock()
&& "No single exit block");<br>
>>>> Â Â }<br>
>>>><br>
>>>> Â Â void LoopVersioning::versionLoop(<br>
>>>> Â Â Â Â Â Â const SmallVectorImpl<Instruction
*> &DefsUsedOutside) {<br>
>>>> -Â
assert(VersionedLoop->getUniqueExitBlock() && "No
single exit <br>
>>>> block");<br>
>>>> Â Â Â Â
assert(VersionedLoop->isLoopSimplifyForm() &&<br>
>>>> Â Â Â Â Â Â Â Â Â Â Â "Loop is not in loop-simplify
form");<br>
>>>><br>
>>>><br>
>>>> diff --git
a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp <br>
>>>>
b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
>>>> index 3277842edbfe..6bce0caeb36f 100644<br>
>>>> ---
a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
>>>> +++
b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
>>>> @@ -852,7 +852,7 @@ class
InnerLoopVectorizer {<br>
>>>> Â Â Â Â /// Middle Block between the vector
and the scalar.<br>
>>>> Â Â Â Â BasicBlock *LoopMiddleBlock;<br>
>>>><br>
>>>> -Â /// The unique ExitBlock of the scalar
loop if one exists. Note <br>
>>>> that<br>
>>>> +Â /// The (unique) ExitBlock of the scalar
loop. Note that<br>
>>>> Â Â Â Â /// there can be multiple exiting
edges reaching this block.<br>
>>>> Â Â Â Â BasicBlock *LoopExitBlock;<br>
>>>><br>
>>>> @@ -3147,13 +3147,9 @@ void <br>
>>>>
InnerLoopVectorizer::emitMinimumIterationCountCheck(Loop *L,<br>
>>>> DT->getNode(Bypass)->getIDom())
&&<br>
>>>> Â Â Â Â Â Â Â Â Â Â Â "TC check is expected to
dominate Bypass");<br>
>>>><br>
>>>> -Â // Update dominator for Bypass &
LoopExit (if needed).<br>
>>>> +Â // Update dominator for Bypass &
LoopExit.<br>
>>>> Â Â Â Â
DT->changeImmediateDominator(Bypass, TCCheckBlock);<br>
>>>> -Â if (!Cost->requiresScalarEpilogue())<br>
>>>> -Â Â Â // If there is an epilogue which must
run, there's no edge <br>
>>>> from the<br>
>>>> -   // middle block to exit blocks and
thus no need to update the <br>
>>>> immediate<br>
>>>> -Â Â Â // dominator of the exit blocks.<br>
>>>> -Â Â Â
DT->changeImmediateDominator(LoopExitBlock,
TCCheckBlock);<br>
>>>> +Â
DT->changeImmediateDominator(LoopExitBlock,
TCCheckBlock);<br>
>>>><br>
>>>> Â Â Â Â ReplaceInstWithInst(<br>
>>>> Â Â Â Â Â Â Â Â TCCheckBlock->getTerminator(),<br>
>>>> @@ -3192,11 +3188,7 @@ void <br>
>>>> InnerLoopVectorizer::emitSCEVChecks(Loop
*L, BasicBlock *Bypass) {<br>
>>>> Â Â Â Â // Update dominator only if this is
first RT check.<br>
>>>> Â Â Â Â if (LoopBypassBlocks.empty()) {<br>
>>>> Â Â Â Â Â Â
DT->changeImmediateDominator(Bypass, SCEVCheckBlock);<br>
>>>> -Â Â Â if
(!Cost->requiresScalarEpilogue())<br>
>>>> -Â Â Â Â Â // If there is an epilogue which
must run, there's no edge <br>
>>>> from the<br>
>>>> -     // middle block to exit blocks and
thus no need to update <br>
>>>> the immediate<br>
>>>> -Â Â Â Â Â // dominator of the exit blocks.<br>
>>>> -Â Â Â Â Â
DT->changeImmediateDominator(LoopExitBlock,
SCEVCheckBlock);<br>
>>>> +Â Â Â
DT->changeImmediateDominator(LoopExitBlock,
SCEVCheckBlock);<br>
>>>> Â Â Â Â }<br>
>>>><br>
>>>> Â Â Â Â ReplaceInstWithInst(<br>
>>>> @@ -3252,11 +3244,7 @@ void <br>
>>>>
InnerLoopVectorizer::emitMemRuntimeChecks(Loop *L,
BasicBlock <br>
>>>> *Bypass) {<br>
>>>> Â Â Â Â // Update dominator only if this is
first RT check.<br>
>>>> Â Â Â Â if (LoopBypassBlocks.empty()) {<br>
>>>> Â Â Â Â Â Â
DT->changeImmediateDominator(Bypass, MemCheckBlock);<br>
>>>> -Â Â Â if
(!Cost->requiresScalarEpilogue())<br>
>>>> -Â Â Â Â Â // If there is an epilogue which
must run, there's no edge <br>
>>>> from the<br>
>>>> -     // middle block to exit blocks and
thus no need to update <br>
>>>> the immediate<br>
>>>> -Â Â Â Â Â // dominator of the exit blocks.<br>
>>>> -Â Â Â Â Â
DT->changeImmediateDominator(LoopExitBlock,
MemCheckBlock);<br>
>>>> +Â Â Â
DT->changeImmediateDominator(LoopExitBlock,
MemCheckBlock);<br>
>>>> Â Â Â Â }<br>
>>>><br>
>>>> Â Â Â Â Instruction *FirstCheckInst;<br>
>>>> @@ -3381,10 +3369,9 @@ Value <br>
>>>> *InnerLoopVectorizer::emitTransformedIndex(<br>
>>>> Â Â Loop
*InnerLoopVectorizer::createVectorLoopSkeleton(StringRef <br>
>>>> Prefix) {<br>
>>>> Â Â Â Â LoopScalarBody =
OrigLoop->getHeader();<br>
>>>> Â Â Â Â LoopVectorPreHeader =
OrigLoop->getLoopPreheader();<br>
>>>> +Â LoopExitBlock =
OrigLoop->getUniqueExitBlock();<br>
>>>> +Â assert(LoopExitBlock && "Must
have an exit block");<br>
>>>> Â Â Â Â assert(LoopVectorPreHeader &&
"Invalid loop structure");<br>
>>>> -Â LoopExitBlock =
OrigLoop->getUniqueExitBlock(); // may be nullptr<br>
>>>> -Â assert((LoopExitBlock ||
Cost->requiresScalarEpilogue()) &&<br>
>>>> -Â Â Â Â Â Â Â Â "multiple exit loop without
required epilogue?");<br>
>>>><br>
>>>> Â Â Â Â LoopMiddleBlock =<br>
>>>> Â Â Â Â Â Â Â Â SplitBlock(LoopVectorPreHeader, <br>
>>>> LoopVectorPreHeader->getTerminator(),
DT,<br>
>>>> @@ -3393,20 +3380,12 @@ Loop <br>
>>>>
*InnerLoopVectorizer::createVectorLoopSkeleton(StringRef
Prefix) {<br>
>>>> Â Â Â Â Â Â Â Â SplitBlock(LoopMiddleBlock, <br>
>>>> LoopMiddleBlock->getTerminator(), DT,
LI,<br>
>>>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â nullptr, Twine(Prefix)
+ "<a href="http://scalar.ph" rel="noreferrer"
target="_blank" moz-do-not-send="true">scalar.ph</a>");<br>
>>>><br>
>>>> +Â // Set up branch from middle block to
the exit and scalar <br>
>>>> preheader blocks.<br>
>>>> +Â // completeLoopSkeleton will update the
condition to use an <br>
>>>> iteration check,<br>
>>>> +Â // if required to decide whether to
execute the remainder.<br>
>>>> +Â BranchInst *BrInst =<br>
>>>> +Â Â Â Â Â BranchInst::Create(LoopExitBlock,
LoopScalarPreHeader, <br>
>>>> Builder.getTrue());<br>
>>>> Â Â Â Â auto *ScalarLatchTerm = <br>
>>>>
OrigLoop->getLoopLatch()->getTerminator();<br>
>>>> -<br>
>>>> -Â // Set up the middle block terminator.Â
Two cases:<br>
>>>> -Â // 1) If we know that we must execute
the scalar epilogue, emit an<br>
>>>> -Â //Â Â Â unconditional branch.<br>
>>>> -Â // 2) Otherwise, we must have a single
unique exit block (due to <br>
>>>> how we<br>
>>>> -Â //Â Â Â implement the multiple exit
case). In this case, set up a <br>
>>>> conditonal<br>
>>>> -Â //Â Â Â branch from the middle block to
the loop scalar preheader, <br>
>>>> and the<br>
>>>> - //   exit block. completeLoopSkeleton
will update the <br>
>>>> condition to use an<br>
>>>> -Â //Â Â Â iteration check, if required to
decide whether to execute <br>
>>>> the remainder.<br>
>>>> -Â BranchInst *BrInst =
Cost->requiresScalarEpilogue() ?<br>
>>>> -Â Â Â
BranchInst::Create(LoopScalarPreHeader) :<br>
>>>> -Â Â Â BranchInst::Create(LoopExitBlock,
LoopScalarPreHeader,<br>
>>>> -Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Builder.getTrue());<br>
>>>>
BrInst->setDebugLoc(ScalarLatchTerm->getDebugLoc());<br>
>>>>
ReplaceInstWithInst(LoopMiddleBlock->getTerminator(),
BrInst);<br>
>>>><br>
>>>> @@ -3418,11 +3397,7 @@ Loop <br>
>>>>
*InnerLoopVectorizer::createVectorLoopSkeleton(StringRef
Prefix) {<br>
>>>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â nullptr, nullptr,
Twine(Prefix) + "vector.body");<br>
>>>><br>
>>>> Â Â Â Â // Update dominator for loop exit.<br>
>>>> -Â if (!Cost->requiresScalarEpilogue())<br>
>>>> -Â Â Â // If there is an epilogue which must
run, there's no edge <br>
>>>> from the<br>
>>>> -   // middle block to exit blocks and
thus no need to update the <br>
>>>> immediate<br>
>>>> -Â Â Â // dominator of the exit blocks.<br>
>>>> -Â Â Â
DT->changeImmediateDominator(LoopExitBlock,
LoopMiddleBlock);<br>
>>>> +Â
DT->changeImmediateDominator(LoopExitBlock,
LoopMiddleBlock);<br>
>>>><br>
>>>> Â Â Â Â // Create and register the new vector
loop.<br>
>>>> Â Â Â Â Loop *Lp = LI->AllocateLoop();<br>
>>>> @@ -3519,14 +3494,10 @@ BasicBlock <br>
>>>>
*InnerLoopVectorizer::completeLoopSkeleton(Loop *L,<br>
>>>> Â Â Â Â auto *ScalarLatchTerm = <br>
>>>>
OrigLoop->getLoopLatch()->getTerminator();<br>
>>>><br>
>>>> Â Â Â Â // Add a check in the middle block to
see if we have completed<br>
>>>> -Â // all of the iterations in the first
vector loop. Three cases:<br>
>>>> -Â // 1) If we require a scalar epilogue,
there is no conditional <br>
>>>> branch as<br>
>>>> -Â //Â Â Â we unconditionally branch to the
scalar preheader. Do <br>
>>>> nothing.<br>
>>>> -Â // 2) If (N - N%VF) == N, then we
*don't* need to run the <br>
>>>> remainder.<br>
>>>> -Â //Â Â Â Thus if tail is to be folded, we
know we don't need to run <br>
>>>> the<br>
>>>> -Â //Â Â Â remainder and we can use the
previous value for the <br>
>>>> condition (true).<br>
>>>> -Â // 3) Otherwise, construct a runtime
check.<br>
>>>> -Â if (!Cost->requiresScalarEpilogue()
&& <br>
>>>> !Cost->foldTailByMasking()) {<br>
>>>> +Â // all of the iterations in the first
vector loop.<br>
>>>> +Â // If (N - N%VF) == N, then we *don't*
need to run the remainder.<br>
>>>> +Â // If tail is to be folded, we know we
don't need to run the <br>
>>>> remainder.<br>
>>>> +Â if (!Cost->foldTailByMasking()) {<br>
>>>> Â Â Â Â Â Â Instruction *CmpN =
CmpInst::Create(Instruction::ICmp, <br>
>>>> CmpInst::ICMP_EQ,<br>
>>>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â
Count, VectorTripCount, <br>
>>>> "cmp.n",<br>
>>>> LoopMiddleBlock->getTerminator());<br>
>>>> @@ -3590,17 +3561,17 @@ BasicBlock <br>
>>>>
*InnerLoopVectorizer::createVectorizedLoopSkeleton() {<br>
>>>> Â Â Â Â |Â Â Â [Â ]_|Â Â <-- vector loop.<br>
>>>> Â Â Â Â |Â Â Â Â |<br>
>>>> Â Â Â Â |Â Â Â Â v<br>
>>>> -Â \Â Â -[ ]Â Â <--- middle-block.<br>
>>>> -Â Â \/Â Â |<br>
>>>> -Â Â /\Â Â v<br>
>>>> -Â Â | ->[ ]Â Â Â Â <--- new preheader.<br>
>>>> +Â |Â Â -[ ]Â Â <--- middle-block.<br>
>>>> +Â |Â /Â |<br>
>>>> +Â | /Â Â v<br>
>>>> +Â -|- >[ ]Â Â Â Â <--- new preheader.<br>
>>>> Â Â Â Â Â |Â Â Â |<br>
>>>> - (opt) v     <-- edge from middle to
exit iff epilogue is not <br>
>>>> required.<br>
>>>> +Â Â |Â Â Â v<br>
>>>> Â Â Â Â Â |Â Â [ ] \<br>
>>>> -Â Â |Â Â [ ]_|Â Â <-- old scalar loop to
handle remainder (scalar <br>
>>>> epilogue).<br>
>>>> +Â Â |Â Â [ ]_|Â Â <-- old scalar loop to
handle remainder.<br>
>>>> Â Â Â Â Â Â \Â Â |<br>
>>>> Â Â Â Â Â Â Â \Â v<br>
>>>> -Â Â Â Â Â >[ ]Â Â Â Â <-- exit block(s).<br>
>>>> +Â Â Â Â Â >[ ]Â Â Â Â <-- exit block.<br>
>>>> Â Â Â Â Â ...<br>
>>>> Â Â Â Â Â */<br>
>>>><br>
>>>> @@ -4021,18 +3992,13 @@ void <br>
>>>> InnerLoopVectorizer::fixVectorizedLoop() {<br>
>>>> Â Â Â Â // Forget the original basic block.<br>
>>>> Â Â Â Â PSE.getSE()->forgetLoop(OrigLoop);<br>
>>>><br>
>>>> -Â // If we inserted an edge from the
middle block to the unique <br>
>>>> exit block,<br>
>>>> -Â // update uses outside the loop (phis)
to account for the newly <br>
>>>> inserted<br>
>>>> -Â // edge.<br>
>>>> -Â if (!Cost->requiresScalarEpilogue())
{<br>
>>>> -Â Â Â // Fix-up external users of the
induction variables.<br>
>>>> -Â Â Â for (auto &Entry :
Legal->getInductionVars())<br>
>>>> -Â Â Â Â Â fixupIVUsers(Entry.first,
Entry.second,<br>
>>>> -
getOrCreateVectorTripCount(LI->getLoopFor(LoopVectorBody)),<br>
>>>> -Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â
IVEndValues[Entry.first], LoopMiddleBlock);<br>
>>>> +Â // Fix-up external users of the
induction variables.<br>
>>>> +Â for (auto &Entry :
Legal->getInductionVars())<br>
>>>> +Â Â Â fixupIVUsers(Entry.first,
Entry.second,<br>
>>>> +
getOrCreateVectorTripCount(LI->getLoopFor(LoopVectorBody)),<br>
>>>> +Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â IVEndValues[Entry.first],
LoopMiddleBlock);<br>
>>>><br>
>>>> -Â Â Â fixLCSSAPHIs();<br>
>>>> -Â }<br>
>>>> +Â fixLCSSAPHIs();<br>
>>>> Â Â Â Â for (Instruction *PI :
PredicatedInstructions)<br>
>>>> Â Â Â Â Â Â sinkScalarOperands(&*PI);<br>
>>>><br>
>>>> @@ -4250,13 +4216,12 @@ void <br>
>>>>
InnerLoopVectorizer::fixFirstOrderRecurrence(PHINode *Phi) {<br>
>>>> Â Â Â Â // recurrence in the exit block, and
then add an edge for the <br>
>>>> middle block.<br>
>>>> Â Â Â Â // Note that LCSSA does not imply
single entry when the <br>
>>>> original scalar loop<br>
>>>> Â Â Â Â // had multiple exiting edges (as we
always run the last <br>
>>>> iteration in the<br>
>>>> -Â // scalar epilogue); in that case, there
is no edge from middle <br>
>>>> to exit and<br>
>>>> -Â // and thus no phis which needed
updated.<br>
>>>> -Â if (!Cost->requiresScalarEpilogue())<br>
>>>> -Â Â Â for (PHINode &LCSSAPhi :
LoopExitBlock->phis())<br>
>>>> -Â Â Â Â Â if
(any_of(LCSSAPhi.incoming_values(),<br>
>>>> -Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â [Phi](Value *V) { return
V == Phi; }))<br>
>>>> -Â Â Â Â Â Â Â
LCSSAPhi.addIncoming(ExtractForPhiUsedOutsideLoop, <br>
>>>> LoopMiddleBlock);<br>
>>>> +Â // scalar epilogue); in that case, the
exiting path through <br>
>>>> middle will be<br>
>>>> +Â // dynamically dead and the value picked
for the phi doesn't <br>
>>>> matter.<br>
>>>> +Â for (PHINode &LCSSAPhi :
LoopExitBlock->phis())<br>
>>>> +Â Â Â if (any_of(LCSSAPhi.incoming_values(),<br>
>>>> +Â Â Â Â Â Â Â Â Â Â Â Â Â Â [Phi](Value *V) { return V
== Phi; }))<br>
>>>> +Â Â Â Â Â
LCSSAPhi.addIncoming(ExtractForPhiUsedOutsideLoop, <br>
>>>> LoopMiddleBlock);<br>
>>>> Â Â }<br>
>>>><br>
>>>> Â Â void
InnerLoopVectorizer::fixReduction(PHINode *Phi) {<br>
>>>> @@ -4421,11 +4386,10 @@ void <br>
>>>> InnerLoopVectorizer::fixReduction(PHINode
*Phi) {<br>
>>>> Â Â Â Â // We know that the loop is in LCSSA
form. We need to update <br>
>>>> the PHI nodes<br>
>>>>     // in the exit blocks. See comment on
analogous loop in<br>
>>>> Â Â Â Â // fixFirstOrderRecurrence for a more
complete explaination of <br>
>>>> the logic.<br>
>>>> -Â if (!Cost->requiresScalarEpilogue())<br>
>>>> -Â Â Â for (PHINode &LCSSAPhi :
LoopExitBlock->phis())<br>
>>>> -Â Â Â Â Â if
(any_of(LCSSAPhi.incoming_values(),<br>
>>>> -Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â [LoopExitInst](Value *V)
{ return V == <br>
>>>> LoopExitInst; }))<br>
>>>> -Â Â Â Â Â Â Â
LCSSAPhi.addIncoming(ReducedPartRdx, LoopMiddleBlock);<br>
>>>> +Â for (PHINode &LCSSAPhi :
LoopExitBlock->phis())<br>
>>>> +Â Â Â if (any_of(LCSSAPhi.incoming_values(),<br>
>>>> +Â Â Â Â Â Â Â Â Â Â Â Â Â Â [LoopExitInst](Value *V) {
return V == <br>
>>>> LoopExitInst; }))<br>
>>>> +Â Â Â Â Â LCSSAPhi.addIncoming(ReducedPartRdx,
LoopMiddleBlock);<br>
>>>><br>
>>>> Â Â Â Â // Fix the scalar loop reduction
variable with the incoming <br>
>>>> reduction sum<br>
>>>> Â Â Â Â // from the vector body and from the
backedge value.<br>
>>>> @@ -8074,11 +8038,7 @@ BasicBlock <br>
>>>>
*EpilogueVectorizerMainLoop::emitMinimumIterationCountCheck(<br>
>>>><br>
>>>> Â Â Â Â Â Â // Update dominator for Bypass &
LoopExit.<br>
>>>> Â Â Â Â Â Â
DT->changeImmediateDominator(Bypass, TCCheckBlock);<br>
>>>> -Â Â Â if
(!Cost->requiresScalarEpilogue())<br>
>>>> -Â Â Â Â Â // For loops with multiple exits,
there's no edge from the <br>
>>>> middle block<br>
>>>> -Â Â Â Â Â // to exit blocks (as the epilogue
must run) and thus no <br>
>>>> need to update<br>
>>>> -Â Â Â Â Â // the immediate dominator of the
exit blocks.<br>
>>>> -Â Â Â Â Â
DT->changeImmediateDominator(LoopExitBlock,
TCCheckBlock);<br>
>>>> +Â Â Â
DT->changeImmediateDominator(LoopExitBlock,
TCCheckBlock);<br>
>>>><br>
>>>> Â Â Â Â Â Â
LoopBypassBlocks.push_back(TCCheckBlock);<br>
>>>><br>
>>>> @@ -8142,12 +8102,7 @@ <br>
>>>>
EpilogueVectorizerEpilogueLoop::createEpilogueVectorizedLoopSkeleton()
<br>
>>>> {<br>
>>>><br>
>>>> Â Â Â Â
DT->changeImmediateDominator(LoopScalarPreHeader,<br>
>>>> EPI.EpilogueIterationCountCheck);<br>
>>>> -Â if (!Cost->requiresScalarEpilogue())<br>
>>>> -Â Â Â // If there is an epilogue which must
run, there's no edge <br>
>>>> from the<br>
>>>> -   // middle block to exit blocks and
thus no need to update the <br>
>>>> immediate<br>
>>>> -Â Â Â // dominator of the exit blocks.<br>
>>>> -Â Â Â
DT->changeImmediateDominator(LoopExitBlock,<br>
>>>> - EPI.EpilogueIterationCountCheck);<br>
>>>> +Â
DT->changeImmediateDominator(LoopExitBlock, <br>
>>>> EPI.EpilogueIterationCountCheck);<br>
>>>><br>
>>>> Â Â Â Â // Keep track of bypass blocks, as
they feed start values to <br>
>>>> the induction<br>
>>>> Â Â Â Â // phis in the scalar loop preheader.<br>
>>>><br>
>>>> diff --git <br>
>>>>
a/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll
<br>
>>>>
b/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
>>>> index ec280bf5d5e4..7d4a3c5c9935 100644<br>
>>>> --- <br>
>>>>
a/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
>>>> +++ <br>
>>>>
b/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
>>>> @@ -471,9 +471,10 @@ define i16
@multiple_exit(i16* %p, i32 %n) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP15:%.*]] = icmp eq
i32 [[INDEX_NEXT]], <br>
>>>> [[N_VEC]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1 [[TMP15]], label
[[MIDDLE_BLOCK:%.*]], <br>
>>>> label [[VECTOR_BODY]], [[LOOP6:!llvm.loop
!.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â middle.block:<br>
>>>> +; CHECK-NEXT:Â Â Â [[CMP_N:%.*]] = icmp eq
i32 [[TMP2]], [[N_VEC]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â
[[VECTOR_RECUR_EXTRACT:%.*]] = extractelement <br>
>>>> <4 x i16> [[WIDE_LOAD]], i32 3<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â
[[VECTOR_RECUR_EXTRACT_FOR_PHI:%.*]] = <br>
>>>> extractelement <4 x i16>
[[WIDE_LOAD]], i32 2<br>
>>>> -; CHECK-NEXT:Â Â Â br label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP_N]], label
[[IF_END:%.*]], label <br>
>>>> [[SCALAR_PH]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â <a
href="http://scalar.ph" rel="noreferrer" target="_blank"
moz-do-not-send="true">scalar.ph</a>:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â
[[SCALAR_RECUR_INIT:%.*]] = phi i16 [ 0, <br>
>>>> [[ENTRY:%.*]] ], [
[[VECTOR_RECUR_EXTRACT]], [[MIDDLE_BLOCK]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_RESUME_VAL:%.*]] =
phi i32 [ [[N_VEC]], <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY]] ]<br>
>>>> @@ -485,14 +486,14 @@ define i16
@multiple_exit(i16* %p, i32 %n) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[B:%.*]] =
getelementptr inbounds i16, i16* <br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[REC_NEXT]] = load
i16, i16* [[B]], align 2<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[CMP:%.*]] = icmp slt
i32 [[I]], [[N]]<br>
>>>> -; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label <br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label [[IF_END]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â for.body:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â store i16
[[SCALAR_RECUR]], i16* [[B]], align 4<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[INC]] = add nsw i32
[[I]], 1<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[CMP2:%.*]] = icmp slt
i32 [[I]], 2096<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1 [[CMP2]], label
[[FOR_COND]], label <br>
>>>> [[IF_END]], [[LOOP7:!llvm.loop !.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â if.end:<br>
>>>> -; CHECK-NEXT:Â Â Â [[REC_LCSSA:%.*]] = phi
i16 [ [[SCALAR_RECUR]], <br>
>>>> [[FOR_BODY]] ], [ [[SCALAR_RECUR]],
[[FOR_COND]] ]<br>
>>>> +; CHECK-NEXT:Â Â Â [[REC_LCSSA:%.*]] = phi
i16 [ [[SCALAR_RECUR]], <br>
>>>> [[FOR_BODY]] ], [ [[SCALAR_RECUR]],
[[FOR_COND]] ], [ <br>
>>>> [[VECTOR_RECUR_EXTRACT_FOR_PHI]],
[[MIDDLE_BLOCK]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â ret i16 [[REC_LCSSA]]<br>
>>>> Â Â ;<br>
>>>> Â Â entry:<br>
>>>> @@ -557,9 +558,10 @@ define i16
@multiple_exit2(i16* %p, i32 %n) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP15:%.*]] = icmp eq
i32 [[INDEX_NEXT]], <br>
>>>> [[N_VEC]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1 [[TMP15]], label
[[MIDDLE_BLOCK:%.*]], <br>
>>>> label [[VECTOR_BODY]], [[LOOP8:!llvm.loop
!.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â middle.block:<br>
>>>> +; CHECK-NEXT:Â Â Â [[CMP_N:%.*]] = icmp eq
i32 [[TMP2]], [[N_VEC]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â
[[VECTOR_RECUR_EXTRACT:%.*]] = extractelement <br>
>>>> <4 x i16> [[WIDE_LOAD]], i32 3<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â
[[VECTOR_RECUR_EXTRACT_FOR_PHI:%.*]] = <br>
>>>> extractelement <4 x i16>
[[WIDE_LOAD]], i32 2<br>
>>>> -; CHECK-NEXT:Â Â Â br label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP_N]], label
[[IF_END:%.*]], label <br>
>>>> [[SCALAR_PH]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â <a
href="http://scalar.ph" rel="noreferrer" target="_blank"
moz-do-not-send="true">scalar.ph</a>:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â
[[SCALAR_RECUR_INIT:%.*]] = phi i16 [ 0, <br>
>>>> [[ENTRY:%.*]] ], [
[[VECTOR_RECUR_EXTRACT]], [[MIDDLE_BLOCK]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_RESUME_VAL:%.*]] =
phi i32 [ [[N_VEC]], <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY]] ]<br>
>>>> @@ -571,14 +573,14 @@ define i16
@multiple_exit2(i16* %p, i32 %n) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[B:%.*]] =
getelementptr inbounds i16, i16* <br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[REC_NEXT]] = load
i16, i16* [[B]], align 2<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[CMP:%.*]] = icmp slt
i32 [[I]], [[N]]<br>
>>>> -; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label <br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label [[IF_END]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â for.body:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â store i16
[[SCALAR_RECUR]], i16* [[B]], align 4<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[INC]] = add nsw i32
[[I]], 1<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[CMP2:%.*]] = icmp slt
i32 [[I]], 2096<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1 [[CMP2]], label
[[FOR_COND]], label <br>
>>>> [[IF_END]], [[LOOP9:!llvm.loop !.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â if.end:<br>
>>>> -; CHECK-NEXT:Â Â Â [[REC_LCSSA:%.*]] = phi
i16 [ [[SCALAR_RECUR]], <br>
>>>> [[FOR_COND]] ], [ 10, [[FOR_BODY]] ]<br>
>>>> +; CHECK-NEXT:Â Â Â [[REC_LCSSA:%.*]] = phi
i16 [ [[SCALAR_RECUR]], <br>
>>>> [[FOR_COND]] ], [ 10, [[FOR_BODY]] ], [ <br>
>>>> [[VECTOR_RECUR_EXTRACT_FOR_PHI]],
[[MIDDLE_BLOCK]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â ret i16 [[REC_LCSSA]]<br>
>>>> Â Â ;<br>
>>>> Â Â entry:<br>
>>>><br>
>>>> diff --git <br>
>>>>
a/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll
<br>
>>>>
b/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
>>>> index f0ba677348ab..0d4bdf0ecac3 100644<br>
>>>> ---
a/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
>>>> +++
b/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
>>>> @@ -447,7 +447,7 @@ define void
@even_load_static_tc(i32* noalias <br>
>>>> nocapture readonly %A, i32* noalia<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP6:%.*]] = icmp eq
i64 [[INDEX_NEXT]], 508<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1 [[TMP6]], label
[[MIDDLE_BLOCK:%.*]], <br>
>>>> label [[VECTOR_BODY]], [[LOOP12:!llvm.loop
!.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â middle.block:<br>
>>>> -; CHECK-NEXT:Â Â Â br label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 false, label
[[FOR_COND_CLEANUP:%.*]], <br>
>>>> label [[SCALAR_PH]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â <a
href="http://scalar.ph" rel="noreferrer" target="_blank"
moz-do-not-send="true">scalar.ph</a>:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_RESUME_VAL:%.*]] =
phi i64 [ 1016, <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br label
[[FOR_BODY:%.*]]<br>
>>>> @@ -463,7 +463,7 @@ define void
@even_load_static_tc(i32* noalias <br>
>>>> nocapture readonly %A, i32* noalia<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â store i32 [[MUL]], i32*
[[ARRAYIDX2]], align 4<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[INDVARS_IV_NEXT]] =
add nuw nsw i64 <br>
>>>> [[INDVARS_IV]], 2<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[CMP:%.*]] = icmp ult
i64 [[INDVARS_IV]], 1022<br>
>>>> -; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label <br>
>>>> [[FOR_COND_CLEANUP:%.*]],
[[LOOP13:!llvm.loop !.*]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label <br>
>>>> [[FOR_COND_CLEANUP]], [[LOOP13:!llvm.loop
!.*]]<br>
>>>> Â Â ;<br>
>>>> Â Â entry:<br>
>>>> Â Â Â Â br label %for.body<br>
>>>> @@ -528,7 +528,7 @@ define void
@even_load_dynamic_tc(i32* noalias <br>
>>>> nocapture readonly %A, i32* noali<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP12:%.*]] = icmp eq
i64 [[INDEX_NEXT]], <br>
>>>> [[N_VEC]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1 [[TMP12]], label
[[MIDDLE_BLOCK:%.*]], <br>
>>>> label [[VECTOR_BODY]], [[LOOP14:!llvm.loop
!.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â middle.block:<br>
>>>> -; CHECK-NEXT:Â Â Â br label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 false, label
[[FOR_COND_CLEANUP:%.*]], <br>
>>>> label [[SCALAR_PH]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â <a
href="http://scalar.ph" rel="noreferrer" target="_blank"
moz-do-not-send="true">scalar.ph</a>:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_RESUME_VAL:%.*]] =
phi i64 [ [[IND_END]], <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br label
[[FOR_BODY:%.*]]<br>
>>>> @@ -544,7 +544,7 @@ define void
@even_load_dynamic_tc(i32* noalias <br>
>>>> nocapture readonly %A, i32* noali<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â store i32 [[MUL]], i32*
[[ARRAYIDX2]], align 4<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[INDVARS_IV_NEXT]] =
add nuw nsw i64 <br>
>>>> [[INDVARS_IV]], 2<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[CMP:%.*]] = icmp ult
i64 [[INDVARS_IV_NEXT]], <br>
>>>> [[N]]<br>
>>>> -; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label <br>
>>>> [[FOR_COND_CLEANUP:%.*]],
[[LOOP15:!llvm.loop !.*]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label <br>
>>>> [[FOR_COND_CLEANUP]], [[LOOP15:!llvm.loop
!.*]]<br>
>>>> Â Â ;<br>
>>>> Â Â entry:<br>
>>>> Â Â Â Â br label %for.body<br>
>>>> @@ -973,7 +973,7 @@ define void
@PR27626_0(%pair.i32 *%p, i32 %z, <br>
>>>> i64 %n) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP19:%.*]] = icmp eq
i64 [[INDEX_NEXT]], <br>
>>>> [[N_VEC]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1 [[TMP19]], label
[[MIDDLE_BLOCK:%.*]], <br>
>>>> label [[VECTOR_BODY]], [[LOOP24:!llvm.loop
!.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â middle.block:<br>
>>>> -; CHECK-NEXT:Â Â Â br label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 false, label
[[FOR_END:%.*]], label <br>
>>>> [[SCALAR_PH]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â <a
href="http://scalar.ph" rel="noreferrer" target="_blank"
moz-do-not-send="true">scalar.ph</a>:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_RESUME_VAL:%.*]] =
phi i64 [ [[N_VEC]], <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br label
[[FOR_BODY:%.*]]<br>
>>>> @@ -985,7 +985,7 @@ define void
@PR27626_0(%pair.i32 *%p, i32 %z, <br>
>>>> i64 %n) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â store i32 [[Z]], i32*
[[P_I_Y]], align 4<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[I_NEXT]] = add nuw
nsw i64 [[I]], 1<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[COND:%.*]] = icmp slt
i64 [[I_NEXT]], [[N]]<br>
>>>> -; CHECK-NEXT:Â Â Â br i1 [[COND]], label
[[FOR_BODY]], label <br>
>>>> [[FOR_END:%.*]], [[LOOP25:!llvm.loop !.*]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[COND]], label
[[FOR_BODY]], label <br>
>>>> [[FOR_END]], [[LOOP25:!llvm.loop !.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â for.end:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â ret void<br>
>>>> Â Â ;<br>
>>>> @@ -1066,7 +1066,7 @@ define i32
@PR27626_1(%pair.i32 *%p, i64 %n) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[RDX_SHUF3:%.*]] =
shufflevector <4 x i32> <br>
>>>> [[BIN_RDX]], <4 x i32> poison, <4
x i32> <i32 1, i32 undef, i32 <br>
>>>> undef, i32 undef><br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BIN_RDX4:%.*]] = add
<4 x i32> [[BIN_RDX]], <br>
>>>> [[RDX_SHUF3]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP19:%.*]] =
extractelement <4 x i32> <br>
>>>> [[BIN_RDX4]], i32 0<br>
>>>> -; CHECK-NEXT:Â Â Â br label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 false, label
[[FOR_END:%.*]], label <br>
>>>> [[SCALAR_PH]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â <a
href="http://scalar.ph" rel="noreferrer" target="_blank"
moz-do-not-send="true">scalar.ph</a>:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_RESUME_VAL:%.*]] =
phi i64 [ [[N_VEC]], <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_MERGE_RDX:%.*]] =
phi i32 [ [[TMP19]], <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY]] ]<br>
>>>> @@ -1081,9 +1081,10 @@ define i32
@PR27626_1(%pair.i32 *%p, i64 %n) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP21]] = add nsw i32
[[TMP20]], [[S]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[I_NEXT]] = add nuw
nsw i64 [[I]], 1<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[COND:%.*]] = icmp slt
i64 [[I_NEXT]], [[N]]<br>
>>>> -; CHECK-NEXT:Â Â Â br i1 [[COND]], label
[[FOR_BODY]], label <br>
>>>> [[FOR_END:%.*]], [[LOOP27:!llvm.loop !.*]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[COND]], label
[[FOR_BODY]], label <br>
>>>> [[FOR_END]], [[LOOP27:!llvm.loop !.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â for.end:<br>
>>>> -; CHECK-NEXT:Â Â Â ret i32 [[TMP21]]<br>
>>>> +; CHECK-NEXT:Â Â Â [[TMP22:%.*]] = phi i32 [
[[TMP21]], [[FOR_BODY]] <br>
>>>> ], [ [[TMP19]], [[MIDDLE_BLOCK]] ]<br>
>>>> +; CHECK-NEXT:Â Â Â ret i32 [[TMP22]]<br>
>>>> Â Â ;<br>
>>>> Â Â entry:<br>
>>>> Â Â Â Â br label %for.body<br>
>>>> @@ -1162,7 +1163,7 @@ define void
@PR27626_2(%pair.i32 *%p, i64 %n, <br>
>>>> i32 %z) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP20:%.*]] = icmp eq
i64 [[INDEX_NEXT]], <br>
>>>> [[N_VEC]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1 [[TMP20]], label
[[MIDDLE_BLOCK:%.*]], <br>
>>>> label [[VECTOR_BODY]], [[LOOP28:!llvm.loop
!.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â middle.block:<br>
>>>> -; CHECK-NEXT:Â Â Â br label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 false, label
[[FOR_END:%.*]], label <br>
>>>> [[SCALAR_PH]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â <a
href="http://scalar.ph" rel="noreferrer" target="_blank"
moz-do-not-send="true">scalar.ph</a>:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_RESUME_VAL:%.*]] =
phi i64 [ [[N_VEC]], <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br label
[[FOR_BODY:%.*]]<br>
>>>> @@ -1176,7 +1177,7 @@ define void
@PR27626_2(%pair.i32 *%p, i64 %n, <br>
>>>> i32 %z) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â store i32 [[TMP21]],
i32* [[P_I_Y]], align 4<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[I_NEXT]] = add nuw
nsw i64 [[I]], 1<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[COND:%.*]] = icmp slt
i64 [[I_NEXT]], [[N]]<br>
>>>> -; CHECK-NEXT:Â Â Â br i1 [[COND]], label
[[FOR_BODY]], label <br>
>>>> [[FOR_END:%.*]], [[LOOP29:!llvm.loop !.*]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[COND]], label
[[FOR_BODY]], label <br>
>>>> [[FOR_END]], [[LOOP29:!llvm.loop !.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â for.end:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â ret void<br>
>>>> Â Â ;<br>
>>>> @@ -1263,7 +1264,7 @@ define i32
@PR27626_3(%pair.i32 *%p, i64 %n, <br>
>>>> i32 %z) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[RDX_SHUF3:%.*]] =
shufflevector <4 x i32> <br>
>>>> [[BIN_RDX]], <4 x i32> poison, <4
x i32> <i32 1, i32 undef, i32 <br>
>>>> undef, i32 undef><br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BIN_RDX4:%.*]] = add
<4 x i32> [[BIN_RDX]], <br>
>>>> [[RDX_SHUF3]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP22:%.*]] =
extractelement <4 x i32> <br>
>>>> [[BIN_RDX4]], i32 0<br>
>>>> -; CHECK-NEXT:Â Â Â br label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 false, label
[[FOR_END:%.*]], label <br>
>>>> [[SCALAR_PH]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â <a
href="http://scalar.ph" rel="noreferrer" target="_blank"
moz-do-not-send="true">scalar.ph</a>:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_RESUME_VAL:%.*]] =
phi i64 [ [[N_VEC]], <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_MERGE_RDX:%.*]] =
phi i32 [ [[TMP22]], <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY]] ]<br>
>>>> @@ -1281,9 +1282,10 @@ define i32
@PR27626_3(%pair.i32 *%p, i64 %n, <br>
>>>> i32 %z) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP25]] = add nsw i32
[[TMP24]], [[S]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[I_NEXT]] = add nuw
nsw i64 [[I]], 1<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[COND:%.*]] = icmp slt
i64 [[I_NEXT]], [[N]]<br>
>>>> -; CHECK-NEXT:Â Â Â br i1 [[COND]], label
[[FOR_BODY]], label <br>
>>>> [[FOR_END:%.*]], [[LOOP31:!llvm.loop !.*]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[COND]], label
[[FOR_BODY]], label <br>
>>>> [[FOR_END]], [[LOOP31:!llvm.loop !.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â for.end:<br>
>>>> -; CHECK-NEXT:Â Â Â ret i32 [[TMP25]]<br>
>>>> +; CHECK-NEXT:Â Â Â [[TMP26:%.*]] = phi i32 [
[[TMP25]], [[FOR_BODY]] <br>
>>>> ], [ [[TMP22]], [[MIDDLE_BLOCK]] ]<br>
>>>> +; CHECK-NEXT:Â Â Â ret i32 [[TMP26]]<br>
>>>> Â Â ;<br>
>>>> Â Â entry:<br>
>>>> Â Â Â Â br label %for.body<br>
>>>><br>
>>>> diff --git
a/llvm/test/Transforms/LoopVectorize/loop-form.ll <br>
>>>>
b/llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
>>>> index f32002fae2b6..91780789088b 100644<br>
>>>> ---
a/llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
>>>> +++
b/llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
>>>> @@ -146,14 +146,15 @@ define void
@early_exit(i16* %p, i32 %n) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP10:%.*]] = icmp eq
i32 [[INDEX_NEXT]], <br>
>>>> [[N_VEC]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1 [[TMP10]], label
[[MIDDLE_BLOCK:%.*]], <br>
>>>> label [[VECTOR_BODY]], [[LOOP4:!llvm.loop
!.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â middle.block:<br>
>>>> -; CHECK-NEXT:Â Â Â br label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:Â Â Â [[CMP_N:%.*]] = icmp eq
i32 [[TMP1]], [[N_VEC]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP_N]], label
[[IF_END:%.*]], label <br>
>>>> [[SCALAR_PH]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â <a
href="http://scalar.ph" rel="noreferrer" target="_blank"
moz-do-not-send="true">scalar.ph</a>:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_RESUME_VAL:%.*]] =
phi i32 [ [[N_VEC]], <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br label
[[FOR_COND:%.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â for.cond:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[I:%.*]] = phi i32 [
[[BC_RESUME_VAL]], <br>
>>>> [[SCALAR_PH]] ], [ [[INC:%.*]],
[[FOR_BODY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[CMP:%.*]] = icmp slt
i32 [[I]], [[N]]<br>
>>>> -; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label <br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label [[IF_END]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â for.body:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[IPROM:%.*]] = sext
i32 [[I]] to i64<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[B:%.*]] =
getelementptr inbounds i16, i16* <br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> @@ -285,14 +286,15 @@ define void
@multiple_unique_exit(i16* %p, <br>
>>>> i32 %n) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP11:%.*]] = icmp eq
i32 [[INDEX_NEXT]], <br>
>>>> [[N_VEC]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1 [[TMP11]], label
[[MIDDLE_BLOCK:%.*]], <br>
>>>> label [[VECTOR_BODY]], [[LOOP6:!llvm.loop
!.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â middle.block:<br>
>>>> -; CHECK-NEXT:Â Â Â br label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:Â Â Â [[CMP_N:%.*]] = icmp eq
i32 [[TMP2]], [[N_VEC]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP_N]], label
[[IF_END:%.*]], label <br>
>>>> [[SCALAR_PH]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â <a
href="http://scalar.ph" rel="noreferrer" target="_blank"
moz-do-not-send="true">scalar.ph</a>:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_RESUME_VAL:%.*]] =
phi i32 [ [[N_VEC]], <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br label
[[FOR_COND:%.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â for.cond:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[I:%.*]] = phi i32 [
[[BC_RESUME_VAL]], <br>
>>>> [[SCALAR_PH]] ], [ [[INC:%.*]],
[[FOR_BODY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[CMP:%.*]] = icmp slt
i32 [[I]], [[N]]<br>
>>>> -; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label <br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label [[IF_END]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â for.body:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[IPROM:%.*]] = sext
i32 [[I]] to i64<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[B:%.*]] =
getelementptr inbounds i16, i16* <br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> @@ -372,14 +374,17 @@ define i32
@multiple_unique_exit2(i16* %p, <br>
>>>> i32 %n) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP11:%.*]] = icmp eq
i32 [[INDEX_NEXT]], <br>
>>>> [[N_VEC]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1 [[TMP11]], label
[[MIDDLE_BLOCK:%.*]], <br>
>>>> label [[VECTOR_BODY]], [[LOOP8:!llvm.loop
!.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â middle.block:<br>
>>>> -; CHECK-NEXT:Â Â Â br label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:Â Â Â [[CMP_N:%.*]] = icmp eq
i32 [[TMP2]], [[N_VEC]]<br>
>>>> +; CHECK-NEXT:Â Â Â [[IND_ESCAPE:%.*]] = sub
i32 [[N_VEC]], 1<br>
>>>> +; CHECK-NEXT:Â Â Â [[IND_ESCAPE1:%.*]] = sub
i32 [[N_VEC]], 1<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP_N]], label
[[IF_END:%.*]], label <br>
>>>> [[SCALAR_PH]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â <a
href="http://scalar.ph" rel="noreferrer" target="_blank"
moz-do-not-send="true">scalar.ph</a>:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_RESUME_VAL:%.*]] =
phi i32 [ [[N_VEC]], <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br label
[[FOR_COND:%.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â for.cond:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[I:%.*]] = phi i32 [
[[BC_RESUME_VAL]], <br>
>>>> [[SCALAR_PH]] ], [ [[INC:%.*]],
[[FOR_BODY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[CMP:%.*]] = icmp slt
i32 [[I]], [[N]]<br>
>>>> -; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label <br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label [[IF_END]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â for.body:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[IPROM:%.*]] = sext
i32 [[I]] to i64<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[B:%.*]] =
getelementptr inbounds i16, i16* <br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> @@ -388,7 +393,7 @@ define i32
@multiple_unique_exit2(i16* %p, i32 <br>
>>>> %n) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[CMP2:%.*]] = icmp slt
i32 [[I]], 2096<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1 [[CMP2]], label
[[FOR_COND]], label <br>
>>>> [[IF_END]], [[LOOP9:!llvm.loop !.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â if.end:<br>
>>>> -; CHECK-NEXT:Â Â Â [[I_LCSSA:%.*]] = phi i32
[ [[I]], [[FOR_BODY]] <br>
>>>> ], [ [[I]], [[FOR_COND]] ]<br>
>>>> +; CHECK-NEXT:Â Â Â [[I_LCSSA:%.*]] = phi i32
[ [[I]], [[FOR_BODY]] <br>
>>>> ], [ [[I]], [[FOR_COND]] ], [
[[IND_ESCAPE1]], [[MIDDLE_BLOCK]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â ret i32 [[I_LCSSA]]<br>
>>>> Â Â ;<br>
>>>> Â Â ; TAILFOLD-LABEL:
@multiple_unique_exit2(<br>
>>>> @@ -461,14 +466,15 @@ define i32
@multiple_unique_exit3(i16* %p, <br>
>>>> i32 %n) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP11:%.*]] = icmp eq
i32 [[INDEX_NEXT]], <br>
>>>> [[N_VEC]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1 [[TMP11]], label
[[MIDDLE_BLOCK:%.*]], <br>
>>>> label [[VECTOR_BODY]], [[LOOP10:!llvm.loop
!.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â middle.block:<br>
>>>> -; CHECK-NEXT:Â Â Â br label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:Â Â Â [[CMP_N:%.*]] = icmp eq
i32 [[TMP2]], [[N_VEC]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP_N]], label
[[IF_END:%.*]], label <br>
>>>> [[SCALAR_PH]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â <a
href="http://scalar.ph" rel="noreferrer" target="_blank"
moz-do-not-send="true">scalar.ph</a>:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_RESUME_VAL:%.*]] =
phi i32 [ [[N_VEC]], <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br label
[[FOR_COND:%.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â for.cond:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[I:%.*]] = phi i32 [
[[BC_RESUME_VAL]], <br>
>>>> [[SCALAR_PH]] ], [ [[INC:%.*]],
[[FOR_BODY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[CMP:%.*]] = icmp slt
i32 [[I]], [[N]]<br>
>>>> -; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label <br>
>>>> [[IF_END:%.*]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP]], label
[[FOR_BODY]], label [[IF_END]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â for.body:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[IPROM:%.*]] = sext
i32 [[I]] to i64<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[B:%.*]] =
getelementptr inbounds i16, i16* <br>
>>>> [[P]], i64 [[IPROM]]<br>
>>>> @@ -477,7 +483,7 @@ define i32
@multiple_unique_exit3(i16* %p, i32 <br>
>>>> %n) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[CMP2:%.*]] = icmp slt
i32 [[I]], 2096<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1 [[CMP2]], label
[[FOR_COND]], label <br>
>>>> [[IF_END]], [[LOOP11:!llvm.loop !.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â if.end:<br>
>>>> -; CHECK-NEXT:Â Â Â [[EXIT:%.*]] = phi i32 [
0, [[FOR_COND]] ], [ 1, <br>
>>>> [[FOR_BODY]] ]<br>
>>>> +; CHECK-NEXT:Â Â Â [[EXIT:%.*]] = phi i32 [
0, [[FOR_COND]] ], [ 1, <br>
>>>> [[FOR_BODY]] ], [ 0, [[MIDDLE_BLOCK]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â ret i32 [[EXIT]]<br>
>>>> Â Â ;<br>
>>>> Â Â ; TAILFOLD-LABEL:
@multiple_unique_exit3(<br>
>>>> @@ -994,7 +1000,8 @@ define void
@scalar_predication(float* %addr) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP10:%.*]] = icmp eq
i64 [[INDEX_NEXT]], 200<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1 [[TMP10]], label
[[MIDDLE_BLOCK:%.*]], <br>
>>>> label [[VECTOR_BODY]], [[LOOP12:!llvm.loop
!.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â middle.block:<br>
>>>> -; CHECK-NEXT:Â Â Â br label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:Â Â Â [[CMP_N:%.*]] = icmp eq
i64 201, 200<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP_N]], label
[[EXIT:%.*]], label <br>
>>>> [[SCALAR_PH]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â <a
href="http://scalar.ph" rel="noreferrer" target="_blank"
moz-do-not-send="true">scalar.ph</a>:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_RESUME_VAL:%.*]] =
phi i64 [ 200, <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br label
[[LOOP_HEADER:%.*]]<br>
>>>> @@ -1002,7 +1009,7 @@ define void
@scalar_predication(float* %addr) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[IV:%.*]] = phi i64 [
[[BC_RESUME_VAL]], <br>
>>>> [[SCALAR_PH]] ], [ [[IV_NEXT:%.*]],
[[LOOP_LATCH:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[GEP:%.*]] =
getelementptr float, float* <br>
>>>> [[ADDR]], i64 [[IV]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[EXITCOND_NOT:%.*]] =
icmp eq i64 [[IV]], 200<br>
>>>> -; CHECK-NEXT:Â Â Â br i1 [[EXITCOND_NOT]],
label [[EXIT:%.*]], label <br>
>>>> [[LOOP_BODY:%.*]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[EXITCOND_NOT]],
label [[EXIT]], label <br>
>>>> [[LOOP_BODY:%.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â loop.body:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP11:%.*]] = load
float, float* [[GEP]], <br>
>>>> align 4<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[PRED:%.*]] = fcmp oeq
float [[TMP11]], <br>
>>>> 0.000000e+00<br>
>>>> @@ -1088,7 +1095,8 @@ define i32
@me_reduction(i32* %addr) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[RDX_SHUF:%.*]] =
shufflevector <2 x i32> <br>
>>>> [[TMP5]], <2 x i32> poison, <2 x
i32> <i32 1, i32 undef><br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BIN_RDX:%.*]] = add
<2 x i32> [[TMP5]], <br>
>>>> [[RDX_SHUF]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP7:%.*]] =
extractelement <2 x i32> <br>
>>>> [[BIN_RDX]], i32 0<br>
>>>> -; CHECK-NEXT:Â Â Â br label [[SCALAR_PH]]<br>
>>>> +; CHECK-NEXT:Â Â Â [[CMP_N:%.*]] = icmp eq
i64 201, 200<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[CMP_N]], label
[[EXIT:%.*]], label <br>
>>>> [[SCALAR_PH]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â <a
href="http://scalar.ph" rel="noreferrer" target="_blank"
moz-do-not-send="true">scalar.ph</a>:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_RESUME_VAL:%.*]] =
phi i64 [ 200, <br>
>>>> [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[BC_MERGE_RDX:%.*]] =
phi i32 [ 0, [[ENTRY]] <br>
>>>> ], [ [[TMP7]], [[MIDDLE_BLOCK]] ]<br>
>>>> @@ -1098,7 +1106,7 @@ define i32
@me_reduction(i32* %addr) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[ACCUM:%.*]] = phi i32
[ [[BC_MERGE_RDX]], <br>
>>>> [[SCALAR_PH]] ], [ [[ACCUM_NEXT:%.*]],
[[LOOP_LATCH]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[GEP:%.*]] =
getelementptr i32, i32* [[ADDR]], <br>
>>>> i64 [[IV]]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[EXITCOND_NOT:%.*]] =
icmp eq i64 [[IV]], 200<br>
>>>> -; CHECK-NEXT:Â Â Â br i1 [[EXITCOND_NOT]],
label [[EXIT:%.*]], label <br>
>>>> [[LOOP_LATCH]]<br>
>>>> +; CHECK-NEXT:Â Â Â br i1 [[EXITCOND_NOT]],
label [[EXIT]], label <br>
>>>> [[LOOP_LATCH]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â loop.latch:<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[TMP8:%.*]] = load
i32, i32* [[GEP]], align 4<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[ACCUM_NEXT]] = add
i32 [[ACCUM]], [[TMP8]]<br>
>>>> @@ -1106,7 +1114,7 @@ define i32
@me_reduction(i32* %addr) {<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â [[EXITCOND2_NOT:%.*]] =
icmp eq i64 [[IV]], 400<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â br i1
[[EXITCOND2_NOT]], label [[EXIT]], label <br>
>>>> [[LOOP_HEADER]], [[LOOP15:!llvm.loop !.*]]<br>
>>>> Â Â ; CHECK:Â Â Â Â Â Â exit:<br>
>>>> -; CHECK-NEXT:Â Â Â [[LCSSA:%.*]] = phi i32 [
0, [[LOOP_HEADER]] ], [ <br>
>>>> [[ACCUM_NEXT]], [[LOOP_LATCH]] ]<br>
>>>> +; CHECK-NEXT:Â Â Â [[LCSSA:%.*]] = phi i32 [
0, [[LOOP_HEADER]] ], [ <br>
>>>> [[ACCUM_NEXT]], [[LOOP_LATCH]] ], [
[[TMP7]], [[MIDDLE_BLOCK]] ]<br>
>>>> Â Â ; CHECK-NEXT:Â Â Â ret i32 [[LCSSA]]<br>
>>>> Â Â ;<br>
>>>> Â Â ; TAILFOLD-LABEL: @me_reduction(<br>
>>>><br>
>>>><br>
>>>><br>
>>>>
_______________________________________________<br>
>>>> llvm-commits mailing list<br>
>>>> <a
href="mailto:llvm-commits@lists.llvm.org" target="_blank"
moz-do-not-send="true">llvm-commits@lists.llvm.org</a><br>
>>>> <a
href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
>>> _______________________________________________<br>
>>> llvm-commits mailing list<br>
>>> <a href="mailto:llvm-commits@lists.llvm.org"
target="_blank" moz-do-not-send="true">llvm-commits@lists.llvm.org</a><br>
>>> <a
href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
> _______________________________________________<br>
> llvm-commits mailing list<br>
> <a href="mailto:llvm-commits@lists.llvm.org"
target="_blank" moz-do-not-send="true">llvm-commits@lists.llvm.org</a><br>
> <a
href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a></blockquote>
</div>
</div>
</blockquote>
</body>
</html>