<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p>The fix mentioned below has landed.  In a couple of days, I plan
      to restore the original patch and see what if anything else falls
      out.  I'll probably wait for the weekend so that it's a slow day
      to minimize bot fallout if any.  <br>
    </p>
    <p>@Stefan - I remember seeing a set of test results from a combined
      patch (fix plus original) on the PPC builders, but can't for the
      life of me find that email now.  Do you remember where that was? 
      Or what the results were?  I don't want to resubmit if we already
      know about additional breakage.</p>
    <p>Philip<br>
    </p>
    <div class="moz-cite-prefix">On 6/4/21 8:53 AM, Philip Reames via
      llvm-commits wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:ec4fc021-c919-bef7-55dc-fc26fdfd0b74@philipreames.com">
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <p>I have a fix for what I think is the root issue here out for
        review: <a class="moz-txt-link-freetext"
          href="https://reviews.llvm.org/D103700" moz-do-not-send="true">https://reviews.llvm.org/D103700</a></p>
      <p>As I suspected, the reverted patch was exposing an existing
        issue, not introducing a new one.  However, the bug does still
        appear to have been mine, it was just introduced back in
        4b33b2387787a, and hasn't otherwise come up.</p>
      <p>Thanks again for the help here.  Hugely appreciated.<br>
      </p>
      <p>Philip<br>
      </p>
      <div class="moz-cite-prefix">On 5/27/21 11:29 AM, Stefan Pintilie
        wrote:<br>
      </div>
      <blockquote type="cite"
cite="mid:OF19D50B3B.B1905522-ON002586E2.00654CFA-002586E2.00659820@notes.na.collabserv.com">
        <meta http-equiv="content-type" content="text/html;
          charset=UTF-8">
        <div class="socmaildefaultfont" dir="ltr"
          style="font-family:Arial, Helvetica,
          sans-serif;font-size:10pt">
          <div dir="ltr"> </div>
          <div dir="ltr">Hi Philip,</div>
          <div dir="ltr"> </div>
          <div dir="ltr">We have managed to track down more information
            on the issue causing the failure with the LV patch.<br>
            It seems that Cost->requiresScalarEpilogue() sometimes
            returns true when the scalar epilogue is actually not
            required.<br>
            One of the failing tests is:<br>
            test-suite/SingleSource/Benchmarks/Misc/oourafft.c<br>
            That test has a loop that runs a fixed 1024 times. If the
            max vector interleave is a factor of 1024 then
            Cost->requiresScalarEpilogue() still returns true even
            though the scalar epilogue is not actually required.<br>
            The following examples were run on a Little Endian machine:<br>
            FAIL - clang -DNDEBUG -fuse-ld=ld  -O3 -DNDEBUG  -w
            -Werror=date-time -mllvm -force-vector-interleave=4
            ${TESTSUITE}/SingleSource/Benchmarks/Misc/oourafft.c -lm<br>
            FAIL - clang -DNDEBUG -fuse-ld=ld  -O3 -DNDEBUG  -w
            -Werror=date-time -mllvm -force-vector-interleave=8
            ${TESTSUITE}/SingleSource/Benchmarks/Misc/oourafft.c -lm<br>
            PASS - clang -DNDEBUG -fuse-ld=ld  -O3 -DNDEBUG  -w
            -Werror=date-time -mllvm -force-vector-interleave=12
            ${TESTSUITE}/SingleSource/Benchmarks/Misc/oourafft.c -lm<br>
            Looking at the IR immediately following LV we see this.</div>
          <div dir="ltr"> </div>
          <div dir="ltr">Working IR:<br>
            middle.block:<br>
              %cmp.n = icmp eq i64 1024, 1024<br>
              br i1 %cmp.n, label %for.end, label %scalar.ph </div>
          <div dir="ltr"> </div>
          <div dir="ltr">Failing IR:<br>
            middle.block:<br>
              br label %scalar.ph</div>
          <div dir="ltr"><br>
            We traced the problem to
            InterleavedAccessInfo::analyzeInterleaving where we set
            RequiresScalarEpilogue to true.<br>
            We probably shouldn't be setting that to TRUE for this test
            case. Perhaps it matters that the loop wasn't actually
            vectorized in this case (VF == 1). That may impact the
            computation of RequiresScalarEpilogue.</div>
          <div dir="ltr"> </div>
          <div dir="ltr">Hope this helps,<br>
            Stefan</div>
          <div dir="ltr"> </div>
          <blockquote data-history-content-modified="1" dir="ltr"
            style="border-left:solid #aaaaaa 2px; margin-left:5px;
            padding-left:5px; direction:ltr; margin-right:0px">-----
            Original message -----<br>
            From: Stefan Pintilie via llvm-commits <a
              class="moz-txt-link-rfc2396E"
              href="mailto:llvm-commits@lists.llvm.org"
              moz-do-not-send="true"><llvm-commits@lists.llvm.org></a><br>
            Sent by: "llvm-commits" <a class="moz-txt-link-rfc2396E"
              href="mailto:llvm-commits-bounces@lists.llvm.org"
              moz-do-not-send="true"><llvm-commits-bounces@lists.llvm.org></a><br>
            To: <a class="moz-txt-link-abbreviated"
              href="mailto:listmail@philipreames.com"
              moz-do-not-send="true">listmail@philipreames.com</a><br>
            Cc: <a class="moz-txt-link-abbreviated"
              href="mailto:llvmlistbot@llvm.org" moz-do-not-send="true">llvmlistbot@llvm.org</a>,
            <a class="moz-txt-link-abbreviated"
              href="mailto:llvm-commits@lists.llvm.org"
              moz-do-not-send="true">llvm-commits@lists.llvm.org</a>, <a
              class="moz-txt-link-abbreviated"
              href="mailto:benny.kra@gmail.com" moz-do-not-send="true">benny.kra@gmail.com</a>,
            Nemanja Ivanovic <a class="moz-txt-link-rfc2396E"
              href="mailto:nemanjai@ca.ibm.com" moz-do-not-send="true"><nemanjai@ca.ibm.com></a>,
            <a class="moz-txt-link-abbreviated"
              href="mailto:akuegel@google.com" moz-do-not-send="true">akuegel@google.com</a>,
            LLVM on Power <a class="moz-txt-link-rfc2396E"
              href="mailto:powerllvm@ca.ibm.com" moz-do-not-send="true"><powerllvm@ca.ibm.com></a><br>
            Subject: [EXTERNAL] RE: [llvm] 7fe41ac - Revert "[LV]
            Unconditionally branch from middle to scalar preheader if
            the scalar loop must execute"<br>
            Date: Tue, May 25, 2021 7:53 PM<br>
             
            <div dir="ltr" style="font-family:Arial, Helvetica,
              sans-serif;font-size:10pt">
              <div dir="ltr"> </div>
              <div dir="ltr">Hi Philip,<br>
                <br>
                I have run a -print-after-all and I'm going to send you
                the IR before and after the Loop Vectorize pass.
                Hopefully this will help.<br>
                Let me know if you need more information or if you want
                me to run something on my end.<br>
                 </div>
              <div dir="ltr">Best,</div>
              <div dir="ltr">Stefan</div>
              <div dir="ltr"> </div>
              <blockquote data-history-content-modified="1" dir="ltr"
                style="border-left:solid #aaaaaa 2px; margin-left:5px;
                padding-left:5px; direction:ltr; margin-right:0px">-----
                Original message -----<br>
                From: Philip Reames <a class="moz-txt-link-rfc2396E"
                  href="mailto:listmail@philipreames.com"
                  moz-do-not-send="true"><listmail@philipreames.com></a><br>
                To: Nemanja Ivanovic <a class="moz-txt-link-rfc2396E"
                  href="mailto:nemanjai@ca.ibm.com"
                  moz-do-not-send="true"><nemanjai@ca.ibm.com></a><br>
                Cc: <a class="moz-txt-link-abbreviated"
                  href="mailto:akuegel@google.com"
                  moz-do-not-send="true">akuegel@google.com</a>, <a
                  class="moz-txt-link-abbreviated"
                  href="mailto:benny.kra@gmail.com"
                  moz-do-not-send="true">benny.kra@gmail.com</a>, <a
                  class="moz-txt-link-abbreviated"
                  href="mailto:llvm-commits@lists.llvm.org"
                  moz-do-not-send="true">llvm-commits@lists.llvm.org</a>,
                <a class="moz-txt-link-abbreviated"
                  href="mailto:llvmlistbot@llvm.org"
                  moz-do-not-send="true">llvmlistbot@llvm.org</a>, LLVM
                on Power <a class="moz-txt-link-rfc2396E"
                  href="mailto:powerllvm@ca.ibm.com"
                  moz-do-not-send="true"><powerllvm@ca.ibm.com></a>,
                Stefan Pintilie <a class="moz-txt-link-rfc2396E"
                  href="mailto:stefanp@ca.ibm.com"
                  moz-do-not-send="true"><stefanp@ca.ibm.com></a><br>
                Subject: [EXTERNAL] Re: [llvm] 7fe41ac - Revert "[LV]
                Unconditionally branch from middle to scalar preheader
                if the scalar loop must execute"<br>
                Date: Tue, May 25, 2021 5:38 PM<br>
                 <br>
                <!-- BaNnErBlUrFlE-HeAdEr-start -->
                <!-- BaNnErBlUrFlE-HeAdEr-end -->
                <!--Notes ACF
<meta http-equiv="Content-Type" content="text/html; charset=utf8" >-->
                <!-- BaNnErBlUrFlE-BoDy-start -->
                <!-- Preheader Text : BEGIN --> <span
                  style="display:none
!important;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;">
                  If you have the before and after IR, that's really all
                  I probably need. If you share those, I can take the
                  investigation from there. Philip On 5/25/21 2:12 PM,
                  Nemanja Ivanovic wrote: ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍
                  ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍
                  ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ </span>
                <!-- Preheader Text : END -->
                <!-- Email Banner : BEGIN --> <span style="display:none
!important;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerStart</span>
                <!--[if ((ie)|(mso))]>
  <table border="0" cellspacing="0" cellpadding="0" width="100%" style="padding: 3px 0px 16px 0px;" ><tr><td>
    <table class="pfptBannerTableMSO" border="0" cellspacing="0" cellpadding="0" style="padding: 0px 10px 5px 6px; width: 100%;border-radius:4px;border-top:4px solid #90a4ae;background-color:#d0d8dc;" ><tr><td valign="top" >
      <table align="left" border="0" cellspacing="0" cellpadding="0" style="padding: 4px 8px 4px 8px" >
        <tr><td><span class="pfptTitleMSO" style="color:#000000 !important;font-family: 'Arial', sans-serif;font-weight:bold !important;font-size:14px !important;" >
          This Message Is From an External Sender
        </span></td></tr>
        <tr><td><span class="pfptSubtitleMSO" style="color:#000000 !important;font-weight:normal !important;font-family: 'Arial', sans-serif; font-size:12px !important;" >
          This message came from outside your organization.
        </span></td></tr>

      </table>

    </td></tr></table>
  </td></tr></table>
<![endif]-->
                <!--[if !((ie)|(mso))]-->
                <div style="margin:16px 0px 16px 0px; padding:8px 16px
                  8px 16px; border-radius: 4px; min-width:
                  200px;background-color: #d0d8dc; border-top: 4px solid
                  #90a4ae;">
                  <div style="display: inline-block; margin: 0px 0px 1px
                    0px; max-width: 600px;">
                    <div style="color:#000000 !important;font-family:
                      'Arial', sans-serif !important;font-weight:bold
                      !important;font-size:14px
                      !important;line-height:18px;display:block;">This
                      Message Is From an External Sender</div>
                    <div style="color:#000000
                      !important;font-weight:normal
                      !important;font-family: 'Arial', sans-serif
                      !important;font-size:12px
                      !important;line-height:18px;margin-top:2px;display:block">This
                      message came from outside your organization.</div>
                  </div>
                </div>
                <!--[endif]--> <span style="display:none
!important;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerEnd</span><br>
                <!-- Email Banner : END -->
                <!-- BaNnErBlUrFlE-BoDy-end -->
                <p>If you have the before and after IR, that's really
                  all I probably need.  If you share those, I can take
                  the investigation from there.</p>
                <p>Philip</p>
                <div>On 5/25/21 2:12 PM, Nemanja Ivanovic wrote:</div>
                <blockquote
cite="mid:OFC99B9E23.4F6BE4A3-ON002586E0.00745021-002586E0.00748300@notes.na.collabserv.com"
                  type="cite"><!--Notes ACF
<meta http-equiv="content-type" content="text/html; charset=utf8" >-->
                  <div dir="ltr" style="font-family:Arial, Helvetica,
                    sans-serif;font-size:10pt">
                    <div dir="ltr">Hi Philip,</div>
                    <div dir="ltr">I am sorry about the late reply -
                      this was a 4 day weekend here in Toronto. We kind
                      of started with looking at the difference in the
                      post-LV IR with and without your patch for the
                      SingleSource test case. Stefan is currently
                      looking at it and we're hoping to figure out
                      what's going on soon. Would it help you if we
                      shared the two IR files with you to get your
                      opinion on what might be happening as well?</div>
                    <div dir="ltr"> </div>
                    <div dir="ltr">Nemanja Ivanovic<br>
                      LLVM PPC Backend Development<br>
                      IBM Toronto Lab<br>
                      Email: <a href="mailto:nemanjai@ca.ibm.com"
                        target="_blank" moz-do-not-send="true">nemanjai@ca.ibm.com</a><br>
                      Phone: 905-413-3388</div>
                    <div dir="ltr"> </div>
                    <div dir="ltr"> </div>
                    <blockquote data-history-content-modified="1"
                      dir="ltr" style="border-left:solid #aaaaaa 2px;
                      margin-left:5px; padding-left:5px; direction:ltr;
                      margin-right:0px">----- Original message -----<br>
                      From: Philip Reames <a
                        href="mailto:listmail@philipreames.com"
                        target="_blank" moz-do-not-send="true"><listmail@philipreames.com></a><br>
                      To: Nemanja Ivanovic <a
                        href="mailto:nemanjai@ca.ibm.com"
                        target="_blank" moz-do-not-send="true"><nemanjai@ca.ibm.com></a><br>
                      Cc: <a href="mailto:akuegel@google.com"
                        target="_blank" moz-do-not-send="true">akuegel@google.com</a>,
                      <a href="mailto:benny.kra@gmail.com"
                        target="_blank" moz-do-not-send="true">benny.kra@gmail.com</a>,
                      <a href="mailto:llvm-commits@lists.llvm.org"
                        target="_blank" moz-do-not-send="true">llvm-commits@lists.llvm.org</a>,
                      <a href="mailto:llvmlistbot@llvm.org"
                        target="_blank" moz-do-not-send="true">llvmlistbot@llvm.org</a>,
                      LLVM on Power <a
                        href="mailto:powerllvm@ca.ibm.com"
                        target="_blank" moz-do-not-send="true"><powerllvm@ca.ibm.com></a><br>
                      Subject: [EXTERNAL] Re: [llvm] 7fe41ac - Revert
                      "[LV] Unconditionally branch from middle to scalar
                      preheader if the scalar loop must execute"<br>
                      Date: Wed, May 19, 2021 2:15 PM<br>
                       <br>
                      <!--Notes ACF
<meta http-equiv="Content-Type" content="text/html; charset=utf8" >-->
                      <p>Looking at the various failures, I see both LE
                        and BE bots failing.  The BE bot shows a
                        miscompile in one of the test suite benchmarks. 
                        The LE bots appear to be showing a crash while
                        using a stage1 build clang to build stage2
                        clang. </p>
                      <p>LE example: <a
                          href="https://lab.llvm.org/buildbot#builders/19/builds/4236"
                          moz-do-not-send="true" target="_blank">https://lab.llvm.org/buildbot#builders/19/builds/4236</a><br>
                        BE example: <a
                          href="https://lab.llvm.org/buildbot#builders/100/builds/5762"
                          moz-do-not-send="true" target="_blank">https://lab.llvm.org/buildbot#builders/100/builds/5762</a></p>
                      <p>So both are showing miscompiles, just with
                        different symptoms.  Frankly, the BE looks
                        easier to debug (much code making it into the
                        miscompiled binary.)</p>
                      <p>Oddly, pretty much only PPC bots are failing. 
                        The one exception is a stage2 failure on AArch64
                        (<a
                          href="https://lab.llvm.org/buildbot/#/builders/111/builds/2027"
                          moz-do-not-send="true" target="_blank">https://lab.llvm.org/buildbot/#/builders/111/builds/2027</a>)
                        which looks similar to the LE failure above.  </p>
                      <p>Given this appears to be target specific, I am
                        *guessing* there's some vectorizer hook which is
                        causing a different codepath to be executed. 
                        Before we start trying to get me access to
                        hardware, do you have any guesses on what that
                        hook might be?  If you can give me a good hint
                        on where to look, I suspect I can probably find
                        the issue that way. </p>
                      <p>Philip</p>
                      <div>On 5/18/21 3:07 AM, Nemanja Ivanovic wrote:</div>
                      <blockquote
cite="mid:OF443062AC.F5998286-ON002586D9.00373B50-002586D9.0037A584@notes.na.collabserv.com"
                        type="cite"><!--Notes ACF
<meta http-equiv="content-type" content="text/html; charset=utf8" >-->
                        <div dir="ltr" style="font-family:Arial,
                          Helvetica, sans-serif;font-size:10pt">
                          <div dir="ltr">Hi Philip,</div>
                          <div dir="ltr">I am not sure what happened
                            with your first attempt to contact us and
                            how we missed it. We would be more than
                            happy to help you debug this issue. Do you
                            know if this only affects the big endian bot
                            or if it also fails on little endian bots?
                            In the latter case, we can certainly provide
                            access to a little endian machine hosted at
                            OSU/OSL. In the former case, we don't have a
                            machine available and we'll have to do the
                            debugging and report to you (which might
                            take a bit longer).</div>
                          <div dir="ltr"> </div>
                          <div dir="ltr">Nemanja Ivanovic<br>
                            LLVM PPC Backend Development<br>
                            IBM Toronto Lab<br>
                            Email: <a href="mailto:nemanjai@ca.ibm.com"
                              moz-do-not-send="true" target="_blank">nemanjai@ca.ibm.com</a><br>
                            Phone: 905-413-3388</div>
                          <div dir="ltr"> </div>
                          <div dir="ltr"> </div>
                          <blockquote data-history-content-modified="1"
                            data-history-expanded="1" dir="ltr"
                            style="border-left:solid #aaaaaa 2px;
                            margin-left:5px; padding-left:5px;
                            direction:ltr; margin-right:0px">-----
                            Original message -----<br>
                            From: Philip Reames <a
                              href="mailto:listmail@philipreames.com"
                              moz-do-not-send="true" target="_blank"><listmail@philipreames.com></a><br>
                            To: Adrian Kuegel <a
                              href="mailto:akuegel@google.com"
                              moz-do-not-send="true" target="_blank"><akuegel@google.com></a><br>
                            Cc: Benjamin Kramer <a
                              href="mailto:benny.kra@gmail.com"
                              moz-do-not-send="true" target="_blank"><benny.kra@gmail.com></a>,
                            Adrian Kuegel <a
                              href="mailto:llvmlistbot@llvm.org"
                              moz-do-not-send="true" target="_blank"><llvmlistbot@llvm.org></a>,
                            llvm-commits <a
                              href="mailto:llvm-commits@lists.llvm.org"
                              moz-do-not-send="true" target="_blank"><llvm-commits@lists.llvm.org></a>,
                            <a href="mailto:powerllvm@ca.ibm.com"
                              moz-do-not-send="true" target="_blank">powerllvm@ca.ibm.com</a><br>
                            Subject: [EXTERNAL] Re: [llvm] 7fe41ac -
                            Revert "[LV] Unconditionally branch from
                            middle to scalar preheader if the scalar
                            loop must execute"<br>
                            Date: Mon, May 17, 2021 11:59 PM<br>
                             <br>
                            <!-- BaNnErBlUrFlE-HeAdEr-start -->
                            <!-- BaNnErBlUrFlE-HeAdEr-end -->
                            <!--Notes ACF
<meta http-equiv="Content-Type" content="text/html; charset=utf8" >-->
                            <!-- BaNnErBlUrFlE-BoDy-start -->
                            <!-- Preheader Text : BEGIN --> <span
style="display:none!important;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;">
                              I tried another cycle to see if this had
                              been resolved, but am still seeing build
                              bot failures, nearly exclusively on PPC. 
                              (And one arm self host bot.)  @PPC Bot
                              Owner - I need help reducing a test case
                              for the failure seen here: ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍
                            </span>
                            <!-- Preheader Text : END -->
                            <!-- Email Banner : BEGIN --> <span
style="display:none!important;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerStart</span>
                            <!--[if ((ie)|(mso))]>
  <table border="0" cellspacing="0" cellpadding="0" width="100%" style="padding: 3px 0px 16px 0px;" ><tr><td>
    <table class="pfptBannerTableMSO" border="0" cellspacing="0" cellpadding="0" style="padding: 0px 10px 5px 6px; width: 100%;border-radius:4px;border-top:4px solid #90a4ae;background-color:#d0d8dc;" ><tr><td valign="top" >
      <table align="left" border="0" cellspacing="0" cellpadding="0" style="padding: 4px 8px 4px 8px" >
        <tr><td><span class="pfptTitleMSO" style="color:#000000 !important;font-family: 'Arial', sans-serif;font-weight:bold !important;font-size:14px !important;" >
          This Message Is From an External Sender
        </span></td></tr>
        <tr><td><span class="pfptSubtitleMSO" style="color:#000000 !important;font-weight:normal !important;font-family: 'Arial', sans-serif; font-size:12px !important;" >
          This message came from outside your organization.
        </span></td></tr>

      </table>

    </td></tr></table>
  </td></tr></table>
<![endif]-->
                            <!--[if !((ie)|(mso))]-->
                            <div style="margin:16px 0px 16px 0px;
                              padding:8px 16px 8px 16px; border-radius:
                              4px; min-width: 200px;background-color:
                              #d0d8dc; border-top: 4px solid #90a4ae;">
                              <div style="display: inline-block; margin:
                                0px 0px 1px 0px; max-width: 600px;">
                                <div style="color:#000000
                                  !important;font-family: 'Arial',
                                  sans-serif !important;font-weight:bold
                                  !important;font-size:14px
                                  !important;line-height:18px;display:block;">This
                                  Message Is From an External Sender</div>
                                <div style="color:#000000
                                  !important;font-weight:normal
                                  !important;font-family: 'Arial',
                                  sans-serif !important;font-size:12px
                                  !important;line-height:18px;margin-top:2px;display:block">This
                                  message came from outside your
                                  organization.</div>
                              </div>
                            </div>
                            <!--[endif]--> <span
style="display:none!important;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;max-height:0px;max-width:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerEnd</span><br>
                            <!-- Email Banner : END -->
                            <!-- BaNnErBlUrFlE-BoDy-end -->
                            <p>I tried another cycle to see if this had
                              been resolved, but am still seeing build
                              bot failures, nearly exclusively on PPC. 
                              (And one arm self host bot.) </p>
                            <p>@PPC Bot Owner - I need help reducing a
                              test case for the failure seen here: <a
                                href="https://lab.llvm.org/buildbot#builders/100/builds/5762"
                                moz-do-not-send="true" target="_blank">https://lab.llvm.org/buildbot#builders/100/builds/5762</a>. 
                              I am stuck, and unable to make progress
                              for nearly 3 months now.  I would greatly
                              appreciate help. </p>
                            <p>Philip</p>
                            <div>On 5/5/21 11:59 PM, Adrian Kuegel
                              wrote:</div>
                            <blockquote
cite="mid:CAHhV=OX3NbAExy4YsQWLH0fC-zXiG_VpToAtB72NhkUwwYekRQ@mail.gmail.com"
                              type="cite"><!--Notes ACF
<meta http-equiv="content-type" content="text/html; charset=utf8" >-->
                              <div dir="ltr">
                                <div dir="ltr">Sounds good to me, and
                                  thanks for sending the heads up :)</div>
                                 
                                <div>
                                  <div dir="ltr">On Wed, May 5, 2021 at
                                    10:01 PM Philip Reames <<a
                                      href="mailto:listmail@philipreames.com"
                                      moz-do-not-send="true"
                                      target="_blank">listmail@philipreames.com</a>>
                                    wrote:</div>
                                  <blockquote style="margin:0px 0px 0px
                                    0.8ex;border-left:1px solid
                                    rgb(204,204,204);padding-left:1ex">FYI,
                                    I'm going to try recommitting this
                                    without changes in a day or so.<br>
                                    <br>
                                    I never heard back from a PPC bot
                                    owner, and I don't have enough<br>
                                    information to really debug anything
                                    from the builtbot log.  I did run<br>
                                    across a latent issue which this
                                    patch may very well have exposed at<br>
                                    much higher frequency; the previous
                                    patch in this series (which is much<br>
                                    more restrictive) does appear to
                                    have increased frequency.  That was<br>
                                    worked around in 80e80250.  My
                                    educated guess is that same issue<br>
                                    triggered the miscompile seen on the
                                    ppc bot, but that is more of a<br>
                                    guess than I'd really prefer.<br>
                                    <br>
                                    I'm going to submit this during off
                                    hours, and watch the bots fairly<br>
                                    closely after submit.  Hopefully
                                    this either cycles clean or I get a<br>
                                    better clue as to what the root
                                    issue is.<br>
                                    <br>
                                    Philip<br>
                                    <br>
                                    On 2/8/21 9:09 PM, Philip Reames via
                                    llvm-commits wrote:<br>
                                    > Ben,<br>
                                    ><br>
                                    > Thanks for the clarification. 
                                    The log does not make the fact this
                                    is<br>
                                    > an execution failure obvious.<br>
                                    ><br>
                                    > No, I don't have access to a
                                    PPC machine.<br>
                                    ><br>
                                    > I am going to need some
                                    assistance from the bot owner on
                                    this. At a<br>
                                    > minimum, IR for the test in
                                    question (before optimization, but
                                    on<br>
                                    > target platform) seems like a
                                    reasonable ask.<br>
                                    ><br>
                                    > I strongly suspect this change
                                    is simply exposing another latent<br>
                                    > issue.  Or at least, I've
                                    reviewed the change and don't see
                                    anything<br>
                                    > likely to cause runtime crashes
                                    w/o also tripping compiler asserts.<br>
                                    ><br>
                                    > Philip<br>
                                    ><br>
                                    ><br>
                                    > On 2/8/21 5:21 AM, Benjamin
                                    Kramer wrote:<br>
                                    >> `execution_time` failures
                                    mean that the bot succeeded building
                                    a test<br>
                                    >> but it failed when running
                                    it. I'm relatively certain that this
                                    is the<br>
                                    >> same issue Adrian is seeing
                                    -- binaries segfaulting early on
                                    PPC.<br>
                                    >><br>
                                    >> The bot log output isn't
                                    helpful at all for investigating why
                                    this is<br>
                                    >> happening. Do you happen to
                                    have access to a PPC machine?<br>
                                    >><br>
                                    >> On Fri, Feb 5, 2021 at 6:05
                                    PM Philip Reames via llvm-commits<br>
                                    >> <<a
                                      href="mailto:llvm-commits@lists.llvm.org"
                                      moz-do-not-send="true"
                                      target="_blank">llvm-commits@lists.llvm.org</a>>
                                    wrote:<br>
                                    >>> Adrian,<br>
                                    >>><br>
                                    >>> I'm going to need you
                                    to provide a bit more information
                                    here. The test<br>
                                    >>> failure in stage1 was
                                    fixed at the time you reverted this
                                    patch.  The<br>
                                    >>> remaining failure in
                                    the bot is very unclear.  What is a
                                    execution_time<br>
                                    >>> failure? From the log
                                    output, the "failing" run finished
                                    in 0.5<br>
                                    >>> seconds,<br>
                                    >>> whereas the previous
                                    "succeeding" run finished in 11
                                    seconds. Without<br>
                                    >>> further context, I'd
                                    say that's no failure.<br>
                                    >>><br>
                                    >>> I'll also note that I
                                    did not receive email from this
                                    bot.  I received<br>
                                    >>> notice from the various
                                    other bots and fixed the ARM test
                                    issue, but<br>
                                    >>> unless I missed it in
                                    with the others, this bot is not
                                    notifying.<br>
                                    >>><br>
                                    >>> In general, I'm a fan
                                    of fast reverts, but I have to
                                    admit, this one<br>
                                    >>> appears borderline at
                                    the moment.<br>
                                    >>><br>
                                    >>> Philip<br>
                                    >>><br>
                                    >>> On 2/5/21 3:53 AM,
                                    Adrian Kuegel via llvm-commits
                                    wrote:<br>
                                    >>>> Author: Adrian
                                    Kuegel<br>
                                    >>>> Date:
                                    2021-02-05T12:51:03+01:00<br>
                                    >>>> New Revision:
                                    7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c<br>
                                    >>>><br>
                                    >>>> URL:<br>
                                    >>>> <a
href="https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c</a><br>
                                    >>>> DIFF:<br>
                                    >>>> <a
href="https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c.diff"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">https://github.com/llvm/llvm-project/commit/7fe41ac3dff2d44c3d2c31b28554fbe4a86eaa6c.diff</a><br>
                                    >>>><br>
                                    >>>> LOG: Revert "[LV]
                                    Unconditionally branch from middle
                                    to scalar<br>
                                    >>>> preheader if the
                                    scalar loop must execute"<br>
                                    >>>><br>
                                    >>>> This reverts commit
3e5ce49e5371ce4feadbf97dd5c2b652d9db3d1d.<br>
                                    >>>><br>
                                    >>>> Tests started
                                    failing on PPC, for example:<br>
                                    >>>> <a
                                      href="http://lab.llvm.org:8011/#/builders/105/builds/5569"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">http://lab.llvm.org:8011/#/builders/105/builds/5569</a><br>
                                    >>>><br>
                                    >>>> Added:<br>
                                    >>>><br>
                                    >>>><br>
                                    >>>> Modified:<br>
                                    >>>>      
                                    llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
                                    >>>>      
                                    llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
                                    >>>>
                                    llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
                                    >>>>
                                    llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
                                    >>>>      
                                    llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
                                    >>>><br>
                                    >>>> Removed:<br>
                                    >>>><br>
                                    >>>><br>
                                    >>>><br>
                                    >>>>
################################################################################<br>
                                    >>>><br>
                                    >>>> diff  --git
                                    a/llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
                                    >>>>
                                    b/llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
                                    >>>> index
                                    8a89158788cf..de4fb446fdf2 100644<br>
                                    >>>> ---
                                    a/llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
                                    >>>> +++
                                    b/llvm/lib/Transforms/Utils/LoopVersioning.cpp<br>
                                    >>>> @@ -44,11 +44,11 @@
                                    LoopVersioning::LoopVersioning(const<br>
                                    >>>> LoopAccessInfo
                                    &LAI,<br>
                                    >>>>         
                                    AliasChecks(Checks.begin(),
                                    Checks.end()),<br>
                                    >>>>         
                                    Preds(LAI.getPSE().getUnionPredicate()),
                                    LAI(LAI), LI(LI),<br>
                                    >>>> DT(DT),<br>
                                    >>>>          SE(SE) {<br>
                                    >>>> + 
                                    assert(L->getUniqueExitBlock()
                                    && "No single exit block");<br>
                                    >>>>    }<br>
                                    >>>><br>
                                    >>>>    void
                                    LoopVersioning::versionLoop(<br>
                                    >>>>        const
                                    SmallVectorImpl<Instruction *>
                                    &DefsUsedOutside) {<br>
                                    >>>> - 
                                    assert(VersionedLoop->getUniqueExitBlock()
                                    && "No single exit<br>
                                    >>>> block");<br>
                                    >>>>     
                                    assert(VersionedLoop->isLoopSimplifyForm()
                                    &&<br>
                                    >>>>             "Loop
                                    is not in loop-simplify form");<br>
                                    >>>><br>
                                    >>>><br>
                                    >>>> diff  --git
                                    a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
                                    >>>>
                                    b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
                                    >>>> index
                                    3277842edbfe..6bce0caeb36f 100644<br>
                                    >>>> ---
                                    a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
                                    >>>> +++
                                    b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
                                    >>>> @@ -852,7 +852,7 @@
                                    class InnerLoopVectorizer {<br>
                                    >>>>      /// Middle
                                    Block between the vector and the
                                    scalar.<br>
                                    >>>>      BasicBlock
                                    *LoopMiddleBlock;<br>
                                    >>>><br>
                                    >>>> -  /// The unique
                                    ExitBlock of the scalar loop if one
                                    exists.  Note<br>
                                    >>>> that<br>
                                    >>>> +  /// The (unique)
                                    ExitBlock of the scalar loop.  Note
                                    that<br>
                                    >>>>      /// there can
                                    be multiple exiting edges reaching
                                    this block.<br>
                                    >>>>      BasicBlock
                                    *LoopExitBlock;<br>
                                    >>>><br>
                                    >>>> @@ -3147,13 +3147,9
                                    @@ void<br>
                                    >>>>
                                    InnerLoopVectorizer::emitMinimumIterationCountCheck(Loop
                                    *L,<br>
                                    >>>>
                                    DT->getNode(Bypass)->getIDom())
                                    &&<br>
                                    >>>>             "TC
                                    check is expected to dominate
                                    Bypass");<br>
                                    >>>><br>
                                    >>>> -  // Update
                                    dominator for Bypass & LoopExit
                                    (if needed).<br>
                                    >>>> +  // Update
                                    dominator for Bypass & LoopExit.<br>
                                    >>>>     
                                    DT->changeImmediateDominator(Bypass,
                                    TCCheckBlock);<br>
                                    >>>> -  if
                                    (!Cost->requiresScalarEpilogue())<br>
                                    >>>> -    // If there is
                                    an epilogue which must run, there's
                                    no edge<br>
                                    >>>> from the<br>
                                    >>>> -    // middle
                                    block to exit blocks  and thus no
                                    need to update the<br>
                                    >>>> immediate<br>
                                    >>>> -    // dominator
                                    of the exit blocks.<br>
                                    >>>> -   
                                    DT->changeImmediateDominator(LoopExitBlock,
                                    TCCheckBlock);<br>
                                    >>>> + 
                                    DT->changeImmediateDominator(LoopExitBlock,
                                    TCCheckBlock);<br>
                                    >>>><br>
                                    >>>>     
                                    ReplaceInstWithInst(<br>
                                    >>>>         
                                    TCCheckBlock->getTerminator(),<br>
                                    >>>> @@ -3192,11 +3188,7
                                    @@ void<br>
                                    >>>>
                                    InnerLoopVectorizer::emitSCEVChecks(Loop
                                    *L, BasicBlock *Bypass) {<br>
                                    >>>>      // Update
                                    dominator only if this is first RT
                                    check.<br>
                                    >>>>      if
                                    (LoopBypassBlocks.empty()) {<br>
                                    >>>>       
                                    DT->changeImmediateDominator(Bypass,
                                    SCEVCheckBlock);<br>
                                    >>>> -    if
                                    (!Cost->requiresScalarEpilogue())<br>
                                    >>>> -      // If there
                                    is an epilogue which must run,
                                    there's no edge<br>
                                    >>>> from the<br>
                                    >>>> -      // middle
                                    block to exit blocks  and thus no
                                    need to update<br>
                                    >>>> the immediate<br>
                                    >>>> -      // dominator
                                    of the exit blocks.<br>
                                    >>>> -     
                                    DT->changeImmediateDominator(LoopExitBlock,
                                    SCEVCheckBlock);<br>
                                    >>>> +   
                                    DT->changeImmediateDominator(LoopExitBlock,
                                    SCEVCheckBlock);<br>
                                    >>>>      }<br>
                                    >>>><br>
                                    >>>>     
                                    ReplaceInstWithInst(<br>
                                    >>>> @@ -3252,11 +3244,7
                                    @@ void<br>
                                    >>>>
                                    InnerLoopVectorizer::emitMemRuntimeChecks(Loop
                                    *L, BasicBlock<br>
                                    >>>> *Bypass) {<br>
                                    >>>>      // Update
                                    dominator only if this is first RT
                                    check.<br>
                                    >>>>      if
                                    (LoopBypassBlocks.empty()) {<br>
                                    >>>>       
                                    DT->changeImmediateDominator(Bypass,
                                    MemCheckBlock);<br>
                                    >>>> -    if
                                    (!Cost->requiresScalarEpilogue())<br>
                                    >>>> -      // If there
                                    is an epilogue which must run,
                                    there's no edge<br>
                                    >>>> from the<br>
                                    >>>> -      // middle
                                    block to exit blocks  and thus no
                                    need to update<br>
                                    >>>> the immediate<br>
                                    >>>> -      // dominator
                                    of the exit blocks.<br>
                                    >>>> -     
                                    DT->changeImmediateDominator(LoopExitBlock,
                                    MemCheckBlock);<br>
                                    >>>> +   
                                    DT->changeImmediateDominator(LoopExitBlock,
                                    MemCheckBlock);<br>
                                    >>>>      }<br>
                                    >>>><br>
                                    >>>>      Instruction
                                    *FirstCheckInst;<br>
                                    >>>> @@ -3381,10 +3369,9
                                    @@ Value<br>
                                    >>>>
                                    *InnerLoopVectorizer::emitTransformedIndex(<br>
                                    >>>>    Loop
                                    *InnerLoopVectorizer::createVectorLoopSkeleton(StringRef<br>
                                    >>>> Prefix) {<br>
                                    >>>>      LoopScalarBody
                                    = OrigLoop->getHeader();<br>
                                    >>>>     
                                    LoopVectorPreHeader =
                                    OrigLoop->getLoopPreheader();<br>
                                    >>>> +  LoopExitBlock =
                                    OrigLoop->getUniqueExitBlock();<br>
                                    >>>> + 
                                    assert(LoopExitBlock &&
                                    "Must have an exit block");<br>
                                    >>>>     
                                    assert(LoopVectorPreHeader
                                    && "Invalid loop
                                    structure");<br>
                                    >>>> -  LoopExitBlock =
                                    OrigLoop->getUniqueExitBlock();
                                    // may be nullptr<br>
                                    >>>> - 
                                    assert((LoopExitBlock ||
                                    Cost->requiresScalarEpilogue())
                                    &&<br>
                                    >>>> -         "multiple
                                    exit loop without required
                                    epilogue?");<br>
                                    >>>><br>
                                    >>>>     
                                    LoopMiddleBlock =<br>
                                    >>>>         
                                    SplitBlock(LoopVectorPreHeader,<br>
                                    >>>>
                                    LoopVectorPreHeader->getTerminator(),
                                    DT,<br>
                                    >>>> @@ -3393,20
                                    +3380,12 @@ Loop<br>
                                    >>>>
                                    *InnerLoopVectorizer::createVectorLoopSkeleton(StringRef
                                    Prefix) {<br>
                                    >>>>         
                                    SplitBlock(LoopMiddleBlock,<br>
                                    >>>>
                                    LoopMiddleBlock->getTerminator(),
                                    DT, LI,<br>
                                    >>>>                    
                                    nullptr, Twine(Prefix) + "<a
                                      href="http://scalar.ph"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">scalar.ph</a>");<br>
                                    >>>><br>
                                    >>>> +  // Set up branch
                                    from middle block to the exit and
                                    scalar<br>
                                    >>>> preheader blocks.<br>
                                    >>>> +  //
                                    completeLoopSkeleton will update the
                                    condition to use an<br>
                                    >>>> iteration check,<br>
                                    >>>> +  // if required
                                    to decide whether to execute the
                                    remainder.<br>
                                    >>>> +  BranchInst
                                    *BrInst =<br>
                                    >>>> +     
                                    BranchInst::Create(LoopExitBlock,
                                    LoopScalarPreHeader,<br>
                                    >>>> Builder.getTrue());<br>
                                    >>>>      auto
                                    *ScalarLatchTerm =<br>
                                    >>>>
                                    OrigLoop->getLoopLatch()->getTerminator();<br>
                                    >>>> -<br>
                                    >>>> -  // Set up the
                                    middle block terminator.  Two cases:<br>
                                    >>>> -  // 1) If we know
                                    that we must execute the scalar
                                    epilogue, emit an<br>
                                    >>>> -  //   
                                    unconditional branch.<br>
                                    >>>> -  // 2) Otherwise,
                                    we must have a single unique exit
                                    block (due to<br>
                                    >>>> how we<br>
                                    >>>> -  //    implement
                                    the multiple exit case).  In this
                                    case, set up a<br>
                                    >>>> conditonal<br>
                                    >>>> -  //    branch
                                    from the middle block to the loop
                                    scalar preheader,<br>
                                    >>>> and the<br>
                                    >>>> -  //    exit
                                    block.  completeLoopSkeleton will
                                    update the<br>
                                    >>>> condition to use an<br>
                                    >>>> -  //    iteration
                                    check, if required to decide whether
                                    to execute<br>
                                    >>>> the remainder.<br>
                                    >>>> -  BranchInst
                                    *BrInst =
                                    Cost->requiresScalarEpilogue() ?<br>
                                    >>>> -   
                                    BranchInst::Create(LoopScalarPreHeader)
                                    :<br>
                                    >>>> -   
                                    BranchInst::Create(LoopExitBlock,
                                    LoopScalarPreHeader,<br>
                                    >>>>
                                    -                      
                                    Builder.getTrue());<br>
                                    >>>>
                                    BrInst->setDebugLoc(ScalarLatchTerm->getDebugLoc());<br>
                                    >>>>
                                    ReplaceInstWithInst(LoopMiddleBlock->getTerminator(),
                                    BrInst);<br>
                                    >>>><br>
                                    >>>> @@ -3418,11 +3397,7
                                    @@ Loop<br>
                                    >>>>
                                    *InnerLoopVectorizer::createVectorLoopSkeleton(StringRef
                                    Prefix) {<br>
                                    >>>>                    
                                    nullptr, nullptr, Twine(Prefix) +
                                    "vector.body");<br>
                                    >>>><br>
                                    >>>>      // Update
                                    dominator for loop exit.<br>
                                    >>>> -  if
                                    (!Cost->requiresScalarEpilogue())<br>
                                    >>>> -    // If there is
                                    an epilogue which must run, there's
                                    no edge<br>
                                    >>>> from the<br>
                                    >>>> -    // middle
                                    block to exit blocks  and thus no
                                    need to update the<br>
                                    >>>> immediate<br>
                                    >>>> -    // dominator
                                    of the exit blocks.<br>
                                    >>>> -   
                                    DT->changeImmediateDominator(LoopExitBlock,
                                    LoopMiddleBlock);<br>
                                    >>>> + 
                                    DT->changeImmediateDominator(LoopExitBlock,
                                    LoopMiddleBlock);<br>
                                    >>>><br>
                                    >>>>      // Create and
                                    register the new vector loop.<br>
                                    >>>>      Loop *Lp =
                                    LI->AllocateLoop();<br>
                                    >>>> @@ -3519,14
                                    +3494,10 @@ BasicBlock<br>
                                    >>>>
                                    *InnerLoopVectorizer::completeLoopSkeleton(Loop
                                    *L,<br>
                                    >>>>      auto
                                    *ScalarLatchTerm =<br>
                                    >>>>
                                    OrigLoop->getLoopLatch()->getTerminator();<br>
                                    >>>><br>
                                    >>>>      // Add a check
                                    in the middle block to see if we
                                    have completed<br>
                                    >>>> -  // all of the
                                    iterations in the first vector
                                    loop.  Three cases:<br>
                                    >>>> -  // 1) If we
                                    require a scalar epilogue, there is
                                    no conditional<br>
                                    >>>> branch as<br>
                                    >>>> -  //    we
                                    unconditionally branch to the scalar
                                    preheader. Do<br>
                                    >>>> nothing.<br>
                                    >>>> -  // 2) If (N -
                                    N%VF) == N, then we *don't* need to
                                    run the<br>
                                    >>>> remainder.<br>
                                    >>>> -  //    Thus if
                                    tail is to be folded, we know we
                                    don't need to run<br>
                                    >>>> the<br>
                                    >>>> -  //    remainder
                                    and we can use the previous value
                                    for the<br>
                                    >>>> condition (true).<br>
                                    >>>> -  // 3) Otherwise,
                                    construct a runtime check.<br>
                                    >>>> -  if
                                    (!Cost->requiresScalarEpilogue()
                                    &&<br>
                                    >>>>
                                    !Cost->foldTailByMasking()) {<br>
                                    >>>> +  // all of the
                                    iterations in the first vector loop.<br>
                                    >>>> +  // If (N - N%VF)
                                    == N, then we *don't* need to run
                                    the remainder.<br>
                                    >>>> +  // If tail is to
                                    be folded, we know we don't need to
                                    run the<br>
                                    >>>> remainder.<br>
                                    >>>> +  if
                                    (!Cost->foldTailByMasking()) {<br>
                                    >>>>        Instruction
                                    *CmpN =
                                    CmpInst::Create(Instruction::ICmp,<br>
                                    >>>> CmpInst::ICMP_EQ,<br>
                                    >>>>
                                                                              
                                    Count, VectorTripCount,<br>
                                    >>>> "cmp.n",<br>
                                    >>>>
                                    LoopMiddleBlock->getTerminator());<br>
                                    >>>> @@ -3590,17
                                    +3561,17 @@ BasicBlock<br>
                                    >>>>
                                    *InnerLoopVectorizer::createVectorizedLoopSkeleton()
                                    {<br>
                                    >>>>      |    [  ]_|  
                                    <-- vector loop.<br>
                                    >>>>      |     |<br>
                                    >>>>      |     v<br>
                                    >>>> -  \   -[ ]  
                                    <--- middle-block.<br>
                                    >>>> -   \/   |<br>
                                    >>>> -   /\   v<br>
                                    >>>> -   | ->[ ]    
                                    <--- new preheader.<br>
                                    >>>> +  |   -[ ]  
                                    <--- middle-block.<br>
                                    >>>> +  |  /  |<br>
                                    >>>> +  | /   v<br>
                                    >>>> +  -|- >[ ]    
                                    <--- new preheader.<br>
                                    >>>>       |    |<br>
                                    >>>> - (opt)  v     
                                    <-- edge from middle to exit iff
                                    epilogue is not<br>
                                    >>>> required.<br>
                                    >>>> +   |    v<br>
                                    >>>>       |   [ ] \<br>
                                    >>>> -   |   [ ]_|  
                                    <-- old scalar loop to handle
                                    remainder (scalar<br>
                                    >>>> epilogue).<br>
                                    >>>> +   |   [ ]_|  
                                    <-- old scalar loop to handle
                                    remainder.<br>
                                    >>>>        \   |<br>
                                    >>>>         \  v<br>
                                    >>>> -      >[ ]    
                                    <-- exit block(s).<br>
                                    >>>> +      >[ ]    
                                    <-- exit block.<br>
                                    >>>>       ...<br>
                                    >>>>       */<br>
                                    >>>><br>
                                    >>>> @@ -4021,18
                                    +3992,13 @@ void<br>
                                    >>>>
                                    InnerLoopVectorizer::fixVectorizedLoop()
                                    {<br>
                                    >>>>      // Forget the
                                    original basic block.<br>
                                    >>>>     
                                    PSE.getSE()->forgetLoop(OrigLoop);<br>
                                    >>>><br>
                                    >>>> -  // If we
                                    inserted an edge from the middle
                                    block to the unique<br>
                                    >>>> exit block,<br>
                                    >>>> -  // update uses
                                    outside the loop (phis) to account
                                    for the newly<br>
                                    >>>> inserted<br>
                                    >>>> -  // edge.<br>
                                    >>>> -  if
                                    (!Cost->requiresScalarEpilogue())
                                    {<br>
                                    >>>> -    // Fix-up
                                    external users of the induction
                                    variables.<br>
                                    >>>> -    for (auto
                                    &Entry :
                                    Legal->getInductionVars())<br>
                                    >>>> -     
                                    fixupIVUsers(Entry.first,
                                    Entry.second,<br>
                                    >>>> -
                                    getOrCreateVectorTripCount(LI->getLoopFor(LoopVectorBody)),<br>
                                    >>>> -                  
                                    IVEndValues[Entry.first],
                                    LoopMiddleBlock);<br>
                                    >>>> +  // Fix-up
                                    external users of the induction
                                    variables.<br>
                                    >>>> +  for (auto
                                    &Entry :
                                    Legal->getInductionVars())<br>
                                    >>>> +   
                                    fixupIVUsers(Entry.first,
                                    Entry.second,<br>
                                    >>>> +
                                    getOrCreateVectorTripCount(LI->getLoopFor(LoopVectorBody)),<br>
                                    >>>> +                
                                    IVEndValues[Entry.first],
                                    LoopMiddleBlock);<br>
                                    >>>><br>
                                    >>>> -   
                                    fixLCSSAPHIs();<br>
                                    >>>> -  }<br>
                                    >>>> +  fixLCSSAPHIs();<br>
                                    >>>>      for
                                    (Instruction *PI :
                                    PredicatedInstructions)<br>
                                    >>>>       
                                    sinkScalarOperands(&*PI);<br>
                                    >>>><br>
                                    >>>> @@ -4250,13
                                    +4216,12 @@ void<br>
                                    >>>>
                                    InnerLoopVectorizer::fixFirstOrderRecurrence(PHINode
                                    *Phi) {<br>
                                    >>>>      // recurrence
                                    in the exit block, and then add an
                                    edge for the<br>
                                    >>>> middle block.<br>
                                    >>>>      // Note that
                                    LCSSA does not imply single entry
                                    when the<br>
                                    >>>> original scalar
                                    loop<br>
                                    >>>>      // had
                                    multiple exiting edges (as we always
                                    run the last<br>
                                    >>>> iteration in the<br>
                                    >>>> -  // scalar
                                    epilogue); in that case, there is no
                                    edge from middle<br>
                                    >>>> to exit and<br>
                                    >>>> -  // and thus no
                                    phis which needed updated.<br>
                                    >>>> -  if
                                    (!Cost->requiresScalarEpilogue())<br>
                                    >>>> -    for (PHINode
                                    &LCSSAPhi :
                                    LoopExitBlock->phis())<br>
                                    >>>> -      if
                                    (any_of(LCSSAPhi.incoming_values(),<br>
                                    >>>> -                
                                    [Phi](Value *V) { return V == Phi;
                                    }))<br>
                                    >>>> -       
                                    LCSSAPhi.addIncoming(ExtractForPhiUsedOutsideLoop,<br>
                                    >>>> LoopMiddleBlock);<br>
                                    >>>> +  // scalar
                                    epilogue); in that case, the exiting
                                    path through<br>
                                    >>>> middle will be<br>
                                    >>>> +  // dynamically
                                    dead and the value picked for the
                                    phi doesn't<br>
                                    >>>> matter.<br>
                                    >>>> +  for (PHINode
                                    &LCSSAPhi :
                                    LoopExitBlock->phis())<br>
                                    >>>> +    if
                                    (any_of(LCSSAPhi.incoming_values(),<br>
                                    >>>> +              
                                    [Phi](Value *V) { return V == Phi;
                                    }))<br>
                                    >>>> +     
                                    LCSSAPhi.addIncoming(ExtractForPhiUsedOutsideLoop,<br>
                                    >>>> LoopMiddleBlock);<br>
                                    >>>>    }<br>
                                    >>>><br>
                                    >>>>    void
                                    InnerLoopVectorizer::fixReduction(PHINode
                                    *Phi) {<br>
                                    >>>> @@ -4421,11
                                    +4386,10 @@ void<br>
                                    >>>>
                                    InnerLoopVectorizer::fixReduction(PHINode
                                    *Phi) {<br>
                                    >>>>      // We know
                                    that the loop is in LCSSA form. We
                                    need to update<br>
                                    >>>> the PHI nodes<br>
                                    >>>>      // in the exit
                                    blocks.  See comment on analogous
                                    loop in<br>
                                    >>>>      //
                                    fixFirstOrderRecurrence for a more
                                    complete explaination of<br>
                                    >>>> the logic.<br>
                                    >>>> -  if
                                    (!Cost->requiresScalarEpilogue())<br>
                                    >>>> -    for (PHINode
                                    &LCSSAPhi :
                                    LoopExitBlock->phis())<br>
                                    >>>> -      if
                                    (any_of(LCSSAPhi.incoming_values(),<br>
                                    >>>> -                
                                    [LoopExitInst](Value *V) { return V
                                    ==<br>
                                    >>>> LoopExitInst; }))<br>
                                    >>>> -       
                                    LCSSAPhi.addIncoming(ReducedPartRdx,
                                    LoopMiddleBlock);<br>
                                    >>>> +  for (PHINode
                                    &LCSSAPhi :
                                    LoopExitBlock->phis())<br>
                                    >>>> +    if
                                    (any_of(LCSSAPhi.incoming_values(),<br>
                                    >>>> +              
                                    [LoopExitInst](Value *V) { return V
                                    ==<br>
                                    >>>> LoopExitInst; }))<br>
                                    >>>> +     
                                    LCSSAPhi.addIncoming(ReducedPartRdx,
                                    LoopMiddleBlock);<br>
                                    >>>><br>
                                    >>>>      // Fix the
                                    scalar loop reduction variable with
                                    the incoming<br>
                                    >>>> reduction sum<br>
                                    >>>>      // from the
                                    vector body and from the backedge
                                    value.<br>
                                    >>>> @@ -8074,11 +8038,7
                                    @@ BasicBlock<br>
                                    >>>>
                                    *EpilogueVectorizerMainLoop::emitMinimumIterationCountCheck(<br>
                                    >>>><br>
                                    >>>>        // Update
                                    dominator for Bypass & LoopExit.<br>
                                    >>>>       
                                    DT->changeImmediateDominator(Bypass,
                                    TCCheckBlock);<br>
                                    >>>> -    if
                                    (!Cost->requiresScalarEpilogue())<br>
                                    >>>> -      // For loops
                                    with multiple exits, there's no edge
                                    from the<br>
                                    >>>> middle block<br>
                                    >>>> -      // to exit
                                    blocks (as the epilogue must run)
                                    and thus no<br>
                                    >>>> need to update<br>
                                    >>>> -      // the
                                    immediate dominator of the exit
                                    blocks.<br>
                                    >>>> -     
                                    DT->changeImmediateDominator(LoopExitBlock,
                                    TCCheckBlock);<br>
                                    >>>> +   
                                    DT->changeImmediateDominator(LoopExitBlock,
                                    TCCheckBlock);<br>
                                    >>>><br>
                                    >>>>       
                                    LoopBypassBlocks.push_back(TCCheckBlock);<br>
                                    >>>><br>
                                    >>>> @@ -8142,12 +8102,7
                                    @@<br>
                                    >>>>
                                    EpilogueVectorizerEpilogueLoop::createEpilogueVectorizedLoopSkeleton()<br>
                                    >>>> {<br>
                                    >>>><br>
                                    >>>>     
                                    DT->changeImmediateDominator(LoopScalarPreHeader,<br>
                                    >>>>
                                    EPI.EpilogueIterationCountCheck);<br>
                                    >>>> -  if
                                    (!Cost->requiresScalarEpilogue())<br>
                                    >>>> -    // If there is
                                    an epilogue which must run, there's
                                    no edge<br>
                                    >>>> from the<br>
                                    >>>> -    // middle
                                    block to exit blocks  and thus no
                                    need to update the<br>
                                    >>>> immediate<br>
                                    >>>> -    // dominator
                                    of the exit blocks.<br>
                                    >>>> -   
                                    DT->changeImmediateDominator(LoopExitBlock,<br>
                                    >>>> -
                                    EPI.EpilogueIterationCountCheck);<br>
                                    >>>> + 
                                    DT->changeImmediateDominator(LoopExitBlock,<br>
                                    >>>>
                                    EPI.EpilogueIterationCountCheck);<br>
                                    >>>><br>
                                    >>>>      // Keep track
                                    of bypass blocks, as they feed start
                                    values to<br>
                                    >>>> the induction<br>
                                    >>>>      // phis in the
                                    scalar loop preheader.<br>
                                    >>>><br>
                                    >>>> diff  --git<br>
                                    >>>>
                                    a/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
                                    >>>>
                                    b/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
                                    >>>> index
                                    ec280bf5d5e4..7d4a3c5c9935 100644<br>
                                    >>>> ---<br>
                                    >>>>
                                    a/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
                                    >>>> +++<br>
                                    >>>>
                                    b/llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll<br>
                                    >>>> @@ -471,9 +471,10
                                    @@ define i16 @multiple_exit(i16*
                                    %p, i32 %n) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP15:%.*]] = icmp eq i32
                                    [[INDEX_NEXT]],<br>
                                    >>>> [[N_VEC]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[TMP15]], label
                                    [[MIDDLE_BLOCK:%.*]],<br>
                                    >>>> label
                                    [[VECTOR_BODY]], [[LOOP6:!llvm.loop
                                    !.*]]<br>
                                    >>>>    ; CHECK:      
                                    middle.block:<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[CMP_N:%.*]] = icmp eq i32
                                    [[TMP2]], [[N_VEC]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[VECTOR_RECUR_EXTRACT:%.*]] =
                                    extractelement<br>
                                    >>>> <4 x i16>
                                    [[WIDE_LOAD]], i32 3<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[VECTOR_RECUR_EXTRACT_FOR_PHI:%.*]]
                                    =<br>
                                    >>>> extractelement
                                    <4 x i16> [[WIDE_LOAD]], i32 2<br>
                                    >>>> -; CHECK-NEXT:   
                                    br label [[SCALAR_PH]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP_N]], label
                                    [[IF_END:%.*]], label<br>
                                    >>>> [[SCALAR_PH]]<br>
                                    >>>>    ; CHECK:       <a
                                      href="http://scalar.ph"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">scalar.ph</a>:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[SCALAR_RECUR_INIT:%.*]] = phi i16
                                    [ 0,<br>
                                    >>>> [[ENTRY:%.*]] ], [
                                    [[VECTOR_RECUR_EXTRACT]],
                                    [[MIDDLE_BLOCK]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_RESUME_VAL:%.*]] = phi i32 [
                                    [[N_VEC]],<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY]] ]<br>
                                    >>>> @@ -485,14 +486,14
                                    @@ define i16 @multiple_exit(i16*
                                    %p, i32 %n) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[B:%.*]] = getelementptr inbounds
                                    i16, i16*<br>
                                    >>>> [[P]], i64
                                    [[IPROM]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[REC_NEXT]] = load i16, i16* [[B]],
                                    align 2<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[CMP:%.*]] = icmp slt i32 [[I]],
                                    [[N]]<br>
                                    >>>> -; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label<br>
                                    >>>> [[IF_END:%.*]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label [[IF_END]]<br>
                                    >>>>    ; CHECK:      
                                    for.body:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    store i16 [[SCALAR_RECUR]], i16*
                                    [[B]], align 4<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[INC]] = add nsw i32 [[I]], 1<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[CMP2:%.*]] = icmp slt i32 [[I]],
                                    2096<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[CMP2]], label [[FOR_COND]],
                                    label<br>
                                    >>>> [[IF_END]],
                                    [[LOOP7:!llvm.loop !.*]]<br>
                                    >>>>    ; CHECK:      
                                    if.end:<br>
                                    >>>> -; CHECK-NEXT:   
                                    [[REC_LCSSA:%.*]] = phi i16 [
                                    [[SCALAR_RECUR]],<br>
                                    >>>> [[FOR_BODY]] ], [
                                    [[SCALAR_RECUR]], [[FOR_COND]] ]<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[REC_LCSSA:%.*]] = phi i16 [
                                    [[SCALAR_RECUR]],<br>
                                    >>>> [[FOR_BODY]] ], [
                                    [[SCALAR_RECUR]], [[FOR_COND]] ], [<br>
                                    >>>>
                                    [[VECTOR_RECUR_EXTRACT_FOR_PHI]],
                                    [[MIDDLE_BLOCK]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    ret i16 [[REC_LCSSA]]<br>
                                    >>>>    ;<br>
                                    >>>>    entry:<br>
                                    >>>> @@ -557,9 +558,10
                                    @@ define i16 @multiple_exit2(i16*
                                    %p, i32 %n) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP15:%.*]] = icmp eq i32
                                    [[INDEX_NEXT]],<br>
                                    >>>> [[N_VEC]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[TMP15]], label
                                    [[MIDDLE_BLOCK:%.*]],<br>
                                    >>>> label
                                    [[VECTOR_BODY]], [[LOOP8:!llvm.loop
                                    !.*]]<br>
                                    >>>>    ; CHECK:      
                                    middle.block:<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[CMP_N:%.*]] = icmp eq i32
                                    [[TMP2]], [[N_VEC]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[VECTOR_RECUR_EXTRACT:%.*]] =
                                    extractelement<br>
                                    >>>> <4 x i16>
                                    [[WIDE_LOAD]], i32 3<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[VECTOR_RECUR_EXTRACT_FOR_PHI:%.*]]
                                    =<br>
                                    >>>> extractelement
                                    <4 x i16> [[WIDE_LOAD]], i32 2<br>
                                    >>>> -; CHECK-NEXT:   
                                    br label [[SCALAR_PH]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP_N]], label
                                    [[IF_END:%.*]], label<br>
                                    >>>> [[SCALAR_PH]]<br>
                                    >>>>    ; CHECK:       <a
                                      href="http://scalar.ph"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">scalar.ph</a>:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[SCALAR_RECUR_INIT:%.*]] = phi i16
                                    [ 0,<br>
                                    >>>> [[ENTRY:%.*]] ], [
                                    [[VECTOR_RECUR_EXTRACT]],
                                    [[MIDDLE_BLOCK]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_RESUME_VAL:%.*]] = phi i32 [
                                    [[N_VEC]],<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY]] ]<br>
                                    >>>> @@ -571,14 +573,14
                                    @@ define i16 @multiple_exit2(i16*
                                    %p, i32 %n) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[B:%.*]] = getelementptr inbounds
                                    i16, i16*<br>
                                    >>>> [[P]], i64
                                    [[IPROM]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[REC_NEXT]] = load i16, i16* [[B]],
                                    align 2<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[CMP:%.*]] = icmp slt i32 [[I]],
                                    [[N]]<br>
                                    >>>> -; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label<br>
                                    >>>> [[IF_END:%.*]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label [[IF_END]]<br>
                                    >>>>    ; CHECK:      
                                    for.body:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    store i16 [[SCALAR_RECUR]], i16*
                                    [[B]], align 4<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[INC]] = add nsw i32 [[I]], 1<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[CMP2:%.*]] = icmp slt i32 [[I]],
                                    2096<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[CMP2]], label [[FOR_COND]],
                                    label<br>
                                    >>>> [[IF_END]],
                                    [[LOOP9:!llvm.loop !.*]]<br>
                                    >>>>    ; CHECK:      
                                    if.end:<br>
                                    >>>> -; CHECK-NEXT:   
                                    [[REC_LCSSA:%.*]] = phi i16 [
                                    [[SCALAR_RECUR]],<br>
                                    >>>> [[FOR_COND]] ], [
                                    10, [[FOR_BODY]] ]<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[REC_LCSSA:%.*]] = phi i16 [
                                    [[SCALAR_RECUR]],<br>
                                    >>>> [[FOR_COND]] ], [
                                    10, [[FOR_BODY]] ], [<br>
                                    >>>>
                                    [[VECTOR_RECUR_EXTRACT_FOR_PHI]],
                                    [[MIDDLE_BLOCK]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    ret i16 [[REC_LCSSA]]<br>
                                    >>>>    ;<br>
                                    >>>>    entry:<br>
                                    >>>><br>
                                    >>>> diff  --git<br>
                                    >>>>
                                    a/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
                                    >>>>
                                    b/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
                                    >>>> index
                                    f0ba677348ab..0d4bdf0ecac3 100644<br>
                                    >>>> ---
                                    a/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
                                    >>>> +++
                                    b/llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll<br>
                                    >>>> @@ -447,7 +447,7 @@
                                    define void
                                    @even_load_static_tc(i32* noalias<br>
                                    >>>> nocapture readonly
                                    %A, i32* noalia<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP6:%.*]] = icmp eq i64
                                    [[INDEX_NEXT]], 508<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[TMP6]], label
                                    [[MIDDLE_BLOCK:%.*]],<br>
                                    >>>> label
                                    [[VECTOR_BODY]], [[LOOP12:!llvm.loop
                                    !.*]]<br>
                                    >>>>    ; CHECK:      
                                    middle.block:<br>
                                    >>>> -; CHECK-NEXT:   
                                    br label [[SCALAR_PH]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 false, label
                                    [[FOR_COND_CLEANUP:%.*]],<br>
                                    >>>> label [[SCALAR_PH]]<br>
                                    >>>>    ; CHECK:       <a
                                      href="http://scalar.ph"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">scalar.ph</a>:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_RESUME_VAL:%.*]] = phi i64 [
                                    1016,<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br label [[FOR_BODY:%.*]]<br>
                                    >>>> @@ -463,7 +463,7 @@
                                    define void
                                    @even_load_static_tc(i32* noalias<br>
                                    >>>> nocapture readonly
                                    %A, i32* noalia<br>
                                    >>>>    ; CHECK-NEXT:   
                                    store i32 [[MUL]], i32*
                                    [[ARRAYIDX2]], align 4<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[INDVARS_IV_NEXT]] = add nuw nsw
                                    i64<br>
                                    >>>> [[INDVARS_IV]], 2<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[CMP:%.*]] = icmp ult i64
                                    [[INDVARS_IV]], 1022<br>
                                    >>>> -; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label<br>
                                    >>>>
                                    [[FOR_COND_CLEANUP:%.*]],
                                    [[LOOP13:!llvm.loop !.*]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label<br>
                                    >>>>
                                    [[FOR_COND_CLEANUP]],
                                    [[LOOP13:!llvm.loop !.*]]<br>
                                    >>>>    ;<br>
                                    >>>>    entry:<br>
                                    >>>>      br label
                                    %for.body<br>
                                    >>>> @@ -528,7 +528,7 @@
                                    define void
                                    @even_load_dynamic_tc(i32* noalias<br>
                                    >>>> nocapture readonly
                                    %A, i32* noali<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP12:%.*]] = icmp eq i64
                                    [[INDEX_NEXT]],<br>
                                    >>>> [[N_VEC]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[TMP12]], label
                                    [[MIDDLE_BLOCK:%.*]],<br>
                                    >>>> label
                                    [[VECTOR_BODY]], [[LOOP14:!llvm.loop
                                    !.*]]<br>
                                    >>>>    ; CHECK:      
                                    middle.block:<br>
                                    >>>> -; CHECK-NEXT:   
                                    br label [[SCALAR_PH]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 false, label
                                    [[FOR_COND_CLEANUP:%.*]],<br>
                                    >>>> label [[SCALAR_PH]]<br>
                                    >>>>    ; CHECK:       <a
                                      href="http://scalar.ph"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">scalar.ph</a>:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_RESUME_VAL:%.*]] = phi i64 [
                                    [[IND_END]],<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br label [[FOR_BODY:%.*]]<br>
                                    >>>> @@ -544,7 +544,7 @@
                                    define void
                                    @even_load_dynamic_tc(i32* noalias<br>
                                    >>>> nocapture readonly
                                    %A, i32* noali<br>
                                    >>>>    ; CHECK-NEXT:   
                                    store i32 [[MUL]], i32*
                                    [[ARRAYIDX2]], align 4<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[INDVARS_IV_NEXT]] = add nuw nsw
                                    i64<br>
                                    >>>> [[INDVARS_IV]], 2<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[CMP:%.*]] = icmp ult i64
                                    [[INDVARS_IV_NEXT]],<br>
                                    >>>> [[N]]<br>
                                    >>>> -; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label<br>
                                    >>>>
                                    [[FOR_COND_CLEANUP:%.*]],
                                    [[LOOP15:!llvm.loop !.*]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label<br>
                                    >>>>
                                    [[FOR_COND_CLEANUP]],
                                    [[LOOP15:!llvm.loop !.*]]<br>
                                    >>>>    ;<br>
                                    >>>>    entry:<br>
                                    >>>>      br label
                                    %for.body<br>
                                    >>>> @@ -973,7 +973,7 @@
                                    define void @PR27626_0(%pair.i32
                                    *%p, i32 %z,<br>
                                    >>>> i64 %n) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP19:%.*]] = icmp eq i64
                                    [[INDEX_NEXT]],<br>
                                    >>>> [[N_VEC]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[TMP19]], label
                                    [[MIDDLE_BLOCK:%.*]],<br>
                                    >>>> label
                                    [[VECTOR_BODY]], [[LOOP24:!llvm.loop
                                    !.*]]<br>
                                    >>>>    ; CHECK:      
                                    middle.block:<br>
                                    >>>> -; CHECK-NEXT:   
                                    br label [[SCALAR_PH]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 false, label [[FOR_END:%.*]],
                                    label<br>
                                    >>>> [[SCALAR_PH]]<br>
                                    >>>>    ; CHECK:       <a
                                      href="http://scalar.ph"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">scalar.ph</a>:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_RESUME_VAL:%.*]] = phi i64 [
                                    [[N_VEC]],<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br label [[FOR_BODY:%.*]]<br>
                                    >>>> @@ -985,7 +985,7 @@
                                    define void @PR27626_0(%pair.i32
                                    *%p, i32 %z,<br>
                                    >>>> i64 %n) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    store i32 [[Z]], i32* [[P_I_Y]],
                                    align 4<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[I_NEXT]] = add nuw nsw i64 [[I]],
                                    1<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[COND:%.*]] = icmp slt i64
                                    [[I_NEXT]], [[N]]<br>
                                    >>>> -; CHECK-NEXT:   
                                    br i1 [[COND]], label [[FOR_BODY]],
                                    label<br>
                                    >>>> [[FOR_END:%.*]],
                                    [[LOOP25:!llvm.loop !.*]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[COND]], label [[FOR_BODY]],
                                    label<br>
                                    >>>> [[FOR_END]],
                                    [[LOOP25:!llvm.loop !.*]]<br>
                                    >>>>    ; CHECK:      
                                    for.end:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    ret void<br>
                                    >>>>    ;<br>
                                    >>>> @@ -1066,7 +1066,7
                                    @@ define i32 @PR27626_1(%pair.i32
                                    *%p, i64 %n) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[RDX_SHUF3:%.*]] = shufflevector
                                    <4 x i32><br>
                                    >>>> [[BIN_RDX]], <4
                                    x i32> poison, <4 x i32>
                                    <i32 1, i32 undef, i32<br>
                                    >>>> undef, i32
                                    undef><br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BIN_RDX4:%.*]] = add <4 x
                                    i32> [[BIN_RDX]],<br>
                                    >>>> [[RDX_SHUF3]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP19:%.*]] = extractelement <4
                                    x i32><br>
                                    >>>> [[BIN_RDX4]], i32 0<br>
                                    >>>> -; CHECK-NEXT:   
                                    br label [[SCALAR_PH]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 false, label [[FOR_END:%.*]],
                                    label<br>
                                    >>>> [[SCALAR_PH]]<br>
                                    >>>>    ; CHECK:       <a
                                      href="http://scalar.ph"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">scalar.ph</a>:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_RESUME_VAL:%.*]] = phi i64 [
                                    [[N_VEC]],<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_MERGE_RDX:%.*]] = phi i32 [
                                    [[TMP19]],<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY]] ]<br>
                                    >>>> @@ -1081,9 +1081,10
                                    @@ define i32 @PR27626_1(%pair.i32
                                    *%p, i64 %n) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP21]] = add nsw i32 [[TMP20]],
                                    [[S]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[I_NEXT]] = add nuw nsw i64 [[I]],
                                    1<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[COND:%.*]] = icmp slt i64
                                    [[I_NEXT]], [[N]]<br>
                                    >>>> -; CHECK-NEXT:   
                                    br i1 [[COND]], label [[FOR_BODY]],
                                    label<br>
                                    >>>> [[FOR_END:%.*]],
                                    [[LOOP27:!llvm.loop !.*]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[COND]], label [[FOR_BODY]],
                                    label<br>
                                    >>>> [[FOR_END]],
                                    [[LOOP27:!llvm.loop !.*]]<br>
                                    >>>>    ; CHECK:      
                                    for.end:<br>
                                    >>>> -; CHECK-NEXT:   
                                    ret i32 [[TMP21]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[TMP22:%.*]] = phi i32 [ [[TMP21]],
                                    [[FOR_BODY]]<br>
                                    >>>> ], [ [[TMP19]],
                                    [[MIDDLE_BLOCK]] ]<br>
                                    >>>> +; CHECK-NEXT:   
                                    ret i32 [[TMP22]]<br>
                                    >>>>    ;<br>
                                    >>>>    entry:<br>
                                    >>>>      br label
                                    %for.body<br>
                                    >>>> @@ -1162,7 +1163,7
                                    @@ define void @PR27626_2(%pair.i32
                                    *%p, i64 %n,<br>
                                    >>>> i32 %z) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP20:%.*]] = icmp eq i64
                                    [[INDEX_NEXT]],<br>
                                    >>>> [[N_VEC]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[TMP20]], label
                                    [[MIDDLE_BLOCK:%.*]],<br>
                                    >>>> label
                                    [[VECTOR_BODY]], [[LOOP28:!llvm.loop
                                    !.*]]<br>
                                    >>>>    ; CHECK:      
                                    middle.block:<br>
                                    >>>> -; CHECK-NEXT:   
                                    br label [[SCALAR_PH]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 false, label [[FOR_END:%.*]],
                                    label<br>
                                    >>>> [[SCALAR_PH]]<br>
                                    >>>>    ; CHECK:       <a
                                      href="http://scalar.ph"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">scalar.ph</a>:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_RESUME_VAL:%.*]] = phi i64 [
                                    [[N_VEC]],<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br label [[FOR_BODY:%.*]]<br>
                                    >>>> @@ -1176,7 +1177,7
                                    @@ define void @PR27626_2(%pair.i32
                                    *%p, i64 %n,<br>
                                    >>>> i32 %z) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    store i32 [[TMP21]], i32* [[P_I_Y]],
                                    align 4<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[I_NEXT]] = add nuw nsw i64 [[I]],
                                    1<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[COND:%.*]] = icmp slt i64
                                    [[I_NEXT]], [[N]]<br>
                                    >>>> -; CHECK-NEXT:   
                                    br i1 [[COND]], label [[FOR_BODY]],
                                    label<br>
                                    >>>> [[FOR_END:%.*]],
                                    [[LOOP29:!llvm.loop !.*]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[COND]], label [[FOR_BODY]],
                                    label<br>
                                    >>>> [[FOR_END]],
                                    [[LOOP29:!llvm.loop !.*]]<br>
                                    >>>>    ; CHECK:      
                                    for.end:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    ret void<br>
                                    >>>>    ;<br>
                                    >>>> @@ -1263,7 +1264,7
                                    @@ define i32 @PR27626_3(%pair.i32
                                    *%p, i64 %n,<br>
                                    >>>> i32 %z) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[RDX_SHUF3:%.*]] = shufflevector
                                    <4 x i32><br>
                                    >>>> [[BIN_RDX]], <4
                                    x i32> poison, <4 x i32>
                                    <i32 1, i32 undef, i32<br>
                                    >>>> undef, i32
                                    undef><br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BIN_RDX4:%.*]] = add <4 x
                                    i32> [[BIN_RDX]],<br>
                                    >>>> [[RDX_SHUF3]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP22:%.*]] = extractelement <4
                                    x i32><br>
                                    >>>> [[BIN_RDX4]], i32 0<br>
                                    >>>> -; CHECK-NEXT:   
                                    br label [[SCALAR_PH]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 false, label [[FOR_END:%.*]],
                                    label<br>
                                    >>>> [[SCALAR_PH]]<br>
                                    >>>>    ; CHECK:       <a
                                      href="http://scalar.ph"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">scalar.ph</a>:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_RESUME_VAL:%.*]] = phi i64 [
                                    [[N_VEC]],<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_MERGE_RDX:%.*]] = phi i32 [
                                    [[TMP22]],<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY]] ]<br>
                                    >>>> @@ -1281,9 +1282,10
                                    @@ define i32 @PR27626_3(%pair.i32
                                    *%p, i64 %n,<br>
                                    >>>> i32 %z) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP25]] = add nsw i32 [[TMP24]],
                                    [[S]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[I_NEXT]] = add nuw nsw i64 [[I]],
                                    1<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[COND:%.*]] = icmp slt i64
                                    [[I_NEXT]], [[N]]<br>
                                    >>>> -; CHECK-NEXT:   
                                    br i1 [[COND]], label [[FOR_BODY]],
                                    label<br>
                                    >>>> [[FOR_END:%.*]],
                                    [[LOOP31:!llvm.loop !.*]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[COND]], label [[FOR_BODY]],
                                    label<br>
                                    >>>> [[FOR_END]],
                                    [[LOOP31:!llvm.loop !.*]]<br>
                                    >>>>    ; CHECK:      
                                    for.end:<br>
                                    >>>> -; CHECK-NEXT:   
                                    ret i32 [[TMP25]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[TMP26:%.*]] = phi i32 [ [[TMP25]],
                                    [[FOR_BODY]]<br>
                                    >>>> ], [ [[TMP22]],
                                    [[MIDDLE_BLOCK]] ]<br>
                                    >>>> +; CHECK-NEXT:   
                                    ret i32 [[TMP26]]<br>
                                    >>>>    ;<br>
                                    >>>>    entry:<br>
                                    >>>>      br label
                                    %for.body<br>
                                    >>>><br>
                                    >>>> diff  --git
                                    a/llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
                                    >>>>
                                    b/llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
                                    >>>> index
                                    f32002fae2b6..91780789088b 100644<br>
                                    >>>> ---
                                    a/llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
                                    >>>> +++
                                    b/llvm/test/Transforms/LoopVectorize/loop-form.ll<br>
                                    >>>> @@ -146,14 +146,15
                                    @@ define void @early_exit(i16* %p,
                                    i32 %n) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP10:%.*]] = icmp eq i32
                                    [[INDEX_NEXT]],<br>
                                    >>>> [[N_VEC]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[TMP10]], label
                                    [[MIDDLE_BLOCK:%.*]],<br>
                                    >>>> label
                                    [[VECTOR_BODY]], [[LOOP4:!llvm.loop
                                    !.*]]<br>
                                    >>>>    ; CHECK:      
                                    middle.block:<br>
                                    >>>> -; CHECK-NEXT:   
                                    br label [[SCALAR_PH]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[CMP_N:%.*]] = icmp eq i32
                                    [[TMP1]], [[N_VEC]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP_N]], label
                                    [[IF_END:%.*]], label<br>
                                    >>>> [[SCALAR_PH]]<br>
                                    >>>>    ; CHECK:       <a
                                      href="http://scalar.ph"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">scalar.ph</a>:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_RESUME_VAL:%.*]] = phi i32 [
                                    [[N_VEC]],<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br label [[FOR_COND:%.*]]<br>
                                    >>>>    ; CHECK:      
                                    for.cond:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[I:%.*]] = phi i32 [
                                    [[BC_RESUME_VAL]],<br>
                                    >>>> [[SCALAR_PH]] ], [
                                    [[INC:%.*]], [[FOR_BODY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[CMP:%.*]] = icmp slt i32 [[I]],
                                    [[N]]<br>
                                    >>>> -; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label<br>
                                    >>>> [[IF_END:%.*]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label [[IF_END]]<br>
                                    >>>>    ; CHECK:      
                                    for.body:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[IPROM:%.*]] = sext i32 [[I]] to
                                    i64<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[B:%.*]] = getelementptr inbounds
                                    i16, i16*<br>
                                    >>>> [[P]], i64
                                    [[IPROM]]<br>
                                    >>>> @@ -285,14 +286,15
                                    @@ define void
                                    @multiple_unique_exit(i16* %p,<br>
                                    >>>> i32 %n) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP11:%.*]] = icmp eq i32
                                    [[INDEX_NEXT]],<br>
                                    >>>> [[N_VEC]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[TMP11]], label
                                    [[MIDDLE_BLOCK:%.*]],<br>
                                    >>>> label
                                    [[VECTOR_BODY]], [[LOOP6:!llvm.loop
                                    !.*]]<br>
                                    >>>>    ; CHECK:      
                                    middle.block:<br>
                                    >>>> -; CHECK-NEXT:   
                                    br label [[SCALAR_PH]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[CMP_N:%.*]] = icmp eq i32
                                    [[TMP2]], [[N_VEC]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP_N]], label
                                    [[IF_END:%.*]], label<br>
                                    >>>> [[SCALAR_PH]]<br>
                                    >>>>    ; CHECK:       <a
                                      href="http://scalar.ph"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">scalar.ph</a>:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_RESUME_VAL:%.*]] = phi i32 [
                                    [[N_VEC]],<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br label [[FOR_COND:%.*]]<br>
                                    >>>>    ; CHECK:      
                                    for.cond:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[I:%.*]] = phi i32 [
                                    [[BC_RESUME_VAL]],<br>
                                    >>>> [[SCALAR_PH]] ], [
                                    [[INC:%.*]], [[FOR_BODY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[CMP:%.*]] = icmp slt i32 [[I]],
                                    [[N]]<br>
                                    >>>> -; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label<br>
                                    >>>> [[IF_END:%.*]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label [[IF_END]]<br>
                                    >>>>    ; CHECK:      
                                    for.body:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[IPROM:%.*]] = sext i32 [[I]] to
                                    i64<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[B:%.*]] = getelementptr inbounds
                                    i16, i16*<br>
                                    >>>> [[P]], i64
                                    [[IPROM]]<br>
                                    >>>> @@ -372,14 +374,17
                                    @@ define i32
                                    @multiple_unique_exit2(i16* %p,<br>
                                    >>>> i32 %n) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP11:%.*]] = icmp eq i32
                                    [[INDEX_NEXT]],<br>
                                    >>>> [[N_VEC]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[TMP11]], label
                                    [[MIDDLE_BLOCK:%.*]],<br>
                                    >>>> label
                                    [[VECTOR_BODY]], [[LOOP8:!llvm.loop
                                    !.*]]<br>
                                    >>>>    ; CHECK:      
                                    middle.block:<br>
                                    >>>> -; CHECK-NEXT:   
                                    br label [[SCALAR_PH]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[CMP_N:%.*]] = icmp eq i32
                                    [[TMP2]], [[N_VEC]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[IND_ESCAPE:%.*]] = sub i32
                                    [[N_VEC]], 1<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[IND_ESCAPE1:%.*]] = sub i32
                                    [[N_VEC]], 1<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP_N]], label
                                    [[IF_END:%.*]], label<br>
                                    >>>> [[SCALAR_PH]]<br>
                                    >>>>    ; CHECK:       <a
                                      href="http://scalar.ph"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">scalar.ph</a>:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_RESUME_VAL:%.*]] = phi i32 [
                                    [[N_VEC]],<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br label [[FOR_COND:%.*]]<br>
                                    >>>>    ; CHECK:      
                                    for.cond:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[I:%.*]] = phi i32 [
                                    [[BC_RESUME_VAL]],<br>
                                    >>>> [[SCALAR_PH]] ], [
                                    [[INC:%.*]], [[FOR_BODY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[CMP:%.*]] = icmp slt i32 [[I]],
                                    [[N]]<br>
                                    >>>> -; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label<br>
                                    >>>> [[IF_END:%.*]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label [[IF_END]]<br>
                                    >>>>    ; CHECK:      
                                    for.body:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[IPROM:%.*]] = sext i32 [[I]] to
                                    i64<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[B:%.*]] = getelementptr inbounds
                                    i16, i16*<br>
                                    >>>> [[P]], i64
                                    [[IPROM]]<br>
                                    >>>> @@ -388,7 +393,7 @@
                                    define i32
                                    @multiple_unique_exit2(i16* %p, i32<br>
                                    >>>> %n) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[CMP2:%.*]] = icmp slt i32 [[I]],
                                    2096<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[CMP2]], label [[FOR_COND]],
                                    label<br>
                                    >>>> [[IF_END]],
                                    [[LOOP9:!llvm.loop !.*]]<br>
                                    >>>>    ; CHECK:      
                                    if.end:<br>
                                    >>>> -; CHECK-NEXT:   
                                    [[I_LCSSA:%.*]] = phi i32 [ [[I]],
                                    [[FOR_BODY]]<br>
                                    >>>> ], [ [[I]],
                                    [[FOR_COND]] ]<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[I_LCSSA:%.*]] = phi i32 [ [[I]],
                                    [[FOR_BODY]]<br>
                                    >>>> ], [ [[I]],
                                    [[FOR_COND]] ], [ [[IND_ESCAPE1]],
                                    [[MIDDLE_BLOCK]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    ret i32 [[I_LCSSA]]<br>
                                    >>>>    ;<br>
                                    >>>>    ;
                                    TAILFOLD-LABEL:
                                    @multiple_unique_exit2(<br>
                                    >>>> @@ -461,14 +466,15
                                    @@ define i32
                                    @multiple_unique_exit3(i16* %p,<br>
                                    >>>> i32 %n) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP11:%.*]] = icmp eq i32
                                    [[INDEX_NEXT]],<br>
                                    >>>> [[N_VEC]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[TMP11]], label
                                    [[MIDDLE_BLOCK:%.*]],<br>
                                    >>>> label
                                    [[VECTOR_BODY]], [[LOOP10:!llvm.loop
                                    !.*]]<br>
                                    >>>>    ; CHECK:      
                                    middle.block:<br>
                                    >>>> -; CHECK-NEXT:   
                                    br label [[SCALAR_PH]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[CMP_N:%.*]] = icmp eq i32
                                    [[TMP2]], [[N_VEC]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP_N]], label
                                    [[IF_END:%.*]], label<br>
                                    >>>> [[SCALAR_PH]]<br>
                                    >>>>    ; CHECK:       <a
                                      href="http://scalar.ph"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">scalar.ph</a>:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_RESUME_VAL:%.*]] = phi i32 [
                                    [[N_VEC]],<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br label [[FOR_COND:%.*]]<br>
                                    >>>>    ; CHECK:      
                                    for.cond:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[I:%.*]] = phi i32 [
                                    [[BC_RESUME_VAL]],<br>
                                    >>>> [[SCALAR_PH]] ], [
                                    [[INC:%.*]], [[FOR_BODY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[CMP:%.*]] = icmp slt i32 [[I]],
                                    [[N]]<br>
                                    >>>> -; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label<br>
                                    >>>> [[IF_END:%.*]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP]], label [[FOR_BODY]],
                                    label [[IF_END]]<br>
                                    >>>>    ; CHECK:      
                                    for.body:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[IPROM:%.*]] = sext i32 [[I]] to
                                    i64<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[B:%.*]] = getelementptr inbounds
                                    i16, i16*<br>
                                    >>>> [[P]], i64
                                    [[IPROM]]<br>
                                    >>>> @@ -477,7 +483,7 @@
                                    define i32
                                    @multiple_unique_exit3(i16* %p, i32<br>
                                    >>>> %n) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[CMP2:%.*]] = icmp slt i32 [[I]],
                                    2096<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[CMP2]], label [[FOR_COND]],
                                    label<br>
                                    >>>> [[IF_END]],
                                    [[LOOP11:!llvm.loop !.*]]<br>
                                    >>>>    ; CHECK:      
                                    if.end:<br>
                                    >>>> -; CHECK-NEXT:   
                                    [[EXIT:%.*]] = phi i32 [ 0,
                                    [[FOR_COND]] ], [ 1,<br>
                                    >>>> [[FOR_BODY]] ]<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[EXIT:%.*]] = phi i32 [ 0,
                                    [[FOR_COND]] ], [ 1,<br>
                                    >>>> [[FOR_BODY]] ], [
                                    0, [[MIDDLE_BLOCK]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    ret i32 [[EXIT]]<br>
                                    >>>>    ;<br>
                                    >>>>    ;
                                    TAILFOLD-LABEL:
                                    @multiple_unique_exit3(<br>
                                    >>>> @@ -994,7 +1000,8
                                    @@ define void
                                    @scalar_predication(float* %addr) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP10:%.*]] = icmp eq i64
                                    [[INDEX_NEXT]], 200<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[TMP10]], label
                                    [[MIDDLE_BLOCK:%.*]],<br>
                                    >>>> label
                                    [[VECTOR_BODY]], [[LOOP12:!llvm.loop
                                    !.*]]<br>
                                    >>>>    ; CHECK:      
                                    middle.block:<br>
                                    >>>> -; CHECK-NEXT:   
                                    br label [[SCALAR_PH]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[CMP_N:%.*]] = icmp eq i64 201, 200<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP_N]], label [[EXIT:%.*]],
                                    label<br>
                                    >>>> [[SCALAR_PH]]<br>
                                    >>>>    ; CHECK:       <a
                                      href="http://scalar.ph"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">scalar.ph</a>:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_RESUME_VAL:%.*]] = phi i64 [
                                    200,<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br label [[LOOP_HEADER:%.*]]<br>
                                    >>>> @@ -1002,7 +1009,7
                                    @@ define void
                                    @scalar_predication(float* %addr) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[IV:%.*]] = phi i64 [
                                    [[BC_RESUME_VAL]],<br>
                                    >>>> [[SCALAR_PH]] ], [
                                    [[IV_NEXT:%.*]], [[LOOP_LATCH:%.*]]
                                    ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[GEP:%.*]] = getelementptr float,
                                    float*<br>
                                    >>>> [[ADDR]], i64
                                    [[IV]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[EXITCOND_NOT:%.*]] = icmp eq i64
                                    [[IV]], 200<br>
                                    >>>> -; CHECK-NEXT:   
                                    br i1 [[EXITCOND_NOT]], label
                                    [[EXIT:%.*]], label<br>
                                    >>>> [[LOOP_BODY:%.*]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[EXITCOND_NOT]], label
                                    [[EXIT]], label<br>
                                    >>>> [[LOOP_BODY:%.*]]<br>
                                    >>>>    ; CHECK:      
                                    loop.body:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP11:%.*]] = load float, float*
                                    [[GEP]],<br>
                                    >>>> align 4<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[PRED:%.*]] = fcmp oeq float
                                    [[TMP11]],<br>
                                    >>>> 0.000000e+00<br>
                                    >>>> @@ -1088,7 +1095,8
                                    @@ define i32 @me_reduction(i32*
                                    %addr) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[RDX_SHUF:%.*]] = shufflevector
                                    <2 x i32><br>
                                    >>>> [[TMP5]], <2 x
                                    i32> poison, <2 x i32>
                                    <i32 1, i32 undef><br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BIN_RDX:%.*]] = add <2 x
                                    i32> [[TMP5]],<br>
                                    >>>> [[RDX_SHUF]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP7:%.*]] = extractelement <2
                                    x i32><br>
                                    >>>> [[BIN_RDX]], i32 0<br>
                                    >>>> -; CHECK-NEXT:   
                                    br label [[SCALAR_PH]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[CMP_N:%.*]] = icmp eq i64 201, 200<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[CMP_N]], label [[EXIT:%.*]],
                                    label<br>
                                    >>>> [[SCALAR_PH]]<br>
                                    >>>>    ; CHECK:       <a
                                      href="http://scalar.ph"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">scalar.ph</a>:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_RESUME_VAL:%.*]] = phi i64 [
                                    200,<br>
                                    >>>> [[MIDDLE_BLOCK]] ],
                                    [ 0, [[ENTRY:%.*]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[BC_MERGE_RDX:%.*]] = phi i32 [ 0,
                                    [[ENTRY]]<br>
                                    >>>> ], [ [[TMP7]],
                                    [[MIDDLE_BLOCK]] ]<br>
                                    >>>> @@ -1098,7 +1106,7
                                    @@ define i32 @me_reduction(i32*
                                    %addr) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[ACCUM:%.*]] = phi i32 [
                                    [[BC_MERGE_RDX]],<br>
                                    >>>> [[SCALAR_PH]] ], [
                                    [[ACCUM_NEXT:%.*]], [[LOOP_LATCH]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[GEP:%.*]] = getelementptr i32,
                                    i32* [[ADDR]],<br>
                                    >>>> i64 [[IV]]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[EXITCOND_NOT:%.*]] = icmp eq i64
                                    [[IV]], 200<br>
                                    >>>> -; CHECK-NEXT:   
                                    br i1 [[EXITCOND_NOT]], label
                                    [[EXIT:%.*]], label<br>
                                    >>>> [[LOOP_LATCH]]<br>
                                    >>>> +; CHECK-NEXT:   
                                    br i1 [[EXITCOND_NOT]], label
                                    [[EXIT]], label<br>
                                    >>>> [[LOOP_LATCH]]<br>
                                    >>>>    ; CHECK:      
                                    loop.latch:<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[TMP8:%.*]] = load i32, i32*
                                    [[GEP]], align 4<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[ACCUM_NEXT]] = add i32 [[ACCUM]],
                                    [[TMP8]]<br>
                                    >>>> @@ -1106,7 +1114,7
                                    @@ define i32 @me_reduction(i32*
                                    %addr) {<br>
                                    >>>>    ; CHECK-NEXT:   
                                    [[EXITCOND2_NOT:%.*]] = icmp eq i64
                                    [[IV]], 400<br>
                                    >>>>    ; CHECK-NEXT:   
                                    br i1 [[EXITCOND2_NOT]], label
                                    [[EXIT]], label<br>
                                    >>>> [[LOOP_HEADER]],
                                    [[LOOP15:!llvm.loop !.*]]<br>
                                    >>>>    ; CHECK:      
                                    exit:<br>
                                    >>>> -; CHECK-NEXT:   
                                    [[LCSSA:%.*]] = phi i32 [ 0,
                                    [[LOOP_HEADER]] ], [<br>
                                    >>>> [[ACCUM_NEXT]],
                                    [[LOOP_LATCH]] ]<br>
                                    >>>> +; CHECK-NEXT:   
                                    [[LCSSA:%.*]] = phi i32 [ 0,
                                    [[LOOP_HEADER]] ], [<br>
                                    >>>> [[ACCUM_NEXT]],
                                    [[LOOP_LATCH]] ], [ [[TMP7]],
                                    [[MIDDLE_BLOCK]] ]<br>
                                    >>>>    ; CHECK-NEXT:   
                                    ret i32 [[LCSSA]]<br>
                                    >>>>    ;<br>
                                    >>>>    ;
                                    TAILFOLD-LABEL: @me_reduction(<br>
                                    >>>><br>
                                    >>>><br>
                                    >>>><br>
                                    >>>>
                                    _______________________________________________<br>
                                    >>>> llvm-commits
                                    mailing list<br>
                                    >>>> <a
                                      href="mailto:llvm-commits@lists.llvm.org"
                                      moz-do-not-send="true"
                                      target="_blank">llvm-commits@lists.llvm.org</a><br>
                                    >>>> <a
                                      href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
                                    >>>
                                    _______________________________________________<br>
                                    >>> llvm-commits mailing
                                    list<br>
                                    >>> <a
                                      href="mailto:llvm-commits@lists.llvm.org"
                                      moz-do-not-send="true"
                                      target="_blank">llvm-commits@lists.llvm.org</a><br>
                                    >>> <a
                                      href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
                                    >
                                    _______________________________________________<br>
                                    > llvm-commits mailing list<br>
                                    > <a
                                      href="mailto:llvm-commits@lists.llvm.org"
                                      moz-do-not-send="true"
                                      target="_blank">llvm-commits@lists.llvm.org</a><br>
                                    > <a
                                      href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits"
                                      moz-do-not-send="true"
                                      rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a></blockquote>
                                </div>
                              </div>
                            </blockquote>
                          </blockquote>
                          <div dir="ltr"> </div>
                        </div>
                      </blockquote>
                    </blockquote>
                    <div dir="ltr"> </div>
                  </div>
                </blockquote>
              </blockquote>
              <div dir="ltr"> </div>
            </div>
             
            <div id="MIMEAttachInfoDiv" style="display:none"
              title="octet-stream|AfterLoopVec.out"> </div>
            <div id="MIMEAttachInfoDiv" style="display:none"
              title="octet-stream|BeforeLoopVec.out"> </div>
            <div><font size="2" face="Default Monospace,Courier
                New,Courier,monospace">_______________________________________________<br>
                llvm-commits mailing list<br>
                <a class="moz-txt-link-abbreviated"
                  href="mailto:llvm-commits@lists.llvm.org"
                  moz-do-not-send="true">llvm-commits@lists.llvm.org</a><br>
                <a
                  href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits"
                  target="_blank" moz-do-not-send="true">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a> </font></div>
          </blockquote>
          <div dir="ltr"> </div>
        </div>
        <br>
      </blockquote>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <pre class="moz-quote-pre" wrap="">_______________________________________________
llvm-commits mailing list
<a class="moz-txt-link-abbreviated" href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a>
<a class="moz-txt-link-freetext" href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a>
</pre>
    </blockquote>
  </body>
</html>