[llvm] f1c5f78 - [LoopIdiom] Support 'arithmetic right-shift until zero' idiom
Craig Topper via llvm-commits
llvm-commits at lists.llvm.org
Tue May 25 09:56:00 PDT 2021
Doesn't the old code call isKnownNonNegative for Ashr?
~Craig
On Tue, May 25, 2021 at 6:04 AM Roman Lebedev via llvm-commits <
llvm-commits at lists.llvm.org> wrote:
> On Tue, May 25, 2021 at 3:44 PM Nikita Popov <nikita.ppv at gmail.com> wrote:
> >
> > On Tue, May 25, 2021 at 1:31 PM Roman Lebedev via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
> >>
> >>
> >> Author: Roman Lebedev
> >> Date: 2021-05-25T14:30:49+03:00
> >> New Revision: f1c5f78d3813584f7796f8b84b92fa0725964c17
> >>
> >> URL:
> https://github.com/llvm/llvm-project/commit/f1c5f78d3813584f7796f8b84b92fa0725964c17
> >> DIFF:
> https://github.com/llvm/llvm-project/commit/f1c5f78d3813584f7796f8b84b92fa0725964c17.diff
> >>
> >> LOG: [LoopIdiom] Support 'arithmetic right-shift until zero' idiom
> >>
> >> This adds support for the "count active bits" pattern, i.e.:
> >> ```
> >> int countActiveBits(signed val) {
> >> int cnt = 0;
> >> for( ; (val >> cnt) != 0; ++cnt)
> >> ;
> >> return cnt;
> >> }
> >> ```
> >> but a somewhat more general one:
> >> ```
> >> int countActiveBits(signed val, int start, int off) {
> >> int cnt;
> >> for (cnt = start; val >> (cnt + off); cnt++)
> >> ;
> >> return cnt;
> >> }
> >> ```
> >>
> >> This directly matches the existing 'logical right-shift until zero'
> idiom.
> >> alive2 is happy with all the tests there.
> >>
> >> Note that, again, much like with the original unsigned case,
> >> we don't require the `val != 0` guard.
> >>
> >> The old `detectShiftUntilZeroIdiom()` already supports this pattern,
> >> the idea here is that the `val` must be positive (have at least one
> >> leading zero), because otherwise the loop is non-terminating,
> >> but since it is not `while(1)`, that would have been UB.
> >
> >
> > Just to double check, are you making a "must progress" assumption here
> (which nowadays is illegal without the presence of the attribute/metadata)?
> I'm missing wider context and just going by this one comment, so I'm
> probably misunderstanding.
> That is pretty much spot on. I'm making an assumption that
> either the other/older preexisting variant of this pattern[1]
> is already broken, and they both need fixing, or this is fine.
>
> In the former case, i'm waiting for alive2 to start complaining about it.
> I'm not really sure we can just fix such cases (by actually checking
> for said metadata), because i'm not really sure how well we preserve it.
>
> [1]
> https://github.com/llvm/llvm-project/blob/8f4db14d1c8f12cade77873f8815497026d21f95/llvm/lib/Transforms/Scalar/LoopIdiomRecognize.cpp#L1606
>
>
> > Regards,
> > Nikita
> Roman
>
> >>
> >> Added:
> >>
> >>
> >> Modified:
> >> llvm/lib/Transforms/Scalar/LoopIdiomRecognize.cpp
> >>
> llvm/test/Transforms/LoopIdiom/X86/arithmetic-right-shift-until-zero.ll
> >>
> >> Removed:
> >>
> >>
> >>
> >>
> ################################################################################
> >> diff --git a/llvm/lib/Transforms/Scalar/LoopIdiomRecognize.cpp
> b/llvm/lib/Transforms/Scalar/LoopIdiomRecognize.cpp
> >> index ae8ef00913c4..d02db245cb88 100644
> >> --- a/llvm/lib/Transforms/Scalar/LoopIdiomRecognize.cpp
> >> +++ b/llvm/lib/Transforms/Scalar/LoopIdiomRecognize.cpp
> >> @@ -2416,7 +2416,7 @@ bool
> LoopIdiomRecognize::recognizeShiftUntilBitTest() {
> >> /// loop:
> >> /// %iv = phi i8 [ %start, %entry ], [ %iv.next, %for.cond ]
> >> /// %nbits = add nsw i8 %iv, %extraoffset
> >> -/// %val.shifted = lshr i8 %val, %nbits
> >> +/// %val.shifted = [la]shr i8 %val, %nbits
> >> /// %val.shifted.iszero = icmp eq i8 %val.shifted, 0
> >> /// %iv.next = add i8 %iv, 1
> >> /// <...>
> >> @@ -2469,8 +2469,8 @@ static bool detectShiftUntilZeroIdiom(Loop
> *CurLoop, ScalarEvolution *SE,
> >>
> >> // Step 2: Check if the comparison's operand is in desirable form.
> >> // FIXME: Val could be a one-input PHI node, which we should look
> past.
> >> - if (!match(ValShifted, m_LShr(m_LoopInvariant(m_Value(Val), CurLoop),
> >> - m_Instruction(NBits)))) {
> >> + if (!match(ValShifted, m_Shr(m_LoopInvariant(m_Value(Val), CurLoop),
> >> + m_Instruction(NBits)))) {
> >> LLVM_DEBUG(dbgs() << DEBUG_TYPE " Bad comparisons value
> computation.\n");
> >> return false;
> >> }
> >> @@ -2541,7 +2541,7 @@ static bool detectShiftUntilZeroIdiom(Loop
> *CurLoop, ScalarEvolution *SE,
> >> /// loop:
> >> /// %iv = phi i8 [ %start, %entry ], [ %iv.next, %for.cond ]
> >> /// %nbits = add nsw i8 %iv, %extraoffset
> >> -/// %val.shifted = lshr i8 %val, %nbits
> >> +/// %val.shifted = [la]shr i8 %val, %nbits
> >> /// %val.shifted.iszero = icmp eq i8 %val.shifted, 0
> >> /// %iv.next = add i8 %iv, 1
> >> /// <...>
> >>
> >> diff --git
> a/llvm/test/Transforms/LoopIdiom/X86/arithmetic-right-shift-until-zero.ll
> b/llvm/test/Transforms/LoopIdiom/X86/arithmetic-right-shift-until-zero.ll
> >> index 5f25dfd68614..b1e14f9f299e 100644
> >> ---
> a/llvm/test/Transforms/LoopIdiom/X86/arithmetic-right-shift-until-zero.ll
> >> +++
> b/llvm/test/Transforms/LoopIdiom/X86/arithmetic-right-shift-until-zero.ll
> >> @@ -11,20 +11,29 @@ declare i8 @gen.i8()
> >> define i8 @p0(i8 %val, i8 %start, i8 %extraoffset) {
> >> ; CHECK-LABEL: @p0(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i8
> @llvm.ctlz.i8(i8 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i8 8,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[TMP0:%.*]] = sub i8 0, [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i8
> [[VAL_NUMACTIVEBITS]], [[TMP0]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i8 @llvm.smax.i8(i8
> [[VAL_NUMACTIVEBITS_OFFSET]], i8 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i8
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i8
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i8 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = add nsw i8 [[IV]], [[EXTRAOFFSET:%.*]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i8 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i8
> [[VAL_SHIFTED]], 0
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i8 [[IV]], 1
> >> -; CHECK-NEXT: call void @escape_inner(i8 [[IV]], i8 [[NBITS]], i8
> [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i8 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i8 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i8 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i8 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i8 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = add nsw i8 [[IV]], [[EXTRAOFFSET]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i8 [[VAL]], [[NBITS]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i8 [[IV]], 1
> >> +; CHECK-NEXT: call void @escape_inner(i8 [[IV]], i8 [[NBITS]], i8
> [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i8 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i8 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i8 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i8 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i8 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i8 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer(i8 [[IV_RES]], i8
> [[NBITS_RES]], i8 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i8
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i8 [[IV_RES]]
> >> @@ -59,20 +68,29 @@ end:
> >> define i8 @p1(i8 %val, i8 %start, i8 %extraoffset) {
> >> ; CHECK-LABEL: @p1(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i8
> @llvm.ctlz.i8(i8 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i8 8,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[TMP0:%.*]] = sub i8 0, [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i8
> [[VAL_NUMACTIVEBITS]], [[TMP0]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i8 @llvm.smax.i8(i8
> [[VAL_NUMACTIVEBITS_OFFSET]], i8 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i8
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i8
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i8 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = add nuw i8 [[IV]], [[EXTRAOFFSET:%.*]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i8 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i8
> [[VAL_SHIFTED]], 0
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i8 [[IV]], 1
> >> -; CHECK-NEXT: call void @escape_inner(i8 [[IV]], i8 [[NBITS]], i8
> [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i8 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i8 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i8 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i8 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i8 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = add nuw i8 [[IV]], [[EXTRAOFFSET]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i8 [[VAL]], [[NBITS]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i8 [[IV]], 1
> >> +; CHECK-NEXT: call void @escape_inner(i8 [[IV]], i8 [[NBITS]], i8
> [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i8 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i8 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i8 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i8 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i8 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i8 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer(i8 [[IV_RES]], i8
> [[NBITS_RES]], i8 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i8
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i8 [[IV_RES]]
> >> @@ -107,20 +125,28 @@ end:
> >> define i8 @p2(i8 %val, i8 %start, i8 %extraoffset) {
> >> ; CHECK-LABEL: @p2(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i8
> @llvm.ctlz.i8(i8 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i8 8,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i8
> [[VAL_NUMACTIVEBITS]], [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i8 @llvm.smax.i8(i8
> [[VAL_NUMACTIVEBITS_OFFSET]], i8 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i8
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i8
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i8 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = sub nsw i8 [[IV]], [[EXTRAOFFSET:%.*]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i8 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i8
> [[VAL_SHIFTED]], 0
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i8 [[IV]], 1
> >> -; CHECK-NEXT: call void @escape_inner(i8 [[IV]], i8 [[NBITS]], i8
> [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i8 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i8 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i8 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i8 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i8 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = sub nsw i8 [[IV]], [[EXTRAOFFSET]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i8 [[VAL]], [[NBITS]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i8 [[IV]], 1
> >> +; CHECK-NEXT: call void @escape_inner(i8 [[IV]], i8 [[NBITS]], i8
> [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i8 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i8 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i8 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i8 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i8 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i8 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer(i8 [[IV_RES]], i8
> [[NBITS_RES]], i8 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i8
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i8 [[IV_RES]]
> >> @@ -299,18 +325,26 @@ end:
> >> define i8 @p6(i8 %val, i8 %start) {
> >> ; CHECK-LABEL: @p6(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i8
> @llvm.ctlz.i8(i8 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i8 8,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nuw nsw i8
> [[VAL_NUMACTIVEBITS]], 0
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i8 @llvm.smax.i8(i8
> [[VAL_NUMACTIVEBITS_OFFSET]], i8 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nuw nsw i8
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i8
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i8 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i8 [[VAL:%.*]], [[IV]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i8
> [[VAL_SHIFTED]], 0
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i8 [[IV]], 1
> >> -; CHECK-NEXT: call void @escape_inner(i8 [[IV]], i8 [[IV]], i8
> [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i8 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i8 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i8 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i8 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i8 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i8 [[VAL]], [[IV]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i8 [[IV]], 1
> >> +; CHECK-NEXT: call void @escape_inner(i8 [[IV]], i8 [[IV]], i8
> [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i8 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i8 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i8 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i8 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i8 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer(i8 [[IV_RES]], i8 [[IV_RES]],
> i8 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i8 [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i8 [[IV_RES]]
> >> @@ -346,20 +380,29 @@ declare void @escape_outer.i7(i7, i7, i7, i1, i7)
> >> define i7 @p7(i7 %val, i7 %start, i7 %extraoffset) {
> >> ; CHECK-LABEL: @p7(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i7
> @llvm.ctlz.i7(i7 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i7 7,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[TMP0:%.*]] = sub i7 0, [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i7
> [[VAL_NUMACTIVEBITS]], [[TMP0]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i7 @llvm.smax.i7(i7
> [[VAL_NUMACTIVEBITS_OFFSET]], i7 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i7
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i7
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i7 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = add nsw i7 [[IV]], [[EXTRAOFFSET:%.*]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i7 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i7
> [[VAL_SHIFTED]], 0
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i7 [[IV]], 1
> >> -; CHECK-NEXT: call void @escape_inner.i7(i7 [[IV]], i7 [[NBITS]],
> i7 [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i7 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i7 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i7 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i7 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i7 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = add nsw i7 [[IV]], [[EXTRAOFFSET]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i7 [[VAL]], [[NBITS]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i7 [[IV]], 1
> >> +; CHECK-NEXT: call void @escape_inner.i7(i7 [[IV]], i7 [[NBITS]],
> i7 [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i7 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i7 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i7 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i7 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i7 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i7 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer.i7(i7 [[IV_RES]], i7
> [[NBITS_RES]], i7 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i7
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i7 [[IV_RES]]
> >> @@ -442,17 +485,27 @@ end:
> >> define i8 @t9(i8 %val, i8 %start, i8 %extraoffset) {
> >> ; CHECK-LABEL: @t9(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i8
> @llvm.ctlz.i8(i8 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i8 8,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[TMP0:%.*]] = sub i8 0, [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i8
> [[VAL_NUMACTIVEBITS]], [[TMP0]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i8 @llvm.smax.i8(i8
> [[VAL_NUMACTIVEBITS_OFFSET]], i8 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i8
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i8
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i8 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = add nsw i8 [[IV]], [[EXTRAOFFSET:%.*]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i8 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISNOTZERO:%.*]] = icmp ne i8
> [[VAL_SHIFTED]], 0
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i8 [[IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i8 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i8 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i8 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISNOTZERO:%.*]] = xor i1
> [[LOOP_IVCHECK]], true
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i8 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = add nsw i8 [[IV]], [[EXTRAOFFSET]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i8 [[VAL]], [[NBITS]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i8 [[IV]], 1
> >> ; CHECK-NEXT: call void @escape_inner(i8 [[IV]], i8 [[NBITS]], i8
> [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISNOTZERO]], i8 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISNOTZERO]], label [[LOOP]],
> label [[END:%.*]]
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i8 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i8 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i8 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i8 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_ISNOTZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISNOTZERO]], [[LOOP]] ]
> >> @@ -783,20 +836,29 @@ end:
> >> define i8 @t16(i8 %val, i8 %start, i8 %extraoffset) {
> >> ; CHECK-LABEL: @t16(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i8
> @llvm.ctlz.i8(i8 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i8 8,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[TMP0:%.*]] = sub i8 0, [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i8
> [[VAL_NUMACTIVEBITS]], [[TMP0]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i8 @llvm.smax.i8(i8
> [[VAL_NUMACTIVEBITS_OFFSET]], i8 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i8
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i8
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i8 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = add nsw i8 [[EXTRAOFFSET:%.*]], [[IV]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i8 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i8
> [[VAL_SHIFTED]], 0
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i8 [[IV]], 1
> >> -; CHECK-NEXT: call void @escape_inner(i8 [[IV]], i8 [[NBITS]], i8
> [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i8 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i8 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i8 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i8 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i8 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = add nsw i8 [[EXTRAOFFSET]], [[IV]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i8 [[VAL]], [[NBITS]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i8 [[IV]], 1
> >> +; CHECK-NEXT: call void @escape_inner(i8 [[IV]], i8 [[NBITS]], i8
> [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i8 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i8 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i8 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i8 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i8 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i8 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer(i8 [[IV_RES]], i8
> [[NBITS_RES]], i8 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i8
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i8 [[IV_RES]]
> >> @@ -1130,22 +1192,31 @@ end:
> >> define i8 @n23(i8 %val, i8 %start, i8 %extraoffset) {
> >> ; CHECK-LABEL: @n23(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i8
> @llvm.ctlz.i8(i8 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i8 8,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[TMP0:%.*]] = sub i8 0, [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i8
> [[VAL_NUMACTIVEBITS]], [[TMP0]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i8 @llvm.smax.i8(i8
> [[VAL_NUMACTIVEBITS_OFFSET]], i8 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i8
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i8
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i8 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = add nsw i8 [[IV]], [[EXTRAOFFSET:%.*]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i8 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i8
> [[VAL_SHIFTED]], 0
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i8 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i8 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i8 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i8 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = add nsw i8 [[IV]], [[EXTRAOFFSET]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i8 [[VAL]], [[NBITS]]
> >> ; CHECK-NEXT: [[NOT_IV_NEXT:%.*]] = add i8 [[IV]], 1
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i8 [[IV]], 1
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i8 [[IV]], 1
> >> ; CHECK-NEXT: [[ALSO_IV_NEXT:%.*]] = add i8 [[IV]], 1
> >> -; CHECK-NEXT: call void @escape_inner(i8 [[IV]], i8 [[NBITS]], i8
> [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i8 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: call void @escape_inner(i8 [[IV]], i8 [[NBITS]], i8
> [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i8 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i8 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i8 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i8 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i8 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i8 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer(i8 [[IV_RES]], i8
> [[NBITS_RES]], i8 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i8
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i8 [[IV_RES]]
> >> @@ -1238,18 +1309,26 @@ declare void @escape_outer.i3(i3, i3, i3, i1,
> i3)
> >> define i1 @t25_nooffset_i1(i1 %val, i1 %start) {
> >> ; CHECK-LABEL: @t25_nooffset_i1(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i1
> @llvm.ctlz.i1(i1 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i1 true,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nuw nsw i1
> [[VAL_NUMACTIVEBITS]], false
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i1 @llvm.smax.i1(i1
> [[VAL_NUMACTIVEBITS_OFFSET]], i1 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nuw nsw i1
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i1
> [[LOOP_BACKEDGETAKENCOUNT]], true
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i1 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i1 [[VAL:%.*]], [[IV]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i1
> [[VAL_SHIFTED]], false
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i1 [[IV]], true
> >> -; CHECK-NEXT: call void @escape_inner.i1(i1 [[IV]], i1 [[IV]], i1
> [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i1 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i1 [ false, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i1 [[LOOP_IV]], true
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i1 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i1 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i1 [[VAL]], [[IV]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i1 [[IV]], true
> >> +; CHECK-NEXT: call void @escape_inner.i1(i1 [[IV]], i1 [[IV]], i1
> [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i1 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i1 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i1 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i1 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i1 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer.i1(i1 [[IV_RES]], i1
> [[IV_RES]], i1 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i1
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i1 [[IV_RES]]
> >> @@ -1280,18 +1359,26 @@ end:
> >> define i2 @t26_nooffset_i2(i2 %val, i2 %start) {
> >> ; CHECK-LABEL: @t26_nooffset_i2(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i2
> @llvm.ctlz.i2(i2 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw i2 -2,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nuw nsw i2
> [[VAL_NUMACTIVEBITS]], 0
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i2 @llvm.smax.i2(i2
> [[VAL_NUMACTIVEBITS_OFFSET]], i2 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nuw nsw i2
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw i2
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i2 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i2 [[VAL:%.*]], [[IV]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i2
> [[VAL_SHIFTED]], 0
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i2 [[IV]], 1
> >> -; CHECK-NEXT: call void @escape_inner.i2(i2 [[IV]], i2 [[IV]], i2
> [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i2 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i2 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw i2 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i2 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i2 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i2 [[VAL]], [[IV]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i2 [[IV]], 1
> >> +; CHECK-NEXT: call void @escape_inner.i2(i2 [[IV]], i2 [[IV]], i2
> [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i2 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i2 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i2 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i2 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i2 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer.i2(i2 [[IV_RES]], i2
> [[IV_RES]], i2 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i2
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i2 [[IV_RES]]
> >> @@ -1322,18 +1409,26 @@ end:
> >> define i3 @t27_nooffset_i3(i3 %val, i3 %start) {
> >> ; CHECK-LABEL: @t27_nooffset_i3(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i3
> @llvm.ctlz.i3(i3 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i3 3,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nuw nsw i3
> [[VAL_NUMACTIVEBITS]], 0
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i3 @llvm.smax.i3(i3
> [[VAL_NUMACTIVEBITS_OFFSET]], i3 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nuw nsw i3
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i3
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i3 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i3 [[VAL:%.*]], [[IV]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i3
> [[VAL_SHIFTED]], 0
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i3 [[IV]], 1
> >> -; CHECK-NEXT: call void @escape_inner.i3(i3 [[IV]], i3 [[IV]], i3
> [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i3 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i3 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i3 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i3 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i3 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i3 [[VAL]], [[IV]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i3 [[IV]], 1
> >> +; CHECK-NEXT: call void @escape_inner.i3(i3 [[IV]], i3 [[IV]], i3
> [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i3 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i3 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i3 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i3 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i3 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer.i3(i3 [[IV_RES]], i3
> [[IV_RES]], i3 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i3
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i3 [[IV_RES]]
> >> @@ -1365,20 +1460,28 @@ end:
> >> define i1 @t28_addnsw_i1(i1 %val, i1 %start, i1 %extraoffset) {
> >> ; CHECK-LABEL: @t28_addnsw_i1(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i1
> @llvm.ctlz.i1(i1 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i1 true,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i1
> [[VAL_NUMACTIVEBITS]], [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i1 @llvm.smax.i1(i1
> [[VAL_NUMACTIVEBITS_OFFSET]], i1 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i1
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i1
> [[LOOP_BACKEDGETAKENCOUNT]], true
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i1 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = add nsw i1 [[IV]], [[EXTRAOFFSET:%.*]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i1 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i1
> [[VAL_SHIFTED]], false
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i1 [[IV]], true
> >> -; CHECK-NEXT: call void @escape_inner.i1(i1 [[IV]], i1 [[NBITS]],
> i1 [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i1 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i1 [ false, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i1 [[LOOP_IV]], true
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i1 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i1 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = add nsw i1 [[IV]], [[EXTRAOFFSET]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i1 [[VAL]], [[NBITS]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i1 [[IV]], true
> >> +; CHECK-NEXT: call void @escape_inner.i1(i1 [[IV]], i1 [[NBITS]],
> i1 [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i1 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i1 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i1 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i1 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i1 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i1 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer.i1(i1 [[IV_RES]], i1
> [[NBITS_RES]], i1 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i1
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i1 [[IV_RES]]
> >> @@ -1411,20 +1514,29 @@ end:
> >> define i2 @t29_addnsw_i2(i2 %val, i2 %start, i2 %extraoffset) {
> >> ; CHECK-LABEL: @t29_addnsw_i2(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i2
> @llvm.ctlz.i2(i2 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw i2 -2,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[TMP0:%.*]] = sub i2 0, [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i2
> [[VAL_NUMACTIVEBITS]], [[TMP0]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i2 @llvm.smax.i2(i2
> [[VAL_NUMACTIVEBITS_OFFSET]], i2 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i2
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw i2
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i2 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = add nsw i2 [[IV]], [[EXTRAOFFSET:%.*]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i2 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i2
> [[VAL_SHIFTED]], 0
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i2 [[IV]], 1
> >> -; CHECK-NEXT: call void @escape_inner.i2(i2 [[IV]], i2 [[NBITS]],
> i2 [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i2 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i2 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw i2 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i2 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i2 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = add nsw i2 [[IV]], [[EXTRAOFFSET]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i2 [[VAL]], [[NBITS]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i2 [[IV]], 1
> >> +; CHECK-NEXT: call void @escape_inner.i2(i2 [[IV]], i2 [[NBITS]],
> i2 [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i2 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i2 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i2 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i2 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i2 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i2 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer.i2(i2 [[IV_RES]], i2
> [[NBITS_RES]], i2 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i2
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i2 [[IV_RES]]
> >> @@ -1457,20 +1569,29 @@ end:
> >> define i3 @t30_addnsw_i3(i3 %val, i3 %start, i3 %extraoffset) {
> >> ; CHECK-LABEL: @t30_addnsw_i3(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i3
> @llvm.ctlz.i3(i3 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i3 3,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[TMP0:%.*]] = sub i3 0, [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i3
> [[VAL_NUMACTIVEBITS]], [[TMP0]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i3 @llvm.smax.i3(i3
> [[VAL_NUMACTIVEBITS_OFFSET]], i3 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i3
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i3
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i3 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = add nsw i3 [[IV]], [[EXTRAOFFSET:%.*]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i3 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i3
> [[VAL_SHIFTED]], 0
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i3 [[IV]], 1
> >> -; CHECK-NEXT: call void @escape_inner.i3(i3 [[IV]], i3 [[NBITS]],
> i3 [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i3 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i3 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i3 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i3 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i3 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = add nsw i3 [[IV]], [[EXTRAOFFSET]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i3 [[VAL]], [[NBITS]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i3 [[IV]], 1
> >> +; CHECK-NEXT: call void @escape_inner.i3(i3 [[IV]], i3 [[NBITS]],
> i3 [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i3 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i3 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i3 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i3 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i3 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i3 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer.i3(i3 [[IV_RES]], i3
> [[NBITS_RES]], i3 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i3
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i3 [[IV_RES]]
> >> @@ -1504,20 +1625,28 @@ end:
> >> define i1 @t31_addnuw_i1(i1 %val, i1 %start, i1 %extraoffset) {
> >> ; CHECK-LABEL: @t31_addnuw_i1(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i1
> @llvm.ctlz.i1(i1 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i1 true,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i1
> [[VAL_NUMACTIVEBITS]], [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i1 @llvm.smax.i1(i1
> [[VAL_NUMACTIVEBITS_OFFSET]], i1 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i1
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i1
> [[LOOP_BACKEDGETAKENCOUNT]], true
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i1 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = add nuw i1 [[IV]], [[EXTRAOFFSET:%.*]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i1 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i1
> [[VAL_SHIFTED]], false
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i1 [[IV]], true
> >> -; CHECK-NEXT: call void @escape_inner.i1(i1 [[IV]], i1 [[NBITS]],
> i1 [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i1 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i1 [ false, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i1 [[LOOP_IV]], true
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i1 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i1 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = add nuw i1 [[IV]], [[EXTRAOFFSET]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i1 [[VAL]], [[NBITS]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i1 [[IV]], true
> >> +; CHECK-NEXT: call void @escape_inner.i1(i1 [[IV]], i1 [[NBITS]],
> i1 [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i1 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i1 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i1 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i1 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i1 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i1 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer.i1(i1 [[IV_RES]], i1
> [[NBITS_RES]], i1 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i1
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i1 [[IV_RES]]
> >> @@ -1550,20 +1679,29 @@ end:
> >> define i2 @t32_addnuw_i2(i2 %val, i2 %start, i2 %extraoffset) {
> >> ; CHECK-LABEL: @t32_addnuw_i2(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i2
> @llvm.ctlz.i2(i2 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw i2 -2,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[TMP0:%.*]] = sub i2 0, [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i2
> [[VAL_NUMACTIVEBITS]], [[TMP0]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i2 @llvm.smax.i2(i2
> [[VAL_NUMACTIVEBITS_OFFSET]], i2 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i2
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw i2
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i2 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = add nuw i2 [[IV]], [[EXTRAOFFSET:%.*]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i2 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i2
> [[VAL_SHIFTED]], 0
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i2 [[IV]], 1
> >> -; CHECK-NEXT: call void @escape_inner.i2(i2 [[IV]], i2 [[NBITS]],
> i2 [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i2 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i2 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw i2 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i2 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i2 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = add nuw i2 [[IV]], [[EXTRAOFFSET]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i2 [[VAL]], [[NBITS]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i2 [[IV]], 1
> >> +; CHECK-NEXT: call void @escape_inner.i2(i2 [[IV]], i2 [[NBITS]],
> i2 [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i2 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i2 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i2 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i2 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i2 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i2 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer.i2(i2 [[IV_RES]], i2
> [[NBITS_RES]], i2 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i2
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i2 [[IV_RES]]
> >> @@ -1596,20 +1734,29 @@ end:
> >> define i3 @t33_addnuw_i3(i3 %val, i3 %start, i3 %extraoffset) {
> >> ; CHECK-LABEL: @t33_addnuw_i3(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i3
> @llvm.ctlz.i3(i3 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i3 3,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[TMP0:%.*]] = sub i3 0, [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i3
> [[VAL_NUMACTIVEBITS]], [[TMP0]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i3 @llvm.smax.i3(i3
> [[VAL_NUMACTIVEBITS_OFFSET]], i3 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i3
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i3
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i3 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = add nuw i3 [[IV]], [[EXTRAOFFSET:%.*]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i3 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i3
> [[VAL_SHIFTED]], 0
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i3 [[IV]], 1
> >> -; CHECK-NEXT: call void @escape_inner.i3(i3 [[IV]], i3 [[NBITS]],
> i3 [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i3 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i3 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i3 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i3 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i3 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = add nuw i3 [[IV]], [[EXTRAOFFSET]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i3 [[VAL]], [[NBITS]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i3 [[IV]], 1
> >> +; CHECK-NEXT: call void @escape_inner.i3(i3 [[IV]], i3 [[NBITS]],
> i3 [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i3 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i3 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i3 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i3 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i3 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i3 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer.i3(i3 [[IV_RES]], i3
> [[NBITS_RES]], i3 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i3
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i3 [[IV_RES]]
> >> @@ -1644,20 +1791,28 @@ end:
> >> define i1 @t34_subnsw_i1(i1 %val, i1 %start, i1 %extraoffset) {
> >> ; CHECK-LABEL: @t34_subnsw_i1(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i1
> @llvm.ctlz.i1(i1 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i1 true,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i1
> [[VAL_NUMACTIVEBITS]], [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i1 @llvm.smax.i1(i1
> [[VAL_NUMACTIVEBITS_OFFSET]], i1 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i1
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i1
> [[LOOP_BACKEDGETAKENCOUNT]], true
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i1 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = sub nsw i1 [[IV]], [[EXTRAOFFSET:%.*]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i1 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i1
> [[VAL_SHIFTED]], false
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i1 [[IV]], true
> >> -; CHECK-NEXT: call void @escape_inner.i1(i1 [[IV]], i1 [[NBITS]],
> i1 [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i1 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i1 [ false, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i1 [[LOOP_IV]], true
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i1 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i1 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = sub nsw i1 [[IV]], [[EXTRAOFFSET]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i1 [[VAL]], [[NBITS]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i1 [[IV]], true
> >> +; CHECK-NEXT: call void @escape_inner.i1(i1 [[IV]], i1 [[NBITS]],
> i1 [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i1 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i1 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i1 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i1 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i1 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i1 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer.i1(i1 [[IV_RES]], i1
> [[NBITS_RES]], i1 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i1
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i1 [[IV_RES]]
> >> @@ -1690,20 +1845,28 @@ end:
> >> define i2 @t35_addnuw_i2(i2 %val, i2 %start, i2 %extraoffset) {
> >> ; CHECK-LABEL: @t35_addnuw_i2(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i2
> @llvm.ctlz.i2(i2 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw i2 -2,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i2
> [[VAL_NUMACTIVEBITS]], [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i2 @llvm.smax.i2(i2
> [[VAL_NUMACTIVEBITS_OFFSET]], i2 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i2
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw i2
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i2 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = sub nsw i2 [[IV]], [[EXTRAOFFSET:%.*]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i2 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i2
> [[VAL_SHIFTED]], 0
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i2 [[IV]], 1
> >> -; CHECK-NEXT: call void @escape_inner.i2(i2 [[IV]], i2 [[NBITS]],
> i2 [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i2 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i2 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw i2 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i2 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i2 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = sub nsw i2 [[IV]], [[EXTRAOFFSET]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i2 [[VAL]], [[NBITS]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i2 [[IV]], 1
> >> +; CHECK-NEXT: call void @escape_inner.i2(i2 [[IV]], i2 [[NBITS]],
> i2 [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i2 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i2 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i2 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i2 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i2 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i2 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer.i2(i2 [[IV_RES]], i2
> [[NBITS_RES]], i2 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i2
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i2 [[IV_RES]]
> >> @@ -1736,20 +1899,28 @@ end:
> >> define i3 @t36_addnuw_i3(i3 %val, i3 %start, i3 %extraoffset) {
> >> ; CHECK-LABEL: @t36_addnuw_i3(
> >> ; CHECK-NEXT: entry:
> >> +; CHECK-NEXT: [[VAL_NUMLEADINGZEROS:%.*]] = call i3
> @llvm.ctlz.i3(i3 [[VAL:%.*]], i1 false)
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS:%.*]] = sub nuw nsw i3 3,
> [[VAL_NUMLEADINGZEROS]]
> >> +; CHECK-NEXT: [[VAL_NUMACTIVEBITS_OFFSET:%.*]] = add nsw i3
> [[VAL_NUMACTIVEBITS]], [[EXTRAOFFSET:%.*]]
> >> +; CHECK-NEXT: [[IV_FINAL:%.*]] = call i3 @llvm.smax.i3(i3
> [[VAL_NUMACTIVEBITS_OFFSET]], i3 [[START:%.*]])
> >> +; CHECK-NEXT: [[LOOP_BACKEDGETAKENCOUNT:%.*]] = sub nsw i3
> [[IV_FINAL]], [[START]]
> >> +; CHECK-NEXT: [[LOOP_TRIPCOUNT:%.*]] = add nuw nsw i3
> [[LOOP_BACKEDGETAKENCOUNT]], 1
> >> ; CHECK-NEXT: br label [[LOOP:%.*]]
> >> ; CHECK: loop:
> >> -; CHECK-NEXT: [[IV:%.*]] = phi i3 [ [[START:%.*]], [[ENTRY:%.*]] ],
> [ [[IV_NEXT:%.*]], [[LOOP]] ]
> >> -; CHECK-NEXT: [[NBITS:%.*]] = sub nsw i3 [[IV]], [[EXTRAOFFSET:%.*]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i3 [[VAL:%.*]], [[NBITS]]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO:%.*]] = icmp eq i3
> [[VAL_SHIFTED]], 0
> >> -; CHECK-NEXT: [[IV_NEXT]] = add i3 [[IV]], 1
> >> -; CHECK-NEXT: call void @escape_inner.i3(i3 [[IV]], i3 [[NBITS]],
> i3 [[VAL_SHIFTED]], i1 [[VAL_SHIFTED_ISZERO]], i3 [[IV_NEXT]])
> >> -; CHECK-NEXT: br i1 [[VAL_SHIFTED_ISZERO]], label [[END:%.*]],
> label [[LOOP]]
> >> +; CHECK-NEXT: [[LOOP_IV:%.*]] = phi i3 [ 0, [[ENTRY:%.*]] ], [
> [[LOOP_IV_NEXT:%.*]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[LOOP_IV_NEXT]] = add nuw nsw i3 [[LOOP_IV]], 1
> >> +; CHECK-NEXT: [[LOOP_IVCHECK:%.*]] = icmp eq i3 [[LOOP_IV_NEXT]],
> [[LOOP_TRIPCOUNT]]
> >> +; CHECK-NEXT: [[IV:%.*]] = add nsw i3 [[LOOP_IV]], [[START]]
> >> +; CHECK-NEXT: [[NBITS:%.*]] = sub nsw i3 [[IV]], [[EXTRAOFFSET]]
> >> +; CHECK-NEXT: [[VAL_SHIFTED:%.*]] = ashr i3 [[VAL]], [[NBITS]]
> >> +; CHECK-NEXT: [[IV_NEXT:%.*]] = add i3 [[IV]], 1
> >> +; CHECK-NEXT: call void @escape_inner.i3(i3 [[IV]], i3 [[NBITS]],
> i3 [[VAL_SHIFTED]], i1 [[LOOP_IVCHECK]], i3 [[IV_NEXT]])
> >> +; CHECK-NEXT: br i1 [[LOOP_IVCHECK]], label [[END:%.*]], label
> [[LOOP]]
> >> ; CHECK: end:
> >> -; CHECK-NEXT: [[IV_RES:%.*]] = phi i3 [ [[IV]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[IV_RES:%.*]] = phi i3 [ [[IV_FINAL]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[NBITS_RES:%.*]] = phi i3 [ [[NBITS]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[VAL_SHIFTED_RES:%.*]] = phi i3 [ [[VAL_SHIFTED]],
> [[LOOP]] ]
> >> -; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[VAL_SHIFTED_ISZERO]], [[LOOP]] ]
> >> +; CHECK-NEXT: [[VAL_SHIFTED_ISZERO_RES:%.*]] = phi i1 [
> [[LOOP_IVCHECK]], [[LOOP]] ]
> >> ; CHECK-NEXT: [[IV_NEXT_RES:%.*]] = phi i3 [ [[IV_NEXT]], [[LOOP]] ]
> >> ; CHECK-NEXT: call void @escape_outer.i3(i3 [[IV_RES]], i3
> [[NBITS_RES]], i3 [[VAL_SHIFTED_RES]], i1 [[VAL_SHIFTED_ISZERO_RES]], i3
> [[IV_NEXT_RES]])
> >> ; CHECK-NEXT: ret i3 [[IV_RES]]
> >>
> >>
> >>
> >> _______________________________________________
> >> llvm-commits mailing list
> >> llvm-commits at lists.llvm.org
> >> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210525/00e9e34f/attachment-0001.html>
More information about the llvm-commits
mailing list