[llvm-dev] Remove zext-unfolding from InstCombine

[Matthias Reisinger via llvm-dev]

Hi all,

I have a question regarding a transformation that is carried out in InstCombine, which has been introduced by r48715. It unfolds expressions of the form `zext(or(icmp, (icmp)))` to `or(zext(icmp), zext(icmp)))` to expose pairs of `zext(icmp)`. In a subsequent iteration these `zext(icmp)` pairs could then (possibly) be optimized by another optimization (which has already been there before r48715), to replace them by bitwise or integer operations. The code in question is located in `visitZExt()` in InstCombineCasts.cpp:

```
if (ICmpInst *ICI = dyn_cast<ICmpInst>(Src))
  return transformZExtICmp(ICI, CI);

BinaryOperator *SrcI = dyn_cast<BinaryOperator>(Src);
if (SrcI && SrcI->getOpcode() == Instruction::Or) {
  // zext (or icmp, icmp) --> or (zext icmp), (zext icmp) if at least one
  // of the (zext icmp) will be transformed.
  ICmpInst *LHS = dyn_cast<ICmpInst>(SrcI->getOperand(0));
  ICmpInst *RHS = dyn_cast<ICmpInst>(SrcI->getOperand(1));
  if (LHS && RHS && LHS->hasOneUse() && RHS->hasOneUse() &&
      (transformZExtICmp(LHS, CI, false) ||
       transformZExtICmp(RHS, CI, false))) {
    Value *LCast = Builder->CreateZExt(LHS, CI.getType(), LHS->getName());
    Value *RCast = Builder->CreateZExt(RHS, CI.getType(), RHS->getName());
    return BinaryOperator::Create(Instruction::Or, LCast, RCast);
  }
}
```

The first two lines do the actual `zext(icmp)` optimization. The remaining lines have been added by r48715 and do the mentioned unfolding. However, in `foldCastedBitwiseLogic()` there happens a reverse of the above unfolding, which transforms `logic(cast(A), cast(B))` to `cast(logic(A, B))`:

```
if ((!isa<ICmpInst>(Cast0Src) || !isa<ICmpInst>(Cast1Src)) &&
    shouldOptimizeCast(Cast0) && shouldOptimizeCast(Cast1)) {
  Value *NewOp = Builder->CreateBinOp(LogicOpc, Cast0Src, Cast1Src,
                                      I.getName());
  return CastInst::Create(CastOpcode, NewOp, DestTy);
}
```

The `(!isa<ICmpInst>(Cast0Src) || !isa<ICmpInst>(Cast1Src))` part has been added by r48715 in order to ensure that InstCombine does not enter an endless loop, switching back and forth between the two transformations. There was also added an InstCombine test case `zext-or-icmp.ll` that would enter such an endless loop, if this check would not be present. However, it also generally permits `foldCastedBitwiseLogic()` from folding operations with two icmps as operands. Something like

```
%1 = icmp sgt i64 %a, %b
%2 = zext i1 %1 to i8
%3 = icmp slt i64 %a, %c
%4 = zext i1 %3 to i8
%5 = and i8 %2, %4
```

would not be able to be transformed, even though the contained `zext icmp` pairs will also not be able to be optimized by `transformZExtICmp()` above, and the casts will survive. Therefore, in r275989 (https://reviews.llvm.org/D22511) I tried to narrow the check in `foldCastedBitwiseLogic()` to allow the folding of casts in the presence of icmps. For the above example this would result in:

```
%1 = icmp sgt i64 %a, %b
%2 = icmp slt i64 %a, %c
%3 = and i1 %1, %2
%4 = zext i1 %3 to i8
```

Despite `zext-or-icmp.ll` did not enter an endless loop with my change, it later turned out the assumptions that I took in r275989 were too optimistic and endless loops might still occur, which has been pointed out by Benjamin Kramer in http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20160718/374347.html and it has therefore been reverted.

I apologize in advance for this possibly naive question, but I was wondering whether it is feasible to remove the unfolding introduced by r48715 in order to enable a cast-folding like on the example above. I've also already tested this and run all of LLVM's test cases via the `make check` target and it seems that the only test case relies on this unfolding is the mentioned `zext-or-icmp.ll`. I don't know if Evan Chang is still responsible for this code region, but since he is the original author of the commit, I thought it would be safe to include him here (I hope it's the right email address).

Thank you in advance for your thoughts on this!

Best regards,
Matthias