[llvm] [InstCombine] Canonicalize complex boolean expressions into ~((y | z) ^ x) via 3-input truth table (PR #149530)

[via llvm-commits]

================
@@ -50,6 +51,213 @@ static Value *getFCmpValue(unsigned Code, Value *LHS, Value *RHS,
   return Builder.CreateFCmpFMF(NewPred, LHS, RHS, FMF);
 }
 
+/// This is to create optimal 3-variable boolean logic from truth tables.
+/// currently it supports the cases pertaining to the issue 97044. More cases
+/// can be added based on real-world justification for specific 3 input cases
+///  or with reviewer approval all 256 cases can be added (choose the
+///  canonicalizations found
+/// in x86InstCombine.cpp?)
+static Value *createLogicFromTable3Var(const std::bitset<8> &Table, Value *Op0,
+                                       Value *Op1, Value *Op2, Value *Root,
+                                       IRBuilderBase &Builder) {
+  uint8_t TruthValue = Table.to_ulong();
+  auto FoldConstant = [&](bool Val) {
+    Type *Ty = Op0->getType();
+    return Val ? ConstantInt::getTrue(Ty) : ConstantInt::getFalse(Ty);
+  };
+
+  Value *Result = nullptr;
+  switch (TruthValue) {
+  default:
+    return nullptr;
+  case 0x00: // Always FALSE
+    Result = FoldConstant(false);
+    break;
+  case 0xFF: // Always TRUE
+    Result = FoldConstant(true);
+    break;
+  case 0xE1: // ~((Op1 | Op2) ^ Op0)
+  {
+    Value *Or = Builder.CreateOr(Op1, Op2);
+    Value *Xor = Builder.CreateXor(Or, Op0);
+    Result = Builder.CreateNot(Xor);
+  } break;
+  case 0x60: // Op0 & (Op1 ^ Op2)
+  {
+    Value *Xor = Builder.CreateXor(Op1, Op2);
+    Result = Builder.CreateAnd(Op0, Xor);
+  } break;
+  case 0xD2: // ((Op1 | Op2) ^ Op0) ^ Op1
+  {
+    Value *Or = Builder.CreateOr(Op1, Op2);
+    Value *Xor1 = Builder.CreateXor(Or, Op0);
+    Result = Builder.CreateXor(Xor1, Op1);
+  } break;
+  }
+
+  return Result;
+}
+
+static std::tuple<Value *, Value *, Value *, SmallVector<Instruction *>>
+extractThreeVariablesAndInstructions(Value *Root) {
+  SmallPtrSet<Value *, 3> Variables;
+  SmallPtrSet<Value *, 32> Visited; // Prevent hanging during loop unrolling
+                                    // (see bitreverse-hang.ll)
+  SmallVector<Instruction *> Instructions;
+  SmallVector<Value *> Worklist;
+  Worklist.push_back(Root);
+
+  while (!Worklist.empty()) {
+    Value *V = Worklist.pop_back_val();
+
+    if (!Visited.insert(V).second)
+      continue;
+
+    Value *NotV;
+    if (match(V, m_Not(m_Value(NotV)))) {
+      if (auto *I = dyn_cast<Instruction>(V))
+        Instructions.push_back(I);
+      if (V == Root ||
+          V->hasOneUse()) { // Due to lack of cost-based heuristic, only
+                            // traverse if it belongs to this expression tree
+        Worklist.push_back(NotV);
+      }
+      continue;
+    }
+    if (auto *BO = dyn_cast<BinaryOperator>(V)) {
+      if (!BO->isBitwiseLogicOp())
+        return {nullptr, nullptr, nullptr, {}};
+
+      Instructions.push_back(BO);
+
+      if (V == Root || V->hasOneUse()) {
+        Worklist.push_back(BO->getOperand(0));
----------------
yafet-a wrote:

Thank you for the suggestion! I tested this approach and found some interesting results.

Although many of the problematic cases do involve constant operands, when I implemented the constant RHS check as a replacement to the prior validation approach, I found that the crashes still occur in cases where non-constant operands are missing from the computed map due to the hasOneUse() filtering.

Specific examples from `and-or.ll` and `and-xor-or.ll` that fail due to a missing key in map assertion:
- %3 = and i1 %a_1, %b_1 - %a_1 (non-constant) missing from Computed map
- %not1 = xor i32 %or1, -1 - %or1 = or i32 %a, %b (non-constant) missing from Computed map

e.g.:
 ```
Processing: %not1 = xor i32 %or1, -1
// %or1 is not a constant but missing from Computed map
DenseMap::at() assertion failure
```

I have stuck with the implementation in https://github.com/llvm/llvm-project/pull/149530/commits/1a94bba9746082ec6089085d3100675bc0b4a011 as it catches both constant and non-constant missing operand cases. It also avoids constant RHS expansion already, since constants aren't in the ValidOperands set since the check for them having to be arguments or instructions already filters them out.

https://github.com/llvm/llvm-project/pull/149530