[llvm] [X86][ISel] Improve VPTERNLOG matching for negated logic trees (PR #164863)

[Yi-Chi Lee via llvm-commits]

================
@@ -4740,6 +4737,34 @@ bool X86DAGToDAGISel::tryVPTERNLOG(SDNode *N) {
     return SDValue();
   };
 
+  // Identify and (optionally) peel an outer NOT that wraps a pure logic tree
+  auto tryPeelOuterNotWrappingLogic = [&](SDNode *Op) {
+    if (Op->getOpcode() == ISD::XOR && Op->hasOneUse() &&
+        ISD::isBuildVectorAllOnes(Op->getOperand(1).getNode())) {
+      SDValue InnerOp = Op->getOperand(0);
+
+      if (!getFoldableLogicOp(InnerOp)) {
+        return SDValue();
+      }
+
+      SDValue InnerN0 = InnerOp.getOperand(0);
+      SDValue InnerN1 = InnerOp.getOperand(1);
+      if (getFoldableLogicOp(InnerN1) || getFoldableLogicOp(InnerN0))
+        return InnerOp;
+    }
+    return SDValue();
+  };
+
+  bool PeeledOuterNot = false;
+  SDNode *OriN = N;
+  if (SDValue InnerOp = tryPeelOuterNotWrappingLogic(N)) {
+    PeeledOuterNot = true;
+    N = InnerOp.getNode();
----------------
yichi170 wrote:

If we peel the not out, the immediate (Imm) should be computed using the inner node (InnerOp), since the negation will be applied afterward. Therefore, we should not update this line here.

https://github.com/llvm/llvm-project/pull/164863