[llvm] 5da2c09 - [X86][ISel] Fix VPTERNLOG matching ensuring the InnerOp is logicOp (#166591)

[via llvm-commits]

Author: Yi-Chi Lee
Date: 2025-11-05T21:57:35Z
New Revision: 5da2c09e6ad3d18c14071e3b2833f4da9b70b6b8

URL: https://github.com/llvm/llvm-project/commit/5da2c09e6ad3d18c14071e3b2833f4da9b70b6b8
DIFF: https://github.com/llvm/llvm-project/commit/5da2c09e6ad3d18c14071e3b2833f4da9b70b6b8.diff

LOG: [X86][ISel] Fix VPTERNLOG matching ensuring the InnerOp is logicOp (#166591)

This patch fixes a crash in `tryVPTERNLOG` when trying to peel out the
outer not in cases like `~(A | B | C)`.

Previously, `InnerOp` was taken directly from `Op->getOperand(0)` before
verifying that it was a logical operation. As a result, the code could
later access `InnerOp->getOperand(0)` or `InnerOp->getOperand(1)` even
when `InnerOp` was something like a bitcast, causing an error.

This patch applies `getFoldableLogicOp` to `InnerOp`, ensuring that
`InnerOp` is a valid logic operation before it is dereferenced.

Added: 
    llvm/test/CodeGen/X86/vpternlog.ll

Modified: 
    llvm/lib/Target/X86/X86ISelDAGToDAG.cpp

Removed: 
    llvm/test/CodeGen/X86/issue163738.ll


################################################################################
diff  --git a/llvm/lib/Target/X86/X86ISelDAGToDAG.cpp b/llvm/lib/Target/X86/X86ISelDAGToDAG.cpp
index d4418c8563780..6c16fcfb282e8 100644
--- a/llvm/lib/Target/X86/X86ISelDAGToDAG.cpp
+++ b/llvm/lib/Target/X86/X86ISelDAGToDAG.cpp
@@ -4728,9 +4728,9 @@ bool X86DAGToDAGISel::tryVPTERNLOG(SDNode *N) {
   auto tryPeelOuterNotWrappingLogic = [&](SDNode *Op) {
     if (Op->getOpcode() == ISD::XOR && Op->hasOneUse() &&
         ISD::isBuildVectorAllOnes(Op->getOperand(1).getNode())) {
-      SDValue InnerOp = Op->getOperand(0);
+      SDValue InnerOp = getFoldableLogicOp(Op->getOperand(0));
 
-      if (!getFoldableLogicOp(InnerOp))
+      if (!InnerOp)
         return SDValue();
 
       N0 = InnerOp.getOperand(0);

diff  --git a/llvm/test/CodeGen/X86/issue163738.ll b/llvm/test/CodeGen/X86/vpternlog.ll
similarity index 59%
rename from llvm/test/CodeGen/X86/issue163738.ll
rename to llvm/test/CodeGen/X86/vpternlog.ll
index 61fe043a970dd..bd7478d3a82d5 100644
--- a/llvm/test/CodeGen/X86/issue163738.ll
+++ b/llvm/test/CodeGen/X86/vpternlog.ll
@@ -11,3 +11,15 @@ define <8 x i64> @foo(<8 x i64> %a, <8 x i64> %b, <8 x i64> %c) {
   %and3 = xor <8 x i64> %and3.demorgan, splat (i64 -1)
   ret <8 x i64> %and3
 }
+
+define <8 x i64> @xorbitcast(<64 x i8> %a, <64 x i8> %b, <64 x i8> %c) {
+; CHECK-LABEL: xorbitcast:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vpternlogq {{.*#+}} zmm0 = ~(zmm0 | zmm2 | zmm1)
+; CHECK-NEXT:    retq
+  %or1 = or <64 x i8> %a, %b
+  %or2 = or <64 x i8> %or1, %c
+  %cast = bitcast <64 x i8> %or2 to <8 x i64>
+  %xor = xor <8 x i64> %cast, splat (i64 -1)
+  ret <8 x i64> %xor
+}