[PATCH] X86: Do not select X86 custom vector nodes if operand types don't match

[Ahmed Bougacha]

For the missing testcase, one hack you can use to force an additional DAGCombine round is to introduce illegal vector ops (so there's another vector ops legalization phase, with the associated combining round), since IIRC the BUILD_VECTOR->SHUFP lowering happens in the final ops legalization phase, after vector ops legalization.  See r235243, where I used ctpop.  That's icky though, and shows that our AddUsersToWorklist tricks aren't enough (but that's a discussion for another day ;)


REPOSITORY
  rL LLVM

================
Comment at: lib/Target/X86/X86ISelLowering.cpp:5272-5285
@@ -5271,10 +5271,16 @@
     if (i * 2 < NumElts) {
-      if (V0.getOpcode() == ISD::UNDEF)
+      if (V0.getOpcode() == ISD::UNDEF) {
         V0 = Op0.getOperand(0);
+        if (V0.getValueType() != VT)
+          return false;
+      }
     } else {
-      if (V1.getOpcode() == ISD::UNDEF)
+      if (V1.getOpcode() == ISD::UNDEF) {
         V1 = Op0.getOperand(0);
+        if (V1.getValueType() != VT)
+          return false;
+      }
       if (i * 2 == NumElts)
         ExpectedVExtractIdx = BaseIdx;
     }
 
----------------
Should you put the checks outside the if/else?  Something like

    if (V0.getVT() != VT || V1.getVT != VT)
      return false;

Not sure it's better.

http://reviews.llvm.org/D9120

EMAIL PREFERENCES
  http://reviews.llvm.org/settings/panel/emailpreferences/