[llvm-commits] Fix for assertion in DAGCombiner

[Jan Sjodin]

My expectation of doing a truncate of a vector of the same length, 
but smaller element types is that each element is truncated to the 
smaller size. This is what happens in the X86 backend today. 
  A truncate of a vector of shorter length, but identical 
element types could mean to reduce the size of the vector. I have not
checked if this is the case.

Below I included the code using getSExtOrTrunc as pointed out by Micha. 

- Jan

> I don't know if this is the right approach. Do you have an idea 
> of what kind of code you'd like to get out of a vector 
> truncate?
>
> Dan

> On Apr 26, 2010, at 12:11 PM, Jan Sjodin 
> > wrote:
> 
> > Below is one way of fixing the issue of sign extending a 
> > vector of smaller element types.
> > I suppose it would be fairly trivial to 
> > include the case where the type is larger by replacing
> > truncate with a 
> > second sign extend, but I wanted to get feedback on this code:


Index: lib/CodeGen/SelectionDAG/DAGCombiner.cpp
===================================================================
--- lib/CodeGen/SelectionDAG/DAGCombiner.cpp    (revision 102340)
+++ lib/CodeGen/SelectionDAG/DAGCombiner.cpp    (working copy)
@@ -3495,20 +3495,35 @@

   if (N0.getOpcode() == ISD::SETCC) {
     // sext(setcc) -> sext_in_reg(vsetcc) for vectors.
-    if (VT.isVector() &&
+    // Only do this before legalize for now.
+    if (VT.isVector() && !LegalOperations) {
+      EVT N0VT = N0.getOperand(0).getValueType();
         // We know that the # elements of the results is the same as the
         // # elements of the compare (and the # elements of the compare result
         // for that matter).  Check to see that they are the same size.  If so,
         // we know that the element size of the sext'd result matches the
         // element size of the compare operands.
-        VT.getSizeInBits() == N0.getOperand(0).getValueType().getSizeInBits() &&
-      
-        // Only do this before legalize for now.
-        !LegalOperations) {
-      return DAG.getVSetCC(N->getDebugLoc(), VT, N0.getOperand(0),
-                           N0.getOperand(1),
-                           cast<CondCodeSDNode>(N0.getOperand(2))->get());
-    }
+      if(VT.getSizeInBits() == N0VT.getSizeInBits())
+       return DAG.getVSetCC(N->getDebugLoc(), VT, N0.getOperand(0),
+                            N0.getOperand(1),
+                            cast<CondCodeSDNode>(N0.getOperand(2))->get());
+      // If the desired elements are smaller or larger than the source
+      // elements we can use a matching integer vector type and then
+      // truncate/sign extend
+      else { 
+       EVT MatchingElementType =
+         EVT::getIntegerVT(*DAG.getContext(),
+                           N0VT.getScalarType().getSizeInBits());
+       EVT MatchingVectorType =
+         EVT::getVectorVT(*DAG.getContext(), MatchingElementType,
+                          N0VT.getVectorNumElements());
+       SDValue VsetCC = 
+         DAG.getVSetCC(N->getDebugLoc(), MatchingVectorType, N0.getOperand(0),
+                       N0.getOperand(1),
+                       cast<CondCodeSDNode>(N0.getOperand(2))->get());
+       return  DAG.getSExtOrTrunc(VsetCC, N->getDebugLoc(),  VT);
+      }
+    }
    
     // sext(setcc x, y, cc) -> (select_cc x, y, -1, 0, cc)
     unsigned ElementWidth = VT.getScalarType().getSizeInBits();