[LLVMdev] Vector splitting vs widening

[Hal Finkel]

Hello,

Working on my (currently out-of-tree) BG/Q PPC enhancements, I've run into the following problem with vector type legalization. Here's a quick example:

Scalarize node result 0: 0x2348420: v1f32 = extract_subvector 0x23434a0, 0x2348320 [ID=0]

Scalarize node result 0: 0x2348220: v1f32 = extract_subvector 0x23434a0, 0x23466e0 [ID=0]

Split node result: 0x23469e0: v4f32 = extract_subvector 0x23435a0, 0x23466e0 [ID=0]

Split node operand: 0x2346be0: v4i1 = setcc 0x23467e0, 0x23469e0, 0x23436a0 [ID=0]

Split node result: 0x2348620: v2f32 = extract_subvector 0x23435a0, 0x2346de0 [ID=0]

Widen node result 0: 0x2348820: v2i1 = setcc 0x2346ee0, 0x2348620, 0x23436a0 [ID=0]

llc: lib/CodeGen/SelectionDAG/LegalizeTypes.h:599: llvm::SDValue llvm::DAGTypeLegalizer::GetWidenedVector(llvm::SDValue): Assertion `WidenedOp.getNode() && "Operand wasn't widened?"' failed.

The problem is essentially the following: there are no vector f32 types (yet), so the <v4i1> = setcc <v4f32> node needs to be split and scalarized. The operand splitting seems to start correctly, but because <v4i1> is itself a legal type, after splitting the node into <v2i1> = setcc <v2f32>, the process becomes confused. The operands are again split (as they should be), but it tries to widen the <v2i1> result back to <v4i1> (thus hitting the operand assertion).

In some sense, the problem is that DAGTypeLegalizer::run decides what to do solely based on the result of calling getTypeAction(ResultVT), but it seems that in this case the operand types need to be accounted for in this determination. Enhancing the logic there to consider the result types in this case seems like it should be straightforward, but how general a problem is this? [Can this problem only happen with vsetcc nodes?]

Maybe, for example, after the ScanOperands part of DAGTypeLegalizer::run, if we need to reanalyze, we should actually run ScanOperands again instead of starting with the result-type processing?

Thanks in advance,
Hal

-- 
Hal Finkel
Postdoctoral Appointee
Leadership Computing Facility
Argonne National Laboratory