<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/63700>63700</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            [X86] Assertion failure when combining a nested shuffle involving a bitcast of non-vector type to vector
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            new issue
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          Benjins
      </td>
    </tr>
</table>

<pre>
    Here is a minimal repro of the assertion failure (when compiled with `llc` with assertions enabled):
```llvm
target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

define <4 x i32> @do_stuff(i128 %0) #0 {
entry:
  %vcmp = bitcast i128 %0 to <4 x i32>
  %shuffle.i = shufflevector <4 x i32> %vcmp, <4 x i32> zeroinitializer, <4 x i32> <i32 0, i32 0, i32 undef, i32 undef>
  %shuffle.i11 = shufflevector <4 x i32> %shuffle.i, <4 x i32> zeroinitializer, <4 x i32> <i32 0, i32 4, i32 1, i32 5>
  ret <4 x i32> %shuffle.i11
}

attributes #0 = { "target-cpu"="haswell" }
```

Which gives this during pre-legalize DAG combine:
```
/root/llvm-project/llvm/include/llvm/CodeGen/ValueTypes.h:320: unsigned int llvm::EVT::getVectorNumElements() const: Assertion `isVector() && "Invalid vector type!"' failed.
```

[Godbolt link](https://godbolt.org/z/ohbfGh3j3)

This assertion is due to `combineX86ShufflesRecursively` peeking through the bitcast, and calls `isTargetShuffleEquivalent` on the i128 value. If the mask has zero sentinels, then we call `getVectorNumElements` on the value's type: this crashes since i128 is not a vector

Here is the DAG at the point of the crash:
```
Combining: t17: v2i64 = zero_extend_vector_inreg t10

SelectionDAG has 16 nodes:
  t0: ch,glue = EntryToken
 t2: i64,ch = CopyFromReg t0, Register:i64 %0
              t4: i64,ch = CopyFromReg t0, Register:i64 %1
            t5: i128 = build_pair t2, t4
          t6: v4i32 = bitcast t5
        t10: v4i32 = vector_shuffle<0,0,u,u> t6, undef:v4i32
      t17: v2i64 = zero_extend_vector_inreg t10
    t18: v4i32 = bitcast t17
  t14: ch,glue = CopyToReg t0, Register:v4i32 $xmm0, t18
  t19: i128 = undef
 t15: ch = X86ISD::RET_GLUE t14, TargetConstant:i32<0>, Register:v4i32 $xmm0, t14:1
```

A bisect found that this started happening in 62fc5f16405a7d39e62044bc461752f3f31bdca0, though I'm not sure if the issue is that code, or the shuffle combine code

I was not able to find any cases where this led to a miscompilation, just the assertion failure if they were enabled

I have verified that this repros on the latest trunk, e90ab9148baccda79874267c257bbba8d56e0600

For priority/triage purposes: this was not in manually-written code, but was instead found by a fuzzer to test SIMD codegen
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJysV0tz4j4S_zTi0kXKlp8cOBAI2VTt7mGSnZ1bSrbbWBMheSUZQj79Vssmgcxjt6b-qWAk1C_149dt4ZzcacQly25ZtpmJwXfGLm9Rf5fazSrTnJZ_Q4sgHQjYSy33QoHF3howLfgOQTiH1kujoRVSDRaB8fLYoYba7HupsIGj9B2wPFKqZnk0bt_ZHKAWlcKG8QVLVizasGjF8mj8V-qwH3_ywu7QQyO8UOJkBg8s2QDjHOd7lqxw3vMiYskq4eFB2_h6S6s8DY-5PC_akphiXs51SYv8zJOn88eYl4zzK_3eyl7hWfdrmT_n6XzQL9oc9VxJPbzOd3p45xqfDbZSE9M6hVeQJP8OWBo15tn5oW0ZL2XMS2A8ixhfAONJBKy4HblRe3t6dw0Q1aHe98GGSvpaOA_v7ODNtZ4LLtcNbavwRgbWaXfA2hv72bZRBePrTwdvaI3U0kuh5BvaHwlYspYJh4hOrhaDbrC93vzcuDj-f8x7J_8LbEzPi_i8yC5ss-h_oz6Op0AXm8uIC--trAaPbgom5UtxSzkzJtK87kOWJBvGeSfcEZVinMOHnHMJXIr9dyfrDnbygA58Jx00g5V6B73FucJduDBsVvdUe5XU-GNBTVu-tcZ4xrdUYPPemu9Yn7eMb6Wu1dDgxw9r0-A9asa3X4Ua8OnUo7vpQq1Q_cCgA5A0ILWHwJOsWLK6-_o0Lnbov4ZI_nPY3ynco_aOUXUtoDbaeZKxekcSlkfSjfQTEeM54zm570EfhJINTInhTz0yHpMreREgCJub33iQZbf3pqmM8qCkfmHZhvGy8753ZCjfMr7djec3xu4Y374xvjVd1d53yfeEMOpC2BOF4AMAQzwwVGAeTRH4VuaPY664L1gP1skDqhPBYI_4QrHznTXDrgtgOpUzpaHQDdRCKTd64ylkzSTq7j-DPAiF2pMgowNvgIADRecGHkZs3gv3Ap1woSTAofZSo3Ik3hNCHzGoIA0_DdCH8CCX8cKNDk9WY_rVVrgOHTip68kC6UAbD2IK0KW7zp2EBFKSCh-WvaGkmdpJkPirtF0Hn0q9CwbEBX0duMzTUF90yWd89aib51H5s9QWd-DjqxR4RIU1BYxsIO_EOWjToLvAWB-yuu4YX-_UMOL9HeHwk3lBPVF56hNArYSv6y7QrE1_2lqz_0JqA8B8wZ10Hi1LVsFQgvhJyeWfT_9IVvyjLJ8FSaEhUIcYpGqeeyEt2UuRTz_z-Dw4MiXku2wqPrumJD9eEU5ensCQJWuykj5D-CR3JJqvz3C_CpyXMv8ghiNb-QuL4-I9gnH6YwjJpU_mpw6dhPH0db8PZ6TkXdbiyqfjfaYkiLNRTzj5VuYPj5sR877cPT3f__1fd8EUvoaxhNcEd0IT4oV2so6o1_xPW-gy8W9wbQWVdFh7aM2gG_BdKC7pwHlhPTbQib5Hqh2QGnLe1lkb52mUiaJJFpjzKE2rOs3jIuNt0iZx1dRi1N0FeHpgvNiH0nY05MmxXKVzw1TTwkNtqGesgXC5w3MLPzej8fjC5gc4igktKhVws5W6AaFPUAuHDo4dIUa4Bs2R3oQh1I2DpaASJm3fB-d_MYqOVp7gSHLOc-aVBZ04IBzQylbipdvCjOvO8KeER1JiB_1CKnERiWoRp2Ul6roRxaIsUp4XNc-KqqpE2WQ5Rnl0FaGtsdBbaaz0J8a33kqxQ-gH2xsXsGdUfHaJ1LAXehBKneZHK70PE_Xo32rwgU5q51E0U8yrEwhoh7c3tOSqYPHjwz82gW2HetYsk2aRLMQMl3FeltlikSXRrFs2dVU2cRVnLRaFSNu6ioqqXRRl3SzaIq1ncskjnkRFlPGEZ0l0U9dp2xQ5xosE8zxKWRrhXkh1Q62fGucsJMYyT4oomilRoXLhHYNzjccxa6hlZ5uZXYYRpBp2jqWRks67DyleehVeTr6VOcs2FyPCOcLnN42xL4AAjY7S_Zx7Uh-MOoxHZ5QwLWij5xcTBPlr3M4Gq5af5gHpu6G6qc3-Yx76NDaFCznGt-HC_w0AAP__25YD6g">