<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/61964>61964</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            [x86 isel] Bad codegen for unpck and permute operations
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            new issue
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
            zhuhan0
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          zhuhan0
      </td>
    </tr>
</table>

<pre>
    https://godbolt.org/z/j7Eff1cbj shows some reduced code from a decode/decompression routine. Defining BAD_CODEGEN gives you really bad codegen for usage of the `permute` mask on line 31. It replaces 2 port5 instructions with 5 port5 instructions.

Inlining permute mask:
```
        vpermq  ymm3, ymm2, 136                 # ymm3 = ymm2[0,2,0,2]
        vpshufb ymm3, ymm3, ymm0
        vpermq ymm4, ymm1, 136                 # ymm4 = ymm1[0,2,0,2]
        vpshufb ymm4, ymm4, ymm0
        vpunpcklwd      ymm3, ymm4, ymm3        # ymm3 = ymm4[0],ymm3[0],ymm4[1],ymm3[1],ymm4[2],ymm3[2],ymm4[3],ymm3[3],ymm4[8],ymm3[8],ymm4[9],ymm3[9],ymm4[10],ymm3[10],ymm4[11],ymm3[11]
 vpermq  ymm2, ymm2, 221                 # ymm2 = ymm2[1,3,1,3]
 vpshufb ymm2, ymm2, ymm0
        vpermq  ymm1, ymm1, 221 # ymm1 = ymm1[1,3,1,3]
        vpshufb ymm1, ymm1, ymm0
 vpunpcklwd      ymm1, ymm1, ymm2        # ymm1 = ymm1[0],ymm2[0],ymm1[1],ymm2[1],ymm1[2],ymm2[2],ymm1[3],ymm2[3],ymm1[8],ymm2[8],ymm1[9],ymm2[9],ymm1[10],ymm2[10],ymm1[11],ymm2[11]
```
Outlining permute mask:
```
        vpunpcklwd      ymm3, ymm1, ymm2        # ymm3 = ymm1[0],ymm2[0],ymm1[1],ymm2[1],ymm1[2],ymm2[2],ymm1[3],ymm2[3],ymm1[8],ymm2[8],ymm1[9],ymm2[9],ymm1[10],ymm2[10],ymm1[11],ymm2[11]
 vpunpckhwd      ymm1, ymm1, ymm2        # ymm1 = ymm1[4],ymm2[4],ymm1[5],ymm2[5],ymm1[6],ymm2[6],ymm1[7],ymm2[7],ymm1[12],ymm2[12],ymm1[13],ymm2[13],ymm1[14],ymm2[14],ymm1[15],ymm2[15]
 vpermd  ymm2, ymm0, ymm3
        vpermd  ymm1, ymm0, ymm1
```
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJzkVttyozgQ_Rrx0jUudXOxeeAhCfHWvOx8whYXYZQB5EUiWc_Xb3HL0ExcW5vXoVJpt4_UfTgc5M6s1ZdOqUSEj4LoRz3UWScFkQhTLxtcbfpk-dLLTXlLaueuVvgPgs6CzhdT5qZxB9NfBJ1_CDq_HJ-rCov8BWxt3ixY0yroVTkUqoTClAqq3rSQQanGTNB5_NBee2WtNh30ZnC6UwdIVaU73V3g8SH96-lb-vzH859w0a_Kws0M0KusaW6QZ3PVi-qgMj0MNrsoMBW4WoGI5FX17eCUiCS0mf0OpoNGdwp8PMBXB726NlmhLBBcTe9C0J11_VA4bToLb9rVEH6AHIRMhXyY_3_tmpno0mtqNCo0r4nk8jelsFyv4-K_AW5t6wt6GiONEf0I9pcgf1oHwk_nheGjFPQ0bphjmO6r23qo8m31NX7M49a2wbIA_4NGsNLA_0NjrR7cozF01-J781bO-YbwusO_J0cw8QhTQU_Ttm02YsgwZBgxjBjmM8xn2IlhJ4bFDIs5F04Ud0x3VPGnohu_0NYvRHjvQdHWL-NTHRWd46bs-xNiZe8a5d0haxwJLA1x64w7DX91Bqu26fuBJfZLaXfHyK25akksQ2YCYhkyExDLkJmAWIbMBMQyZCYgliH3APEUuSWIW2J3sHwb3KeOobvv3T2R_d9J5FWg-rMuDFjlgLUNGRYyLGJYxLAjw478VriyyKVFri1ycZGTRc4WOd053RxOJfDTY_3Z-fUUKYG_76uc3Khemfhl7MeZpxKMTvLoy_gUe3USZIRxJTGMSWIeYZipCiMZB1VR4bEiTyckyZeBDAkDX8YHrNQpllVcRnER5MVRBFK1mW4OTfPajrOLp60dVBJhHAVek-WqsctA1Kk3mMBlJOqTcc-XfLhYEchGW2d_VnHaNdMk9c8pAm1VI8IUHvcTyugnyLry_UU1V9Vn01zhDX2zn7C0q4f8UJhW0HnstIQv1968qMIJOk_8rKDzxP_fAAAA__98CrKT">