<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/109641>109641</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            [MLIR][sparse] Error when reshaping CSC tensor
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            mlir
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          mtsokol
      </td>
    </tr>
</table>

<pre>
    Hi!

I'm trying to reshape CSR/CSC tensors. While reshaping CSR tensors works, running the `PassManager` on CSC reshape fails with: 

```
Traceback (most recent call last):
  File "/home/mtsokol/sparse/csc_reshape.py", line 29, in <module>
 pm.run(module.operation)
mlir._mlir_libs._site_initialize.<locals>.MLIRError: Failure while executing pass pipeline:
error: "-":6:16: ConvertOp not staged.
 note: "-":6:16: see current operation: %8 = "sparse_tensor.convert"(%7) : (tensor<25x200xf64, #sparse_tensor.encoding<{ map = (d0, d1) -> (d0 : compressed(nonunique, nonordered), d1 : singleton(nonordered)), posWidth = 64, crdWidth = 64 }>>) -> tensor<25x200xf64, #sparse_tensor.encoding<{ map = (d0, d1) -> (d1 : dense, d0 : compressed), posWidth = 64, crdWidth = 64 }>>
error: "-":6:16: ConvertOp not staged.
 note: "-":6:16: see current operation: %9 = "sparse_tensor.convert"(%8) : (tensor<25x200xf64, #sparse_tensor.encoding<{ map = (d0, d1) -> (d0 : compressed(nonunique, nonordered), d1 : singleton(nonordered)), posWidth = 64, crdWidth = 64 }>>) -> tensor<25x200xf64, #sparse_tensor.encoding<{ map = (d0, d1) -> (d1 : dense, d0 : compressed), posWidth = 64, crdWidth = 64 }>>
error: "-":6:16: ConvertOp not staged.
 note: "-":6:16: see current operation: %6 = "sparse_tensor.convert"(%5) : (tensor<25x200xf64, #sparse_tensor.encoding<{ map = (d0, d1) -> (d0 : compressed(nonunique, nonordered), d1 : singleton(nonordered)), posWidth = 64, crdWidth = 64 }>>) -> tensor<25x200xf64, #sparse_tensor.encoding<{ map = (d0, d1) -> (d1 : dense, d0 : compressed), posWidth = 64, crdWidth = 64 }>>
error: "-":6:16: failed to legalize operation 'sparse_tensor.convert' that was explicitly marked illegal
 note: "-":6:16: see current operation: %44 = "sparse_tensor.convert"(%43) : (tensor<25x200xf64, #sparse_tensor.encoding<{ map = (d0, d1) -> (d0 : compressed(nonunique, nonordered), d1 : singleton(nonordered)), posWidth = 64, crdWidth = 64 }>>) -> tensor<25x200xf64, #sparse_tensor.encoding<{ map = (d0, d1) -> (d1 : dense, d0 : compressed), posWidth = 64, crdWidth = 64 }>>
```

---

Here's a short script that reproduces failing CSC reshape: https://gist.github.com/mtsokol/cc67c576172c67141126307713f4cb96

<details>

```python
import ctypes
import ctypes.util
import pathlib

from mlir.ir import Context, Module
from mlir import execution_engine, passmanager


MLIR_C_RUNNER_UTILS = ctypes.util.find_library("mlir_c_runner_utils")


with Context():
    module = Module.parse(
    """
    #CSC = #sparse_tensor.encoding<{
        map = (i, j) -> (j : dense, i : compressed), posWidth = 64, crdWidth = 64
    }>
    func.func @add(%st_0 : tensor<100x50xf64, #CSC>, %st_1 : tensor<2xi64>) -> tensor<25x200xf64, #CSC> attributes { llvm.emit_c_interface } {
 %dst = tensor.reshape %st_0(%st_1) : (tensor<100x50xf64, #CSC>, tensor<2xi64>) -> tensor<25x200xf64, #CSC>        
        return %dst : tensor<25x200xf64, #CSC>
    }
    """
    )

    CWD = pathlib.Path(".")
    (CWD / "module.mlir").write_text(str(module))

    pm = passmanager.PassManager.parse("builtin.module(sparse-assembler{direct-out=true}, sparsifier{create-sparse-deallocs=1 enable-runtime-library=false})")
 pm.run(module.operation)

    (CWD / "module_opt.mlir").write_text(str(module))

    ee = execution_engine.ExecutionEngine(module, opt_level=2, shared_libs=[MLIR_C_RUNNER_UTILS])
```

</details>

---

And here a script showing CSR reshape that works: https://gist.github.com/mtsokol/a9950a60f39983bab6b365081c8d2b3a

<details>

```python
import ctypes
import ctypes.util
import pathlib

from mlir.ir import Context, Module
from mlir import execution_engine, passmanager


MLIR_C_RUNNER_UTILS = ctypes.util.find_library("mlir_c_runner_utils")


with Context():
    module = Module.parse(
    """
    #CSR = #sparse_tensor.encoding<{
        map = (i, j) -> (i : dense, j : compressed), posWidth = 64, crdWidth = 64
    }>
    func.func @add(%st_0 : tensor<100x50xf64, #CSR>, %st_1 : tensor<2xi64>) -> tensor<25x200xf64, #CSR> attributes { llvm.emit_c_interface } {
 %dst = tensor.reshape %st_0(%st_1) : (tensor<100x50xf64, #CSR>, tensor<2xi64>) -> tensor<25x200xf64, #CSR>        
        return %dst : tensor<25x200xf64, #CSR>
    }
    """
    )

    CWD = pathlib.Path(".")
    (CWD / "module.mlir").write_text(str(module))

    pm = passmanager.PassManager.parse("builtin.module(sparse-assembler{direct-out=true}, sparsifier{create-sparse-deallocs=1 enable-runtime-library=false})")
 pm.run(module.operation)

    (CWD / "module_opt.mlir").write_text(str(module))

    ee = execution_engine.ExecutionEngine(module, opt_level=2, shared_libs=[MLIR_C_RUNNER_UTILS])
```

</details>

</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJzsWV-P2zYS_zT0C2FBIq1_D37YlW00QNMrvC3yKFDU2GJCkTqSyu7epz-Qkv_tbdP0ksOlwAYGs6KHw5nhb-Y3tJi14qgA1ii9R-lmwUbXabPundWftFw0un1e_yQQSVC8QfHdNL5DJO-xM89CHbHT2IDt2AC4etgjsqseKuxAWW1shD90QsIs4KWrh_3pS_yozSeLSIXNqFRQ1QFGWfwrs_Y9U-wIBmUx1gp7ladNDkxIix-F6xC9w9dmoSyeP-HxN8M4NIx_wogUvbYOG-CgHOZMSiyZdYiUiM6LMd55SxEhiOw63QMiuzkKiOzswIz1U9zyerYkGp6DdIWlUIBJ6f8UCiNa9bodJSC6nXUPfWRGFczwX0R6AMOc0MpbEER6KUxU-7GWorFRbYWDWijhBJPiXxAhWknNmbSIbqP3P7_bb43Rxodgx4QcDeDHEGp4Aj46H82BWYsHMYA37-wnnJYhQpbefnqXIXqX-AFXWn0G4_4xYKUdto4doY1mH5R28EfrLADmozE-uhffgnBaYEQ3ftUUw3o6_YhPW4UIFoikOSIlnpYUkwiiFUmfSBw_HbKVjy0i9FYHKK5boY6IVii_xz0b5r2KNvYL2sQrXSK6neaCfq77wYC10CJSKK1GJf45ghdXWmnTgvHflNP6sMIKdZTg_GkVL2SC2KDtB9G6Lmw-mcpNez2FUb7xaKDbs0H_Ix8ni1tQNrj0is9_3eT_D27Kr8NN8YabN9xc4yb7Otykb7j5--LG9wDQ-tZDwjHw4wUGGJH8D44-x65jDj8yi-FpkIILJ59xz8wnaLGQQdc3om-1-jr4regb_n5o_L1oJqdxuVxeP_4EBjzaMMO208Zhy40Y3IQyA4PR7cjBBrhOze-5jfVWds4N1jdmZIfI7iisi47CdWMTcd3ftJ-cZzlP8yzJCc_yZJUkJKNxnif0sOJNmd10wbRqwfkm-eLLrUfDs-u0miZFP3jDuXsewL4yFY1OyJv5gblOiuZa88HoHocOVhg8i1VaOXhyPtLvp274VvQkN_erWtWgjr5R9afFrO3nC8C1A2H0vW9d1fvff_llu69__-3dzw_hCK_sjQ5Ctb6PNsw8h3wjobPmtb9pgKm9kA25WP7nBv5qcTG_uL0kYDy18GHLybFovhsUF5mgOnyupqg__gnLX8T8ZU3Y7pIAwsfm4zX8P96iX_y34L8yc8qC8_NhVDzyA0armLXtVL2sq6dEO2d1EsdP6XVWVw9VSH__4OWTW3nyJLLV19WHSRNmzhnRjA4s9oVBys99BL1wNa-FcmAOjIM3H19CiEjaWhd8nCN9ukTOPpy9SV6rxl_w6Rv8mP_dnrIBNxp1MfjuT_XcHtmXkXcDcj9VfdiEqMy5HP3KXDflSXSdFdPqIkiTndc93199Nk2C0aPxF9U5V6wz50vuzBcvdh76eeNzhkdX1_1LKhHSjEI6oaKTtmLKmSWzFvpGgkH5fSsMcLfUo0N048wIPhakwkFUHEQQ4gaYg-W8vAUmpeYW0U2CQbFGwtKMyokelqeKQTcHJu2krLwJyJ9f5L8cuVoP7huiB1PdeVkzo-1pYjvX0LOWCuvB1RI-g0R0E36tsB0zEMqjDwJK71-pqCjdXLZ_jQl9qSK714nmBUveqRZ3YMCz5MSPttOPpx-DThk5dWbhF6G_RI2sLNOYZfGBlmVBG9ZkDc3SuEh40ZKGsjdq_HtQ4_47U6O4pcaPPxQ17r8bNe5_GGrcfxM17r8TNe7fqPGNGn9Ealy0a9qWtGQLWCc5yeOC5lm66NZFkbCEZYzE6SEpVnlbNKsVa8ghblmbZ_lCrElMVnFJaFySkqTRIYnjpEwPQIAxRjO0iqFnQkYh8bU5LoS1I6yTuMxWyUKyBqQNL3imYu8DnG4WZu3ll814tGgVS2GdvWhwwsnwUshHwLuc3s_vQNINDq8e8GMH6ubNzin3F6OR65cMfkXefpP5v-Vg9EfgDpFdsNkispvN_rwm_w4AAP__DYHFjA">