[Mlir-commits] [mlir] [MLIR][Python] Extend bindings for external projects without duplication (PR #173241)

[Sylvain Noiry]

ElectrikSpace wrote:

Ok thank you. I hadn't tested the full wheel setup for now.....and it actually fails at runtime. I build everything with `*_DYLIB=ON`, so I end up loading only one `MLIRPythonCAP` and one `MLIRPythonCAP` .
But now I get a `ImportError: libMLIRPythonCAPI.so.21.1:`

I had "LD_LIBRARY_PATH=$(realpath ./llvm-project/install/lib/):$LD_LIBRARY_PATH" in my test env, which made it work before.

Here is the tree for `venv/lib/python3.11/site-packages/mlir_sdist`:

```
venv/lib/python3.11/site-packages/mlir_sdist
├── dialects
│   ├── __init__.py
│   ├── __pycache__
│   │   ├── __init__.cpython-311.pyc
│   │   ├── sdist.cpython-311.pyc
│   │   ├── _sdist_enum_gen.cpython-311.pyc
│   │   ├── _sdist_ops_gen.cpython-311.pyc
│   │   ├── _sdist_transform_ops_gen.cpython-311.pyc
│   │   ├── sdist_utils.cpython-311.pyc
│   │   └── _sdist_utils_ops_gen.cpython-311.pyc
│   ├── _sdist_enum_gen.py
│   ├── _sdist_ops_gen.py
│   ├── sdist.py
│   ├── _sdist_transform_ops_gen.py
│   ├── _sdist_utils_ops_gen.py
│   ├── sdist_utils.py
│   └── transform
│       ├── __pycache__
│       │   └── sdist.cpython-311.pyc
│       └── sdist.py
├── extras
│   ...
│   ├── run_aie.py
│   ├── run_mppa.py
│   └── utils.py
└── _mlir_libs
    ├── libSDistPythonCAPI.so
    ├── libSDistPythonCAPI.so.21.1
    └── _sdistDialects.cpython-311-x86_64-linux-gnu.so

```

> If this is working for you then it's purely luck because if `mlir_sdist` links against `SDistPythonCAPI` then it's linking against a completely separate `MLIRCAPIR` and etc and so you're just lucky [`register_dialects` doesn't use any state](https://github.com/llvm/llvm-project/blob/97fc5682112d019230a3ab23cced3bdf093f6094/mlir/lib/CAPI/IR/DialectHandle.cpp#L25-L30) in `SDistPythonCAPI`. Plenty of other APIs [aren't so lucky](https://github.com/llvm/llvm-project/blob/7c4e8c6a273f25b3ef33e9c123b3969632ab59bb/mlir/lib/CAPI/RegisterEverything/RegisterEverything.cpp#L32).

I use a custom `CAPI` function to register the `SDist` dialects.

https://github.com/llvm/llvm-project/pull/173241