[Mlir-commits] [mlir] Added free-threading CPython mode support in MLIR Python bindings (PR #107103)

[llvmlistbot at llvm.org]

vfdev-5 wrote:

@stellaraccident thanks for the discussion, I agree with your suggestion about documenting the scope of what free-threading would enable.

Concerning this part:

> If you want to make them thread-safe, then we will need to add much more locking, likely choosing to use context-scoped locks. The issue is that the underlying MLIR library is thread-compatible but not thread-safe. The GIL is currently making usage from Python more thread safe than MLIR actually is. Concretely, think of two threads attempting to modify the operations in the same function as an example of what would happen without more binding-level locking.

I would like still to highlight the point that free-threading aims:

>  to retains the thread safety guarantees that were in place before the GIL was removed, and it doesn’t either add nor remove any. So that, for instance, my_int += 1 is unsafe both in the free-threaded and in the default build.
The fact that there is no GIL doesn’t mean that everything must now be thread safe.

(source: https://discuss.python.org/t/free-threading-trove-classifier/62406/24)

A similar situation is now with Numpy which is thread-unsafe to use from both the GIL-enabled and GIL-disabled build and user needs to be careful not to mutate shared state (cc @ngoldbaum). 


> I'm supportive of making incremental progress in the right direction, so long as we document where we are trying to get to and are clear with folks what the current safety level is.

Sounds good, so let's to make MLIR Python bindings tread-safe in free threading mode and document it.


https://github.com/llvm/llvm-project/pull/107103