[Mlir-commits] [mlir] Added free-threading CPython mode support in MLIR Python bindings (PR #107103)

[Stella Laurenzo]

stellaraccident wrote:

That locking idiom looks good to me. Thanks. 

For the tests, I won't claim to love the current testing style, but as with most such things, it is costly to change. If this weren't a part of llvm, I never would have written the tests this way, but I digress...

I suppose we should discuss what level of threading compatibility we are targeting (and likely document that). It seems where this patch is going is ensuring that the library itself does not provide any inherent locking hazards if independent contexts are used from multiple threads. Beyond that, the thread safety is approximately what it is for the underlying mlir libraries, which is to say that it is unsafe to perform mutation on any IR structure that you are unsure as to whether it is being exclusively access by a single thread. Most non framework code will just assume thread per context as the safe way to use the API.

That is quite a bit less safe than the CPython itself, which guards mutable data structures (list, shared dict, etc) internally, making it possible to have logical consistency problems is used across threads in an unsafe way, but internally guarding against corruption of the data structures themselves. 

I'm not aware of a consensus that has emerged on the level of safety that a library such as this should provide. Fine grained locking at the binding layer seems like it would be expensive and error prone, especially given the more relaxed stance of the underlying library. 

Thoughts?

https://github.com/llvm/llvm-project/pull/107103