[Mlir-commits] [mlir] [MLIR][Python] remove `liveOperations` (PR #155114)

[John Demme]

teqdruid wrote:

> > From previous discussion on this issue, @ftynse wrote in ...
> 
> I actually did sketch that out @ #97340. If you take a look you'll notice how much starts to change when you try to implement this - specifically everywhere you return an `Operation*` you now need to return a weakref instead. I personally felt it was just too much work for not enough ROI (and also that core wouldn't take the PR). If you want to try to RFC that idea (weakref system) I could probably finish up an MVP but I highly doubt the RFC will be approved 🤷.

That looks like what @stellaraccident was talking about rather than the proposal I had. My proposal is nowhere near as invasive. Simply a callback which one could register in MLIRContext (and plumbed through the CAPI) which would be called when an operation is destructed (pointer is no longer valid). We could use it in the Python bindings to invalidate the `PyOperation`(s) which are fronting the erased operation.



> The bindings aren't designed to be used the way you're using them though - they weren't designed for people to be able to run passes and then still be in a valid state. What's been drilled into my head over the years is they were conceived of and designed only to support emitting IR (i.e., a nicer test writing UX). So we're already where you want to be - just assume they are all already invalid always and don't try to do what you're trying to do 😄. Now certainly the bindings have evolved (e.g., thanks to you, thanks to many others) and they do support some light "industrial use" (rewrites and such) but the founding premise is still there - they're not meant to be safe for this kind of use. And just re-stating the above: the user in this instance isn't a DSL user - it's you the compiler developer and you're writing library code (not user code). So I think the expectations are the same as for C++; i.e., the Python library code you write needs to be as conservative as it would need to be if it were just C++ library code.

I think you're missing my point. It doesn't matter how the bindings are supposed to be used. (I've pivoted to using them "the right way".) When they're used improperly, the correct behavior is to throw an exception rather than just crash. This is the expectation for all Python libraries.

... I wonder if it's possible to "catch" a crash in C++ land and translate it to a Python exception. That would also fix this problem and many others. Yeah, the C++ side is possibly not valid anymore but what's the worst which can happen if you call into it? Another crash which would be caught?

https://github.com/llvm/llvm-project/pull/155114