[Mlir-commits] [mlir] [MLIR][Python] Support Python-defined passes in MLIR (PR #156000)

[llvmlistbot at llvm.org]

================
@@ -20,6 +23,81 @@ using namespace mlir::python;
 
 namespace {
 
+// A base class for defining passes in Python
+// Users are expected to subclass this and implement the `run` method, e.g.
+// ```
+// class MyPass(Pass):
+//   def __init__(self):
+//     super().__init__("MyPass", ..)
+//     # other init stuff..
+//   def run(self, operation):
+//     # do something with operation..
+//     pass
+// ```
+class PyPassBase {
+public:
+  PyPassBase(std::string name, std::string argument, std::string description,
+             std::string opName)
+      : name(std::move(name)), argument(std::move(argument)),
+        description(std::move(description)), opName(std::move(opName)) {
+    callbacks.construct = [](void *obj) {};
+    callbacks.destruct = [](void *obj) {
+      nb::handle(static_cast<PyObject *>(obj)).dec_ref();
+    };
+    callbacks.run = [](MlirOperation op, MlirExternalPass, void *obj) {
+      auto handle = nb::handle(static_cast<PyObject *>(obj));
+      nb::cast<PyPassBase *>(handle)->run(op);
+    };
+    callbacks.clone = [](void *obj) -> void * {
+      nb::object copy = nb::module_::import_("copy");
+      nb::object deepcopy = copy.attr("deepcopy");
+      return deepcopy(obj).release().ptr();
+    };
+    callbacks.initialize = nullptr;
+  }
+
+  // this method should be overridden by subclasses in Python.
+  virtual void run(MlirOperation op) = 0;
+
+  virtual ~PyPassBase() = default;
+
+  // Make an MlirPass instance on-the-fly that wraps this object.
+  // Note that passmanager will take the ownership of the returned
+  // object and release it when appropriate.
+  MlirPass make() {
----------------
PragmaTwice wrote:

The reason that in this design I add a new class `PyPassBase` and lazily construct the `MlirPass` while adding into the pass manager instead of just wrapping `MlirPass` as a python object is that: the lifetime of `MlirPass` is a bit tricky. 

If we don't add it into the pass manager, the owership of `MlirPass` should be in our hand (and we are responsible to release it finally), and after adding into a pass manager, the ownership is transfered into the manager, and `delete myPassPtr` will be called when the manager is destructed on the C++ side.

By lazily constructing the `MlirPass` only when adding to a pass manager, we can avoid to expose the `MlirPass` object into the python world so that we don't need to care about how to interoperate between c++ `delete passPtr` and python object ref-count.

This approach might seem a bit roundabout but works. I’m also open to trying other methods if there are any : )

https://github.com/llvm/llvm-project/pull/156000