[all-commits] [llvm/llvm-project] a6e7d0: [mlir][python] Add pyi stub files to enable auto c...

[Stella Laurenzo via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: a6e7d024a9ebda1564fd78b829c45169add80864
      https://github.com/llvm/llvm-project/commit/a6e7d024a9ebda1564fd78b829c45169add80864
  Author: Stella Laurenzo <stellaraccident at gmail.com>
  Date:   2021-11-29 (Mon, 29 Nov 2021)

  Changed paths:
    M mlir/lib/Bindings/Python/ExecutionEngineModule.cpp
    M mlir/lib/Bindings/Python/IRAffine.cpp
    M mlir/lib/Bindings/Python/IRAttributes.cpp
    M mlir/lib/Bindings/Python/IRCore.cpp
    M mlir/lib/Bindings/Python/IRModule.h
    M mlir/lib/Bindings/Python/IRTypes.cpp
    M mlir/lib/Bindings/Python/MainModule.cpp
    M mlir/lib/Bindings/Python/Pass.cpp
    M mlir/python/CMakeLists.txt
    A mlir/python/mlir/_mlir_libs/_mlir/__init__.pyi
    A mlir/python/mlir/_mlir_libs/_mlir/ir.pyi
    A mlir/python/mlir/_mlir_libs/_mlir/passmanager.pyi
    A mlir/python/mlir/_mlir_libs/_mlirExecutionEngine.pyi

  Log Message:
  -----------
  [mlir][python] Add pyi stub files to enable auto completion.

There is no completely automated facility for generating stubs that are both accurate and comprehensive for native modules. After some experimentation, I found that MyPy's stubgen does the best at generating correct stubs with a few caveats that are relatively easy to fix:
  * Some types resolve to cross module symbols incorrectly.
  * staticmethod and classmethod signatures seem to always be completely generic and need to be manually provided.
  * It does not generate an __all__ which, from testing, causes namespace pollution to be visible to IDE code completion.

As a first step, I did the following:
  * Ran `stubgen` for `_mlir.ir`, `_mlir.passmanager`, and `_mlirExecutionEngine`.
  * Manually looked for all instances where unnamed arguments were being emitted (i.e. as 'arg0', etc) and updated the C++ side to include names (and re-ran stubgen to get a good initial state).
  * Made/noted a few structural changes to each `pyi` file to make it minimally functional.
  * Added the `pyi` files to the CMake rules so they are installed and visible.

To test, I added a `.env` file to the root of the project with `PYTHONPATH=...` set as per instructions. Then reload the developer window (in VsCode) and verify that completion works for various changes to test cases.

There are still a number of overly generic signatures, but I want to check in this low-touch baseline before iterating on more ambiguous changes. This is already a big improvement.

Differential Revision: https://reviews.llvm.org/D114679