[all-commits] [llvm/llvm-project] e31c77: [mlir][python] Reorganize MLIR python into namespa...

[Stella Laurenzo via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: e31c77b1827fa4dd3511f21af11cfab18ecf6d38
      https://github.com/llvm/llvm-project/commit/e31c77b1827fa4dd3511f21af11cfab18ecf6d38
  Author: Stella Laurenzo <stellaraccident at gmail.com>
  Date:   2021-03-08 (Mon, 08 Mar 2021)

  Changed paths:
    M mlir/cmake/modules/AddMLIRPythonExtension.cmake
    M mlir/docs/Bindings/Python.md
    R mlir/lib/Bindings/Python/mlir/__init__.py
    A mlir/lib/Bindings/Python/mlir/_cext_loader.py
    M mlir/lib/Bindings/Python/mlir/conversions/__init__.py
    R mlir/lib/Bindings/Python/mlir/dialects/__init__.py
    R mlir/lib/Bindings/Python/mlir/dialects/_builtin.py
    A mlir/lib/Bindings/Python/mlir/dialects/_builtin_ops_ext.py
    R mlir/lib/Bindings/Python/mlir/dialects/_linalg.py
    A mlir/lib/Bindings/Python/mlir/dialects/_linalg_ops_ext.py
    A mlir/lib/Bindings/Python/mlir/dialects/_ods_common.py
    A mlir/lib/Bindings/Python/mlir/dialects/builtin.py
    A mlir/lib/Bindings/Python/mlir/dialects/linalg/__init__.py
    A mlir/lib/Bindings/Python/mlir/dialects/linalg/opdsl/__init__.py
    A mlir/lib/Bindings/Python/mlir/dialects/linalg/opdsl/dump_oplib.py
    A mlir/lib/Bindings/Python/mlir/dialects/linalg/opdsl/lang/__init__.py
    A mlir/lib/Bindings/Python/mlir/dialects/linalg/opdsl/lang/affine.py
    A mlir/lib/Bindings/Python/mlir/dialects/linalg/opdsl/lang/comprehension.py
    A mlir/lib/Bindings/Python/mlir/dialects/linalg/opdsl/lang/config.py
    A mlir/lib/Bindings/Python/mlir/dialects/linalg/opdsl/lang/dsl.py
    A mlir/lib/Bindings/Python/mlir/dialects/linalg/opdsl/lang/scalar_expr.py
    A mlir/lib/Bindings/Python/mlir/dialects/linalg/opdsl/lang/types.py
    A mlir/lib/Bindings/Python/mlir/dialects/linalg/opdsl/lang/yaml_helper.py
    A mlir/lib/Bindings/Python/mlir/dialects/linalg/opdsl/ops/__init__.py
    A mlir/lib/Bindings/Python/mlir/dialects/linalg/opdsl/ops/core_named_ops.py
    A mlir/lib/Bindings/Python/mlir/dialects/python_test.py
    A mlir/lib/Bindings/Python/mlir/dialects/shape.py
    A mlir/lib/Bindings/Python/mlir/dialects/std.py
    A mlir/lib/Bindings/Python/mlir/dialects/tensor.py
    M mlir/lib/Bindings/Python/mlir/execution_engine.py
    M mlir/lib/Bindings/Python/mlir/ir.py
    M mlir/lib/Bindings/Python/mlir/passmanager.py
    R mlir/lib/Bindings/Python/mlir/tools/__init__.py
    R mlir/lib/Bindings/Python/mlir/tools/linalg_opdsl/__init__.py
    R mlir/lib/Bindings/Python/mlir/tools/linalg_opdsl/dump_oplib.py
    R mlir/lib/Bindings/Python/mlir/tools/linalg_opdsl/lang/__init__.py
    R mlir/lib/Bindings/Python/mlir/tools/linalg_opdsl/lang/affine.py
    R mlir/lib/Bindings/Python/mlir/tools/linalg_opdsl/lang/comprehension.py
    R mlir/lib/Bindings/Python/mlir/tools/linalg_opdsl/lang/config.py
    R mlir/lib/Bindings/Python/mlir/tools/linalg_opdsl/lang/dsl.py
    R mlir/lib/Bindings/Python/mlir/tools/linalg_opdsl/lang/scalar_expr.py
    R mlir/lib/Bindings/Python/mlir/tools/linalg_opdsl/lang/types.py
    R mlir/lib/Bindings/Python/mlir/tools/linalg_opdsl/lang/yaml_helper.py
    R mlir/lib/Bindings/Python/mlir/tools/linalg_opdsl/ops/__init__.py
    R mlir/lib/Bindings/Python/mlir/tools/linalg_opdsl/ops/core_named_ops.py
    M mlir/lib/Bindings/Python/mlir/transforms/__init__.py
    M mlir/test/Bindings/Python/context_lifecycle.py
    M mlir/test/Bindings/Python/dialects.py
    R mlir/test/Bindings/Python/dialects/linalg.py
    A mlir/test/Bindings/Python/dialects/linalg/opdsl/assignments.py
    A mlir/test/Bindings/Python/dialects/linalg/opdsl/doctests.py
    A mlir/test/Bindings/Python/dialects/linalg/opdsl/interfaces.py
    A mlir/test/Bindings/Python/dialects/linalg/opdsl/lit.local.cfg
    A mlir/test/Bindings/Python/dialects/linalg/opdsl/shape_maps_iteration.py
    A mlir/test/Bindings/Python/dialects/linalg/opdsl/test_core_named_ops.py
    A mlir/test/Bindings/Python/dialects/linalg/ops.py
    M mlir/test/Bindings/Python/ir_operation.py
    R mlir/test/Bindings/Python/tools/linalg_opdsl/assignments.py
    R mlir/test/Bindings/Python/tools/linalg_opdsl/doctests.py
    R mlir/test/Bindings/Python/tools/linalg_opdsl/interfaces.py
    R mlir/test/Bindings/Python/tools/linalg_opdsl/lit.local.cfg
    R mlir/test/Bindings/Python/tools/linalg_opdsl/shape_maps_iteration.py
    R mlir/test/Bindings/Python/tools/linalg_opdsl/test_core_named_ops.py
    M mlir/tools/mlir-tblgen/OpPythonBindingGen.cpp

  Log Message:
  -----------
  [mlir][python] Reorganize MLIR python into namespace packages.

* Only leaf packages are non-namespace packages. This allows most of the top levels to be split into different directories or deployment packages. In the previous state, the presence of __init__.py files at each level meant that the entire tree could only ever exist in one physical directory on the path.
* This changes the API usage slightly: `import mlir` will no longer do a deep import of `mlir.ir`, etc. This may necessitate some client code changes.
* Dialect gen code was restructured so that the user is responsible for providing the `my_dialect.py` file, which then must import its peer `_my_dialect_ops_gen`. This gives complete control of the dialect namespace to the user instead of to tablegen code, allowing further dialect-specific python APIs.
* Correspondingly, the previous extension modules `_my_dialect.py` are now `_my_dialect_ops_ext.py`.
* Now that the `linalg` namespace is open, moved the `linalg_opdsl` tool into it.
* This may require some corresponding downstream adjustments to npcomp, circt, et al:
  * Probably some shallow imports need to be converted to deep imports (i.e. not `import mlir` brings in the world).
  * Each tablegen generated dialect now needs an explicit `foo.py` which does a `from ._foo_ops_gen import *`. This is similar to the way that generated code operates in the C++ world.
  * If providing dialect op extensions, those need to be moved from `_foo.py` -> `_foo_ops_ext.py`.

Differential Revision: https://reviews.llvm.org/D98096