[llvm] [bazel]: de-alias pybind11 headers target (PR #79676)

Jeremy Kun via llvm-commits llvm-commits at lists.llvm.org
Fri Jan 26 21:16:44 PST 2024


https://github.com/j2kun created https://github.com/llvm/llvm-project/pull/79676

In trying to set up python headers in an out-of-tree bazel MLIR project, I encountered the `pybind11_bazel` project, and found that the `@python_runtime` target used here is not defined by it.

Instead, it seems that `@python_runtime` is an alias used in some projects like Tensorflow (see
https://github.com/tensorflow/tensorflow/blob/322936ffdd96ee59e27d028467fe458859cf3855/third_party/python_runtime/BUILD#L7-L7), where it is aliased to `@local_config_python`. In fact, `@local_config_python` is defined by `@pybind11_bazel`, and so it seems that this layer of indirection no longer serves a purpose, and instead just prevents anyone who doesn't clone Tensorflow's config from using the python bindings here.

This commit updates the dependent targets to their canonical de-aliased equivalents, and I suspect this will not even break any downstream users since the new target is defined in those projects already.

Without this change, running, for example

```
bazel build @llvm-project//mlir:MLIRBindingsPythonCore
```

gives the error

```
no such package '@python_runtime//': The repository '@python_runtime'
could not be resolved: Repository '@python_runtime' is not defined and
referenced by '@llvm-project//mlir:MLIRBindingsPythonCore'
```

Minimal reproduction in https://github.com/j2kun/test_mlir_bazel_pybind, which, when pointing to a local LLVM repository that has this change (see `bazel/import_llvm.bzl` in that repository), results in that build succeeding.

Hat tip to Maksim Levental for going on an hours-long investigation with me to figure this out.

>From 281c7899edc97b91e4a90333aec4a836e18e3cae Mon Sep 17 00:00:00 2001
From: Jeremy Kun <jkun at google.com>
Date: Fri, 26 Jan 2024 20:52:10 -0800
Subject: [PATCH] [bazel]: de-alias pybind11 headers target

In trying to set up python headers in an out-of-tree bazel MLIR project,
I encountered the `pybind11_bazel` project, and found that the
`@python_runtime` target used here is not defined by it.

Instead, it seems that `@python_runtime` is an alias used in some
projects like Tensorflow (see
https://github.com/tensorflow/tensorflow/blob/322936ffdd96ee59e27d028467fe458859cf3855/third_party/python_runtime/BUILD#L7-L7),
where it is aliased to `@local_config_python`. In fact,
`@local_config_python` is defined by `@pybind11_bazel`, and so it seems
that this layer of indirection no longer serves a purpose, and instead
just prevents anyone who doesn't clone Tensorflow's config from using
the python bindings here.

This commit updates the dependent targets to their canonical de-aliased
equivalents, and I suspect this will not even break any downstream users
since the new target is defined in those projects already.

Without this change, running, for example

```
bazel build @llvm-project//mlir:MLIRBindingsPythonCore
```

gives the error

```
no such package '@python_runtime//': The repository '@python_runtime'
could not be resolved: Repository '@python_runtime' is not defined and
referenced by '@llvm-project//mlir:MLIRBindingsPythonCore'
```

Minimal reproduction in https://github.com/j2kun/test_mlir_bazel_pybind,
which, when pointing to a local LLVM repository that has this change
(see `bazel/import_llvm.bzl` in that repository), results in that build
succeeding.

Hat tip to Maksim Levental for going on an hours-long investigation with
me to figure this out.
---
 .../llvm-project-overlay/mlir/BUILD.bazel     | 25 ++++++++++++++-----
 1 file changed, 19 insertions(+), 6 deletions(-)

diff --git a/utils/bazel/llvm-project-overlay/mlir/BUILD.bazel b/utils/bazel/llvm-project-overlay/mlir/BUILD.bazel
index 217518cf24a7a7..da61176c786b44 100644
--- a/utils/bazel/llvm-project-overlay/mlir/BUILD.bazel
+++ b/utils/bazel/llvm-project-overlay/mlir/BUILD.bazel
@@ -870,6 +870,19 @@ exports_files(
     glob(["lib/Bindings/Python/**/*.cpp"]),
 )
 
+# In the targets related to Python bindings, the projects @pybind11 and
+# @local_config_python are defined by @pybind11_bazel. The latter contains
+# python headers, and can be configured in an out-of-tree bazel project via
+#
+#   load("@pybind11_bazel//:python_configure.bzl", "python_configure")
+#   python_configure(name = "local_config_python")
+#
+# For more up-to-date instructions, see
+# https://github.com/pybind/pybind11_bazel
+#
+# Some out-of-tree projects alias @python_runtime//:headers to
+# @local_config_python//:python_headers.
+
 MLIR_BINDINGS_PYTHON_HEADERS = [
     "lib/Bindings/Python/*.h",
     "include/mlir-c/Bindings/Python/*.h",
@@ -890,7 +903,7 @@ cc_library(
     deps = [
         ":CAPIIRHeaders",
         "@pybind11",
-        "@python_runtime//:headers",
+        "@local_config_python//:python_headers",
     ],
 )
 
@@ -908,7 +921,7 @@ cc_library(
     deps = [
         ":CAPIIR",
         "@pybind11",
-        "@python_runtime//:headers",
+        "@local_config_python//:python_headers",
     ],
 )
 
@@ -951,7 +964,7 @@ cc_library(
         ":Support",
         "//llvm:Support",
         "@pybind11",
-        "@python_runtime//:headers",
+        "@local_config_python//:python_headers",
     ],
 )
 
@@ -971,7 +984,7 @@ cc_library(
         ":MLIRBindingsPythonHeaders",
         "//llvm:Support",
         "@pybind11",
-        "@python_runtime//:headers",
+        "@local_config_python//:python_headers",
     ],
 )
 
@@ -1082,7 +1095,7 @@ cc_binary(
         ":CAPIExecutionEngine",
         ":MLIRBindingsPythonHeadersAndDeps",
         "@pybind11",
-        "@python_runtime//:headers",
+        "@local_config_python//:python_headers",
     ],
 )
 
@@ -1102,7 +1115,7 @@ cc_binary(
         ":CAPILinalg",
         ":MLIRBindingsPythonHeadersAndDeps",
         "@pybind11",
-        "@python_runtime//:headers",
+        "@local_config_python//:python_headers",
     ],
 )
 



More information about the llvm-commits mailing list