[PATCH] D136496: [bazel] Make labels to third-party dependencies explicit

[Aaron Siddhartha Mondal via Phabricator via llvm-commits]

aaronmondal added a comment.

In D136496#3906794 <https://reviews.llvm.org/D136496#3906794>, @GMNGeoffrey wrote:

> I think this is coming from my complete ignorance of bzlmod, but why exactly do we have a separate llvm-project-overlay repository here? Why isn't it pulling from utils/bazel?

The outline is as follows.

1. A user imports `bazel_dep(name="llvm-project-overlay", version = "...")`.
2. bzlmod will now go to whatever bazel registry was specified and pull the sources declared there. While the module name is `@llvm-project-overlay`, the actual sources are the regular sources of ad github.com/llvm/llvm-project, slightly patched with `MODULE.bazel`, and empty `BUILD.bazel` and `WORKSPACE.bazel` files. This behavior is completly independent of anything implemented in the actually fetched repo.
3. Now, an extension is called from the `@llvm-project-overlay`'s `MODULE.bazel` file to configure the llvm project (D137008 <https://reviews.llvm.org/D137008>).  The extension implementation has to be located via `@llvm-project-overlay//utils/bazel:extensions.bzl`, since we currently have no notion of `@llvm-raw` or `@llvm-project`. Two things can happen now.
  - 3a. If no commit is specified, the extension re-registers the `@llvm-project-overlay` archive as `@llvm-raw`.  It then applies the `configure` logic to create `@llvm-project` from these sources. The `@llvm-project-overlay` module makes these workspaces available to importers via `use_repo(...,)` in the `@llvm-project-overlay`'s `MODULE.bazel` file.
  - 3b. If a commit (and optional patches) are specified, we download ANOTHER copy of the llvm sources at the specified commit (and optionally patch them) and register those as `@llvm-raw`. Now the `configure` logic from `@llvm-project-overlay//utils/bazel:configure.bzl` is applied, to the sources of `@llvm-raw` to create `@llvm-project`.

So basically, we always want `configure` to run on `@llvm-raw`, never on `@llvm-project-overlay`, since that would make patching and custom commit checkouts even more complicated than they already are 😅

The **3b** part i there so that we do not need to update the version of this module too often in bazel registries. The implementation in D137008 <https://reviews.llvm.org/D137008> lets us get away with only rarely updating the module in bazel registries, since we can use an older version of llvm to just download a newer version by bumping the commit on the importer side. Without **3b** (i.e. without custom commit checkout and patching) we would have to frequently update the registry version to stay at head in downstream projects.

---

Note that this only covered the kinds of builds downstream users would run. In-tree builds work more or less like the current WORKSPACE in-tree build.

1. Assume we are in `utils/bazel`.
2. Dependencies like `zlib` and `rules_cc` are fetched from a bazel registry.
3. We import the module extension as `//:extensions.bzl`, since we are already in `utils/bazel`.
4. We use essentially the same flow as **3a** from above. I believe there may still be some issues with the current D137008 <https://reviews.llvm.org/D137008> in this case though, since we somehow need to get a `WORKSPACE.bazel` and `BUILD.bazel` into the top level of the repo, but we cannot use native `local_repository` from module extensions. This may resolve itself as bzlmod matures.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D136496/new/

https://reviews.llvm.org/D136496