[llvm] [CI] Switch to a single runtimes build (PR #131913)

[Vlad Serebrennikov via llvm-commits]

Endilll wrote:

> The resources aren't getting diverted. We're running both systems in parallel. Buldkite has the same number of resources as before.

Good to know, but it seems that we run things twice: once via buildkite, once via github. Is that accurate?

> > 10 minutes out of how much?
> 
> Typically 30, so approximately 30% of the build/test time.

> > I can agree with that regarding C++03 build, but as C++20 modules are (slowly) adopted, Clang modules build becomes increasingly more relevant, because Clang modules shares a lot of code paths with C++20 modules.
> 
> They share code paths, sure. But they're a fairly atypical combination. I can understand wanting to test it in premerge CI, but we need to have balance we cannot test everything, and we want to provide results quickly. To me, anything that requires building a project in more than one configuration shifts the balance strongly towards exhaustive testing, which I do not think we should be doing in premerge CI.

This was discussed during Clang WG meeting yesterday. There were no objections to cut C++03 configuration, but C++26 and Clang Modules configurations were deemed important. Can we start with removing C++03?

Also, in the current state this PR removes C++03, C++26, and Modules configurations, leaving C++17 instead. It's not clear to me why it has to be this way. It's as if we don't care about anything from C++20 on, because those parts of libc++ will be disabled in C++17 mode.

> > I don't follow. Even of runtimes can be built today via LLVM_ENABLE_PROJECTS, which I'm not sure about, the code you remove relies on LLVM_ENABLE_RUNTIMES.
> 
> The ability to build runtimes with `LLVM_ENABLE_PROJECTS` is going to be deprecated (see #124009). This means we have to move all the runtimes we currently test to the runtimes build. Trying to support multiple runtime build configurations with a bunch of projects that probably don't care about those features (libc, compiler-rt, etc.) adds complexity, although is doable.

Yes, I'm aware that specifying `libcxx` and other runtimes in `LLVM_ENABLE_PROJECT` is deprecated, and that makes sense. However, the part I'm not following is how this patch help with any of that. CMake invocations that you remove don't use `LLVM_ENABLE_PROJECTS` at all, so they are not going to be affected even after this deprecated feature will be removed.

> It's hard to strike a balance against exhaustive testing and keeping premerge fast. If clang developers have evidence that this catches a significant number of issues, I think we can look at reenabling it just for builds of clang.

I've seen libc++ C++26 build catching errors last year, when I was working on this part of pre-commit CI. This is of course an anecdotal evidence, but I'd be surprised if anyone has more than that for any project in the monorepo. Above Matheus mentioned that Clang Modules build, too, catches issues for him, and given how little contributors we have in this area, testing this part of Clang becomes even more important to prevent bitrot. Also, it seems that lately modules-related bug reports are files almost on a daily basis: https://github.com/llvm/llvm-project/issues?q=is%3Aissue%20state%3Aopen%20label%3Aclang%3Amodules

> With how things are setup right now though, touching MLIR, LLVM, Flang, or Clang will cause all of these runtimes to be rebuilt, which isn't the most tractable in terms of build scalability.

https://github.com/llvm/llvm-project/blob/eb77061a428c28e9297c7a35cdbe568811025b2b/.ci/compute-projects.sh#L59-L73

Looking at the code above, I don't understand why libc++ and friends are built for MLIR, LLVM, and Flang. If that's the case, then I agree that it doesn't bring a significant amount of additional coverage, and can be disabled.

> [libc](https://github.com/llvm/llvm-project/blob/main/.github/workflows/libc-fullbuild-tests.yml) and [libcxx](https://github.com/llvm/llvm-project/blob/main/.github/workflows/libcxx-build-and-test.yaml) already have their own pre-merge tests. They use stock compilers (gcc and clang), so the additional value here is only whether Clang is regressing. If worker load is at capacity, I would even argue there is not need to bootstap-build libcxx/libc at all, for the same reason a 2-stage build of Clang is probably not worth it: A littlle more coverage, but whether can build in principle has already been verified.

I think that "whether Clang is regressing" is a significant value, given how many regressions we're seeing: https://github.com/llvm/llvm-project/issues?q=is%3Aissue%20state%3Aopen%20label%3Aclang%2Cclang%3Afrontend%2Cclang%3APCH%2Cclang%3Amodules%2Cclang%3Adiagnostics%2Cclang%3Adriver%2Cclang%3Acodegen%2Cclang%3Aheaders%2Cclang%3Aanalysis%20label%3Aregression%2Cregression%3A20%2Cregression%3A19%2Cregression%3A18%2Cregression%3A17%2Cregression%3A16.

https://github.com/llvm/llvm-project/pull/131913