[llvm-dev] Bazel support

Tue Aug 13 00:22:46 PDT 2019

Thanks for the responses everyone! Much appreciated!

I have given this some thought and the better solution I could come up with
is to build LLVM with the CMake build system inside a fresh Docker
container, upload the binaries to cloud storage, and use these binaries to
build up toolchains inside Bazel. Does this make sense?

This removes the need to change LLVM, introduce new dependencies, etc.

Also, if anyone of you have more information of how to build LLVM inside a
Docker container for Linux, MacOS, and Windows, I would really like to know
more about it.

The reason I want to do this is because I am developing a system emulator,
kind of like Qemu, but for really old machines like the Amiga and Commodore
64, and I want to build and release binaries all from within Bazel (Bazel
having native cross-compilation, and an awesome test framework).

On Tue, Aug 13, 2019 at 9:09 AM Kristina Brooks <notstina at gmail.com> wrote:

> I agree, IMHO Bazel support in-tree would likely not be amazingly
> useful as it doesn't offer a lot in relation to local builds and is
> pretty large (while GN and Ninja binaries can be compiled and
> statically linked pretty easily). To add to that, while it makes sense
> to integrate LLVM builds into large repositories that use a different
> build system (ie. TF), especially when support for multiple languages
> is desired, the upstream LLVM repositories are pretty much almost
> entirely C/C++. So I don't believe having it upstream would bring much
> overall benefit - its scope is simply too broad.
>
> Also as far as GN goes, I think it was universally agreed on that it
> wasn't a replacement for CMake but rather a second buildsystem, since
> it does bring considerable performance benefits when it comes to
> regenerating the Ninja buildfiles to those that see it as a
> bottleneck, as well as it being relatively independent and with far
> less extra baggage (unlike a typical CMake install), only requiring
> Ninja and a host toolchain to work. In a similar fashion it has
> support for compiler wrappers (ie. through "command_launcher")
> allowing things like gomacc/RBE to be used for distributed
> compilation.
>
> Thank you.
>
> On Mon, Aug 12, 2019 at 10:23 PM Chris Bieneman via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> >
> > Hi Wynand,
> >
> > My big concern is related to what Mehdi mentioned as #4. Bazel doesn't
> really handle the configuration-management that CMake does, so adding Bazel
> support would really mean supporting an extra build system with no path for
> it to replace CMake. We're kinda already in that situation with gn, but gn
> is a developer productivity tool and we don't consider changes that break
> gn to be build-breaking.
> >
> > Anyone who has been working on LLVM long enough will remember the
> headaches we had as a community when we supported both CMake and autoconf
> as primary build systems. Frequently changes would break one or the other
> build system, we had lots of issues with autoconf versioning, and it was
> generally considered an undue burden on the community to have more than one
> build system.
> >
> > To justify adding a new build system I think one big question we would
> need to answer is what does this get us that we *can't* get from CMake, and
> does that justify the burden?
> >
> > In your initial email you mentioned remote builds and caching. CMake
> supports both of these things if the underlying generator does. For many
> releases, CMake supports an option `CMAKE_<LANG>_COMPILER_LAUNCHER` which
> integrates with caching tools like ccache, and distribution tools like
> distcc.
> >
> > Personally I think the best path for a real solution to building LLVM
> with Bazel would likely be for CMake to get support for generating Bazel
> build files. That is a topic that has come up on the CMake developer
> mailing lists, but I don't believe there has been any progress on it due to
> nobody actively working on it.
> >
> > -Chris
> >
> >
> > On Aug 8, 2019, at 4:24 PM, Mehdi AMINI via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > Hi,
> >
> > On Thu, Aug 8, 2019 at 4:15 PM Wynand Pieterse via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >>
> >> Hi all, trust all is well!
> >>
> >> I was wondering if LLVM would have a Bazel integrated build in the
> future? I can imagine the benefits this could bring, especially with
> regards to remote builds and caching.
> >
> >
> > There was a round-table at EuroLLVM on this topic. I couldn't
> participate fully (I was writing my slides...) but I remember some key
> elements that we not playing in favor of Bazel:
> > 1) It is written in Java, and not widely available by default on every
> platform. This is not a nice property to base a key piece of the
> requirement for getting started with LLVM.
> > 2) It is memory heavy (maybe related to the previous point...): it
> consumes multiple GB by itself when building LLVM.
> > 3) There are other solutions to have remote build execution and caching,
> for instance the Chrome Goma Client implements the same protocol for remote
> execution as Bazel and (in theory at least) can integrate well with `CMake`
> and `ninja` to build a project like LLVM (I am actually playing with this
> these days).
> > 4) While sandboxing and the declarative approach that Bazel is using is
> nice, it isn't clear to me that Bazel provides the same ability that CMake
> has to detect the environment and configure the build. The "regular" LLVM
> build is likely easy to mimic with Bazel, but can you plumb all the knob
> and make it work on every platform that LLVM support?
> >
> >>
> >> I'm thinking of dumping the LLVM source-tree into my code-base and
> experiment with building LLVM via Bazel. Will pull requests be accepted for
> this? I won't mind contributing my changes back into the upstream.
> >
> >
> > If you want to get started an play with it, the TensorFlow project
> builds LLVM with Bazel. They maintain their own BUILD file for LLVM (it is
> probably not complete, but maybe a good way to get started).
> >
> > Best,
> >
> > --
> > Mehdi
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190813/652751f6/attachment.html>