[PATCH] D34197: Added Dockerfiles to build clang from sources.

Ilya Biryukov via Phabricator via llvm-commits llvm-commits at lists.llvm.org
Mon Jun 19 10:17:45 PDT 2017


ilya-biryukov added a comment.

> Feel free to advocate for it, but except that stating an opinion above I'm missing you arguments.

For a newcomer or even a contributor, not already familiar with LLVM's build system, it's not at all obvious which arguments one must pass to cmake to make build work for him.
Given that the docker images we're producing is a distribution of LLVM tools, rather than a build/testing framework, there's no point in coupling its interface to the LLVM buildsystem, effectively making it as complicated as the buildsystem itself.

E.g., say someone would like to provide deb packages for clang, lld, etc and is willing to use Dockerfiles we provide. Say, the person is not an LLVM contributor.
They won't really want to dive into the complexity of building LLVM, but it won't take them long to figure out they need to run `build_docker_image.sh debian8 myrepo/clang_deb staging --with_clang --with_lld --with_libc++`.
For software distributions, the things that really matter are:

- Which components do you include in the distribution, i.e. clang, lld, libc++, sanitizers, etc.
- Configurations of those components, i.e. whether to build with sanitizers, whether to use LTO, which backends to include, etc. (This is the trickiest part from my perspective)
- SVN revision/branch to build from.

Everything else can be figured out by the scripts: SVN projects you need to checkout, whether you should use one of 2-stage build script from 'Advanced Build Configuration' you pointed out or just run cmake twice, which cmake targets to build/install for each of the components, which cmake flags to pass, etc.

I do think you make a very good point that all of this 'is already there' in the cmake scripts and providing an interface to them gives greater flexibility.
I think all of those arguments are also true for standalone LLVM builds outside docker containers. But in docker we have a very controllable environment, which makes it possible to make things that actually work **and** have a nice interface.

> You're stating some priority above without clearly defining the goals. I'm missing the big picture: who is the target of these images? What problem are we addressing? In my previous comment I put forward that one the main use case for the LLVM project would be to help people to start using LLVM (which is why I mentioned  http://llvm.org/docs/GettingStarted.html )

Our goal would be to build docker images, containing fresh versions of clang (and later, possibly, other LLVM tools). These Dockerfiles are a step in that direction.
We have plans to add sanitizers into those images afterwards, maybe some other LLVM tools as well.

> Now why should the interface for this script be different from the build interface we *already* have? 
>  You mention that you're looking for a `make a 2-stage bootstrap and install lld+clang` kind of interface, but that's already possible with our cmake AFAIK, it should something like: `cmake  -DCLANG_ENABLE_BOOTSTRAP=On  -DLLVM_ENABLE_PROJECTS="lld;clang" && make install`

It is possible to do this (and other things, of course) with cmake, my point is that cmake is not great at giving you a good and discoverable interface to all of that.

BTW, it would be very nice for us if we had some scripts for our images, and then we could iterate to make them more configurable/flexible and discuss/implement alternative designs.



================
Comment at: utils/docker/debian8/build/build_install_clang.sh:62
+svn co -q $SVN_REV_ARG "http://llvm.org/svn/llvm-project/cfe/$LLVM_BRANCH" \
+  "$CLANG_BUILD_DIR/llvm/tools/clang"
+
----------------
mehdi_amini wrote:
> ilya-biryukov wrote:
> > mehdi_amini wrote:
> > > That's where it becomes to get interesting: why only cfe?
> > > Why wouldn't we package libc++, compiler-rt, lld, lldb, etc.
> > > I understand you may not care, or that it even may be a burden (extra size), but as LLVM developers it isn't clear to me why we wouldn't have by default everything included.
> > > 
> > > It also seems to me that this script `build_install_clang.sh` is `yet another build script` to maintain while we already have some in the repo.
> > > 
> > > I suspect everyone will have different needs, and we will end up with a very complicated script to handle all possibilities (I for one am interested into using this, but with a very different config: bootstrap+LTO+lld+only some targets to begin with).
> > > 
> > > We really need more thoughts around this.
> > The maintenance cost is exactly the reason why we hard-coded the script to build only clang. It makes the script very easy. That said, I'm in favour of more configuration in the future, building clang is just a first iteration.
> > 
> > I can spend some time making the script configurable enough to cover your use-case too. Than we could iterate to make it even more configurable.
> > > I suspect everyone will have different needs, and we will end up with a very complicated script to handle all possibilities
> > Could you elaborate more on what exactly are 'only some targets to begin with'?
> I rather not land this before we settle on this at this point (I don't see any urgency to land it that would motivate doing otherwise).
> 
> > Could you elaborate more on what exactly are 'only some targets to begin with'?
> 
> `-DLLVM_TARGETS_TO_BUILD=X86`
Got it, backend targets.  Thanks for elaborating on that.


================
Comment at: utils/docker/debian8/build/build_install_clang.sh:81
+      -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX="$CLANG_INSTALL_DIR" \
+      "$CLANG_BUILD_DIR/llvm"
+
----------------
mehdi_amini wrote:
> ilya-biryukov wrote:
> > mehdi_amini wrote:
> > > LLVM has a bootstrap process in a single cmake invocation, and can even build LLD during stage 1 and use it to link stage 2 (with LTO for example).
> > Thanks, will look into using it.
> FYI: http://llvm.org/docs/AdvancedBuilds.html
I've already found that, but thanks for the link nevertheless.


https://reviews.llvm.org/D34197





More information about the llvm-commits mailing list