[PATCH] [DOCS] How to Setup a Windows Builder

Tue Nov 12 23:17:58 PST 2013

Hi Sean,

Thanks for your great list of issues with the document. I fully agree, now
that I've been made aware of it, that the document is very poorly
organized.  I am thinking that I could make it like this:

SUPPORTED TOOLCHAINS
Toolchain this and toolchain that.

INSTALLATION MATRIX
          Tool A   Tool B  Tool C
Step 1  X           X
Step 2  X                     X
Step 3               X
Step 4  X           X
Step 5                         X

INSTALLATION STEPS
Install step 1:
Do this.

Install step 2:
Do that.

Install step 3:
Do something.

About using Python: As much I love Python, in comparison to Batch files,
Python programs cannot update the environment of the running shell on
Windows.  Windows has no "source" command.  So to set the environment we
need Batch files.  And that makes it odd to use Python scripts to retrieve
files from Subversion.  For the sake of consistency, I'd like to stick with
Batch.

As for mentioning only one of MinGW32 or MinGW64, I don't think that's
feasible.  I'd much prefer to make a clear list of the toolsets that we
support, including MinGW32 and MinGW64, and then do as outlined above.

Building Ninja from source on Windows requires having Microsoft Visual
Studio installed (AFAIK).  When I first started out on the document, my
goal was to document how to build without using MSVC.  I'll check up on it,
but I don't think you can easily build Ninja using MinGW.  Your idea about
hosting a known good build on LLVM sounds very good to me.

Perhaps a separate document on making a Windows buildbot slave out of a
system already prepared to be a Windows builder would be in order.  I'll
see if such a document is really necessary once this doc is done.

As for all of your other comments, I won't comment on them but rather go
and fix them.  Thanks again!

Sincerely,
Mikael Lyngvig

2013/11/13 Sean Silva <silvas at purdue.edu>

> (There is a summary of the biggest issues at the bottom)
>
> Note, some of my comments may be addressed by content further in the
> document (I'm reading mostly sequentially). Such comments should not be
> brushed off as "oh, he just didn't get there yet"; they should be
> interpreted as meaning that the content should be reorganized so that a
> reader does not encounter such questions in the first place. Also if I make
> a comment that seems to be due to me misunderstanding some part of what is
> written, that should be interpreted as meaning that that part of the
> document should be revised to be clearer.
>
> +Building LLVM on Windows using the MinGW32, MinGW64, and Visual Studio
> toolchains
> +is complicated. [...]
>
> This seems completely generic about building on windows. Where do
> buildbots come into this? This should be clarified in the introduction.
>
> +[...]This document aims to make it easy for you to configure
> +Windows to build LLVM with the major windows toolchains, bourne shells,
> and
> +natively with Microsoft's ``cmd.exe``.
>
> Here you list 3 different things as the major parts of this document. They
> should each be a top-level section (i.e. accessible through the table of
> contents) if they are so prominent.
>
> +In the text that follows, your LLVM working directory will be referred to
> as
> +``%LLVMDIR%``.
>
> This is ambiguous/confusing. A lot of people have a directory with
> different llvm checkouts/builddir's that could be considered an "LLVM
> working directory". Please explicitly say the root directory of the source
> checkout.
>
> +MinGW currently exists in two versions: MinGW32 (32-bit tools) and MinGW64
> +(64-bit tools).  The 32-bit tools are readily available from the `main
> MinGW
> +website <http://www.mingw.org>`_.  The 64-bit tools are available from
> the
> +`MinGW W64 website <http://mingw-w64.sourceforge.net>`_.
>
> Why would a user prefer one or the other? Can we mention only one to make
> things simpler?
>
> +Install GnuWin32 by following the *Looking for the latest version?* link
> at
> +`GnuWin32 on SourceForge.net <gnuwin32.sourceforge.net>`_.  Download and
> run
>
> I don't see a "Looking for the latest version?" link there. Also, it seems
> like these are 32-bit. Does that interact in any way with a user's choice
> for 32/64 bit in e.g. the mingw choice?
>
> +The primary purpose of GnuWin32 is to supply utilities missing in MinGW-32
> +MSYS. GnuWin32 will be after MSYS in the path and handle utilities that
> are not
> +available in MSYS.
>
> Which utilities? Why are they needed?
>
> +Installing Subversion
> +---------------------
>
> Many people are much more comfortable with git. Please include
> instructions for them.
>
> +Ninja is a *very* fast build tool that is one of the builders in CMake.
> +You can download a binary from
> +`CMakeScript <http://sourceforge.net/projects/cmakescript/files/>`_.  You
> +can save it to your standard tools directory or to ``%LLVMDIR%`` and
> invoke it
> +from there.
>
> That link seems really sketchy (and I found it hard to find the ninja
> binary). If it is so vital to provide a binary download (see the comment
> just below), I would prefer to build a known-good version of ninja and host
> it on llvm.org.
>
> +
> +**Notice:** This version of Ninja requires you to have the Microsoft
> Visual
> +C++ 2010 Redistributable installed.  You can get it from `Microsoft's
> Download
> +Center <http://www.microsoft.com/en-us/download/details.aspx?id=5555>`_.
>
> This almost seems more complicated than just building ninja from source?
> It's literally just git clone and ./bootstrap.py.
>
> +   | Devel/DejaGnu.
> +   | Interpreter/tcl.
> +   | Tcl/expect.
>
> Do we still need these?
>
> +**Notice:** If you do not plan to run the test suite, or sshd server, you
> don't
> +need Cygwin. You can build LLVM + Clang with only Subversion, MingwNN,
> and CMake.
>
> I feel like the way you are handling these notices is backwards. Your
> current approach seems to be to spew all things that could possibly need to
> be installed and then have little "Notice" sections telling you whether
> that section is actually of interest to you; this is extremely inefficient
> and confusing for the reader. I recommend turning that upside down: create
> a nested list whose structure shows which components are necessary for what
> you want to do. E.g.
>
> - Required:
>   + Thing1. <http://downloadlink1>. Succinct description of any
> nonobvious specific steps.
>   + Thing2. <http://downloadlink2>. ...
>   + ...
> - Only need these if you want to do X:
>   + OptionalThing1 ...
>   + ...
> - Only need these if you want to do Y:
>   + OtherOptionalThing2 ...
>   + ...
>
> +If you want to setup your Windows build machine as Buildslave in the LLVM
> +Buildbot Infrastructure (Zorg project),
>
> In view of the title of this document, it seems like this "if" is
> pointless since it will always be true.
>
> +[...]For unix style builds, [...]
>
> I just had to read through setup instructions for setting up a bunch of
> unix utilities and now you're telling me that unix-style builds are only
> one of the options, and so I was wasting my time if I wasn't doing a
> unix-style build? Please clarify. Try to achieve the following goal: The
> document is structured/organized in such a way that a reader can know
> exactly which parts of the document are relevant for their situation and
> will spend (i.e. waste) no time looking at instructions that are not
> relevant for their desired configuration.
>
> +[...]MSVC and Intel compilers require setup
> +scripts that are invoked from existing environment variables. An example
> for MSVC10
> +is included. [...]
>
> Ok now this is worrying. This is a HowTo, there shouldn't be a big piece
> of functionality like a crucial script missing. This guide cannot purport
> to be a HowTo for configurations for which it doesn't provide this
> component, so either clarify that this HowTo is only for MSVC10 (and also
> unix-like), or (preferred) provide instructions for the other scenarios.
> Also, AFAIK almost all LLVM MSVC users are running 12.
>
> +    svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm-trunk
>
> Please provide a git option as well, especially since you are assuming git
> is present below.
>
> +.. code-block:: bat
>
> bat sucks, can you just write these scripts in Python (with the subprocess
> module)? It should be much clearer than goto-ridden bat.
>
> +The whole thing takes about five minutes so you can safely go get a cup of
> +coffee or what you normally do when you have to wait for your machine.
>
> I doubt this is consistently five minutes. Checkout time is highly
> dependent on network speed and may take much longer. You don't need to give
> a numerical estimate ("safely go get a cup of coffee" is fine).
>
> +If everything went okay and there were no error messages, you've
> succesfully
> +got all of the relevant sources from the LLVM Subversion repository.
>
> And if there were error messages? Then what do I do? (suggestion: direct
> the reader to the mailing lists)
>
> +There's a complete list of CMake variables in the `Building LLVM with
> CMake
> +<http://llvm.org/docs/CMake.html>`_ document.
>
> Use :ref:`CMake` instead of an HTTP link.
>
> +**Notice:** ``cmake-gui`` is not part of the special Ninja version of
> CMake.
>
> What "special Ninja version of CMake"?
>
> +[...] Preferredly, you can do a ``ninja install`` to make
> +the binaries, libraries, and headers, and have them installed in the
> location
> +you specified as part of the ``CMAKE_INSTALL_PREFIX`` variable above.
>
> Is this really preferred? Especially on a bot (consider the title), this
> doesn't seem like it is "preferred" at all.
>
>
> The biggest issues I see are
> 1) Extremely poor organization. Try to achieve the following goal: The
> document is structured/organized in such a way that a reader can know
> exactly which parts of the document are relevant for their situation and
> will spend (i.e. waste) no time looking at instructions that are not
> relevant for their desired configuration.
>   + For example, I still don't know which "toolchains" this is supposed to
> cover. Intel and MSVC were mentioned in passing?
>   + There's a *ton* of software that this is guiding the reader to
> install. I don't use Windows much, but last I did (this Summer), all it
> took to build LLVM was git, MSVC, CMake, and Ninja, and it was like 1 step,
> and I didn't need to significantly fiddle with any paths or anything. Where
> is that "easy" option?
> 2) This seems to have almost nothing to do with builbots, despite having
> the phrase "for the LLVM Buildbot infrastructure" in the title. I suggest
> just removing that part of the title (and renaming the file).
> 3) Please just obliterate the sentence: "This document aims to make it
> easy for you to configure Windows to build LLVM with the major windows
> toolchains, bourne shells, and natively with Microsoft’s cmd.exe.". I
> literally just wrote like 10 lines of text describing things that are wrong
> with this sentence (I decided it's just not worth getting into). Please
> just nuke it and write a replacement in the context of the (freshly
> organized) document.
>
> Bottom line: This document has lots of good content (clearly gleaned from
> "battle scars" doing this kind of setup in the field), but it doesn't seem
> to have a very good idea of what it is trying to accomplish, and doesn't
> provide any way for the reader to be able to  efficiently navigate the
> content or know what is relevant. It needs some reorganization, but I would
> like to see something like this added to our docs.
>
> -- Sean Silva
>
>
> On Mon, Nov 11, 2013 at 6:50 PM, Mikael Lyngvig <mikael at lyngvig.org>wrote:
>
>> This document describes how to set up a Windows builder and is a document
>> that quite a few have asked about during the past few years.
>>
>> Please find attached diff.
>>
>> Please direct all criticism, error reports, and enhancement requests to
>> me and I'll see what I can do.
>>
>>
>> Sincerely,
>> Mikael Lyngvig
>>
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20131113/1ddd9dd3/attachment.html>