[PATCH] [DOCS] How to Setup a Windows Builder

[Sean Silva]

On Wed, Nov 13, 2013 at 2:17 AM, Mikael Lyngvig <mikael at lyngvig.org> wrote:

> Hi Sean,
>
> Thanks for your great list of issues with the document. I fully agree, now
> that I've been made aware of it, that the document is very poorly
> organized.  I am thinking that I could make it like this:
>
> SUPPORTED TOOLCHAINS
> Toolchain this and toolchain that.
>
> INSTALLATION MATRIX
>           Tool A   Tool B  Tool C
> Step 1  X           X
> Step 2  X                     X
> Step 3               X
> Step 4  X           X
> Step 5                         X
>
> INSTALLATION STEPS
> Install step 1:
> Do this.
>
> Install step 2:
> Do that.
>
> Install step 3:
> Do something.
>

+1 I hadn't thought of the matrix idea, but I like it. If you need info
about the reStructuredText syntax for something, <
http://sphinx-doc.org/rest.html> should have what you need.


>
> About using Python: As much I love Python, in comparison to Batch files,
> Python programs cannot update the environment of the running shell on
> Windows.  Windows has no "source" command.  So to set the environment we
> need Batch files.  And that makes it odd to use Python scripts to retrieve
> files from Subversion.  For the sake of consistency, I'd like to stick with
> Batch.
>

Ok, if there's something that only batch can do, then I guess there's no
choice. Please keep in mind trying to make the batch script as simple as
possible. Those nested goto's and stuff I saw really made my eyes glaze
over. It may be a readability win to have the batch script contain just
something like this <http://stackoverflow.com/a/1195651/394436> (i.e. it
invokes Python to generate a sequence of commands (do the control flow in
Python and output just straight line batch commands) into a temporary file
and then execute and delete the temporary). I'm not sure if that would be a
win though, it's pretty hacky; it's something to keep in mind as a
possibility. I would say that reorganizing the document is a higher
priority though for the moment.


>
> As for mentioning only one of MinGW32 or MinGW64, I don't think that's
> feasible.  I'd much prefer to make a clear list of the toolsets that we
> support, including MinGW32 and MinGW64, and then do as outlined above.
>
> Building Ninja from source on Windows requires having Microsoft Visual
> Studio installed (AFAIK).  When I first started out on the document, my
> goal was to document how to build without using MSVC.  I'll check up on it,
> but I don't think you can easily build Ninja using MinGW.  Your idea about
> hosting a known good build on LLVM sounds very good to me.
>
> Perhaps a separate document on making a Windows buildbot slave out of a
> system already prepared to be a Windows builder would be in order.  I'll
> see if such a document is really necessary once this doc is done.
>

That sounds like a good plan.


>
> As for all of your other comments, I won't comment on them but rather go
> and fix them.  Thanks again!
>

Great, thanks! Like I said this document has a lot of content that I think
we are currently missing from the docs and I would love to get it added; I
look forward to seeing the next iteration on this patch.

-- Sean Silva


>
>
> Sincerely,
> Mikael Lyngvig
>
>
> 2013/11/13 Sean Silva <silvas at purdue.edu>
>
>> (There is a summary of the biggest issues at the bottom)
>>
>> Note, some of my comments may be addressed by content further in the
>> document (I'm reading mostly sequentially). Such comments should not be
>> brushed off as "oh, he just didn't get there yet"; they should be
>> interpreted as meaning that the content should be reorganized so that a
>> reader does not encounter such questions in the first place. Also if I make
>> a comment that seems to be due to me misunderstanding some part of what is
>> written, that should be interpreted as meaning that that part of the
>> document should be revised to be clearer.
>>
>> +Building LLVM on Windows using the MinGW32, MinGW64, and Visual Studio
>> toolchains
>> +is complicated. [...]
>>
>> This seems completely generic about building on windows. Where do
>> buildbots come into this? This should be clarified in the introduction.
>>
>> +[...]This document aims to make it easy for you to configure
>> +Windows to build LLVM with the major windows toolchains, bourne shells,
>> and
>> +natively with Microsoft's ``cmd.exe``.
>>
>> Here you list 3 different things as the major parts of this document.
>> They should each be a top-level section (i.e. accessible through the table
>> of contents) if they are so prominent.
>>
>> +In the text that follows, your LLVM working directory will be referred
>> to as
>> +``%LLVMDIR%``.
>>
>> This is ambiguous/confusing. A lot of people have a directory with
>> different llvm checkouts/builddir's that could be considered an "LLVM
>> working directory". Please explicitly say the root directory of the source
>> checkout.
>>
>> +MinGW currently exists in two versions: MinGW32 (32-bit tools) and
>> MinGW64
>> +(64-bit tools).  The 32-bit tools are readily available from the `main
>> MinGW
>> +website <http://www.mingw.org>`_.  The 64-bit tools are available from
>> the
>> +`MinGW W64 website <http://mingw-w64.sourceforge.net>`_.
>>
>> Why would a user prefer one or the other? Can we mention only one to make
>> things simpler?
>>
>> +Install GnuWin32 by following the *Looking for the latest version?* link
>> at
>> +`GnuWin32 on SourceForge.net <gnuwin32.sourceforge.net>`_.  Download
>> and run
>>
>> I don't see a "Looking for the latest version?" link there. Also, it
>> seems like these are 32-bit. Does that interact in any way with a user's
>> choice for 32/64 bit in e.g. the mingw choice?
>>
>> +The primary purpose of GnuWin32 is to supply utilities missing in
>> MinGW-32
>> +MSYS. GnuWin32 will be after MSYS in the path and handle utilities that
>> are not
>> +available in MSYS.
>>
>> Which utilities? Why are they needed?
>>
>> +Installing Subversion
>> +---------------------
>>
>> Many people are much more comfortable with git. Please include
>> instructions for them.
>>
>> +Ninja is a *very* fast build tool that is one of the builders in CMake.
>> +You can download a binary from
>> +`CMakeScript <http://sourceforge.net/projects/cmakescript/files/>`_.
>>  You
>> +can save it to your standard tools directory or to ``%LLVMDIR%`` and
>> invoke it
>> +from there.
>>
>> That link seems really sketchy (and I found it hard to find the ninja
>> binary). If it is so vital to provide a binary download (see the comment
>> just below), I would prefer to build a known-good version of ninja and host
>> it on llvm.org.
>>
>> +
>> +**Notice:** This version of Ninja requires you to have the Microsoft
>> Visual
>> +C++ 2010 Redistributable installed.  You can get it from `Microsoft's
>> Download
>> +Center <http://www.microsoft.com/en-us/download/details.aspx?id=5555>`_.
>>
>> This almost seems more complicated than just building ninja from source?
>> It's literally just git clone and ./bootstrap.py.
>>
>> +   | Devel/DejaGnu.
>> +   | Interpreter/tcl.
>> +   | Tcl/expect.
>>
>> Do we still need these?
>>
>> +**Notice:** If you do not plan to run the test suite, or sshd server,
>> you don't
>> +need Cygwin. You can build LLVM + Clang with only Subversion, MingwNN,
>> and CMake.
>>
>> I feel like the way you are handling these notices is backwards. Your
>> current approach seems to be to spew all things that could possibly need to
>> be installed and then have little "Notice" sections telling you whether
>> that section is actually of interest to you; this is extremely inefficient
>> and confusing for the reader. I recommend turning that upside down: create
>> a nested list whose structure shows which components are necessary for what
>> you want to do. E.g.
>>
>> - Required:
>>   + Thing1. <http://downloadlink1>. Succinct description of any
>> nonobvious specific steps.
>>   + Thing2. <http://downloadlink2>. ...
>>   + ...
>> - Only need these if you want to do X:
>>   + OptionalThing1 ...
>>   + ...
>> - Only need these if you want to do Y:
>>   + OtherOptionalThing2 ...
>>   + ...
>>
>> +If you want to setup your Windows build machine as Buildslave in the
>> LLVM
>> +Buildbot Infrastructure (Zorg project),
>>
>> In view of the title of this document, it seems like this "if" is
>> pointless since it will always be true.
>>
>> +[...]For unix style builds, [...]
>>
>> I just had to read through setup instructions for setting up a bunch of
>> unix utilities and now you're telling me that unix-style builds are only
>> one of the options, and so I was wasting my time if I wasn't doing a
>> unix-style build? Please clarify. Try to achieve the following goal: The
>> document is structured/organized in such a way that a reader can know
>> exactly which parts of the document are relevant for their situation and
>> will spend (i.e. waste) no time looking at instructions that are not
>> relevant for their desired configuration.
>>
>> +[...]MSVC and Intel compilers require setup
>> +scripts that are invoked from existing environment variables. An example
>> for MSVC10
>> +is included. [...]
>>
>> Ok now this is worrying. This is a HowTo, there shouldn't be a big piece
>> of functionality like a crucial script missing. This guide cannot purport
>> to be a HowTo for configurations for which it doesn't provide this
>> component, so either clarify that this HowTo is only for MSVC10 (and also
>> unix-like), or (preferred) provide instructions for the other scenarios.
>> Also, AFAIK almost all LLVM MSVC users are running 12.
>>
>> +    svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm-trunk
>>
>> Please provide a git option as well, especially since you are assuming
>> git is present below.
>>
>> +.. code-block:: bat
>>
>> bat sucks, can you just write these scripts in Python (with the
>> subprocess module)? It should be much clearer than goto-ridden bat.
>>
>> +The whole thing takes about five minutes so you can safely go get a cup
>> of
>> +coffee or what you normally do when you have to wait for your machine.
>>
>> I doubt this is consistently five minutes. Checkout time is highly
>> dependent on network speed and may take much longer. You don't need to give
>> a numerical estimate ("safely go get a cup of coffee" is fine).
>>
>> +If everything went okay and there were no error messages, you've
>> succesfully
>> +got all of the relevant sources from the LLVM Subversion repository.
>>
>> And if there were error messages? Then what do I do? (suggestion: direct
>> the reader to the mailing lists)
>>
>> +There's a complete list of CMake variables in the `Building LLVM with
>> CMake
>> +<http://llvm.org/docs/CMake.html>`_ document.
>>
>> Use :ref:`CMake` instead of an HTTP link.
>>
>> +**Notice:** ``cmake-gui`` is not part of the special Ninja version of
>> CMake.
>>
>> What "special Ninja version of CMake"?
>>
>> +[...] Preferredly, you can do a ``ninja install`` to make
>> +the binaries, libraries, and headers, and have them installed in the
>> location
>> +you specified as part of the ``CMAKE_INSTALL_PREFIX`` variable above.
>>
>> Is this really preferred? Especially on a bot (consider the title), this
>> doesn't seem like it is "preferred" at all.
>>
>>
>> The biggest issues I see are
>> 1) Extremely poor organization. Try to achieve the following goal: The
>> document is structured/organized in such a way that a reader can know
>> exactly which parts of the document are relevant for their situation and
>> will spend (i.e. waste) no time looking at instructions that are not
>> relevant for their desired configuration.
>>   + For example, I still don't know which "toolchains" this is supposed
>> to cover. Intel and MSVC were mentioned in passing?
>>   + There's a *ton* of software that this is guiding the reader to
>> install. I don't use Windows much, but last I did (this Summer), all it
>> took to build LLVM was git, MSVC, CMake, and Ninja, and it was like 1 step,
>> and I didn't need to significantly fiddle with any paths or anything. Where
>> is that "easy" option?
>> 2) This seems to have almost nothing to do with builbots, despite having
>> the phrase "for the LLVM Buildbot infrastructure" in the title. I suggest
>> just removing that part of the title (and renaming the file).
>> 3) Please just obliterate the sentence: "This document aims to make it
>> easy for you to configure Windows to build LLVM with the major windows
>> toolchains, bourne shells, and natively with Microsoft’s cmd.exe.". I
>> literally just wrote like 10 lines of text describing things that are wrong
>> with this sentence (I decided it's just not worth getting into). Please
>> just nuke it and write a replacement in the context of the (freshly
>> organized) document.
>>
>> Bottom line: This document has lots of good content (clearly gleaned from
>> "battle scars" doing this kind of setup in the field), but it doesn't seem
>> to have a very good idea of what it is trying to accomplish, and doesn't
>> provide any way for the reader to be able to  efficiently navigate the
>> content or know what is relevant. It needs some reorganization, but I would
>> like to see something like this added to our docs.
>>
>> -- Sean Silva
>>
>>
>> On Mon, Nov 11, 2013 at 6:50 PM, Mikael Lyngvig <mikael at lyngvig.org>wrote:
>>
>>> This document describes how to set up a Windows builder and is a
>>> document that quite a few have asked about during the past few years.
>>>
>>> Please find attached diff.
>>>
>>> Please direct all criticism, error reports, and enhancement requests to
>>> me and I'll see what I can do.
>>>
>>>
>>> Sincerely,
>>> Mikael Lyngvig
>>>
>>>
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20131113/70586121/attachment.html>