<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Thanks for the feedback.<div class=""><br class=""></div><div class="">Updated in r263840 & r263841.</div><div class=""><br class=""></div><div class="">-Chris</div><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Mar 18, 2016, at 2:39 PM, Sean Silva <<a href="mailto:chisophugis@gmail.com" class="">chisophugis@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="">Nice!</div><div class=""><br class=""></div><div class="">For consistency with other docs, can you rename this to AdvancedBuilds or AdvancedBuildConfigurations?</div><div class=""><br class=""></div>A couple comments about non-ascii characters inline:<div class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Fri, Mar 18, 2016 at 2:16 PM, Chris Bieneman via llvm-commits<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:llvm-commits@lists.llvm.org" target="_blank" class="">llvm-commits@lists.llvm.org</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;">Author: cbieneman<br class="">Date: Fri Mar 18 16:16:26 2016<br class="">New Revision: 263834<br class=""><br class="">URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project?rev=263834&view=rev" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-project?rev=263834&view=rev</a><br class="">Log:<br class="">[Docs] New documentation for advanced build configurations<br class=""><br class="">This document covers how to use some of the new complex build configurations CMake supports.<br class=""><br class="">Feedback and improvements welcomed!<br class=""><br class="">Added:<br class=""> <span class="Apple-converted-space"> </span>llvm/trunk/docs/Advanced_Builds.rst<br class="">Modified:<br class=""> <span class="Apple-converted-space"> </span>llvm/trunk/docs/CMake.rst<br class=""> <span class="Apple-converted-space"> </span>llvm/trunk/docs/index.rst<br class=""><br class="">Added: llvm/trunk/docs/Advanced_Builds.rst<br class="">URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/Advanced_Builds.rst?rev=263834&view=auto" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/Advanced_Builds.rst?rev=263834&view=auto</a><br class="">==============================================================================<br class="">--- llvm/trunk/docs/Advanced_Builds.rst (added)<br class="">+++ llvm/trunk/docs/Advanced_Builds.rst Fri Mar 18 16:16:26 2016<br class="">@@ -0,0 +1,174 @@<br class="">+=============================<br class="">+Advanced Build Configurations<br class="">+=============================<br class="">+<br class="">+.. contents::<br class="">+ :local:<br class="">+<br class="">+Introduction<br class="">+============<br class="">+<br class="">+`CMake <<a href="http://www.cmake.org/" rel="noreferrer" target="_blank" class="">http://www.cmake.org/</a>>`_ is a cross-platform build-generator tool. CMake<br class="">+does not build the project, it generates the files needed by your build tool<br class="">+(GNU make, Visual Studio, etc.) for building LLVM.<br class="">+<br class="">+If **you are a new contributor**, please start with the :doc:`GettingStarted` or<br class="">+:doc:`CMake` pages. This page is intended for users doing more complex builds.<br class="">+<br class="">+Many of the examples below are written assuming specific CMake Generators.<br class="">+Unless otherwise explicitly called out these commands should work with any CMake<br class="">+generator.<br class="">+<br class="">+Bootstrap Builds<br class="">+================<br class="">+<br class="">+The Clang CMake build system supports bootstrap (aka multi-stage) builds. At a<br class="">+high level a multi-stage build is a chain of builds that pass data from one<br class="">+stage into the next. The most common and simple version of this is a traditional<br class="">+bootstrap build.<br class="">+<br class="">+In a simple two-stage bootstrap build, we build clang using the system compiler,<br class="">+then use that just-built clang to build clang again. In CMake this simplest form<br class="">+of a bootstrap build can be configured with a single option,<br class="">+CLANG_ENABLE_BOOTSTRAP.<br class="">+<br class="">+.. code-block:: console<br class="">+<br class="">+ $ make -G Ninja -DCLANG_ENABLE_BOOTSTRAP=On <path to source><br class="">+ $ ninja stage2<br class="">+<br class="">+This command itself isn’t terribly useful because it assumes default<br class=""></blockquote><div class=""><br class=""></div><div class="">Non-ascii</div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;">+configurations for each stage. The next series of examples utilize CMake cache<br class="">+scripts to provide more complex options.<br class="">+<br class="">+The clang build system refers to builds as stages. A stage1 build is a standard<br class="">+build using the compiler installed on the host, and a stage2 build is built<br class="">+using the stage1 compiler. This nomenclature holds up to more stages too. In<br class="">+general a stage*n* build is built using the output from stage*n-1*.<br class="">+<br class="">+Apple Clang Builds (A More Complex Bootstrap)<br class="">+=============================================<br class="">+<br class="">+Apple’s Clang builds are a slightly more complicated example of the simple<br class=""></blockquote><div class=""><br class=""></div><div class="">Non-ascii</div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;">+bootstrapping scenario. Apple Clang is built using a 2-stage build.<br class="">+<br class="">+The stage1 compiler is a host-only compiler with some options set. The stage1<br class="">+compiler is a balance of optimization vs build time because it is a throwaway.<br class="">+The stage2 compiler is the fully optimized compiler intended to ship to users.<br class="">+<br class="">+Setting up these compilers requires a lot of options. To simplify the<br class="">+configuration the Apple Clang build settings are contained in CMake Cache files.<br class="">+You can build an Apple Clang compiler using the following commands:<br class="">+<br class="">+.. code-block:: console<br class="">+<br class="">+ $ make -G Ninja -C <path to clang>/cmake/caches/Apple-stage1.cmake <path to source><br class="">+ $ ninja stage2-distribution<br class="">+<br class="">+This CMake invocation configures the stage1 host compiler, and sets<br class="">+CLANG_BOOTSTRAP_CMAKE_ARGS to pass the Apple-stage2.cmake cache script to the<br class="">+stage2 configuration step.<br class="">+<br class="">+When you build the stage2-distribution target it builds the minimal stage1<br class="">+compiler and required tools, then configures and builds the stage2 compiler<br class="">+based on the settings in Apple-stage2.cmake.<br class="">+<br class="">+This pattern of using cache scripts to set complex settings, and specifically to<br class="">+make later stage builds include cache scripts is common in our more advanced<br class="">+build configurations.<br class="">+<br class="">+Multi-stage PGO<br class="">+===============<br class="">+<br class="">+Profile-Guided Optimizations (PGO) is a really great way to optimize the code<br class="">+clang generates. Our multi-stage PGO builds are a workflow for generating PGO<br class="">+profiles that can be used to optimize clang.<br class="">+<br class="">+At a high level, the way PGO works is that you build an instrumented compiler,<br class="">+then you run the instrumented compiler against sample source files. While the<br class="">+instrumented compiler runs it will output a bunch of files containing<br class="">+performance counters (.profraw files). After generating all the profraw files<br class="">+you use llvm-profdata to merge the files into a single profdata file that you<br class="">+can feed into the LLVM_PROFDATA_FILE option.<br class="">+<br class="">+Our PGO.cmake cache script automates that whole process. You can use it by<br class="">+running:<br class="">+<br class="">+.. code-block:: console<br class="">+<br class="">+ $ make -G Ninja -C <path_to_clang>/cmake/caches/PGO.cmake <source dir><br class="">+ $ ninja stage2-instrumented-generate-profdata<br class="">+<br class="">+If you let that run for a few hours or so, it will place a profdata file in your<br class="">+build directory. This takes a really long time because it builds clang twice,<br class="">+and you *must* have compiler-rt in your build tree.<br class="">+<br class="">+This process uses any source files under the perf-training directory as training<br class="">+data as long as the source files are marked up with LIT-style RUN lines.<br class="">+<br class="">+After it finishes you can use “find . -name clang.profdata†to find it, but it<br class="">+should be at a path something like:<br class="">+<br class="">+.. code-block:: console<br class="">+<br class="">+ <build dir>/tools/clang/stage2-instrumented-bins/utils/perf-training/clang.profdata<br class="">+<br class="">+You can feed that file into the LLVM_PROFDATA_FILE option when you build your<br class="">+optimized compiler.<br class="">+<br class="">+The PGO came cache has a slightly different stage naming scheme than other<br class="">+multi-stage builds. It generates three stages; stage1, stage2-instrumented, and<br class="">+stage2. Both of the stage2 builds are built using the stage1 compiler.<br class="">+<br class="">+The PGO came cache generates the following additional targets:<br class="">+<br class="">+**stage2-instrumented**<br class="">+ Builds a stage1 x86 compiler, runtime, and required tools (llvm-config,<br class="">+ llvm-profdata) then uses that compiler to build an instrumented stage2 compiler.<br class="">+<br class="">+**stage2-instrumented-generate-profdata**<br class="">+ Depends on “stage2-instrumented†and will use the instrumented compiler to<br class="">+ generate profdata based on the training files in <clang>/utils/perf-training<br class="">+<br class="">+**stage2**<br class="">+ Depends of “stage2-instrumented-generate-profdata†and will use the stage1<br class=""></blockquote><div class=""><br class=""></div><div class="">Non-ascii</div><div class=""><br class=""></div><div class="">-- Sean Silva</div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;">+ compiler with the stage2 profdata to build a PGO-optimized compiler.<br class="">+<br class="">+**stage2-check-llvm**<br class="">+ Depends on stage2 and runs check-llvm using the stage2 compiler.<br class="">+<br class="">+**stage2-check-clang**<br class="">+ Depends on stage2 and runs check-clang using the stage2 compiler.<br class="">+<br class="">+**stage2-check-all**<br class="">+ Depends on stage2 and runs check-all using the stage2 compiler.<br class="">+<br class="">+**stage2-test-suite**<br class="">+ Depends on stage2 and runs the test-suite using the stage3 compiler (requires<br class="">+ in-tree test-suite).<br class="">+<br class="">+3-Stage Non-Determinism<br class="">+=======================<br class="">+<br class="">+In the ancient lore of compilers non-determinism is like the multi-headed hydra.<br class="">+Whenever it's head pops up, terror and chaos ensue.<br class="">+<br class="">+Historically one of the tests to verify that a compiler was deterministic would<br class="">+be a three stage build. The idea of a three stage build is you take your sources<br class="">+and build a compiler (stage1), then use that compiler to rebuild the sources<br class="">+(stage2), then you use that compiler to rebuild the sources a third time<br class="">+(stage3) with an identical configuration to the stage2 build. At the end of<br class="">+this, you have a stage2 and stage3 compiler that should be bit-for-bit<br class="">+identical.<br class="">+<br class="">+You can perform one of these 3-stage builds with LLVM & clang using the<br class="">+following commands:<br class="">+<br class="">+.. code-block:: console<br class="">+<br class="">+ $ cmake -G Ninja -C <path_to_clang>/cmake/caches/3-stage.cmake <source dir><br class="">+ $ ninja stage3<br class="">+<br class="">+After the build you can compare the stage2 & stage3 compilers. We have a bot<br class="">+setup `here <<a href="http://lab.llvm.org:8011/builders/clang-3stage-ubuntu" rel="noreferrer" target="_blank" class="">http://lab.llvm.org:8011/builders/clang-3stage-ubuntu</a>>`_ that runs<br class="">+this build and compare configuration.<br class=""><br class="">Modified: llvm/trunk/docs/CMake.rst<br class="">URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/CMake.rst?rev=263834&r1=263833&r2=263834&view=diff" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/CMake.rst?rev=263834&r1=263833&r2=263834&view=diff</a><br class="">==============================================================================<br class="">--- llvm/trunk/docs/CMake.rst (original)<br class="">+++ llvm/trunk/docs/CMake.rst Fri Mar 18 16:16:26 2016<br class="">@@ -474,6 +474,41 @@ LLVM-specific variables<br class=""> If you want to build LLVM as a shared library, you should use the<br class=""> ``LLVM_BUILD_LLVM_DYLIB`` option.<br class=""><br class="">+CMake Caches<br class="">+============<br class="">+<br class="">+Recently LLVM and Clang have been adding some more complicated build system<br class="">+features. Utilizing these new features often involves a complicated chain of<br class="">+CMake variables passed on the command line. Clang provides a collection of CMake<br class="">+cache scripts to make these features more approachable.<br class="">+<br class="">+CMake cache files are utilized using CMake's -C flag:<br class="">+<br class="">+.. code-block:: console<br class="">+<br class="">+ $ cmake -C <path to cache file> <path to sources><br class="">+<br class="">+CMake cache scripts are processed in an isolated scope, only cached variables<br class="">+remain set when the main configuration runs. CMake cached variables do not reset<br class="">+variables that are already set unless the FORCE option is specified.<br class="">+<br class="">+A few notes about CMake Caches:<br class="">+<br class="">+- Order of command line arguments is important<br class="">+<br class="">+ - -D arguments specified before -C are set before the cache is processed and<br class="">+ can be read inside the cache file<br class="">+ - -D arguments specified after -C are set after the cache is processed and<br class="">+ are unset inside the cache file<br class="">+<br class="">+- All -D arguments will override cache file settings<br class="">+- CMAKE_TOOLCHAIN_FILE is evaluated after both the cache file and the command<br class="">+ line arguments<br class="">+- It is recommended that all -D options should be specified *before* -C<br class="">+<br class="">+For more information about some of the advanced build configurations supported<br class="">+via Cache files see :doc:`Advanced_Builds`.<br class="">+<br class=""> Executing the test suite<br class=""> ========================<br class=""><br class=""><br class="">Modified: llvm/trunk/docs/index.rst<br class="">URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/index.rst?rev=263834&r1=263833&r2=263834&view=diff" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/index.rst?rev=263834&r1=263833&r2=263834&view=diff</a><br class="">==============================================================================<br class="">--- llvm/trunk/docs/index.rst (original)<br class="">+++ llvm/trunk/docs/index.rst Fri Mar 18 16:16:26 2016<br class="">@@ -65,6 +65,7 @@ representation.<br class=""> :hidden:<br class=""><br class=""> CMake<br class="">+ Advanced_Builds<br class=""> HowToBuildOnARM<br class=""> HowToCrossCompileLLVM<br class=""> CommandGuide/index<br class=""><br class=""><br class="">_______________________________________________<br class="">llvm-commits mailing list<br class=""><a href="mailto:llvm-commits@lists.llvm.org" class="">llvm-commits@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a></blockquote></div></div></div></div></div></blockquote></div><br class=""></div></body></html>