[LLVMdev] llvmlab (phased buildmaster) is in production mode!

[Reed Kotler]

Most of the selections, like "console" for example, do not work when I 
click on them.

On 03/27/2013 03:57 PM, Michael Gottesman wrote:
> Hello LLVM Dev and Clang Dev!
>
> David Dean and I just finished bringing up a new build master on
> lab.llvm.org <http://lab.llvm.org/>, llvmlab, which is located at the
> url http://lab.llvm.org:8013 <http://lab.llvm.org:8013/> and is in #llvm
> under the username llvmlab.
>
> llvmlab is different than the current buildbot based continuous
> integration systems llvm uses; llvmlab is a phased builder based system.
> The high level details of the phased builder system are as follows:
>
> 1. Builders are grouped together into phases as sub builders. Each phase
> is a builder itself and triggers said grouped ``sub builders'' and
> assuming that all of the sub builders complete, triggers its successor
> phased builder if one exists. This creates a gating like effect.
>
> 2. All phases are gated together in a linear fashion, yielding via the
> linearity the phenomena that if an earlier phase does not succeed, then
> no later phases run. The key idea here is if we know that there is a
> fundamental issue with the compiler why try to build 20 compilers, when
> performing one quick build is all that is needed to ascertain such a
> fact? Also if we can not build a compiler successfully, why try to do
> LNT performance runs? This gets rid of pointless work, stops excessive
> emails from being sent out for 1 bad commit, and reduces cycle time
> especially if certain builds take significantly longer than others to fail.
>
> 3. Later phases do broader, longer lasting testing than earlier phases.
> Thus the 4 phases we currently have are:
>
> a. Phase 1 (sanity):          Phase 1 is a quick non-bootstrapped,
> non-lto compiler build, to check the ``basic sanity'' of the code base
> and build process. This generally takes 15-20 minutes to complete.
> b. Phase 2 (living on):      Phase 2 builds bootstrapped compilers for
> the different configurations that can be used for ``living on'', i.e.
> good enough for common compilation tasks. This is meant to cycle in up
> to an hour.
> c. Phase 3 (tree health):  Phase 3 runs performance tests, i.e., LNT
> (which are not live yet) as well as other compiler builds which take a
> longer amount of time (a full clang build with LTO enabled for instance).
> d. Phase 4 (validation):   Phase 4 runs longer running validation tests.
> Currently we have nothing in phase 4, but I am sure that will change = p.
>
> 4. Builders in later phases rely on outputs from earlier phases. If we
> are doing performance runs, why should we compile a new compiler for
> such a performance run? This is duplicated work! Instead the phased
> build system stores ``artifacts'', i.e., built compilers, in earlier
> phases and uses them as compilers for builds in later phases. Thus we
> could have 30 different Phase 3 LNT performance runs with different
> configurations all using the same compiler artifacts built in phase 2.
> This significantly deduplicates work yielding a decreased cycle time.
>
> As time moves on we will be moving more and more builders to llvmlab
> including LNT performance builders and a builder which runs the libcxx
> test suite.
>
> Michael
>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev-Tmj1lob9twqVc3sceRu5cw at public.gmane.org
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>