[LLVMdev] llvmlab (phased buildmaster) is in production mode!

[Michael Gottesman]

This is a known issue. I have actually screwed with this before it is just a matter of getting it into zorg in the right manner.

On Mar 27, 2013, at 5:50 PM, Reed Kotler <rkotler at mips.com> wrote:

> Most of the selections, like "console" for example, do not work when I click on them.
> 
> On 03/27/2013 03:57 PM, Michael Gottesman wrote:
>> Hello LLVM Dev and Clang Dev!
>> 
>> David Dean and I just finished bringing up a new build master on
>> lab.llvm.org <http://lab.llvm.org/>, llvmlab, which is located at the
>> url http://lab.llvm.org:8013 <http://lab.llvm.org:8013/> and is in #llvm
>> under the username llvmlab.
>> 
>> llvmlab is different than the current buildbot based continuous
>> integration systems llvm uses; llvmlab is a phased builder based system.
>> The high level details of the phased builder system are as follows:
>> 
>> 1. Builders are grouped together into phases as sub builders. Each phase
>> is a builder itself and triggers said grouped ``sub builders'' and
>> assuming that all of the sub builders complete, triggers its successor
>> phased builder if one exists. This creates a gating like effect.
>> 
>> 2. All phases are gated together in a linear fashion, yielding via the
>> linearity the phenomena that if an earlier phase does not succeed, then
>> no later phases run. The key idea here is if we know that there is a
>> fundamental issue with the compiler why try to build 20 compilers, when
>> performing one quick build is all that is needed to ascertain such a
>> fact? Also if we can not build a compiler successfully, why try to do
>> LNT performance runs? This gets rid of pointless work, stops excessive
>> emails from being sent out for 1 bad commit, and reduces cycle time
>> especially if certain builds take significantly longer than others to fail.
>> 
>> 3. Later phases do broader, longer lasting testing than earlier phases.
>> Thus the 4 phases we currently have are:
>> 
>> a. Phase 1 (sanity):          Phase 1 is a quick non-bootstrapped,
>> non-lto compiler build, to check the ``basic sanity'' of the code base
>> and build process. This generally takes 15-20 minutes to complete.
>> b. Phase 2 (living on):      Phase 2 builds bootstrapped compilers for
>> the different configurations that can be used for ``living on'', i.e.
>> good enough for common compilation tasks. This is meant to cycle in up
>> to an hour.
>> c. Phase 3 (tree health):  Phase 3 runs performance tests, i.e., LNT
>> (which are not live yet) as well as other compiler builds which take a
>> longer amount of time (a full clang build with LTO enabled for instance).
>> d. Phase 4 (validation):   Phase 4 runs longer running validation tests.
>> Currently we have nothing in phase 4, but I am sure that will change = p.
>> 
>> 4. Builders in later phases rely on outputs from earlier phases. If we
>> are doing performance runs, why should we compile a new compiler for
>> such a performance run? This is duplicated work! Instead the phased
>> build system stores ``artifacts'', i.e., built compilers, in earlier
>> phases and uses them as compilers for builds in later phases. Thus we
>> could have 30 different Phase 3 LNT performance runs with different
>> configurations all using the same compiler artifacts built in phase 2.
>> This significantly deduplicates work yielding a decreased cycle time.
>> 
>> As time moves on we will be moving more and more builders to llvmlab
>> including LNT performance builders and a builder which runs the libcxx
>> test suite.
>> 
>> Michael
>> 
>> 
>> 
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev-Tmj1lob9twqVc3sceRu5cw at public.gmane.org
>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>> 
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev