[LLVMdev] [cfe-dev] ARM failures

[NAKAMURA Takumi]

2013/1/9 David Blaikie <dblaikie at gmail.com>:
> On Tue, Jan 8, 2013 at 10:01 AM, David Tweed <David.Tweed at arm.com> wrote:
>> | Certainly slow builders are problematic. The phase-based building
>> | system David Dean is setting up may help mitigate some of this (it
>> | should make better use of the resources we have, as well as allowing
>> | us to benefit (in the form of smaller blame lists, though not
>> | necessarily lower buildbot result latency) from additional resources
>> | by allowing greater parallelism).
>>
>> There's something I've always meant to ask: when you've got a stable buildot setup (same compiler, etc)
>> all the buildbots are still set up to do a configure and "make clean". I can easily understand if there was the possibility that
>> any other part of the system might also be being changed you'd want to be sure changes were due to repo changes
>> and so do a make clean. But those changes aren't too frequent and you could do manual clean at those times.
>>
>> So all the make clean appears to be doing is guarding against an error in llvm's build system dependency checking. Is the tiny
>> probability of this worth the effect on the build times? (Personally I'd say no, but maybe there's an argument for it I haven't spotted.)
>
> I don't believe so, no, and in fact when I went to setup my own
> builders (for the GDB 7.5 test suite - I've got it running on the
> public builder, but also my own internal one with a bit more hardware
> so I can get results sooner (not ideal - I'd prefer to have things
> publically, but I had the hardware lying around so figured I'd use
> it)) I tried to do this. I seemed to have trouble with the 'configure'
> step invalidating the whole build anyway - I don't want the configure
> step to be something someone has to do manually on a new builder, but
> I'm not sure how to run it in such a way that it doesn't cause an
> otherwise incremental build to become a full rebuild either. Any ideas
> would be most welcome.

Since most of lab's builders have been doing clean-rebuild, I let my
builders do incremental build.
In principle, the tree should be rebuilt (and reconfigured) from every
revision. I am addressing it on.
(That said, the real life is tough against the principle.)

I also suggest to introduce ccache, if reconfiguration could update
too many dependent files.

> (I believe Takumi's bots do incremental rebuilds though, so I guess he
> has some way of doing that)

With CMake builds, each builder watches
$(builddir)/{Makefile|ALL_BUILD.vcxproj|build.ninja}.
CMake runs only when they are missed.
I know a few issues around rebuild for reconfigration to fail.

With configure, each builder might watch $(builddir)/Makefile. It
works in most cases.
But it fails in a few cases. An extreme case is transition between
master and release.

FYI:
def CheckMakefile(factory, makefile="Makefile"):
    factory.addStep(SetProperty(name="Makefile_isready",
                                command=["sh", "-c",
                                         "test -e " + makefile + "&& echo OK"],
                                flunkOnFailure=False,
                                property="exists_Makefile"))

def Makefile_not_ready(step):
    return step.build.getProperty("exists_Makefile") != "OK"

factory.addStep(... doStepIf=Makefile_not_ready)