[LLVMdev] please stabilize the trunk

[Nick Lewycky]

2009/7/15 Dale Johannesen <dalej at apple.com>

>
> On Jul 15, 2009, at 4:48 PMPDT, Daniel Dunbar wrote:
>
> > That depends on what you call a false positive. The public buildbot
> > regularly fails because of mailing Frontend tests, and I have had
> > continues failures of some DejaGNU tests for a long time on some
> > builders. Its not a false positive per se, but one starts to ignore
> > the failures because they aren't unexpected.
>
> Yes.  Probably the only way this will work better is if we get the
> testsuite to 0 failures, everywhere, conditionalizing as necessary to
> get rid of expected failures.  Then regressions will be more visible.
> I doubt that will happen unless we freeze the tree for a while and get
> everybody to fix bugs, or disable tests, instead of doing new stuff
> (at least, that was the case for gcc).


This is exactly what we're supposed to do for releases, and in theory, all
of the time.

We've been having a lot of churn lately. This is a good thing overall, since
it means there's lots of contributions going into the project. What's
different about this is that we have a lot of large-scale, sweeping changes
that touch a lot of code. In the past we've generally serialized this sort
of thing between contributors, or broken changes up to be extremely
incremental. The reason this is happening less now is that we, as
developers, are growing more ambitious with our fixes to LLVM systematic
problems, and doing so on a tigher schedule. Once again, this is a good
thing.

There's two issues with buildbots. Firstly, we need more buildbots on more
platforms. For example, there are no Darwin buildbots, so if I commit a
change that breaks Darwin I won't get immediate notice about it, nor a log
of the failure. We could even consider having a buildbot a prerequisite to
being a release-blocking platform. The other is that we need some level of
quality control on buildbots. We can accomplish this by either publishing a
few buildbot guidelines (ie., don't install llvm-gcc on your buildbot
machine because it will cause false-positives as llvm and llvm-gcc get out
of step) and by enhancing the buildbot system to let us mark problems as
expected. We already have part of that by XFAILing tests.

Even so, better buildbots will improve visibility into how the tree is
progressing on a commit-by-commit basis, but it does nothing to prevent
breakage in the first place. I suspect most of our grief will go away as
some of the current major changes finish. If not, we'll have to come up with
a better way to handle so many large changes, maybe something like a
"schedule of merges" so that committers don't step all over each other. I
think GCC does something like this already?

We've deferred imposing structure like that until we discover that we need
it, and I'm not conviced we're quite there yet, but perhaps it's time to
start thinking about it.

Nick

 > - Daniel
> >
> > On Wed, Jul 15, 2009 at 4:10 PM, Bill Wendling<isanbard at gmail.com>
> > wrote:
> >> On Wed, Jul 15, 2009 at 3:43 PM, Eli
> >> Friedman<eli.friedman at gmail.com> wrote:
> >>> On Wed, Jul 15, 2009 at 3:01 PM, Bill Wendling<isanbard at gmail.com>
> >>> wrote:
> >>>> The core problem, in my opinion, is that people *don't* pay
> >>>> attention
> >>>> to the build bot failure messages that come along.
> >>>
> >>> That's largely because of the number of false positives.
> >>>
> >> There have been fewer and fewer of these in recent times.
> >>
> >> -bw
> >> _______________________________________________
> >> LLVM Developers mailing list
> >> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> >>
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20090715/8fb490eb/attachment.html>