[LLVMdev] Validating LLVM

[David Greene]

On Wednesday 12 November 2008 02:20, Bill Wendling wrote:
> On Nov 11, 2008, at 11:00 PM, Óscar Fuentes wrote:
> > Bill Wendling <isanbard at gmail.com> writes:
> >
> > [snip]
> >
> >> All four of the above should be run on at least a nightly basis (more
> >> frequently for some, like the regression tests). Each of these are
> >> automated, making that easy. If there are no regressions from the
> >> above four, we could tag that revision as being potentially "valid".
> >
> > If a new test case is created (coming from a bug report or a code
> > review, not from adding a new feature) and it fails for a previously
> > "valid" revision, is the tag removed?
>
> There would probably be some sort of concept of "last known good". In
> the above case, either the tag could be removed and/or a new valid
> revision tagged. It was "good" up until that point, at least. :-)

I would say that once a validation tag is created, it stays.  We don't want to 
be in the business of going back through lots of tag history and checking
against new tests that didn't exist when the tags were originally created.

Perhaps for a "last known good" tag we could re-check but I'm even hesitent
to do that.  It seems like a lot of extra work for little real gain.

                                                           -Dave