[cfe-dev] The effects of exact diagnostic checking

[Sebastian Redl]

Mike Stump wrote:
> :-(  This is kinda annoying.  If 100 people did this with 10 test
> cases apiece, at any given time, there would be about 1000 failing
> test cases, and the poor developers would then have to puzzle out
> which failures are theirs, which failures mark real bugs in their work
> that they now can't catch and so on.  Imagine we have a cron job that
> reverts all work that doesn't result in a clean test suite run.
Then my change would be reverted :-)
The problem is that there is no good way of doing such a thing without
temporarily breaking the tree. If I just published my patch to the
mailing list and asked people to apply it and see what test failures
they have and if they can fix them, would it happen? The people have the
same work as before - puzzle out which failures are there, and which are
real bugs - plus they have to download and manually apply the patch,
instead of having SVN do it for them.
>
> I now have 13 failing test cases, you only listed nine, and before I
> updated, I had 0.  :-(  That leave me wondering what happened with 4
> of them.
Me too. Can you post some more info?

I can revert the change for the time being, but that's no solution. The
bugs are there - they just weren't visible before. I've already filtered
all non-bugs I found.

Anyway, if you have test cases that fail that didn't fail for me, that
just means there are test cases which aren't portable between our
systems, which is also a real problem.

Sebastian