[LLVMdev] NewNightlyTester.pl: split into phases?

[Joachim Durchholz]

Am Samstag, den 08.03.2008, 14:02 -0800 schrieb Tanya Lattner:
> - ability to check out llvm-gcc or update llvm-gcc and build it before  
> running tests. In addition to using a prebuilt binary.

Does it need a prebuilt binary?
I have been suspecting so since it has been failing with BUILD ERROR for
me. I just haven't found the time to verify that yet.

I have yet to try building llvm-gcc from sources though. The
instructions seem to be clear enough, but it's still possible that I'll
hit a hidden snag.
Well, I'll try that anyway. I like my software well-done after all :-)


> If you fix the two things above then you should be able to just rerun  
> the script multiple times for different targets like you want. It will  
> also solve problems many others have experienced.

Ensuring that the tree is pristine could be a bit of a challenge. It's
always possible that the user inadvertently changed or deleted a file,
giving a false alarm for all future checks.
Options to avoid that seem to be:
1. Do a 'make dist-clean' before running the tests. Of course, if
dist-clean has bugs, this may create trouble. On the other hand, it
would exercise dist-clean on a regular basis, which might be a Good
Thing.
2. Recreate $BUILDDIR as a copy of a pristine check-out directory each
time the tester is run. This would be much faster than a download, but
still a lot of unnecessary load.
3. For each test, first run svn revert, then ask svn about any
unversioned files and delete them, finally svn update.

(2) would be easiest. (3) would be most failsafe. (1) might be more
useful for the project because it could uncover bugs in the dist-clean
target though. (OTOH one could argue that this should be a separate
regression test.)


> A -noreport option is probably a good idea too.

OK, that should be easy to add.

> Just be sure not to change what it sends to the website. That needs to  
> stay the same.

Sure. I think wrapping an if statement around the HTTP send should be
easy; the code of NewNightlyTester.pl isn't very complicated.


On a tangent:
What's the best way to submit patches: Bugzilla? Something else?

Regards,
Jo