[LLVMdev] NewNightlyTester.pl: split into phases?

[Tanya Lattner]

On Mar 8, 2008, at 2:45 PM, Joachim Durchholz wrote:

>
> Am Samstag, den 08.03.2008, 14:02 -0800 schrieb Tanya Lattner:
>> - ability to check out llvm-gcc or update llvm-gcc and build it  
>> before
>> running tests. In addition to using a prebuilt binary.
>
> Does it need a prebuilt binary?
> I have been suspecting so since it has been failing with BUILD ERROR  
> for
> me. I just haven't found the time to verify that yet.
>
> I have yet to try building llvm-gcc from sources though. The
> instructions seem to be clear enough, but it's still possible that  
> I'll
> hit a hidden snag.
> Well, I'll try that anyway. I like my software well-done after all :-)
>

You need to have an llvm-gcc, so you either build it yourself or use  
one of the prebuilt binaries. The nightly tester just needs to know  
where its located, but it doesn't know anything about llvm-gcc  
otherwise. Since most people are testing TOT and llvm/llvm-gcc are  
tied together it makes sense to update llvm-gcc before running the  
nightly tester. However, it takes quite a bit of time to checkout and  
build llvm-gcc, so having the option to just update existing sources  
would be nice.

>
>> If you fix the two things above then you should be able to just rerun
>> the script multiple times for different targets like you want. It  
>> will
>> also solve problems many others have experienced.
>
> Ensuring that the tree is pristine could be a bit of a challenge. It's
> always possible that the user inadvertently changed or deleted a file,
> giving a false alarm for all future checks.
> Options to avoid that seem to be:
> 1. Do a 'make dist-clean' before running the tests. Of course, if
> dist-clean has bugs, this may create trouble. On the other hand, it
> would exercise dist-clean on a regular basis, which might be a Good
> Thing.
> 2. Recreate $BUILDDIR as a copy of a pristine check-out directory each
> time the tester is run. This would be much faster than a download, but
> still a lot of unnecessary load.
> 3. For each test, first run svn revert, then ask svn about any
> unversioned files and delete them, finally svn update.
>
> (2) would be easiest. (3) would be most failsafe. (1) might be more
> useful for the project because it could uncover bugs in the dist-clean
> target though. (OTOH one could argue that this should be a separate
> regression test.)
>

I'd rather not use the nightly tester to test make dist-clean. I'm not  
sure how many people actually use make dist anything right now so it  
probably has bugs too. This should probably be investigated separately.

I think the only way to ensure the tree is pristine is to do a clean  
checkout like it does now. So I think that should be the preferred  
way. However, I think its worthwhile to be able to update your tree  
and build using the nightly tester. Checking out and building llvm can  
be a time consuming process for people with slow network connections  
and slow machines.  This is worthwhile in my opinion, but its not  
necessarily the solution to your problem if you want to make sure your  
tree is perfect.

However, I think #2 is the better solution for your particular problem  
and I don't think you need to do any changes to the nightly tester,  
just tell it to use a new build directory.

>> A -noreport option is probably a good idea too.
>
> OK, that should be easy to add.
>
>> Just be sure not to change what it sends to the website. That needs  
>> to
>> stay the same.
>
> Sure. I think wrapping an if statement around the HTTP send should be
> easy; the code of NewNightlyTester.pl isn't very complicated.
>

Cool.

> On a tangent:
> What's the best way to submit patches: Bugzilla? Something else?
>

Send to llvm-commits or llvm-dev (as long as its not too big).

-Tanya

P.S. I'm not sure if someone has gotten a chance to look at your  
dejagnu bugs you filed, but if not.. I'll try to take a look this  
weekend. Unfortunately, I've been preoccupied this week with family  
matters.


> Regards,
> Jo
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev