[LLVMdev] Test-Suite Bots failing randomly

[Daniel Dunbar]

On Mon, Apr 22, 2013 at 10:56 AM, Renato Golin <renato.golin at linaro.org>wrote:

> On 22 April 2013 18:02, Daniel Dunbar <daniel at zuster.org> wrote:
>
>> Ok, so we are already downloading from a private server (lab.llvm.org),
>> which IMHO we should be doing because we want tight control over the
>> dependencies etc.
>>
>
> Agreed.
>
>
> I forgot that your machine is outside the lab, so that may be involved
>> here. I don't fully understand the networking setup but it is very
>> different outside vs inside. That said, I ran some timings from our network
>> here to lab.llvm.org, and I couldn't reproduce any connectivity issues
>> from here.
>>
>
> By the same time you replied I found out that there were networking issues
> in our lab, so it's probably our fault. Sorry for the noise. :(
>
> In theory, it should be fixed, so let's wait until tomorrow and I'll
> follow the bot closely to see if there's any problem.
>

Great.


>
> Renato, can you try running some testing on your side to quantify where
>> the connectivity issue is? Things like:
>> $ ab -c 3 -n 100 http://lab.llvm.org/packages/SQLAlchemy-0.6.6.tar.gz
>> work fine from here.
>>
>
> Works on our lab, too. That is, now...
>
>
> That said, it *would* be nice to harden this code against that failure
>> mode -- one reasonable way would be to have the buildbot slave mirror the
>> packages directory locally (using rsync or so) and do the install from
>> there. We can make the rsync command not cause a buildbot failure, since we
>> rarely change the packages. Any interest in making these changes to the
>> buildbot config?
>>
>
> I wonder if the Python code that downloads it doesn't already provide a
> mechanism to cache the files. I don't think we need an external rsync, just
> save the temporary files on /tmp and retrieve from there on failure to
> resolve the external host. A warning would probably be in order.
>
> I can have a look at the script. Would be great if you could tell me where
> I can find the basic stuff. ;)
>

The buildbot scripts are in the 'zorg' repo, the one in question is here:

http://llvm.org/viewvc/llvm-project/zorg/trunk/zorg/buildbot/builders/LNTBuilder.py?view=log

 - Daniel

cheers,
> --renato
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130422/ad78b244/attachment.html>