[LLVMdev] Validation Update

[Tanya M. Lattner]

>>> Right now the buildbot is set up to build every N commits (N=10 for
>>> testing purposes) and also build every Sunday at midnight.  The buildbot
>>> does not automatically tag successful validations.  That will take some
>>> more work and we don't want that automation quite yet anyway.  We'll do
>>> it manually for a while and then once we get the gist of it we can make
>>> it automatic.
>>
>> How exactly do you plan to validate it? What point of reference will you
>> be using as a start?
>
> It's all in the proposal I posted a week or so ago.

Yes, your proposal said that the community must agree on the set of tests 
used to validate it. You haven't said what the conclusion was. Also, in 
your proposal you did not specify what reference point you were going to 
use. Not everything in llvm-test passes. I will suggest that you use a 
release to get a starting reference point.

>>> One thing that would be nice to have is a commit hook on trunk (and
>>> eventually release branches) that lets buildbot know when changes are
>>> submitted.  As of now, buildbot polls the svn server every 10 minutes,
>>> meaning it could miss one of the "every 10" commits (revision mod 10 ==
>>> 0).  The buildbot manual talks a bit about what the svn server needs to
>>> do here:
>>>
>>> http://buildbot.net/repos/release/docs/buildbot.html#PBChangeSource
>>>
>>> Could we get that set up in the llvm.org svn commit hook?
>>
>> What are the pros and cons of this? How much does it tax the server?
>
> It's an extra TCP connection out to the buildbot on every commit.  I would
> think that would be low cost compared to all the other processing that has to
> go on.

I'll have to talk to some people about this before committing to getting 
this done. So ping me later.

>>> Eventually we'll want to migrate the buidlbot master to some publicly
>>> accessible machine so others can contribute machines to do validations
>>> (for various architectures, etc.).  Where should this get hosted?
>>> Hosting at llvm.org would be great but I'm not sure that's possible.
>>> Daniel has a machine set up so that's a possibility too.  Wherever it
>>> goes, I'll need an account so I can muck around with the buildbot config
>>> as build machines are added, targets change, etc.  Eventually that
>>> account should migrate to whomever is the overall validation coordinator
>>> (in the proposal I volunteered myself to play that role for now).
>>
>> The LLVM server is not the place to host this. Could Cray host a machine
>> to do this?
>
> I can certainly ask.  I'll let you know what I hear but be mindful that if
> someone else steps up to coordinate validations, Cray isn't going to give them
> access to an internal machine.  That means we'd have to revisit this question
> in the future to find another home.  I'd rather put this somewhere that is
> under control of the wider LLVM community.  I think that's more in the
> interest of the LLVM community.

This is exactly why we can't do it at LLVM. We can't give arbitrary people 
accounts.

-Tanya