[lldb-dev] buildbot master configs [Re: buildbot deployment: gsutil: Anonymous caller does not have storage.objects.create access to lldb_test_traces]

[Jan Kratochvil via lldb-dev]

Hello,

I need a testing local buildbot instance to develop a buildbot slave config:

On Thu, 02 Aug 2018 14:47:42 +0200, Pavel Labath via lldb-dev wrote:
> On Thu, 2 Aug 2018 at 13:39, Jan Kratochvil <jan.kratochvil at redhat.com> wrote:
> > On Thu, 02 Aug 2018 13:47:25 +0200, Pavel Labath wrote:
> > > *However*, for setting up a new bot, I'd recommend not using this
> > > particular slave factory (getLLDBScriptCommandsFactory) at all,
> > > because it's heavily customized for our use case (*), and very
> > > different from how typical llvm buildbots are set up. You might be
> > > better off setting up a new factory, which just does the typical
> > > checkout+build+(optional) test steps, and avoids all of this mess.
> >
> > OK. For development of these new steps I guess I should run my own buildbot
> > master instance? As otherwise that will be probably several/many commits to
> > zorg repo (+requested buildbot master restarts) and I may screw up something
> > along.
> 
> Yes, that would definitely be the best, but last time I tried that, I
> couldn't get my master instance to run, for any approximation of the
> word "run" (which is part of the reason why I haven't done anything
> about this slave factory, even though I really don't like it)..

I have found buildbot versions different than 0.8.5 are incompatibile with
LLVM infrastructure/configs so to run 0.8.5 on Fedora 28 x86_64 I have
backported:
	https://people.redhat.com/jkratoch/buildbot-0.8.5-fix.patch
	https://people.redhat.com/jkratoch/buildbot-0.8.5-fix2.patch

So I downloaded zorg from LLVM and set it up
	[buildbot at host1 ~]$ ls -l lldbmaster
	lrwxrwxrwx 1 buildbot buildbot 32 Aug 14 18:55 lldbmaster -> zorg-git/buildbot/osuosl/master/
	[buildbot at host1 ~]$ ls -l lldbmaster/
	total 76
	-rw-r--r-- 1 buildbot buildbot   878 Aug 14 15:25 buildbot.tac
	drwxr-xr-x 2 buildbot buildbot  4096 Aug 14 19:01 config
	-rw-r--r-- 1 buildbot buildbot  9552 Aug 14 15:25 master.cfg
	drwxr-xr-x 2 buildbot buildbot  4096 Aug 14 15:25 public_html
	-rw-r--r-- 1 buildbot buildbot   465 Aug 14 15:25 README.txt
	drwxr-xr-x 2 buildbot buildbot  4096 Aug 14 15:25 templates
	-rw-r--r-- 1 buildbot buildbot 34088 Aug 14 19:01 twistd.log
	-rw------- 1 buildbot buildbot     7 Aug 14 19:01 twistd.pid
	lrwxrwxrwx 1 buildbot buildbot    28 Aug 14 19:00 zorg -> /home/buildbot/zorg-git/zorg
with zorg-git directory from https://llvm.org/git/zorg.git patched as attached
but then I still get:

------------------------------------------------------------------------------
$ buildbot start ~/lldbmaster
Following twistd.log until startup finished..
/home/buildbot/.local/lib/python2.7/site-packages/buildbot-latest-py2.7.egg/buildbot/schedulers/base.py:111: DeprecationWarning: twisted.internet.defer.deferredGenerator was deprecated in Twisted 15.0.0; please use twisted.internet.defer.inlineCallbacks instead
  @defer.deferredGenerator
... ^^^ this looks harmless

2018-08-14 14:35:04+0200 [-] error while parsing config file
2018-08-14 14:35:04+0200 [-] Unhandled Error
	Traceback (most recent call last):
	  File "/home/buildbot/.local/lib/python2.7/site-packages/buildbot-latest-py2.7.egg/buildbot/master.py", line 197, in loadTheConfigFile
	    d = self.loadConfig(f)
	  File "/home/buildbot/.local/lib/python2.7/site-packages/buildbot-latest-py2.7.egg/buildbot/master.py", line 579, in loadConfig
	    d.addCallback(do_load)
	  File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 317, in addCallback
	    callbackKeywords=kw)
	  File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 306, in addCallbacks
	    self._runCallbacks()
	--- <exception caught here> ---
	  File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 587, in _runCallbacks
	    current.result = callback(current.result, *args, **kw)
	  File "/home/buildbot/.local/lib/python2.7/site-packages/buildbot-latest-py2.7.egg/buildbot/master.py", line 226, in do_load
	    exec f in localDict
	  File "/quad/home/buildbot/lldbmaster/master.cfg", line 104, in <module>
	    standard_categories)
	  File "/quad/home/buildbot/lldbmaster/config/status.py", line 31, in get_status_targets
	    default_email = config.options.get('Master Options', 'default_email')
	  File "/usr/lib64/python2.7/ConfigParser.py", line 330, in get
	    raise NoSectionError(section)
	ConfigParser.NoSectionError: No section: 'Master Options'
------------------------------------------------------------------------------

I have found last zorg files containing '[Master Options]' have been removed by:
	https://reviews.llvm.org/D30503
	commit a4a7c00a15e94bf2a26ec209d27e6ece5c20a16b
	git-svn-id: https://llvm.org/svn/llvm-project/zorg/trunk@296756 91177308-0d34-0410-b5e6-96231b3b80d8
	Delete zorg/buildbot/llvmlab, smooshlab was replaced by Jenkins
	The last change to this directory was in 2014, so I'm assuming this
	buildbot config code is all dead.

But then what the current buildbot slaves use as their master?
It looks as a buildbot for me, where to find some configs for it?


Thanks,
Jan Kratochvil
-------------- next part --------------
diff --git a/buildbot/osuosl/master/config/slaves.py b/buildbot/osuosl/master/config/slaves.py
index 91e1a306..cc1443f8 100644
--- a/buildbot/osuosl/master/config/slaves.py
+++ b/buildbot/osuosl/master/config/slaves.py
@@ -5,7 +5,8 @@ import os
 import config
 
 def create_slave(name, *args, **kwargs):
-    password = config.options.get('Slave Passwords', name)
+    #password = config.options.get('Slave Passwords', name)
+    password = 'somepassword'
     return buildbot.buildslave.BuildSlave(name, password=password, *args, **kwargs)
 
 def get_build_slaves():