[LLVMdev] Contributing a buildbot for the BPF backend

[Marco Leogrande]

Hi Galina,
I can see the buildslave in the waterfall display, and one build was
already issued and completed successfully.

I will give it a deeper look at the output and monitor in the next few days.


Thanks!



Regards,

Marco Leogrande

Sent by a carbon-based life form; hence, it may contain repetitions,
inaccuracies, logical fallacies and repetitions.


On Thu, Jun 18, 2015 at 10:31 AM, Galina Kistanova <gkistanova at gmail.com> wrote:
> Hi Marco,
>
> Your slave should be able to attach to llvm master now.
> Please let me know if there is any problems with this.
>
> Thanks
>
> Galina
>
> On Wed, Jun 17, 2015 at 5:38 PM, Marco Leogrande <marcol at plumgrid.com>
> wrote:
>>
>> Thank you.
>> Alexei just committed the change to SVN.
>>
>> Regards,
>>
>> Marco Leogrande
>>
>> Sent by a carbon-based life form; hence, it may contain repetitions,
>> inaccuracies, logical fallacies and repetitions.
>>
>>
>> On Wed, Jun 17, 2015 at 2:52 PM, Galina Kistanova <gkistanova at gmail.com>
>> wrote:
>> > LGTM.
>> > Please commit.
>> >
>> > Thanks
>> >
>> > Galina
>> >
>> > On Tue, Jun 16, 2015 at 8:32 PM, Marco Leogrande <marcol at plumgrid.com>
>> > wrote:
>> >>
>> >> Galina,
>> >> thanks again for your help.
>> >>
>> >> I have cooked a new patch that should make a bit more sense.
>> >> I have added the new builder under
>> >> _get_experimental_scheduled_builders(), as the comment above that
>> >> function suggested me it would be a good idea. :)
>> >> Does it look a reasonable start?
>> >>
>> >>
>> >> Regards,
>> >>
>> >> Marco Leogrande
>> >>
>> >> Sent by a carbon-based life form; hence, it may contain repetitions,
>> >> inaccuracies, logical fallacies and repetitions.
>> >>
>> >>
>> >> On Wed, Jun 10, 2015 at 7:06 PM, Marco Leogrande <marcol at plumgrid.com>
>> >> wrote:
>> >> > Hi Galina,
>> >> >
>> >> > thanks for the info!
>> >> >
>> >> > I think we will go for the "LLVM+clang with cmake" approach, since
>> >> > the
>> >> > BPF target has been marked non-experimental in the meantime, and we
>> >> > are interested in building clang as well.
>> >> > I will provide the info requested in step #10 separately and
>> >> > experiment
>> >> > a bit.
>> >> >
>> >> > Thanks.
>> >> >
>> >> >
>> >> > Regards,
>> >> >
>> >> > Marco Leogrande
>> >> >
>> >> > Sent by a carbon-based life form; hence, it may contain repetitions,
>> >> > inaccuracies, logical fallacies and repetitions.
>> >> >
>> >> >
>> >> > On Wed, Jun 10, 2015 at 4:05 PM, Galina Kistanova
>> >> > <gkistanova at gmail.com>
>> >> > wrote:
>> >> >> Hello Marco,
>> >> >>
>> >> >> Welcome aboard!
>> >> >>
>> >> >> slaves.py keeps the build slaves definitions, builders.py keeps the
>> >> >> builders
>> >> >> definitions. You have to have both.
>> >> >> The steps of adding a new slave is here:
>> >> >> http://llvm.org/docs/HowToAddABuilder.html. Please make sure you
>> >> >> done
>> >> >> the
>> >> >> step # 10 before bringing your slave up, otherwise it wouldn't be
>> >> >> authorized
>> >> >> by the master and will be blacklisted after multiple failures.
>> >> >>
>> >> >> To make sure the slave is ready, you can start from manually
>> >> >> building
>> >> >> and
>> >> >> testing everything you want the build to do on that slave in context
>> >> >> of
>> >> >> the
>> >> >> account the build slave will be running in.
>> >> >> If you succeed, then the slave is likely ready.
>> >> >>
>> >> >> You patch doesn't look right.
>> >> >> getLLVMBuildFactory does not support cmake, so it wouldn't do what
>> >> >> you
>> >> >> think
>> >> >> it will. For the same reason your slave properties (jobs and
>> >> >> loadaverage)
>> >> >> wouldn't work.
>> >> >> See zorg/buildbot/builders/LLVMBuilder.py for more details.
>> >> >>
>> >> >> You either build LLVM with configure (this is what the existing LLVM
>> >> >> factory
>> >> >> does), add a new CMakeBuildFactory to support cmake, or build
>> >> >> LLVM+clang
>> >> >> with cmake by using a clang build factory.
>> >> >>
>> >> >> If you use any of the existing build factories, you do not really
>> >> >> need
>> >> >> a
>> >> >> local master. You can add your builder and your slave to the
>> >> >> lab.llvm.org
>> >> >> master and move from there.
>> >> >> As the first step you will be adding your builder as experimental
>> >> >> anyway, so
>> >> >> you will have a chance to work out possible issues and instabilities
>> >> >> before
>> >> >> promoting it to a stable one.
>> >> >>
>> >> >> You do need a local master to be able to debug and test if you
>> >> >> develop
>> >> >> zorg,
>> >> >> i.e. write a new CMakeBuildFactory and such, though.
>> >> >>
>> >> >> Hope this helps.
>> >> >>
>> >> >> Thanks
>> >> >>
>> >> >> Galina
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Mon, Jun 8, 2015 at 9:04 PM, Marco Leogrande
>> >> >> <marcol at plumgrid.com>
>> >> >> wrote:
>> >> >>>
>> >> >>> Hi all,
>> >> >>> I was trying to make some progress on this front and I spent some
>> >> >>> more
>> >> >>> time on it.
>> >> >>>
>> >> >>> * I think I managed to configure the buildslave correctly; any
>> >> >>> chance
>> >> >>> there is a way to test that before the buildmaster is configured?
>> >> >>>
>> >> >>> * I was trying to figure out where/how to add the configuration in
>> >> >>> zorg. slaves.py looks reasonably easy to configure, but builders.py
>> >> >>> looks more convoluted. I think the attached patch is a start for
>> >> >>> the
>> >> >>> build part, but of course I have no idea if it makes any sense :)
>> >> >>> The idea would be to replicate the following workflow (build
>> >> >>> experimental backend + tests):
>> >> >>>
>> >> >>> cmake -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=BPF ../path/to/src/
>> >> >>> make -jX
>> >> >>> ./bin/llvm-lit test/CodeGen/BPF/
>> >> >>>
>> >> >>>
>> >> >>> Any suggestion would be appreciated.
>> >> >>>
>> >> >>> Thanks!
>> >> >>>
>> >> >>> Regards,
>> >> >>>
>> >> >>> Marco Leogrande
>> >> >>>
>> >> >>> Sent by a carbon-based life form; hence, it may contain
>> >> >>> repetitions,
>> >> >>> inaccuracies, logical fallacies and repetitions.
>> >> >>>
>> >> >>>
>> >> >>> On Thu, May 7, 2015 at 8:56 PM, Marco Leogrande
>> >> >>> <marcol at plumgrid.com>
>> >> >>> wrote:
>> >> >>> > Hi everyone,
>> >> >>> > I am working with Alexei Starovoitov to contribute an LLVM
>> >> >>> > buildbot
>> >> >>> > for the experimental BPF backend.
>> >> >>> > I am following the steps at [1] to setup a buildbot and I was
>> >> >>> > mostly
>> >> >>> > successful: I was able to setup a slave and a temporary master to
>> >> >>> > check its base config.
>> >> >>> >
>> >> >>> > Now I'm not sure about the next step: patching the "slaves.py"
>> >> >>> > and
>> >> >>> > "builders.py" files in zorg.
>> >> >>> >
>> >> >>> > * Is there a guide I can follow to understand the format of these
>> >> >>> > files? For example, being BPF an experimental backend, I need to
>> >> >>> > provide extra parameters to cmake at build time.
>> >> >>> >
>> >> >>> > * Also, is there a way to test these changes in zorg in a local
>> >> >>> > environment before submitting the patch for review?
>> >> >>> >
>> >> >>> > Thank you!
>> >> >>> >
>> >> >>> >
>> >> >>> > [1] http://llvm.org/docs/HowToAddABuilder.html
>> >> >>> >
>> >> >>> > Regards,
>> >> >>> >
>> >> >>> > Marco Leogrande
>> >> >>> >
>> >> >>> > Sent by a carbon-based life form; hence, it may contain
>> >> >>> > repetitions,
>> >> >>> > inaccuracies, logical fallacies and repetitions.
>> >> >>
>> >> >>
>> >
>> >
>
>