[llvm-dev] Zorg migration to GitHub/monorepo

Mon Oct 28 17:08:53 PDT 2019

I think what she is referring to was that the build seemed to be triggered
by a commit to a project that shouldn't trigger builds on a libcxx bot
(i.e. the change was in llvm).

I have a somewhat orthogonal but related question. In the past, commits to
compiler-rt did not trigger builds on llvm/clang/sanitizer bots. Has this
behaviour been rectified with the move to github? I am really sorry if you
already answered this question and I just missed it.

On Mon, Oct 28, 2019 at 2:37 PM Galina Kistanova via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Hi Diana,
>
> It is not clear from your description of what is the problem. Could
> you elaborate, please?
>
> I have looked at the build history closer and see that this build
> configuration depends on libcxx, libcxxabi, libunwind, llvm projects,
> and changes to any of these would trigger a build. Depending on a bot
> performance, patches could be grouped to a longer blame list. To me,
> this is exactly how it supposedly should be. Are you missing any
> particular changes in libcxx, libcxxabi,or libunwind project which
> should trigger a build but they didn't? If so, could you point me to
> such change, please?
>
> Thanks
>
> Galina
>
>
>
> On Mon, Oct 28, 2019 at 5:16 AM Diana Picus <diana.picus at linaro.org>
> wrote:
> >
> > Hi Galina,
> >
> > It seems that our libcxx bots are now triggering builds for any changes
> to llvm:
> >
> http://lab.llvm.org:8011/builders/libcxx-libcxxabi-libunwind-aarch64-linux/builds/2434
> >
> > Should I file a bug report for this?
> >
> > Thanks,
> > Diana
> >
> > On Sat, 19 Oct 2019 at 11:36, Galina Kistanova via cfe-commits
> > <cfe-commits at lists.llvm.org> wrote:
> > >
> > > Hello everyone,
> > >
> > > The staging master is ready to accept bots from the list I have sent
> yesterday. Don't wait too long.
> > >
> > > The master has been updated and works with both SVN and Github
> monorepo now.
> > >
> > > The following builders are already listening for changes in monorepo
> and building monorepo. More are coming.
> > >
> > > * clang-sphinx-docs
> > > * clang-tools-sphinx-docs
> > > * clang-x86_64-linux-abi-test
> > > * clang-lld-x86_64-2stage
> > > * libcxx-libcxxabi-singlethreaded-x86_64-linux-debian
> > > * libcxx-sphinx-docs
> > > * libunwind-sphinx-docs
> > > * lld-sphinx-docs
> > > * lld-x86_64-darwin13
> > > * lld-x86_64-ubuntu-fast
> > > * lldb-sphinx-docs
> > > * llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast
> > > * llvm-clang-x86_64-win-fast <<-- ?
> > > * llvm-sphinx-docs
> > > * clang-x86_64-debian-fast
> > > * libcxx-libcxxabi-libunwind-x86_64-linux-debian
> > > * libcxx-libcxxabi-singlethreaded-x86_64-linux-debian
> > > * libcxx-libcxxabi-x86_64-linux-debian
> > > * libcxx-libcxxabi-x86_64-linux-debian-noexceptions
> > >
> > > A friendly reminder. If your bots are using one of these build
> factories, you would need either update your build configurations to use
> one of the currently supported build factories, or port that factory to
> work with github and monorepo.
> > >
> > > * LLVMBuilder (3 bots)
> > > * PollyBuilder (3 bots)
> > > * LLDBBuilder (6 bots)
> > > * SanitizerBuilder (10 bots)
> > > * CUDATestsuiteBuilder (1 bot) - depends on
> ClangBuilder.getClangBuildFactory
> > > * AOSPBuilder (1 bot) - depends on PollyBuilder
> > > * AnnotatedBuilder (2 bots)
> > > * OpenMPBuilder (2 bots)
> > > * FuchsiaBuilder (1 bot)
> > >
> > > Thanks
> > >
> > > Galina
> > >
> > >
> > > On Fri, Oct 18, 2019 at 12:05 AM Galina Kistanova <
> gkistanova at gmail.com> wrote:
> > >>
> > >> Hello build bot owners!
> > >>
> > >> The staging master is ready. Please feel free to use it to make sure
> your bots would work well with the monorepo and github.
> > >>
> > >> The following builders could be configured to build monorepo:
> > >>
> > >> * clang-atom-d525-fedora-rel
> > >> * clang-native-arm-lnt-perf
> > >> * clang-cmake-armv7-lnt
> > >> * clang-cmake-armv7-selfhost-neon
> > >> * clang-cmake-armv7-quick
> > >> * clang-cmake-armv7-global-isel
> > >> * clang-cmake-armv7-selfhost
> > >> * clang-cmake-aarch64-quick
> > >> * clang-cmake-aarch64-lld
> > >> * clang-cmake-aarch64-global-isel
> > >> * clang-ppc64be-linux-lnt
> > >> * clang-ppc64be-linux-multistage
> > >> * clang-ppc64le-linux-lnt
> > >> * clang-ppc64le-linux-multistage
> > >> * clang-ppc64be-linux
> > >> * clang-ppc64le-linux
> > >> * clang-s390x-linux
> > >> * clang-s390x-linux-multistage
> > >> * clang-s390x-linux-lnt
> > >> * clang-hexagon-elf
> > >> * clang-cmake-x86_64-avx2-linux
> > >> * clang-cmake-x86_64-avx2-linux-perf
> > >> * clang-cmake-x86_64-sde-avx512-linux
> > >> * clang-solaris11-amd64
> > >> * clang-x64-ninja-win7
> > >> * clang-solaris11-sparcv9
> > >> * clang-cmake-armv7-full
> > >> * clang-cmake-thumbv7-full-sh
> > >> * clang-cmake-armv8-lld
> > >> * clang-cmake-aarch64-full
> > >> * clang-armv7-linux-build-cache
> > >> * clang-aarch64-linux-build-cache
> > >> * libcxx-libcxxabi-x86_64-linux-debian
> > >> * libcxx-libcxxabi-x86_64-linux-debian-noexceptions
> > >> * libcxx-libcxxabi-libunwind-x86_64-linux-debian
> > >> * libcxx-libcxxabi-singlethreaded-x86_64-linux-debian
> > >> * libcxx-libcxxabi-x86_64-linux-ubuntu-cxx03
> > >> * libcxx-libcxxabi-x86_64-linux-ubuntu-cxx11
> > >> * libcxx-libcxxabi-x86_64-linux-ubuntu-cxx14
> > >> * libcxx-libcxxabi-x86_64-linux-ubuntu-cxx17
> > >> * libcxx-libcxxabi-x86_64-linux-ubuntu-cxx2a
> > >> * libcxx-libcxxabi-x86_64-linux-ubuntu-32bit
> > >> * libcxx-libcxxabi-x86_64-linux-ubuntu-asan
> > >> * libcxx-libcxxabi-x86_64-linux-ubuntu-ubsan
> > >> * libcxx-libcxxabi-x86_64-linux-ubuntu-msan
> > >> * libcxx-libcxxabi-libunwind-x86_64-linux-ubuntu
> > >> * libcxx-libcxxabi-x86_64-linux-ubuntu-tsan
> > >> * libcxx-libcxxabi-x86_64-linux-ubuntu-gcc5-cxx11
> > >> * libcxx-libcxxabi-x86_64-linux-ubuntu-gcc-tot-latest-std
> > >> * libcxx-libcxxabi-libunwind-armv7-linux
> > >> * libcxx-libcxxabi-libunwind-armv8-linux
> > >> * libcxx-libcxxabi-libunwind-armv7-linux-noexceptions
> > >> * libcxx-libcxxabi-libunwind-armv8-linux-noexceptions
> > >> * libcxx-libcxxabi-libunwind-aarch64-linux
> > >> * libcxx-libcxxabi-libunwind-aarch64-linux-noexceptions
> > >> * ppc64le-lld-multistage-test
> > >>
> > >> These builders are already on the staging master. So, please ping me
> if you would like to configure any of them to work with monorepo:
> > >>
> > >> * clang-freebsd11-amd64
> > >>
> > >> These builders have been already tested and could be reconfigured
> without staging as soon as public master is ready:
> > >>
> > >> * llvm-sphinx-docs
> > >> * clang-sphinx-docs
> > >> * clang-tools-sphinx-docs
> > >> * lld-sphinx-docs
> > >> * lldb-sphinx-docs
> > >> * libcxx-sphinx-docs
> > >> * libunwind-sphinx-docs
> > >> * clang-x86_64-debian-fast
> > >> * libcxx-libcxxabi-x86_64-linux-debian
> > >> * libcxx-libcxxabi-x86_64-linux-debian-noexceptions
> > >> * libcxx-libcxxabi-libunwind-x86_64-linux-debian
> > >> * libcxx-libcxxabi-singlethreaded-x86_64-linux-debian
> > >> * lld-x86_64-darwin13
> > >> * lld-x86_64-win7
> > >> * llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast
> > >> * llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast
> > >> * clang-x86_64-linux-abi-test
> > >> * clang-with-lto-ubuntu
> > >> * clang-with-thin-lto-ubuntu
> > >> * llvm-clang-x86_64-expensive-checks-win
> > >> * llvm-clang-x86_64-win-fast
> > >> * lld-x86_64-ubuntu-fast
> > >> * clang-lld-x86_64-2stage
> > >> * lld-perf-testsuite
> > >>
> > >> Thanks for your patience and help!
> > >>
> > >> Galina
> > >>
> > >> On Thu, Oct 17, 2019 at 5:04 PM Galina Kistanova <
> gkistanova at gmail.com> wrote:
> > >>>
> > >>> Hello everyone,
> > >>>
> > >>> The build bot is almost ready to move to github.
> > >>>
> > >>> As the next step we would migrate the staging master to listen for
> the both changes, from SVN as it was before, and from github.
> > >>>
> > >>> Tonight I am going to update the staging (aka silent) master and it
> will start working with github.
> > >>>
> > >>> I will send an email when it is done.
> > >>>
> > >>> If you bots use one of the ported build factories from the list in
> my previous e-mail, please feel free to stage them for working with
> monorepo/github and once you are happy, let me know, and I'll configure
> them accordingly on the production master. Once that's done, you could move
> the bots back to the production master and you are done.
> > >>>
> > >>> When you stage your bot, please follow these steps:
> > >>> 1. Stop your bot between builds, if possible,
> > >>> 2. Remove the bot working directory (usually this directory has a
> builder name and is under directory where your bot is installed; if you are
> not sure, check the 'builddir' param of your bot in
> zorg/buildbot/osuosl/master/config/builders.py),
> > >>> 3. Edit buildbot.tac in the directory where your bot is installed.
> Change "port = 9990" line to "port = 9994". Save the change.
> > >>> 4. Start your bot, make sure it connects to the staging master.
> > >>> 5. Send me a mail with the staged bot names.
> > >>>
> > >>> Once you are happy with your bot building monorepo changes from
> github, please send me an e-mail and I'll respond with the instructions of
> how to get your bot back to production.
> > >>>
> > >>> Please be aware that staging master could restart often. Please let
> me know if you are having long running builds.
> > >>>
> > >>> Feel free to ask if you have questions.
> > >>> Please me know if you will see issues with the staging master.
> > >>>
> > >>> Thanks
> > >>>
> > >>> Galina
> > >>>
> > >>> On Mon, Oct 14, 2019 at 6:16 PM Galina Kistanova <
> gkistanova at gmail.com> wrote:
> > >>>>
> > >>>> Hello everyone,
> > >>>>
> > >>>>
> > >>>>
> > >>>> We are in the middle of porting the majority of zorg to
> GitHub/monorepo. The following build factories will be ported and if you
> use one of those for your bots, you are all covered:
> > >>>>
> > >>>>
> > >>>>
> > >>>> * ClangBuilder.getClangCMakeBuildFactory (31 bots)
> > >>>>
> > >>>> * ClangBuilder.getClangCMakeGCSBuildFactory (2 bots)
> > >>>>
> > >>>> * LibcxxAndAbiBuilder (23 bots)
> > >>>>
> > >>>> * SphinxDocsBuilder (7 bots)
> > >>>>
> > >>>> * UnifiedTreeBuilder (11 bots)
> > >>>>
> > >>>> * ABITestsuitBuilder (1 bot) - based on UnifiedTreeBuilder
> > >>>>
> > >>>> * ClangLTOBuilder (2 bots) - based on UnifiedTreeBuilder
> > >>>>
> > >>>> * LLDPerformanceTesuiteBuilder (1 bot) - based on UnifiedTreeBuilder
> > >>>>
> > >>>>
> > >>>>
> > >>>> Some build factories will be deprecated. If you use one of these,
> please change your bot to use something else instead. Here is the list of
> deprecated build factories:
> > >>>>
> > >>>>
> > >>>>
> > >>>> * ClangBuilder.getClangBuildFactory (0 bots)
> > >>>>
> > >>>> * LLDBuilder (0 bots)
> > >>>>
> > >>>> * ClangAndLLDBuilder (0 bots)
> > >>>>
> > >>>>
> > >>>>
> > >>>> However, some special build factories and build factories with a
> few bots would need your attention.
> > >>>>
> > >>>> Here is the list of build factories in need of porting. Patches are
> welcome.
> > >>>>
> > >>>>
> > >>>>
> > >>>> * LLVMBuilder (3 bots)
> > >>>>
> > >>>> * PollyBuilder (3 bots)
> > >>>>
> > >>>> * LLDBBuilder (6 bots)
> > >>>>
> > >>>> * SanitizerBuilder (10 bots)
> > >>>>
> > >>>> * CUDATestsuiteBuilder (1 bot) - depends on
> ClangBuilder.getClangBuildFactory
> > >>>>
> > >>>> * AOSPBuilder (1 bot) - depends on PollyBuilder
> > >>>>
> > >>>> * AnnotatedBuilder (2 bots)
> > >>>>
> > >>>> * OpenMPBuilder (2 bots)
> > >>>>
> > >>>> * FuchsiaBuilder (1 bot)
> > >>>>
> > >>>>
> > >>>>
> > >>>> Please feel free to ask if you have questions.
> > >>>>
> > >>>>
> > >>>>
> > >>>> Thanks
> > >>>>
> > >>>>
> > >>>>
> > >>>> Galina
> > >>>>
> > >>>>
> > > _______________________________________________
> > > cfe-commits mailing list
> > > cfe-commits at lists.llvm.org
> > > https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191028/3a242950/attachment.html>