[llvm-dev] beta: cloud compiler bisection tool

[Chris Matthews via llvm-dev]

I am happy to announce we are contributing one of our favorite internal tools: llvm bisect!  

First: I want to thank Daniel Dunbar for writing all the code for this tool, and Google for providing cloud storage and bandwidth to host it!

We keep the compilers we build in the Green Dragon CI cluster, and now upload them to the Google Storage Cloud.  The llvmlab bisect tool, takes those compilers and allows you to quickly bisect on a predicate.  Instead of compiling those compilers again, the tool downloads them from the cloud.  The tool allows you to down download and run compilers, or bisect on a range of commits with a predicate.   Example, find where a crash started:

$ llvmlab bisect bash -c "%(path)s/bin/clang -c crashy.c”

FAIL: clang-r219899-t2014-10-15_21-42-48-b809
FAIL: clang-r219778-t2014-10-15_06-18-17-b787
FAIL: clang-r219773-t2014-10-14_21-18-07-b779
FAIL: clang-r219746-t2014-10-14_17-01-07-b775
FAIL: clang-r219739-t2014-10-14_16-09-35-b771
FAIL: clang-r219729-t2014-10-14_15-21-29-b757
clang-r219719-t2014-10-14_14-46-50-b756: first working build
clang-r219729-t2014-10-14_15-21-29-b757: next failing build

Grab the most recent compiler from a build:

$ llvmlab fetch "clang-stage1-configure-RA_build”
downloaded root: clang-r249752-b13228.tar.gz
extracted path : clang-r249752-b13228

For this initial release, I am uploading the ~18000 compilers produced on Green Dragon, which span the last 8 months of builds. As well, all new builds on Green Dragon upload their compilers right away.  These are Darwin compilers, in release asserts and release + LTO configurations.  More configurations will be made available soon, including ASANified compilers and release branch compilers.

The tool is in the zorg repo, in the llvmbisect directory. You install it like a regular python program.  There are extensive docs with many examples included.

I invite other bot owners to to upload their compilers as well.  You will need the Google Storage tool installed on your bot, as well as an account.  Contact me if you are interested, and I can help you get setup.

Happy bisecting!