<div dir="ltr"><span style="font-family:monospace,monospace">Hello everyone,<br><br>Below are some buildbot numbers for the last week of 1/31/2016 - 2/6/2016.<br>I have added a new report that shows a "status change ratio" for each builder. <br>The idea is to provide some kind of metrics for a noise level a builder  produces. The number is a percent of builds that changed the builder status from greed to red or from red to green.<br>The high number is not that bad, as faster builders would show a higher ratio in general, just because they build more often and more granular. We also have a different number of broken builds day to day.<br>But I hope this would give us another data point to tell our build bot health.<br><br>Thanks<br><br>Galina<br><br><br> buildername                                               | builds | changes | status change ratio<br>-----------------------------------------------------------+--------+---------+---------------------<br> sanitizer-x86_64-linux                                    |     99 |      51 |                51.5<br> clang-ppc64le-linux-lnt                                   |     81 |      28 |                34.6<br> lldb-x86_64-darwin-13.4                                   |     84 |      25 |                29.8<br> lldb-x86_64-ubuntu-14.04-android                          |    108 |      32 |                29.6<br> lldb-windows7-android                                     |    116 |      34 |                29.3<br> clang-native-aarch64-full                                 |     18 |       4 |                22.2<br> clang-cmake-aarch64-full                                  |     42 |       8 |                19.0<br> perf-x86_64-penryn-O3-polly                               |    135 |      18 |                13.3<br> clang-cmake-armv7-a15-selfhost-neon                       |     24 |       3 |                12.5<br> sanitizer-windows                                         |    330 |      41 |                12.4<br> sanitizer-ppc64le-linux                                   |     50 |       6 |                12.0<br> sanitizer-ppc64be-linux                                   |    100 |      12 |                12.0<br> clang-native-arm-lnt-perf                                 |      9 |       1 |                11.1<br> clang-cmake-mips                                          |     82 |       8 |                 9.8<br> clang-x64-ninja-win7                                      |    177 |      16 |                 9.0<br> clang-cmake-thumbv7-a15-full-sh                           |     23 |       2 |                 8.7<br> sanitizer-x86_64-linux-bootstrap                          |     58 |       5 |                 8.6<br> clang-cmake-aarch64-42vma                                 |    151 |      10 |                 6.6<br> lldb-x86_64-ubuntu-14.04-cmake                            |    225 |      14 |                 6.2<br> clang-ppc64le-linux-multistage                            |     84 |       5 |                 6.0<br> perf-x86_64-penryn-O3-polly-fast                          |    105 |       6 |                 5.7<br> perf-x86_64-penryn-O3                                     |    195 |      11 |                 5.6<br> clang-x86-win2008-selfhost                                |    173 |       9 |                 5.2<br> llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast                |    356 |      17 |                 4.8<br> clang-cmake-armv7-a15-full                                |     83 |       4 |                 4.8<br> clang-cmake-mipsel                                        |     22 |       1 |                 4.5<br> llvm-mips-linux                                           |     44 |       2 |                 4.5<br> lldb-x86-windows-msvc2015                                 |    221 |      10 |                 4.5<br> perf-x86_64-penryn-O3-polly-before-vectorizer             |     91 |       4 |                 4.4<br> clang-ppc64be-linux-multistage                            |    140 |       6 |                 4.3<br> clang-native-arm-lnt                                      |     95 |       4 |                 4.2<br> clang-cmake-aarch64-quick                                 |    143 |       6 |                 4.2<br> lld-x86_64-freebsd                                        |    197 |       8 |                 4.1<br> sanitizer-x86_64-linux-fast                               |    176 |       7 |                 4.0<br> clang-cmake-thumbv7-a15                                   |    177 |       6 |                 3.4<br> polly-amd64-linux                                         |    235 |       8 |                 3.4<br> perf-x86_64-penryn-O3-polly-before-vectorizer-detect-only |     91 |       3 |                 3.3<br> lldb-x86_64-ubuntu-14.04-buildserver                      |    129 |       4 |                 3.1<br> sanitizer-x86_64-linux-fuzzer                             |    205 |       6 |                 2.9<br> lldb-amd64-ninja-netbsd7                                  |    142 |       4 |                 2.8<br> clang-bpf-build                                           |    292 |       7 |                 2.4<br> clang-cmake-armv7-a15                                     |    170 |       4 |                 2.4<br> clang-ppc64be-linux                                       |    271 |       6 |                 2.2<br> clang-s390x-linux                                         |    274 |       6 |                 2.2<br> clang-ppc64be-linux-lnt                                   |    192 |       4 |                 2.1<br> sanitizer-x86_64-linux-autoconf                           |    306 |       6 |                 2.0<br> clang-ppc64le-linux                                       |    198 |       4 |                 2.0<br> clang-x86_64-linux-abi-test                               |    295 |       6 |                 2.0<br> clang-x86_64-linux-selfhost-modules                       |    262 |       5 |                 1.9<br> clang-x86_64-debian-fast                                  |    110 |       2 |                 1.8<br> perf-x86_64-penryn-O3-polly-unprofitable                  |    247 |       4 |                 1.6<br> lld-x86_64-win7                                           |    184 |       3 |                 1.6<br> llvm-sphinx-docs                                          |     70 |       1 |                 1.4<br> llvm-clang-lld-x86_64-debian-fast                         |    148 |       2 |                 1.4<br> lldb-x86_64-debian-clang                                  |    325 |       0 |                 0.0<br> libomp-gcc-x86_64-linux-debian                            |      2 |       0 |                 0.0<br> libomp-clang-x86_64-linux-debian                          |      3 |       0 |                 0.0<br> libcxx-sphinx-docs                                        |      2 |       0 |                 0.0<br> libcxx-libcxxabi-x86_64-linux-ubuntu-unstable-abi         |      2 |       0 |                 0.0<br> llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast          |      1 |       0 |                 0.0<br> libcxx-libcxxabi-x86_64-linux-ubuntu-tsan                 |      2 |       0 |                 0.0<br> libcxx-libcxxabi-x86_64-linux-ubuntu-msan                 |      2 |       0 |                 0.0<br> libcxx-libcxxabi-x86_64-linux-ubuntu-gcc49-cxx11          |      2 |       0 |                 0.0<br> libcxx-libcxxabi-x86_64-linux-ubuntu-cxx1z                |      2 |       0 |                 0.0<br> libcxx-libcxxabi-x86_64-linux-ubuntu-cxx14                |      2 |       0 |                 0.0<br> libcxx-libcxxabi-x86_64-linux-ubuntu-cxx11                |      2 |       0 |                 0.0<br> perf-x86_64-penryn-O3-polly-parallel-fast                 |    241 |       0 |                 0.0<br> clang-atom-d525-fedora                                    |     68 |       0 |                 0.0<br> libcxx-libcxxabi-x86_64-linux-ubuntu-cxx03                |      2 |       0 |                 0.0<br> libcxx-libcxxabi-x86_64-linux-ubuntu-asan                 |      2 |       0 |                 0.0<br> libcxx-libcxxabi-x86_64-linux-debian-noexceptions         |      2 |       0 |                 0.0<br> libcxx-libcxxabi-x86_64-linux-debian                      |      2 |       0 |                 0.0<br> libcxx-libcxxabi-singlethreaded-x86_64-linux-debian       |      2 |       0 |                 0.0<br> libcxx-libcxxabi-arm-linux                                |      2 |       0 |                 0.0<br> clang-sphinx-docs                                         |     84 |       0 |                 0.0<br> clang-cmake-armv7-a15-selfhost                            |     35 |       0 |                 0.0<br> clang-atom-d525-fedora-rel                                |    118 |       0 |                 0.0<br> lldb-x86-win7-msvc                                        |    325 |       0 |                 0.0<br> sanitizer_x86_64-freebsd                                  |    234 |       0 |                 0.0<br> lld-sphinx-docs                                           |     74 |       0 |                 0.0<br>(80 rows)<br><br><br>Number of commits by project:<br><br> project           |           commits<br>-------------------+-------------------<br> llvm              |               248<br> cfe               |               101<br> lld               |                75<br> lldb              |                59<br> polly             |                39<br> compiler-rt       |                28<br> clang-tools-extra |                12<br> libcxx            |                 2<br> openmp            |                 2<br> -------------------+-------------------<br>                                   566<br><br><br>Number of completed builds, failed builds and average build time for successful builds per active builder:<br><br> buildername                                               | completed  | failed | time<br>-----------------------------------------------------------+------------+--------+--------------------<br> clang-atom-d525-fedora                                    |         68 |     68 |<br> clang-atom-d525-fedora-rel                                |        120 |    118 |<br> clang-bpf-build                                           |        292 |     44 | 00:03:10<br> clang-cmake-aarch64-42vma                                 |        151 |     10 | 00:42:32<br> clang-cmake-aarch64-full                                  |         42 |     12 | 03:42:29<br> clang-cmake-aarch64-quick                                 |        143 |     21 | 00:43:13<br> clang-cmake-armv7-a15                                     |        170 |      4 | 00:31:53<br> clang-cmake-armv7-a15-full                                |         83 |      2 | 01:29:14<br> clang-cmake-armv7-a15-selfhost                            |         35 |        | 04:14:19<br> clang-cmake-armv7-a15-selfhost-neon                       |         26 |      2 | 05:36:14<br> clang-cmake-mips                                          |         82 |      8 | 01:37:29<br> clang-cmake-mipsel                                        |         22 |      1 | 06:52:43<br> clang-cmake-thumbv7-a15                                   |        177 |      5 | 00:28:02<br> clang-cmake-thumbv7-a15-full-sh                           |         23 |      1 | 06:32:26<br> clang-native-aarch64-full                                 |         18 |      4 | 08:42:51<br> clang-native-arm-lnt                                      |         95 |     13 | 01:14:08<br> clang-native-arm-lnt-perf                                 |          9 |      2 | 09:57:46<br> clang-ppc64be-linux                                       |        271 |     11 | 00:08:43<br> clang-ppc64be-linux-lnt                                   |        192 |     34 | 00:27:28<br> clang-ppc64be-linux-multistage                            |        140 |      5 | 00:46:12<br> clang-ppc64le-linux                                       |        198 |      9 | 00:25:35<br> clang-ppc64le-linux-lnt                                   |         82 |     34 | 01:25:46<br> clang-ppc64le-linux-multistage                            |         84 |      3 | 01:10:34<br> clang-s390x-linux                                         |        274 |     11 | 00:08:09<br> clang-sphinx-docs                                         |         84 |     84 |<br> clang-x64-ninja-win7                                      |        177 |     55 | 00:36:07<br> clang-x86-win2008-selfhost                                |        174 |    126 | 01:21:55<br> clang-x86_64-debian-fast                                  |        110 |    107 | 00:18:32<br> clang-x86_64-linux-abi-test                               |        295 |      4 | 00:10:01<br> clang-x86_64-linux-selfhost-modules                       |        262 |    116 | 00:16:54<br> libcxx-libcxxabi-arm-linux                                |          2 |        | 01:09:09<br> libcxx-libcxxabi-singlethreaded-x86_64-linux-debian       |          2 |      2 |<br> libcxx-libcxxabi-x86_64-linux-debian                      |          2 |        | 00:10:32<br> libcxx-libcxxabi-x86_64-linux-debian-noexceptions         |          2 |        | 00:10:26<br> libcxx-libcxxabi-x86_64-linux-ubuntu-asan                 |          2 |        | 00:07:11<br> libcxx-libcxxabi-x86_64-linux-ubuntu-cxx03                |          2 |        | 00:03:15<br> libcxx-libcxxabi-x86_64-linux-ubuntu-cxx11                |          2 |        | 00:03:43<br> libcxx-libcxxabi-x86_64-linux-ubuntu-cxx14                |          2 |        | 00:03:44<br> libcxx-libcxxabi-x86_64-linux-ubuntu-cxx1z                |          2 |        | 00:03:51<br> libcxx-libcxxabi-x86_64-linux-ubuntu-gcc49-cxx11          |          2 |      2 |<br> libcxx-libcxxabi-x86_64-linux-ubuntu-msan                 |          2 |        | 00:17:27<br> libcxx-libcxxabi-x86_64-linux-ubuntu-tsan                 |          2 |        | 00:10:42<br> libcxx-libcxxabi-x86_64-linux-ubuntu-unstable-abi         |          2 |        | 00:03:33<br> libcxx-sphinx-docs                                        |          2 |        | 00:00:14<br> libomp-clang-x86_64-linux-debian                          |          3 |        | 00:50:12<br> libomp-gcc-x86_64-linux-debian                            |          2 |        | 00:04:57<br> lld-sphinx-docs                                           |         74 |     74 |<br> lld-x86_64-freebsd                                        |        197 |      4 | 00:14:36<br> lld-x86_64-win7                                           |        185 |     31 | 00:09:07<br> lldb-amd64-ninja-netbsd7                                  |        142 |      4 | 00:47:18<br> lldb-windows7-android                                     |        116 |     30 | 00:55:44<br> lldb-x86-win7-msvc                                        |        325 |    325 |<br> lldb-x86-windows-msvc2015                                 |        221 |     69 | 00:23:01<br> lldb-x86_64-darwin-13.4                                   |         84 |     19 | 01:36:08<br> lldb-x86_64-debian-clang                                  |        325 |    325 |<br> lldb-x86_64-ubuntu-14.04-android                          |        108 |     24 | 01:08:41<br> lldb-x86_64-ubuntu-14.04-buildserver                      |        129 |      4 | 00:54:44<br> lldb-x86_64-ubuntu-14.04-cmake                            |        226 |     27 | 00:23:36<br> llvm-clang-lld-x86_64-debian-fast                         |        148 |    144 | 00:28:26<br> llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast                |        356 |     20 | 00:03:21<br> llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast          |          2 |      1 |<br> llvm-mips-linux                                           |         44 |      1 | 02:42:47<br> llvm-sphinx-docs                                          |         70 |      1 | 00:00:21<br> perf-x86_64-penryn-O3                                     |        195 |    149 | 05:16:12<br> perf-x86_64-penryn-O3-polly                               |        135 |    109 | 05:04:35<br> perf-x86_64-penryn-O3-polly-before-vectorizer             |         91 |     87 | 05:53:02<br> perf-x86_64-penryn-O3-polly-before-vectorizer-detect-only |         91 |     82 | 05:34:34<br> perf-x86_64-penryn-O3-polly-fast                          |        105 |     53 | 00:11:03<br> perf-x86_64-penryn-O3-polly-parallel-fast                 |        241 |    241 |<br> perf-x86_64-penryn-O3-polly-unprofitable                  |        247 |    209 | 00:10:02<br> polly-amd64-linux                                         |        235 |     14 | 00:11:21<br> sanitizer-ppc64be-linux                                   |        100 |      8 | 01:14:28<br> sanitizer-ppc64le-linux                                   |         50 |      4 | 02:45:45<br> sanitizer-windows                                         |        331 |     51 | 00:06:05<br> sanitizer-x86_64-linux                                    |         99 |     34 | 01:18:33<br> sanitizer-x86_64-linux-autoconf                           |        306 |     12 | 00:09:51<br> sanitizer-x86_64-linux-bootstrap                          |         58 |      8 | 02:26:20<br> sanitizer-x86_64-linux-fast                               |        176 |     23 | 00:35:48<br> sanitizer-x86_64-linux-fuzzer                             |        205 |     10 | 00:29:04<br> sanitizer_x86_64-freebsd                                  |        234 |    234 |<br>(80 rows)<br><br><br>Average waiting time for a revision to get build result per active builder (response time):<br><br> buildername                                               | time<br>-----------------------------------------------------------+-------------------<br> libcxx-sphinx-docs                                        | 00:00:41<br> lldb-x86_64-debian-clang                                  | 00:02:11<br> lldb-x86-win7-msvc                                        | 00:03:16<br> llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast                | 00:03:28<br> libcxx-libcxxabi-x86_64-linux-ubuntu-unstable-abi         | 00:04:01<br> libcxx-libcxxabi-x86_64-linux-ubuntu-cxx14                | 00:04:11<br> clang-bpf-build                                           | 00:05:42<br> libcxx-libcxxabi-x86_64-linux-ubuntu-asan                 | 00:07:40<br> libcxx-libcxxabi-x86_64-linux-ubuntu-cxx03                | 00:07:42<br> sanitizer-windows                                         | 00:10:08<br> libcxx-libcxxabi-x86_64-linux-ubuntu-cxx11                | 00:11:31<br> libcxx-libcxxabi-x86_64-linux-ubuntu-cxx1z                | 00:11:55<br> clang-s390x-linux                                         | 00:12:56<br> clang-x86_64-linux-abi-test                               | 00:14:14<br> libcxx-libcxxabi-x86_64-linux-ubuntu-tsan                 | 00:15:00<br> clang-ppc64be-linux                                       | 00:15:05<br> perf-x86_64-penryn-O3-polly-unprofitable                  | 00:16:03<br> sanitizer-x86_64-linux-autoconf                           | 00:16:09<br> clang-x86_64-linux-selfhost-modules                       | 00:17:40<br> libcxx-libcxxabi-x86_64-linux-ubuntu-msan                 | 00:18:09<br> libcxx-libcxxabi-x86_64-linux-ubuntu-gcc49-cxx11          | 00:18:49<br> lld-x86_64-freebsd                                        | 00:19:33<br> perf-x86_64-penryn-O3-polly-parallel-fast                 | 00:21:08<br> sanitizer_x86_64-freebsd                                  | 00:24:14<br> polly-amd64-linux                                         | 00:33:57<br> lldb-x86_64-ubuntu-14.04-cmake                            | 00:34:00<br> lldb-x86-windows-msvc2015                                 | 00:37:02<br> clang-ppc64le-linux                                       | 00:40:59<br> lld-sphinx-docs                                           | 00:42:08<br> sanitizer-x86_64-linux-fuzzer                             | 00:42:51<br> clang-sphinx-docs                                         | 00:44:35<br> llvm-sphinx-docs                                          | 00:46:42<br> clang-x86_64-debian-fast                                  | 00:49:51<br> clang-ppc64be-linux-lnt                                   | 00:52:01<br> clang-x64-ninja-win7                                      | 00:56:46<br> llvm-clang-lld-x86_64-debian-fast                         | 00:58:43<br> sanitizer-x86_64-linux-fast                               | 00:59:08<br> libcxx-libcxxabi-singlethreaded-x86_64-linux-debian       | 01:02:15<br> clang-cmake-armv7-a15                                     | 01:04:48<br> clang-cmake-thumbv7-a15                                   | 01:05:30<br> clang-x86-win2008-selfhost                                | 01:07:46<br> libomp-clang-x86_64-linux-debian                          | 01:09:10<br> libcxx-libcxxabi-arm-linux                                | 01:09:37<br> lldb-amd64-ninja-netbsd7                                  | 01:10:37<br> libcxx-libcxxabi-x86_64-linux-debian-noexceptions         | 01:13:17<br> clang-ppc64be-linux-multistage                            | 01:15:46<br> clang-cmake-aarch64-quick                                 | 01:18:27<br> clang-cmake-aarch64-42vma                                 | 01:20:15<br> lldb-x86_64-ubuntu-14.04-buildserver                      | 01:20:35<br> libomp-gcc-x86_64-linux-debian                            | 01:23:33<br> libcxx-libcxxabi-x86_64-linux-debian                      | 01:23:59<br> lldb-x86_64-ubuntu-14.04-android                          | 01:36:31<br> clang-ppc64le-linux-multistage                            | 01:43:10<br> sanitizer-x86_64-linux                                    | 01:55:29<br> sanitizer-ppc64be-linux                                   | 02:01:01<br> lldb-x86_64-darwin-13.4                                   | 02:24:48<br> clang-native-arm-lnt                                      | 02:34:30<br> clang-cmake-mips                                          | 02:43:18<br> clang-atom-d525-fedora-rel                                | 03:04:25<br> clang-ppc64le-linux-lnt                                   | 03:09:12<br> clang-cmake-armv7-a15-full                                | 03:43:54<br> sanitizer-x86_64-linux-bootstrap                          | 03:45:06<br> perf-x86_64-penryn-O3                                     | 03:52:48<br> clang-atom-d525-fedora                                    | 04:00:03<br> perf-x86_64-penryn-O3-polly-fast                          | 04:39:05<br> perf-x86_64-penryn-O3-polly                               | 04:54:00<br> clang-cmake-aarch64-full                                  | 05:10:15<br> llvm-mips-linux                                           | 05:11:41<br> sanitizer-ppc64le-linux                                   | 05:12:21<br> lldb-windows7-android                                     | 05:39:37<br> clang-cmake-armv7-a15-selfhost                            | 06:56:10<br> perf-x86_64-penryn-O3-polly-before-vectorizer-detect-only | 08:52:49<br> perf-x86_64-penryn-O3-polly-before-vectorizer             | 08:55:15<br> clang-cmake-armv7-a15-selfhost-neon                       | 09:13:55<br> clang-cmake-thumbv7-a15-full-sh                           | 11:04:37<br> lld-x86_64-win7                                           | 13:35:41<br> clang-native-aarch64-full                                 | 13:47:54<br> clang-cmake-mipsel                                        | 14:06:45<br> llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast          | 15:32:55<br> clang-native-arm-lnt-perf                                 | 15:50:58<br>(80 rows)<br></span></div>