<div dir="ltr">Wow Tamas, this is perfect. Thanks for pulling that together!<div><br></div><div>Don't worry about the bigger file.</div><div><br></div><div>Thanks much.</div><div><br></div><div>-Todd</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Sep 15, 2015 at 8:56 AM, Tamas Berghammer <span dir="ltr"><<a href="mailto:tberghammer@google.com" target="_blank">tberghammer@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I created a new statistic what separates the data based on compiler and architecture and I also extended it to the last 250 builds on the Linux build bot. If you would like to see the build IDs for the different outcomes then let me know, because I have them collected out, but it is a quite big file.<span class="HOEnZb"><font color="#888888"><div><br></div><div><span style="line-height:1.5;font-size:13.2px">Tamas</span><br></div></font></span><div><div class="h5"><br><div class="gmail_quote"><div dir="ltr">On Tue, Sep 15, 2015 at 3:37 PM Todd Fiala <<a href="mailto:todd.fiala@gmail.com" target="_blank">todd.fiala@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Tue, Sep 15, 2015 at 2:57 AM, Tamas Berghammer <span dir="ltr"><<a href="mailto:tberghammer@google.com" target="_blank">tberghammer@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Todd,<div><br></div><div>I attached the statistic of the last 100 test run on the Linux x86_64 builder (<a href="http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake" target="_blank">http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake</a>). The data might be a little bit noisy because of the actual test failures happening because of a temporary regression, but they should give you a general idea about what is happening. </div><div><br></div></div></blockquote><div><br></div></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>Thanks, Tamas! I'll have a look.</div></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div></div><div>I will try to create a statistic where the results are displayed separately for each compiler and architecture to get a bit more detailed view, but it will take some time. If you want I can include the list of build numbers for all outcome, but it will be a very log list (currently only included for Timeout and Failure)</div><div><br></div></div></blockquote><div><br></div></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>I'll know better when I have a look at what you provided. The hole I see right now is we're not adequately dealing with unexpected successes for different configurations. Any reporting around that is helpful.</div><div><br></div><div>Thanks!</div></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div></div><div>Tamas</div><br><div class="gmail_quote"><div><div><div dir="ltr">On Mon, Sep 14, 2015 at 11:24 PM Todd Fiala via lldb-dev <<a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a>> wrote:<br></div></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr">On an Ubuntu 14.04 x86_64 system, I'm seeing the following results:<div><br></div><div><b>cmake/ninja/clang-3.6:</b></div><div><br></div><div><div>Testing: 395 test suites, 24 threads</div><div>395 out of 395 test suites processed - TestGdbRemoteKill.py</div><div>Ran 395 test suites (0 failed) (0.000000%)</div><div>Ran 478 test cases (0 failed) (0.000000%)</div><div><br></div><div>Unexpected Successes (6)</div><div>UNEXPECTED SUCCESS: LLDB (suite) :: TestConstVariables.py</div><div>UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py</div><div>UNEXPECTED SUCCESS: LLDB (suite) :: TestMiBreak.py</div><div>UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py</div><div>UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py</div><div>UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py</div><div><br></div><div><br></div><div><b>cmake/ninja/gcc-4.9.2:</b></div><div><br></div><div><div>395 out of 395 test suites processed - TestMultithreaded.py</div><div>Ran 395 test suites (1 failed) (0.253165%)</div><div>Ran 457 test cases (1 failed) (0.218818%)</div><div>Failing Tests (1)</div><div>FAIL: LLDB (suite) :: TestRegisterVariables.py</div><div><br></div><div>Unexpected Successes (6)</div><div>UNEXPECTED SUCCESS: LLDB (suite) :: TestDataFormatterSynth.py</div><div>UNEXPECTED SUCCESS: LLDB (suite) :: TestMiBreak.py</div><div>UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py</div><div>UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py</div><div>UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py</div><div>UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py</div></div><div><br></div><div><br></div><div>I will look into those. I suspect some of them are compiler-version specific, much like some of the OS X ones I dug into earlier.</div>-- <br><div><div dir="ltr">-Todd</div></div>
</div></div></div></div>
_______________________________________________<br>
lldb-dev mailing list<br>
<a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev</a><br>
</blockquote></div></div>
</blockquote></div></div></div><div dir="ltr"><div class="gmail_extra"><br><br clear="all"><div><br></div>-- <br><div><div dir="ltr">-Todd</div></div>
</div></div></blockquote></div></div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr">-Todd</div></div>
</div>