<div dir="ltr">+dvyukov</div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 13, 2015 at 11:08 AM, David Blaikie <span dir="ltr"><<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Wed, May 13, 2015 at 10:39 AM, Reid Kleckner <span dir="ltr"><<a href="mailto:rnk@google.com" target="_blank">rnk@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>It's a 20m timeout without output.</div><div><br></div>If you back up to the build and look at the 'annotate' step output, there's this text:<div><a href="http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/17916/steps/annotate/logs/stdio" target="_blank">http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/17916/steps/annotate/logs/stdio</a></div><div><pre style="font-family:'Courier New',courier,monotype,monospace;color:rgb(0,0,0);font-size:medium"><span>-- Testing: 258 tests, 16 threads --
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..</span><span style="color:blue">
command timed out: 1200 seconds without output, attempting to kill
process killed by signal 9
program finished with exit code -1
elapsedTime=3507.624426</span></pre><div>The annotator should probably include that timeout text in the failing step, so that sounds like a bug.<br></div></div><div><br></div><div>Another issue is that tsan times out sometimes. </div></div></blockquote></span><div><br>Also - how often are the timeouts actually indicative of regressions. Perhaps we could flag them as "exceptional" results, shown in purple (& possibly not emailing anyone except the buildbot owner) - rather than red failures somehow.<br> </div><div><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Should we be sending tsan build failures to upstream developers? How often do they break tsan? I suspect that when LLVM breaks tsan, it also breaks ASan, which isn't as flaky. It might be better to mail the tsan failures to Dmitry or someone and not upstream LLVM devs.</div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 13, 2015 at 9:59 AM, Diego Novillo <span dir="ltr"><<a href="mailto:dnovillo@google.com" target="_blank">dnovillo@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Alexey, I got mail from one of the tsan buildbots, claiming a breakage<br>
in tsan tests. But I cannot see anything on the logs it has for the<br>
build.<br>
<br>
<a href="http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/17916/steps/run%2064-bit%20tsan%20unit%20tests/logs/stdio" target="_blank">http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/17916/steps/run%2064-bit%20tsan%20unit%20tests/logs/stdio</a><br>
<br>
Any ideas? Thanks. Diego.<br>
<br>
<br>
---------- Forwarded message ----------<br>
From: <<a href="mailto:llvm.buildmaster@lab.llvm.org" target="_blank">llvm.buildmaster@lab.llvm.org</a>><br>
Date: Wed, May 13, 2015 at 12:53 PM<br>
Subject: buildbot failure in LLVM on sanitizer-x86_64-linux<br>
To: Brendon Cahoon <<a href="mailto:bcahoon@codeaurora.org" target="_blank">bcahoon@codeaurora.org</a>>, Diego Novillo<br>
<<a href="mailto:dnovillo@google.com" target="_blank">dnovillo@google.com</a>>, Teresa Johnson <<a href="mailto:tejohnson@google.com" target="_blank">tejohnson@google.com</a>>, Yaron<br>
Keren <<a href="mailto:yaron.keren@gmail.com" target="_blank">yaron.keren@gmail.com</a>><br>
Cc: <a href="mailto:gkistanova@gmail.com" target="_blank">gkistanova@gmail.com</a><br>
<br>
<br>
The Buildbot has detected a new failure on builder<br>
sanitizer-x86_64-linux while building llvm.<br>
Full details are available at:<br>
<a href="http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/17916" target="_blank">http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/17916</a><br>
<br>
Buildbot URL: <a href="http://lab.llvm.org:8011/" target="_blank">http://lab.llvm.org:8011/</a><br>
<br>
Buildslave for this Build: sanitizer-buildbot1<br>
<br>
Build Reason: scheduler<br>
Build Source Stamp: [branch trunk] 237261<br>
Blamelist: bcahoon,dnovillo,tejohnson,yrnkrn<br>
<br>
BUILD FAILED: failed annotate failed run 64-bit tsan unit tests<br>
<br>
sincerely,<br>
-The Buildbot<br>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:LLVMdev@cs.uiuc.edu" target="_blank">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
</blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:LLVMdev@cs.uiuc.edu" target="_blank">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
<br></blockquote></div></div></div><br></div></div>
<br>_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:LLVMdev@cs.uiuc.edu">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
<br></blockquote></div><br></div>