[LLVMdev] Confusing buildbot failure in LLVM on sanitizer-x86_64-linux

[Kostya Serebryany]

+dvyukov

On Wed, May 13, 2015 at 11:08 AM, David Blaikie <dblaikie at gmail.com> wrote:

>
>
> On Wed, May 13, 2015 at 10:39 AM, Reid Kleckner <rnk at google.com> wrote:
>
>> It's a 20m timeout without output.
>>
>> If you back up to the build and look at the 'annotate' step output,
>> there's this text:
>>
>> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/17916/steps/annotate/logs/stdio
>>
>> -- Testing: 258 tests, 16 threads --
>> Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
>> command timed out: 1200 seconds without output, attempting to kill
>> process killed by signal 9
>> program finished with exit code -1
>> elapsedTime=3507.624426
>>
>> The annotator should probably include that timeout text in the failing
>> step, so that sounds like a bug.
>>
>> Another issue is that tsan times out sometimes.
>>
>
> Also - how often are the timeouts actually indicative of regressions.
> Perhaps we could flag them as "exceptional" results, shown in purple (&
> possibly not emailing anyone except the buildbot owner) - rather than red
> failures somehow.
>
>
>> Should we be sending tsan build failures to upstream developers? How
>> often do they break tsan? I suspect that when LLVM breaks tsan, it also
>> breaks ASan, which isn't as flaky. It might be better to mail the tsan
>> failures to Dmitry or someone and not upstream LLVM devs.
>>
>> On Wed, May 13, 2015 at 9:59 AM, Diego Novillo <dnovillo at google.com>
>> wrote:
>>
>>> Alexey, I got mail from one of the tsan buildbots, claiming a breakage
>>> in tsan tests. But I cannot see anything on the logs it has for the
>>> build.
>>>
>>>
>>> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/17916/steps/run%2064-bit%20tsan%20unit%20tests/logs/stdio
>>>
>>> Any ideas?  Thanks. Diego.
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From:  <llvm.buildmaster at lab.llvm.org>
>>> Date: Wed, May 13, 2015 at 12:53 PM
>>> Subject: buildbot failure in LLVM on sanitizer-x86_64-linux
>>> To: Brendon Cahoon <bcahoon at codeaurora.org>, Diego Novillo
>>> <dnovillo at google.com>, Teresa Johnson <tejohnson at google.com>, Yaron
>>> Keren <yaron.keren at gmail.com>
>>> Cc: gkistanova at gmail.com
>>>
>>>
>>> The Buildbot has detected a new failure on builder
>>> sanitizer-x86_64-linux while building llvm.
>>> Full details are available at:
>>>  http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/17916
>>>
>>> Buildbot URL: http://lab.llvm.org:8011/
>>>
>>> Buildslave for this Build: sanitizer-buildbot1
>>>
>>> Build Reason: scheduler
>>> Build Source Stamp: [branch trunk] 237261
>>> Blamelist: bcahoon,dnovillo,tejohnson,yrnkrn
>>>
>>> BUILD FAILED: failed annotate failed run 64-bit tsan unit tests
>>>
>>> sincerely,
>>>  -The Buildbot
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150514/c295079d/attachment.html>