<div dir="ltr">Should be flake-free in r275948 (and r275946).<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jul 18, 2016 at 6:38 PM, Hans Wennborg <span dir="ltr"><<a href="mailto:hans@chromium.org" target="_blank">hans@chromium.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I just ran into this. It's the only test failing on the 3.9 branch<br>
(so, based on r275826), which seems unfortunate.<br>
<br>
It seems this test is flaky by nature. Is there some way it could be<br>
made not to fail if the event it's waiting for doesn't occur, or<br>
alternatively could it be made to wait for that event to happen?<br>
<br>
Thanks,<br>
Hans<br>
<br>
<br>
******************** TEST 'EfficiencySanitizer-x86_64 ::<br>
TestCases/workingset-samples.cpp' FAILED ********************<br>
Script:<br>
--<br>
/work/llvm-3.9/build.release/./bin/clang<br>
-fsanitize=efficiency-working-set -m64 -gline-tables-only -O0<br>
/usr/local/google/work/llvm-3.9/compiler-rt.src/test/esan/TestCases/workingset-samples.cpp<br>
-o /usr/local/google/work/llvm-3.9/build.release/projects/compiler-rt/test/esan/X86_64Config/TestCases/Output/workingset-samples.cpp.tmp<br>
2>&1<br>
/usr/local/google/work/llvm-3.9/build.release/projects/compiler-rt/test/esan/X86_64Config/TestCases/Output/workingset-samples.cpp.tmp<br>
2>&1 | FileCheck<br>
/usr/local/google/work/llvm-3.9/compiler-rt.src/test/esan/TestCases/workingset-samples.cpp<br>
--<br>
Exit Code: 1<br>
<br>
Command Output (stderr):<br>
--<br>
/usr/local/google/work/llvm-3.9/compiler-rt.src/test/esan/TestCases/workingset-samples.cpp:29:17:<br>
<span class="">error: expected string not found in input<br>
</span> // CHECK-NEXT: =={{[0-9]+}}==# 0: {{[ 0-9]+}} {{KB|MB|Bytes}} ({{[<br>
<span class="">0-9]+}} cache lines)<br>
^<br>
<stdin>:4:1: note: scanning from here<br>
</span>==31948== Samples array #1 at period 80 ms<br>
<span class="">^<br>
<stdin>:11:36: note: possible intended match here<br>
</span>==31948== EfficiencySanitizer: the total working set size: 32 MB<br>
(524289 cache lines)<br>
^<br>
<br>
--<br>
<br>
********************<br>
<br>
<br>
<br>
On Wed, Jul 6, 2016 at 1:37 PM, Derek Bruening via llvm-commits<br>
<div class="HOEnZb"><div class="h5"><<a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a>> wrote:<br>
> I put in r274669 to improve the test. I'll watch the bot to ensure this<br>
> results in no more flakes.<br>
><br>
> On Tue, Jul 5, 2016 at 9:29 PM, Derek Bruening <<a href="mailto:bruening@google.com">bruening@google.com</a>> wrote:<br>
>><br>
>> It's not clear though, b/c if it matched sample #3 in array #0, there<br>
>> should be a sample in array #1 (downsampled every 4). Basically the<br>
>> FileCheck output doesn't make sense to me: it would really help to see the<br>
>> complete stdout. Is that available anywhere on the bot in a log or<br>
>> something?<br>
>><br>
>> On Tue, Jul 5, 2016 at 9:27 PM, Derek Bruening <<a href="mailto:bruening@google.com">bruening@google.com</a>><br>
>> wrote:<br>
>>><br>
>>> I believe you mis-interpreted the problem: the MB is the total size, and<br>
>>> that line matches. The problem looks like (though FileCheck output is<br>
>>> confusing) there are not enough samples to have any in array #1, which is a<br>
>>> tougher problem to solve.<br>
>>><br>
>>> On Tue, Jul 5, 2016 at 8:20 PM, David Majnemer <<a href="mailto:david.majnemer@gmail.com">david.majnemer@gmail.com</a>><br>
>>> wrote:<br>
>>>><br>
>>>> I took a wild stab in the dark at fixing this in r274587. Hopefully it<br>
>>>> is not just papering over an existing bug but its either that or reverting<br>
>>>> the changes that added the test in the first place.<br>
>>>><br>
>>>> On Tue, Jul 5, 2016 at 3:12 PM, Justin Bogner via llvm-commits<br>
>>>> <<a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a>> wrote:<br>
>>>>><br>
>>>>> Derek Bruening via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a>> writes:<br>
>>>>> > Please provide a pointer to which bots. It is not easy to find which<br>
>>>>> > ones<br>
>>>>> > are being referred to at <a href="http://lab.llvm.org:8011/waterfall" rel="noreferrer" target="_blank">http://lab.llvm.org:8011/waterfall</a> in order<br>
>>>>> > to see<br>
>>>>> > the precise test output.<br>
>>>>><br>
>>>>> The output is as David wrote, but here's a run that failed like this<br>
>>>>> today:<br>
>>>>><br>
>>>>><br>
>>>>> <a href="http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-debian-fast/builds/39465" rel="noreferrer" target="_blank">http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-debian-fast/builds/39465</a><br>
>>>>><br>
>>>>> This seems to be happening fairly frequently right now.<br>
>>>>><br>
>>>>> > On Fri, Jul 1, 2016 at 1:51 PM, Chandler Carruth<br>
>>>>> > <<a href="mailto:chandlerc@gmail.com">chandlerc@gmail.com</a>><br>
>>>>> > wrote:<br>
>>>>> ><br>
>>>>> >> Any update here? the build bots are flaking because of this test.<br>
>>>>> >><br>
>>>>> >> On Thu, Jun 23, 2016 at 11:22 PM David Majnemer via llvm-commits <<br>
>>>>> >> <a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a>> wrote:<br>
>>>>> >><br>
>>>>> >>> Still seems flaky:<br>
>>>>> >>><br>
>>>>> >>><br>
>>>>> >>> /home/llvmbb/llvm-build-dir/llvm-clang-lld-x86_64-debian-fast/llvm.src/projects/compiler-rt/test/esan/TestCases/workingset-samples.cpp:25:17:<br>
>>>>> >>> error: expected string not found in input<br>
>>>>> >>> // CHECK-NEXT: =={{[0-9]+}}==# 0: {{[ 0-9]+}} {{KB|Bytes}} ({{[<br>
>>>>> >>> 0-9]+}}<br>
>>>>> >>> cache lines)<br>
>>>>> >>> ^<br>
>>>>> >>> <stdin>:4:1: note: scanning from here<br>
>>>>> >>> ==21268== Samples array #1 at period 80 ms<br>
>>>>> >>> ^<br>
>>>>> >>> <stdin>:11:36: note: possible intended match here<br>
>>>>> >>> ==21268== EfficiencySanitizer: the total working set size: 32 MB<br>
>>>>> >>> (524289<br>
>>>>> >>> cache lines)<br>
>>>>> >>><br>
>>>>> >>> Looks like the regex pattern should be expanded further?<br>
>>>>> >>><br>
>>>>> >>> On Wed, Jun 8, 2016 at 10:35 AM, Derek Bruening via llvm-commits <<br>
>>>>> >>> <a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a>> wrote:<br>
>>>>> >>><br>
>>>>> >>>> Author: bruening<br>
>>>>> >>>> Date: Wed Jun 8 12:35:52 2016<br>
>>>>> >>>> New Revision: 272175<br>
>>>>> >>>><br>
>>>>> >>>> URL: <a href="http://llvm.org/viewvc/llvm-project?rev=272175&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=272175&view=rev</a><br>
>>>>> >>>> Log:<br>
>>>>> >>>> [esan|wset] Reduce flakiness in samples test<br>
>>>>> >>>><br>
>>>>> >>>> Generalizes the workingset-samples test to pass when a sample has<br>
>>>>> >>>> a<br>
>>>>> >>>> size of 0, which can happen on a loaded machine.<br>
>>>>> >>>><br>
>>>>> >>>> Modified:<br>
>>>>> >>>> compiler-rt/trunk/test/esan/TestCases/workingset-samples.cpp<br>
>>>>> >>>><br>
>>>>> >>>> Modified:<br>
>>>>> >>>> compiler-rt/trunk/test/esan/TestCases/workingset-samples.cpp<br>
>>>>> >>>> URL:<br>
>>>>> >>>><br>
>>>>> >>>> <a href="http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/test/esan/TestCases/workingset-samples.cpp?rev=272175&r1=272174&r2=272175&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/test/esan/TestCases/workingset-samples.cpp?rev=272175&r1=272174&r2=272175&view=diff</a><br>
>>>>> >>>><br>
>>>>> >>>><br>
>>>>> >>>> ==============================================================================<br>
>>>>> >>>> --- compiler-rt/trunk/test/esan/TestCases/workingset-samples.cpp<br>
>>>>> >>>> (original)<br>
>>>>> >>>> +++ compiler-rt/trunk/test/esan/TestCases/workingset-samples.cpp<br>
>>>>> >>>> Wed<br>
>>>>> >>>> Jun 8 12:35:52 2016<br>
>>>>> >>>> @@ -22,12 +22,12 @@ int main(int argc, char **argv) {<br>
>>>>> >>>> // We only check for a few samples here to reduce the chance of<br>
>>>>> >>>> flakiness.<br>
>>>>> >>>> // CHECK: =={{[0-9]+}}== Total number of samples:<br>
>>>>> >>>> {{[0-9]+}}<br>
>>>>> >>>> // CHECK-NEXT: =={{[0-9]+}}== Samples array #0 at period 20 ms<br>
>>>>> >>>> - // CHECK-NEXT: =={{[0-9]+}}==# 0: {{[ 0-9]+}} KB ({{[ 0-9]+}}<br>
>>>>> >>>> cache<br>
>>>>> >>>> lines)<br>
>>>>> >>>> - // CHECK-NEXT: =={{[0-9]+}}==# 1: {{[ 0-9]+}} KB ({{[ 0-9]+}}<br>
>>>>> >>>> cache<br>
>>>>> >>>> lines)<br>
>>>>> >>>> - // CHECK-NEXT: =={{[0-9]+}}==# 2: {{[ 0-9]+}} KB ({{[ 0-9]+}}<br>
>>>>> >>>> cache<br>
>>>>> >>>> lines)<br>
>>>>> >>>> - // CHECK-NEXT: =={{[0-9]+}}==# 3: {{[ 0-9]+}} KB ({{[ 0-9]+}}<br>
>>>>> >>>> cache<br>
>>>>> >>>> lines)<br>
>>>>> >>>> + // CHECK-NEXT: =={{[0-9]+}}==# 0: {{[ 0-9]+}} {{KB|Bytes}}<br>
>>>>> >>>> ({{[<br>
>>>>> >>>> 0-9]+}} cache lines)<br>
>>>>> >>>> + // CHECK-NEXT: =={{[0-9]+}}==# 1: {{[ 0-9]+}} {{KB|Bytes}}<br>
>>>>> >>>> ({{[<br>
>>>>> >>>> 0-9]+}} cache lines)<br>
>>>>> >>>> + // CHECK-NEXT: =={{[0-9]+}}==# 2: {{[ 0-9]+}} {{KB|Bytes}}<br>
>>>>> >>>> ({{[<br>
>>>>> >>>> 0-9]+}} cache lines)<br>
>>>>> >>>> + // CHECK-NEXT: =={{[0-9]+}}==# 3: {{[ 0-9]+}} {{KB|Bytes}}<br>
>>>>> >>>> ({{[<br>
>>>>> >>>> 0-9]+}} cache lines)<br>
>>>>> >>>> // CHECK: =={{[0-9]+}}== Samples array #1 at period 80 ms<br>
>>>>> >>>> - // CHECK-NEXT: =={{[0-9]+}}==# 0: {{[ 0-9]+}} KB ({{[ 0-9]+}}<br>
>>>>> >>>> cache<br>
>>>>> >>>> lines)<br>
>>>>> >>>> + // CHECK-NEXT: =={{[0-9]+}}==# 0: {{[ 0-9]+}} {{KB|Bytes}}<br>
>>>>> >>>> ({{[<br>
>>>>> >>>> 0-9]+}} cache lines)<br>
>>>>> >>>> // CHECK: =={{[0-9]+}}== Samples array #2 at period 320 ms<br>
>>>>> >>>> // CHECK: =={{[0-9]+}}== Samples array #3 at period 1280<br>
>>>>> >>>> ms<br>
>>>>> >>>> // CHECK: =={{[0-9]+}}== Samples array #4 at period 5120<br>
>>>>> >>>> ms<br>
>>>>> >>>><br>
>>>>> >>>><br>
>>>>> >>>> _______________________________________________<br>
>>>>> >>>> llvm-commits mailing list<br>
>>>>> >>>> <a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a><br>
>>>>> >>>> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
>>>>> >>>><br>
>>>>> >>><br>
>>>>> >>> _______________________________________________<br>
>>>>> >>> llvm-commits mailing list<br>
>>>>> >>> <a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a><br>
>>>>> >>> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
>>>>> >>><br>
>>>>> >><br>
>>>>> > _______________________________________________<br>
>>>>> > llvm-commits mailing list<br>
>>>>> > <a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a><br>
>>>>> > <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
>>>>> _______________________________________________<br>
>>>>> llvm-commits mailing list<br>
>>>>> <a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a><br>
>>>>> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
>>>><br>
>>>><br>
>>><br>
>><br>
><br>
><br>
> _______________________________________________<br>
> llvm-commits mailing list<br>
> <a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a><br>
> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
><br>
</div></div></blockquote></div><br></div>