[compiler-rt] r272175 - [esan|wset] Reduce flakiness in samples test
Hans Wennborg via llvm-commits
llvm-commits at lists.llvm.org
Mon Jul 18 15:38:58 PDT 2016
I just ran into this. It's the only test failing on the 3.9 branch
(so, based on r275826), which seems unfortunate.
It seems this test is flaky by nature. Is there some way it could be
made not to fail if the event it's waiting for doesn't occur, or
alternatively could it be made to wait for that event to happen?
Thanks,
Hans
******************** TEST 'EfficiencySanitizer-x86_64 ::
TestCases/workingset-samples.cpp' FAILED ********************
Script:
--
/work/llvm-3.9/build.release/./bin/clang
-fsanitize=efficiency-working-set -m64 -gline-tables-only -O0
/usr/local/google/work/llvm-3.9/compiler-rt.src/test/esan/TestCases/workingset-samples.cpp
-o /usr/local/google/work/llvm-3.9/build.release/projects/compiler-rt/test/esan/X86_64Config/TestCases/Output/workingset-samples.cpp.tmp
2>&1
/usr/local/google/work/llvm-3.9/build.release/projects/compiler-rt/test/esan/X86_64Config/TestCases/Output/workingset-samples.cpp.tmp
2>&1 | FileCheck
/usr/local/google/work/llvm-3.9/compiler-rt.src/test/esan/TestCases/workingset-samples.cpp
--
Exit Code: 1
Command Output (stderr):
--
/usr/local/google/work/llvm-3.9/compiler-rt.src/test/esan/TestCases/workingset-samples.cpp:29:17:
error: expected string not found in input
// CHECK-NEXT: =={{[0-9]+}}==# 0: {{[ 0-9]+}} {{KB|MB|Bytes}} ({{[
0-9]+}} cache lines)
^
<stdin>:4:1: note: scanning from here
==31948== Samples array #1 at period 80 ms
^
<stdin>:11:36: note: possible intended match here
==31948== EfficiencySanitizer: the total working set size: 32 MB
(524289 cache lines)
^
--
********************
On Wed, Jul 6, 2016 at 1:37 PM, Derek Bruening via llvm-commits
<llvm-commits at lists.llvm.org> wrote:
> I put in r274669 to improve the test. I'll watch the bot to ensure this
> results in no more flakes.
>
> On Tue, Jul 5, 2016 at 9:29 PM, Derek Bruening <bruening at google.com> wrote:
>>
>> It's not clear though, b/c if it matched sample #3 in array #0, there
>> should be a sample in array #1 (downsampled every 4). Basically the
>> FileCheck output doesn't make sense to me: it would really help to see the
>> complete stdout. Is that available anywhere on the bot in a log or
>> something?
>>
>> On Tue, Jul 5, 2016 at 9:27 PM, Derek Bruening <bruening at google.com>
>> wrote:
>>>
>>> I believe you mis-interpreted the problem: the MB is the total size, and
>>> that line matches. The problem looks like (though FileCheck output is
>>> confusing) there are not enough samples to have any in array #1, which is a
>>> tougher problem to solve.
>>>
>>> On Tue, Jul 5, 2016 at 8:20 PM, David Majnemer <david.majnemer at gmail.com>
>>> wrote:
>>>>
>>>> I took a wild stab in the dark at fixing this in r274587. Hopefully it
>>>> is not just papering over an existing bug but its either that or reverting
>>>> the changes that added the test in the first place.
>>>>
>>>> On Tue, Jul 5, 2016 at 3:12 PM, Justin Bogner via llvm-commits
>>>> <llvm-commits at lists.llvm.org> wrote:
>>>>>
>>>>> Derek Bruening via llvm-commits <llvm-commits at lists.llvm.org> writes:
>>>>> > Please provide a pointer to which bots. It is not easy to find which
>>>>> > ones
>>>>> > are being referred to at http://lab.llvm.org:8011/waterfall in order
>>>>> > to see
>>>>> > the precise test output.
>>>>>
>>>>> The output is as David wrote, but here's a run that failed like this
>>>>> today:
>>>>>
>>>>>
>>>>> http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-debian-fast/builds/39465
>>>>>
>>>>> This seems to be happening fairly frequently right now.
>>>>>
>>>>> > On Fri, Jul 1, 2016 at 1:51 PM, Chandler Carruth
>>>>> > <chandlerc at gmail.com>
>>>>> > wrote:
>>>>> >
>>>>> >> Any update here? the build bots are flaking because of this test.
>>>>> >>
>>>>> >> On Thu, Jun 23, 2016 at 11:22 PM David Majnemer via llvm-commits <
>>>>> >> llvm-commits at lists.llvm.org> wrote:
>>>>> >>
>>>>> >>> Still seems flaky:
>>>>> >>>
>>>>> >>>
>>>>> >>> /home/llvmbb/llvm-build-dir/llvm-clang-lld-x86_64-debian-fast/llvm.src/projects/compiler-rt/test/esan/TestCases/workingset-samples.cpp:25:17:
>>>>> >>> error: expected string not found in input
>>>>> >>> // CHECK-NEXT: =={{[0-9]+}}==# 0: {{[ 0-9]+}} {{KB|Bytes}} ({{[
>>>>> >>> 0-9]+}}
>>>>> >>> cache lines)
>>>>> >>> ^
>>>>> >>> <stdin>:4:1: note: scanning from here
>>>>> >>> ==21268== Samples array #1 at period 80 ms
>>>>> >>> ^
>>>>> >>> <stdin>:11:36: note: possible intended match here
>>>>> >>> ==21268== EfficiencySanitizer: the total working set size: 32 MB
>>>>> >>> (524289
>>>>> >>> cache lines)
>>>>> >>>
>>>>> >>> Looks like the regex pattern should be expanded further?
>>>>> >>>
>>>>> >>> On Wed, Jun 8, 2016 at 10:35 AM, Derek Bruening via llvm-commits <
>>>>> >>> llvm-commits at lists.llvm.org> wrote:
>>>>> >>>
>>>>> >>>> Author: bruening
>>>>> >>>> Date: Wed Jun 8 12:35:52 2016
>>>>> >>>> New Revision: 272175
>>>>> >>>>
>>>>> >>>> URL: http://llvm.org/viewvc/llvm-project?rev=272175&view=rev
>>>>> >>>> Log:
>>>>> >>>> [esan|wset] Reduce flakiness in samples test
>>>>> >>>>
>>>>> >>>> Generalizes the workingset-samples test to pass when a sample has
>>>>> >>>> a
>>>>> >>>> size of 0, which can happen on a loaded machine.
>>>>> >>>>
>>>>> >>>> Modified:
>>>>> >>>> compiler-rt/trunk/test/esan/TestCases/workingset-samples.cpp
>>>>> >>>>
>>>>> >>>> Modified:
>>>>> >>>> compiler-rt/trunk/test/esan/TestCases/workingset-samples.cpp
>>>>> >>>> URL:
>>>>> >>>>
>>>>> >>>> http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/test/esan/TestCases/workingset-samples.cpp?rev=272175&r1=272174&r2=272175&view=diff
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> ==============================================================================
>>>>> >>>> --- compiler-rt/trunk/test/esan/TestCases/workingset-samples.cpp
>>>>> >>>> (original)
>>>>> >>>> +++ compiler-rt/trunk/test/esan/TestCases/workingset-samples.cpp
>>>>> >>>> Wed
>>>>> >>>> Jun 8 12:35:52 2016
>>>>> >>>> @@ -22,12 +22,12 @@ int main(int argc, char **argv) {
>>>>> >>>> // We only check for a few samples here to reduce the chance of
>>>>> >>>> flakiness.
>>>>> >>>> // CHECK: =={{[0-9]+}}== Total number of samples:
>>>>> >>>> {{[0-9]+}}
>>>>> >>>> // CHECK-NEXT: =={{[0-9]+}}== Samples array #0 at period 20 ms
>>>>> >>>> - // CHECK-NEXT: =={{[0-9]+}}==# 0: {{[ 0-9]+}} KB ({{[ 0-9]+}}
>>>>> >>>> cache
>>>>> >>>> lines)
>>>>> >>>> - // CHECK-NEXT: =={{[0-9]+}}==# 1: {{[ 0-9]+}} KB ({{[ 0-9]+}}
>>>>> >>>> cache
>>>>> >>>> lines)
>>>>> >>>> - // CHECK-NEXT: =={{[0-9]+}}==# 2: {{[ 0-9]+}} KB ({{[ 0-9]+}}
>>>>> >>>> cache
>>>>> >>>> lines)
>>>>> >>>> - // CHECK-NEXT: =={{[0-9]+}}==# 3: {{[ 0-9]+}} KB ({{[ 0-9]+}}
>>>>> >>>> cache
>>>>> >>>> lines)
>>>>> >>>> + // CHECK-NEXT: =={{[0-9]+}}==# 0: {{[ 0-9]+}} {{KB|Bytes}}
>>>>> >>>> ({{[
>>>>> >>>> 0-9]+}} cache lines)
>>>>> >>>> + // CHECK-NEXT: =={{[0-9]+}}==# 1: {{[ 0-9]+}} {{KB|Bytes}}
>>>>> >>>> ({{[
>>>>> >>>> 0-9]+}} cache lines)
>>>>> >>>> + // CHECK-NEXT: =={{[0-9]+}}==# 2: {{[ 0-9]+}} {{KB|Bytes}}
>>>>> >>>> ({{[
>>>>> >>>> 0-9]+}} cache lines)
>>>>> >>>> + // CHECK-NEXT: =={{[0-9]+}}==# 3: {{[ 0-9]+}} {{KB|Bytes}}
>>>>> >>>> ({{[
>>>>> >>>> 0-9]+}} cache lines)
>>>>> >>>> // CHECK: =={{[0-9]+}}== Samples array #1 at period 80 ms
>>>>> >>>> - // CHECK-NEXT: =={{[0-9]+}}==# 0: {{[ 0-9]+}} KB ({{[ 0-9]+}}
>>>>> >>>> cache
>>>>> >>>> lines)
>>>>> >>>> + // CHECK-NEXT: =={{[0-9]+}}==# 0: {{[ 0-9]+}} {{KB|Bytes}}
>>>>> >>>> ({{[
>>>>> >>>> 0-9]+}} cache lines)
>>>>> >>>> // CHECK: =={{[0-9]+}}== Samples array #2 at period 320 ms
>>>>> >>>> // CHECK: =={{[0-9]+}}== Samples array #3 at period 1280
>>>>> >>>> ms
>>>>> >>>> // CHECK: =={{[0-9]+}}== Samples array #4 at period 5120
>>>>> >>>> ms
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> _______________________________________________
>>>>> >>>> llvm-commits mailing list
>>>>> >>>> llvm-commits at lists.llvm.org
>>>>> >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>>> >>>>
>>>>> >>>
>>>>> >>> _______________________________________________
>>>>> >>> llvm-commits mailing list
>>>>> >>> llvm-commits at lists.llvm.org
>>>>> >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>>> >>>
>>>>> >>
>>>>> > _______________________________________________
>>>>> > llvm-commits mailing list
>>>>> > llvm-commits at lists.llvm.org
>>>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>>> _______________________________________________
>>>>> llvm-commits mailing list
>>>>> llvm-commits at lists.llvm.org
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>>
>>>>
>>>
>>
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
More information about the llvm-commits
mailing list