[compiler-rt] r315777 - [asan] Deflake one test by running it 3 times.

Mon Oct 16 14:09:06 PDT 2017

You cannot fix a flakey test by just running it more than once, can you? I
mean, the probability of test failure decreases from *p* to *p*^3, but it
is still not zero.

On Mon, Oct 16, 2017 at 2:03 PM, Vitaly Buka via llvm-commits <
llvm-commits at lists.llvm.org> wrote:

> It does not makes test slower as it passes on all platforms from the first
> time.
>
> We don't suspect issues with functionally, rather some specific platform
> bug.
> Also on Android we know only one device were it fails. And I guess it
> fails only in Asan mode?
> So if we mark it unsupported, it should be very specific.
> E.g. // UNSUPPORTED: android && asan && x86 (not sure if that one is
> 64bit and how we exactly recognize x86 on android)
>
> On Mon, Oct 16, 2017 at 1:49 PM, David Blaikie via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
>
>> If this functionality isn't reliable on Android, I think
>> UNSUPPORTED/XFAIL is more appropriate than deflaking like this. The
>> functionality's no good if it fails that much, so why test/support it,
>> making tests slower and risk flakiness creeping in on other platforms due
>> to the more generous testing?
>>
>> On Fri, Oct 13, 2017 at 4:57 PM Evgeniy Stepanov via llvm-commits <
>> llvm-commits at lists.llvm.org> wrote:
>>
>>> Author: eugenis
>>> Date: Fri Oct 13 16:57:08 2017
>>> New Revision: 315777
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=315777&view=rev
>>> Log:
>>> [asan] Deflake one test by running it 3 times.
>>>
>>> The test seems to trigger an android platform bug under load.
>>>
>>> Modified:
>>>     compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/
>>> allow_user_segv.cc
>>>
>>> Modified: compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/
>>> allow_user_segv.cc
>>> URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/test/
>>> sanitizer_common/TestCases/Linux/allow_user_segv.cc?rev=
>>> 315777&r1=315776&r2=315777&view=diff
>>> ============================================================
>>> ==================
>>> --- compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc
>>> (original)
>>> +++ compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc
>>> Fri Oct 13 16:57:08 2017
>>> @@ -4,16 +4,34 @@
>>>  // clang-format off
>>>  // RUN: %clangxx -O0 %s -o %t
>>>
>>> +// RUN: %env_tool_opts=handle_segv=0 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK0 || \
>>> +// RUN: %env_tool_opts=handle_segv=0 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK0 || \
>>>  // RUN: %env_tool_opts=handle_segv=0 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK0
>>> +// RUN: %env_tool_opts=handle_segv=1 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK1 || \
>>> +// RUN: %env_tool_opts=handle_segv=1 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK1 || \
>>>  // RUN: %env_tool_opts=handle_segv=1 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK1
>>> +// RUN: %env_tool_opts=handle_segv=2 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK2 || \
>>> +// RUN: %env_tool_opts=handle_segv=2 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK2 || \
>>>  // RUN: %env_tool_opts=handle_segv=2 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK2
>>>
>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=0 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=0 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
>>>  // RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=0 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK0
>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=0 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=0 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>>>  // RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=0 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK2
>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=0 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=0 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>>>  // RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=0 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK2
>>>
>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=1 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=1 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
>>>  // RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=1 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK0
>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=1 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK1 || \
>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=1 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK1 || \
>>>  // RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=1 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK1
>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=1 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=1 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>>>  // RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=1 not
>>> %run %t 2>&1 | FileCheck %s --check-prefix=CHECK2
>>>  // clang-format on
>>>
>>> @@ -21,9 +39,6 @@
>>>  // XFAIL: msan
>>>  // XFAIL: tsan
>>>
>>> -// Flaky errors in debuggerd with "waitpid returned unexpected pid (0)"
>>> in logcat.
>>> -// UNSUPPORTED: android
>>> -
>>>  #include <signal.h>
>>>  #include <stdio.h>
>>>  #include <stdlib.h>
>>>
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>
>>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20171016/122e914b/attachment-0001.html>