[compiler-rt] r315777 - [asan] Deflake one test by running it 3 times.

Mon Oct 16 14:08:41 PDT 2017

On Mon, Oct 16, 2017 at 2:04 PM Vitaly Buka <vitalybuka at google.com> wrote:

> It does not makes test slower as it passes on all platforms from the first
> time.
>
> We don't suspect issues with functionally, rather some specific platform
> bug.
>

But what I mean is this test now could pass anywhere it became flaky, not
just on Android. That seems like a bad thing - that the test could hide
problems we'd want to know about.

> Also on Android we know only one device were it fails. And I guess it
> fails only in Asan mode?
> So if we mark it unsupported, it should be very specific.
> E.g. // UNSUPPORTED: android && asan && x86 (not sure if that one is
> 64bit and how we exactly recognize x86 on android)
>

If there's not a feature to test for the platform (android), one could be
added, perhaps?

- Dave

>
> On Mon, Oct 16, 2017 at 1:49 PM, David Blaikie via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
>
>> If this functionality isn't reliable on Android, I think
>> UNSUPPORTED/XFAIL is more appropriate than deflaking like this. The
>> functionality's no good if it fails that much, so why test/support it,
>> making tests slower and risk flakiness creeping in on other platforms due
>> to the more generous testing?
>>
>> On Fri, Oct 13, 2017 at 4:57 PM Evgeniy Stepanov via llvm-commits <
>> llvm-commits at lists.llvm.org> wrote:
>>
>>> Author: eugenis
>>> Date: Fri Oct 13 16:57:08 2017
>>> New Revision: 315777
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=315777&view=rev
>>> Log:
>>> [asan] Deflake one test by running it 3 times.
>>>
>>> The test seems to trigger an android platform bug under load.
>>>
>>> Modified:
>>>
>>> compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc
>>>
>>> Modified:
>>> compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc?rev=315777&r1=315776&r2=315777&view=diff
>>>
>>> ==============================================================================
>>> ---
>>> compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc
>>> (original)
>>> +++
>>> compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc
>>> Fri Oct 13 16:57:08 2017
>>> @@ -4,16 +4,34 @@
>>>  // clang-format off
>>>  // RUN: %clangxx -O0 %s -o %t
>>>
>>> +// RUN: %env_tool_opts=handle_segv=0 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK0 || \
>>> +// RUN: %env_tool_opts=handle_segv=0 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK0 || \
>>>  // RUN: %env_tool_opts=handle_segv=0 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK0
>>> +// RUN: %env_tool_opts=handle_segv=1 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK1 || \
>>> +// RUN: %env_tool_opts=handle_segv=1 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK1 || \
>>>  // RUN: %env_tool_opts=handle_segv=1 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK1
>>> +// RUN: %env_tool_opts=handle_segv=2 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK2 || \
>>> +// RUN: %env_tool_opts=handle_segv=2 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK2 || \
>>>  // RUN: %env_tool_opts=handle_segv=2 not %run %t 2>&1 | FileCheck %s
>>> --check-prefix=CHECK2
>>>
>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=0 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=0 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
>>>  // RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=0 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0
>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=0 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=0 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>>>  // RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=0 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2
>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=0 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=0 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>>>  // RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=0 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2
>>>
>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=1 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=1 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
>>>  // RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=1 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0
>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=1 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK1 || \
>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=1 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK1 || \
>>>  // RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=1 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK1
>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=1 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=1 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>>>  // RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=1 not %run
>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2
>>>  // clang-format on
>>>
>>> @@ -21,9 +39,6 @@
>>>  // XFAIL: msan
>>>  // XFAIL: tsan
>>>
>>> -// Flaky errors in debuggerd with "waitpid returned unexpected pid (0)"
>>> in logcat.
>>> -// UNSUPPORTED: android
>>> -
>>>  #include <signal.h>
>>>  #include <stdio.h>
>>>  #include <stdlib.h>
>>>
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20171016/2a786419/attachment.html>