[compiler-rt] r315777 - [asan] Deflake one test by running it 3 times.

Mon Oct 16 16:26:41 PDT 2017

Well, it's an x86 device, so a bit of an oddity, but it is still
supported. It runs an OS release from 2016 (N). It's a reasonable
configuration.

On Mon, Oct 16, 2017 at 2:42 PM, David Blaikie <dblaikie at gmail.com> wrote:
> Could the device be removed from testing? (is it likely the device that's at
> fault, more than the test - or is it a truly representative, valid device
> where any device in that configuration would have this problem and it's a
> supported configuration (not too old, etc)?)
>
> On Mon, Oct 16, 2017 at 2:38 PM Evgenii Stepanov via llvm-commits
> <llvm-commits at lists.llvm.org> wrote:
>>
>> For some value of x, p^x is low enough :)
>>
>> Seriously speaking, the test is failing on a particular device and I
>> suspect it could be a kernel issue: it is running 3.10, which is quite
>> old. I don't have time to debug it, but I could make the test as
>> failing on (android && i686) which is not the real cause of the error,
>> but it would make the bot green and keep the test exercised on the
>> arm-android device.
>>
>> On Mon, Oct 16, 2017 at 2:09 PM, Rui Ueyama via llvm-commits
>> <llvm-commits at lists.llvm.org> wrote:
>> > You cannot fix a flakey test by just running it more than once, can you?
>> > I
>> > mean, the probability of test failure decreases from p to p^3, but it is
>> > still not zero.
>> >
>> > On Mon, Oct 16, 2017 at 2:03 PM, Vitaly Buka via llvm-commits
>> > <llvm-commits at lists.llvm.org> wrote:
>> >>
>> >> It does not makes test slower as it passes on all platforms from the
>> >> first
>> >> time.
>> >>
>> >> We don't suspect issues with functionally, rather some specific
>> >> platform
>> >> bug.
>> >> Also on Android we know only one device were it fails. And I guess it
>> >> fails only in Asan mode?
>> >> So if we mark it unsupported, it should be very specific.
>> >> E.g. // UNSUPPORTED: android && asan && x86 (not sure if that one is
>> >> 64bit
>> >> and how we exactly recognize x86 on android)
>> >>
>> >> On Mon, Oct 16, 2017 at 1:49 PM, David Blaikie via llvm-commits
>> >> <llvm-commits at lists.llvm.org> wrote:
>> >>>
>> >>> If this functionality isn't reliable on Android, I think
>> >>> UNSUPPORTED/XFAIL is more appropriate than deflaking like this. The
>> >>> functionality's no good if it fails that much, so why test/support it,
>> >>> making tests slower and risk flakiness creeping in on other platforms
>> >>> due to
>> >>> the more generous testing?
>> >>>
>> >>> On Fri, Oct 13, 2017 at 4:57 PM Evgeniy Stepanov via llvm-commits
>> >>> <llvm-commits at lists.llvm.org> wrote:
>> >>>>
>> >>>> Author: eugenis
>> >>>> Date: Fri Oct 13 16:57:08 2017
>> >>>> New Revision: 315777
>> >>>>
>> >>>> URL: http://llvm.org/viewvc/llvm-project?rev=315777&view=rev
>> >>>> Log:
>> >>>> [asan] Deflake one test by running it 3 times.
>> >>>>
>> >>>> The test seems to trigger an android platform bug under load.
>> >>>>
>> >>>> Modified:
>> >>>>
>> >>>>
>> >>>> compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc
>> >>>>
>> >>>> Modified:
>> >>>>
>> >>>> compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc
>> >>>> URL:
>> >>>>
>> >>>> http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc?rev=315777&r1=315776&r2=315777&view=diff
>> >>>>
>> >>>>
>> >>>> ==============================================================================
>> >>>> ---
>> >>>>
>> >>>> compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc
>> >>>> (original)
>> >>>> +++
>> >>>>
>> >>>> compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc
>> >>>> Fri Oct 13 16:57:08 2017
>> >>>> @@ -4,16 +4,34 @@
>> >>>>  // clang-format off
>> >>>>  // RUN: %clangxx -O0 %s -o %t
>> >>>>
>> >>>> +// RUN: %env_tool_opts=handle_segv=0 not %run %t 2>&1 | FileCheck %s
>> >>>> --check-prefix=CHECK0 || \
>> >>>> +// RUN: %env_tool_opts=handle_segv=0 not %run %t 2>&1 | FileCheck %s
>> >>>> --check-prefix=CHECK0 || \
>> >>>>  // RUN: %env_tool_opts=handle_segv=0 not %run %t 2>&1 | FileCheck %s
>> >>>> --check-prefix=CHECK0
>> >>>> +// RUN: %env_tool_opts=handle_segv=1 not %run %t 2>&1 | FileCheck %s
>> >>>> --check-prefix=CHECK1 || \
>> >>>> +// RUN: %env_tool_opts=handle_segv=1 not %run %t 2>&1 | FileCheck %s
>> >>>> --check-prefix=CHECK1 || \
>> >>>>  // RUN: %env_tool_opts=handle_segv=1 not %run %t 2>&1 | FileCheck %s
>> >>>> --check-prefix=CHECK1
>> >>>> +// RUN: %env_tool_opts=handle_segv=2 not %run %t 2>&1 | FileCheck %s
>> >>>> --check-prefix=CHECK2 || \
>> >>>> +// RUN: %env_tool_opts=handle_segv=2 not %run %t 2>&1 | FileCheck %s
>> >>>> --check-prefix=CHECK2 || \
>> >>>>  // RUN: %env_tool_opts=handle_segv=2 not %run %t 2>&1 | FileCheck %s
>> >>>> --check-prefix=CHECK2
>> >>>>
>> >>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=0 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
>> >>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=0 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
>> >>>>  // RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=0 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0
>> >>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=0 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>> >>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=0 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>> >>>>  // RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=0 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2
>> >>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=0 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>> >>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=0 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>> >>>>  // RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=0 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2
>> >>>>
>> >>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=1 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
>> >>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=1 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
>> >>>>  // RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=1 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0
>> >>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=1 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK1 || \
>> >>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=1 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK1 || \
>> >>>>  // RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=1 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK1
>> >>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=1 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>> >>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=1 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
>> >>>>  // RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=1 not
>> >>>> %run
>> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2
>> >>>>  // clang-format on
>> >>>>
>> >>>> @@ -21,9 +39,6 @@
>> >>>>  // XFAIL: msan
>> >>>>  // XFAIL: tsan
>> >>>>
>> >>>> -// Flaky errors in debuggerd with "waitpid returned unexpected pid
>> >>>> (0)"
>> >>>> in logcat.
>> >>>> -// UNSUPPORTED: android
>> >>>> -
>> >>>>  #include <signal.h>
>> >>>>  #include <stdio.h>
>> >>>>  #include <stdlib.h>
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> llvm-commits mailing list
>> >>>> llvm-commits at lists.llvm.org
>> >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> llvm-commits mailing list
>> >>> llvm-commits at lists.llvm.org
>> >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>> >>>
>> >>
>> >>
>> >> _______________________________________________
>> >> llvm-commits mailing list
>> >> llvm-commits at lists.llvm.org
>> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>> >>
>> >
>> >
>> > _______________________________________________
>> > llvm-commits mailing list
>> > llvm-commits at lists.llvm.org
>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>> >
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits