[compiler-rt] r315777 - [asan] Deflake one test by running it 3 times.

Mon Oct 16 18:56:17 PDT 2017

But if it's supported, is this a bug that's going to be fixed? (on either
side - platform or compiler) - or is this functionality unsupported on this
platform? (if it's flaky, it's hard to say it's supported, that's not a
good user experience)

On Mon, Oct 16, 2017 at 4:27 PM Evgenii Stepanov <eugeni.stepanov at gmail.com>
wrote:

> Well, it's an x86 device, so a bit of an oddity, but it is still
> supported. It runs an OS release from 2016 (N). It's a reasonable
> configuration.
>
>
> On Mon, Oct 16, 2017 at 2:42 PM, David Blaikie <dblaikie at gmail.com> wrote:
> > Could the device be removed from testing? (is it likely the device
> that's at
> > fault, more than the test - or is it a truly representative, valid device
> > where any device in that configuration would have this problem and it's a
> > supported configuration (not too old, etc)?)
> >
> > On Mon, Oct 16, 2017 at 2:38 PM Evgenii Stepanov via llvm-commits
> > <llvm-commits at lists.llvm.org> wrote:
> >>
> >> For some value of x, p^x is low enough :)
> >>
> >> Seriously speaking, the test is failing on a particular device and I
> >> suspect it could be a kernel issue: it is running 3.10, which is quite
> >> old. I don't have time to debug it, but I could make the test as
> >> failing on (android && i686) which is not the real cause of the error,
> >> but it would make the bot green and keep the test exercised on the
> >> arm-android device.
> >>
> >> On Mon, Oct 16, 2017 at 2:09 PM, Rui Ueyama via llvm-commits
> >> <llvm-commits at lists.llvm.org> wrote:
> >> > You cannot fix a flakey test by just running it more than once, can
> you?
> >> > I
> >> > mean, the probability of test failure decreases from p to p^3, but it
> is
> >> > still not zero.
> >> >
> >> > On Mon, Oct 16, 2017 at 2:03 PM, Vitaly Buka via llvm-commits
> >> > <llvm-commits at lists.llvm.org> wrote:
> >> >>
> >> >> It does not makes test slower as it passes on all platforms from the
> >> >> first
> >> >> time.
> >> >>
> >> >> We don't suspect issues with functionally, rather some specific
> >> >> platform
> >> >> bug.
> >> >> Also on Android we know only one device were it fails. And I guess it
> >> >> fails only in Asan mode?
> >> >> So if we mark it unsupported, it should be very specific.
> >> >> E.g. // UNSUPPORTED: android && asan && x86 (not sure if that one is
> >> >> 64bit
> >> >> and how we exactly recognize x86 on android)
> >> >>
> >> >> On Mon, Oct 16, 2017 at 1:49 PM, David Blaikie via llvm-commits
> >> >> <llvm-commits at lists.llvm.org> wrote:
> >> >>>
> >> >>> If this functionality isn't reliable on Android, I think
> >> >>> UNSUPPORTED/XFAIL is more appropriate than deflaking like this. The
> >> >>> functionality's no good if it fails that much, so why test/support
> it,
> >> >>> making tests slower and risk flakiness creeping in on other
> platforms
> >> >>> due to
> >> >>> the more generous testing?
> >> >>>
> >> >>> On Fri, Oct 13, 2017 at 4:57 PM Evgeniy Stepanov via llvm-commits
> >> >>> <llvm-commits at lists.llvm.org> wrote:
> >> >>>>
> >> >>>> Author: eugenis
> >> >>>> Date: Fri Oct 13 16:57:08 2017
> >> >>>> New Revision: 315777
> >> >>>>
> >> >>>> URL: http://llvm.org/viewvc/llvm-project?rev=315777&view=rev
> >> >>>> Log:
> >> >>>> [asan] Deflake one test by running it 3 times.
> >> >>>>
> >> >>>> The test seems to trigger an android platform bug under load.
> >> >>>>
> >> >>>> Modified:
> >> >>>>
> >> >>>>
> >> >>>>
> compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc
> >> >>>>
> >> >>>> Modified:
> >> >>>>
> >> >>>>
> compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc
> >> >>>> URL:
> >> >>>>
> >> >>>>
> http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc?rev=315777&r1=315776&r2=315777&view=diff
> >> >>>>
> >> >>>>
> >> >>>>
> ==============================================================================
> >> >>>> ---
> >> >>>>
> >> >>>>
> compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc
> >> >>>> (original)
> >> >>>> +++
> >> >>>>
> >> >>>>
> compiler-rt/trunk/test/sanitizer_common/TestCases/Linux/allow_user_segv.cc
> >> >>>> Fri Oct 13 16:57:08 2017
> >> >>>> @@ -4,16 +4,34 @@
> >> >>>>  // clang-format off
> >> >>>>  // RUN: %clangxx -O0 %s -o %t
> >> >>>>
> >> >>>> +// RUN: %env_tool_opts=handle_segv=0 not %run %t 2>&1 | FileCheck
> %s
> >> >>>> --check-prefix=CHECK0 || \
> >> >>>> +// RUN: %env_tool_opts=handle_segv=0 not %run %t 2>&1 | FileCheck
> %s
> >> >>>> --check-prefix=CHECK0 || \
> >> >>>>  // RUN: %env_tool_opts=handle_segv=0 not %run %t 2>&1 | FileCheck
> %s
> >> >>>> --check-prefix=CHECK0
> >> >>>> +// RUN: %env_tool_opts=handle_segv=1 not %run %t 2>&1 | FileCheck
> %s
> >> >>>> --check-prefix=CHECK1 || \
> >> >>>> +// RUN: %env_tool_opts=handle_segv=1 not %run %t 2>&1 | FileCheck
> %s
> >> >>>> --check-prefix=CHECK1 || \
> >> >>>>  // RUN: %env_tool_opts=handle_segv=1 not %run %t 2>&1 | FileCheck
> %s
> >> >>>> --check-prefix=CHECK1
> >> >>>> +// RUN: %env_tool_opts=handle_segv=2 not %run %t 2>&1 | FileCheck
> %s
> >> >>>> --check-prefix=CHECK2 || \
> >> >>>> +// RUN: %env_tool_opts=handle_segv=2 not %run %t 2>&1 | FileCheck
> %s
> >> >>>> --check-prefix=CHECK2 || \
> >> >>>>  // RUN: %env_tool_opts=handle_segv=2 not %run %t 2>&1 | FileCheck
> %s
> >> >>>> --check-prefix=CHECK2
> >> >>>>
> >> >>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=0 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
> >> >>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=0 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
> >> >>>>  // RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=0 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0
> >> >>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=0 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
> >> >>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=0 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
> >> >>>>  // RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=0 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2
> >> >>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=0 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
> >> >>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=0 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
> >> >>>>  // RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=0 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2
> >> >>>>
> >> >>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=1 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
> >> >>>> +// RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=1 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0 || \
> >> >>>>  // RUN: %env_tool_opts=handle_segv=0:allow_user_segv_handler=1 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK0
> >> >>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=1 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK1 || \
> >> >>>> +// RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=1 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK1 || \
> >> >>>>  // RUN: %env_tool_opts=handle_segv=1:allow_user_segv_handler=1 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK1
> >> >>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=1 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
> >> >>>> +// RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=1 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2 || \
> >> >>>>  // RUN: %env_tool_opts=handle_segv=2:allow_user_segv_handler=1 not
> >> >>>> %run
> >> >>>> %t 2>&1 | FileCheck %s --check-prefix=CHECK2
> >> >>>>  // clang-format on
> >> >>>>
> >> >>>> @@ -21,9 +39,6 @@
> >> >>>>  // XFAIL: msan
> >> >>>>  // XFAIL: tsan
> >> >>>>
> >> >>>> -// Flaky errors in debuggerd with "waitpid returned unexpected pid
> >> >>>> (0)"
> >> >>>> in logcat.
> >> >>>> -// UNSUPPORTED: android
> >> >>>> -
> >> >>>>  #include <signal.h>
> >> >>>>  #include <stdio.h>
> >> >>>>  #include <stdlib.h>
> >> >>>>
> >> >>>>
> >> >>>> _______________________________________________
> >> >>>> llvm-commits mailing list
> >> >>>> llvm-commits at lists.llvm.org
> >> >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
> >> >>>
> >> >>>
> >> >>> _______________________________________________
> >> >>> llvm-commits mailing list
> >> >>> llvm-commits at lists.llvm.org
> >> >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
> >> >>>
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> llvm-commits mailing list
> >> >> llvm-commits at lists.llvm.org
> >> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
> >> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > llvm-commits mailing list
> >> > llvm-commits at lists.llvm.org
> >> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
> >> >
> >> _______________________________________________
> >> llvm-commits mailing list
> >> llvm-commits at lists.llvm.org
> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20171017/e807128c/attachment.html>