[llvm] r292862 - [sanitizer-coverage] emit __sanitizer_cov_trace_pc_guard w/o a preceding 'if' by default. Update the docs, also add deprecation notes around other parts of sanitizer coverage

Tue Jan 24 15:50:34 PST 2017

So, we've lost over a day of build bot coverage on PPC.

Kostya, before detailed investigation, can you at least let the bots return
to green? I don't really care between revert and disable on PPC. I just
want these bots to go green:
http://lab.llvm.org:8011/builders/clang-ppc64be-linux-lnt
http://lab.llvm.org:8011/builders/clang-ppc64le-linux-lnt
http://lab.llvm.org:8011/builders/clang-ppc64be-linux-multistage
http://lab.llvm.org:8011/builders/clang-ppc64le-linux-multistage
http://lab.llvm.org:8011/builders/clang-ppc64be-linux
http://lab.llvm.org:8011/builders/clang-ppc64le-linux

On Tue, Jan 24, 2017 at 3:48 PM Kostya Serebryany via llvm-commits <
llvm-commits at lists.llvm.org> wrote:

> turns out one of the x86 bots is failing too (gcc build fails, clang build
> passes).
>
> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/721/steps/check-asan%20in%20gcc%20build/logs/stdio
> This gives me a way to reproduce, looking.
>
>
> On Tue, Jan 24, 2017 at 1:43 PM, Eric Christopher <echristo at gmail.com>
> wrote:
>
>
>
> On Tue, Jan 24, 2017 at 1:37 PM Kostya Serebryany <kcc at google.com> wrote:
>
> On Tue, Jan 24, 2017 at 1:34 PM, Eric Christopher <echristo at gmail.com>
> wrote:
>
>
>
> On Tue, Jan 24, 2017 at 1:23 PM Kostya Serebryany <kcc at google.com> wrote:
>
> On Tue, Jan 24, 2017 at 1:17 PM, Eric Christopher <echristo at gmail.com>
> wrote:
>
>
>
> On Tue, Jan 24, 2017 at 11:49 AM Kostya Serebryany <kcc at google.com> wrote:
>
> On Tue, Jan 24, 2017 at 11:45 AM, Eric Christopher <echristo at gmail.com>
> wrote:
>
> This seems problematic in general - we have a bot that is failing due to a
> particular test. Is this patch necessary for correctness somewhere else?
>
>
> Correctness -- no. Performance -- yes, and in some cases very significant.
>
>
> OK. Thanks.
>
>
>
>
> Can it be reverted until we can figure out what's going on?
>
>
> Sure we can revert it, but who is going to analyze the problem and how
> soon?
>
>
> I'm hoping someone at IBM can take a look at it. I've got limited ability
> to debug it myself at the moment.
>
> Kit: Would you mind terribly?
> Kostya: It might be nice to revert in the mean time if you don't mind?
>
>
> I do mind, sorry.
> Again, this functionality is not used by anyone on PPC (afaict) but is
> important for libFuzzer users on x86 side.
>
>
> I believe this could be causing problems not just in libFuzzer and so
> affect anyone trying to use the sanitizer on ppc at all?
>
>
> I doubt so. This change affects sanitizer coverage (
> https://clang.llvm.org/docs/SanitizerCoverage.html),
> which is usually used together with *san and libFuzzer.
>
>
> OK. Thanks for the clarification.
>
> In general, we have a revert first policy and so this should probably have
> fallen under there without letting people know what the plan was for making
> sure the bots come back - it's important that we continue to get bot
> testing for asan in general.
>
> Thanks!
>
> -eric
>
>
>
>
>
> If not, can you explain the scope of this a bit better - especially as
> it's now a flaky failure?
>
> -eric
>
>
>
>
>
> Thanks!
>
> -eric
>
>
> Note that the functionality tested in this test is not the core asan, but
> the coverage instrumentation used by libFuzzer,
> and afaict there are no users of libFuzzer on PPC, so we can just disable
> the test on PPC and no users will be harmed.
>
>
>
> -eric
>
> On Tue, Jan 24, 2017 at 11:29 AM Kostya Serebryany via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
>
> Hi Bill,
>
> thanks for the report.
> Unfortunately, I don't have anyone working on the PPC side at the moment
> and the test passes on x86.
> So, the only thing I can propose now is to disable this test on PPC.
> (Maybe it's not that bad, I was going to kill at least some of the related
> code this year).
>
> --kcc
>
> On Tue, Jan 24, 2017 at 8:44 AM, Bill Seurer <seurer at linux.vnet.ibm.com>
> wrote:
>
> This broke the powerpc sanitizers.
>
> updatellvm 292861
> ninja
> ninja check-asan
> Testing Time: 29.65s
>   Expected Passes    : 457
>   Expected Failures  : 1
>   Unsupported Tests  : 157
>
> updatellvm 292862
> ninja
> ninja check-asan
>
> FAIL: AddressSanitizer-powerpc64le-linux ::
> TestCases/Posix/coverage-direct.cc (528 of 615)
> ******************** TEST 'AddressSanitizer-powerpc64le-linux ::
> TestCases/Posix/coverage-direct.cc' FAILED ********************
> Script:
> --
> /home/seurer/llvm/build/llvm-test2/./bin/clang --driver-mode=g++
> -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer
> -fno-optimize-sibling-calls -gline-tables-only -m64
> -fsanitize-coverage=func -DSHARED
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/test/asan/TestCases/Posix/coverage-direct.cc
> -shared -o
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/lib$(basename
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp).dynamic.so
> -fPIC
> /home/seurer/llvm/build/llvm-test2/./bin/clang --driver-mode=g++
> -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer
> -fno-optimize-sibling-calls -gline-tables-only -m64
> -fsanitize-coverage=func
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/test/asan/TestCases/Posix/coverage-direct.cc
> -ldl -o
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp
> rm -rf
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct
> mkdir -p
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal
> env
> ASAN_OPTIONS=coverage=1:coverage_direct=0:coverage_dir=/home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal:verbosity=1
>
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/lib$(basename
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp).dynamic.so
> /usr/bin/python2.7
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/lib/sanitizer_common/scripts/sancov.py
> print
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal/*.sancov
> >/home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal/out.txt
> mkdir -p
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/direct
> env
> ASAN_OPTIONS=coverage=1:coverage_direct=1:coverage_dir=/home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/direct:verbosity=1
>
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/lib$(basename
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp).dynamic.so
> cd
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/direct
> /usr/bin/python2.7
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/lib/sanitizer_common/scripts/sancov.py
> rawunpack *.sancov.raw
> /usr/bin/python2.7
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/lib/sanitizer_common/scripts/sancov.py
> print *.sancov >out.txt
> cd ../..
> diff -u coverage-direct/normal/out.txt coverage-direct/direct/out.txt
> /home/seurer/llvm/build/llvm-test2/./bin/clang --driver-mode=g++
> -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer
> -fno-optimize-sibling-calls -gline-tables-only -m64 -fsanitize-coverage=bb
> -DSHARED
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/test/asan/TestCases/Posix/coverage-direct.cc
> -shared -o
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/lib$(basename
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp).dynamic.so
> -fPIC
> /home/seurer/llvm/build/llvm-test2/./bin/clang --driver-mode=g++
> -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer
> -fno-optimize-sibling-calls -gline-tables-only -m64 -fsanitize-coverage=bb
> -DSO_DIR=\"/home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output\"
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/test/asan/TestCases/Posix/coverage-direct.cc
> -ldl -o
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp
> rm -rf
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct
> mkdir -p
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal
> env
> ASAN_OPTIONS=coverage=1:coverage_direct=0:coverage_dir=/home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal:verbosity=1
>
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/lib$(basename
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp).dynamic.so
> /usr/bin/python2.7
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/lib/sanitizer_common/scripts/sancov.py
> print
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal/*.sancov
> >/home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal/out.txt
> mkdir -p
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/direct
> env
> ASAN_OPTIONS=coverage=1:coverage_direct=1:coverage_dir=/home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/direct:verbosity=1
>
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/lib$(basename
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp).dynamic.so
> cd
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/direct
> /usr/bin/python2.7
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/lib/sanitizer_common/scripts/sancov.py
> rawunpack *.sancov.raw
> /usr/bin/python2.7
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/lib/sanitizer_common/scripts/sancov.py
> print *.sancov >out.txt
> cd ../..
> diff -u coverage-direct/normal/out.txt coverage-direct/direct/out.txt
> /home/seurer/llvm/build/llvm-test2/./bin/clang --driver-mode=g++
> -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer
> -fno-optimize-sibling-calls -gline-tables-only -m64
> -fsanitize-coverage=edge -DSHARED
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/test/asan/TestCases/Posix/coverage-direct.cc
> -shared -o
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/lib$(basename
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp).dynamic.so
> -fPIC
> /home/seurer/llvm/build/llvm-test2/./bin/clang --driver-mode=g++
> -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer
> -fno-optimize-sibling-calls -gline-tables-only -m64
> -fsanitize-coverage=edge
> -DSO_DIR=\"/home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output\"
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/test/asan/TestCases/Posix/coverage-direct.cc
> -ldl -o
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp
> rm -rf
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct
> mkdir -p
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal
> env
> ASAN_OPTIONS=coverage=1:coverage_direct=0:coverage_dir=/home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal:verbosity=1
>
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/lib$(basename
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp).dynamic.so
> /usr/bin/python2.7
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/lib/sanitizer_common/scripts/sancov.py
> print
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal/*.sancov
> >/home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal/out.txt
> mkdir -p
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/direct
> env
> ASAN_OPTIONS=coverage=1:coverage_direct=1:coverage_dir=/home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/direct:verbosity=1
>
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/lib$(basename
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct.cc.tmp).dynamic.so
> cd
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/direct
> /usr/bin/python2.7
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/lib/sanitizer_common/scripts/sancov.py
> rawunpack *.sancov.raw
> /usr/bin/python2.7
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/lib/sanitizer_common/scripts/sancov.py
> print *.sancov >out.txt
> cd ../..
> diff -u coverage-direct/normal/out.txt coverage-direct/direct/out.txt
> --
> Exit Code: 1
>
> Command Output (stdout):
> --
> bar
> bar
> bar
> bar
> --- coverage-direct/normal/out.txt      2017-01-24 10:20:41.258917000 -0600
> +++ coverage-direct/direct/out.txt      2017-01-24 10:20:41.638917000 -0600
> @@ -1 +1,2 @@
>  0x10128bb0
> +0x10128bb0
>
> --
> Command Output (stderr):
> --
> ==18701==AddressSanitizer: failed to intercept '__isoc99_printf'
> ==18701==AddressSanitizer: failed to intercept '__isoc99_sprintf'
> ==18701==AddressSanitizer: failed to intercept '__isoc99_snprintf'
> ==18701==AddressSanitizer: failed to intercept '__isoc99_fprintf'
> ==18701==AddressSanitizer: failed to intercept '__isoc99_vprintf'
> ==18701==AddressSanitizer: failed to intercept '__isoc99_vsprintf'
> ==18701==AddressSanitizer: failed to intercept '__isoc99_vsnprintf'
> ==18701==AddressSanitizer: failed to intercept '__isoc99_vfprintf'
> ==18701==AddressSanitizer: libc interceptors initialized
> || `[0x0a0000000000, 0x3fffffffffff]` || HighMem    ||
> || `[0x034000000000, 0x09ffffffffff]` || HighShadow ||
> || `[0x024000000000, 0x033fffffffff]` || ShadowGap  ||
> || `[0x020000000000, 0x023fffffffff]` || LowShadow  ||
> || `[0x000000000000, 0x01ffffffffff]` || LowMem     ||
> MemToShadow(shadow): 0x024000000000 0x0247ffffffff 0x026800000000
> 0x033fffffffff
> redzone=16
> max_redzone=2048
> quarantine_size_mb=256M
> thread_local_quarantine_size_kb=1024K
> malloc_context_size=30
> SHADOW_SCALE: 3
> SHADOW_GRANULARITY: 8
> SHADOW_OFFSET: 0x20000000000
> ==18701==Installed the sigaction for signal 11
> ==18701==Installed the sigaction for signal 7
> ==18701==Installed the sigaction for signal 8
> ==18701==T0: stack [0x3fffdd8c0000,0x3fffde0c0000) size 0x800000;
> local=0x3fffde0ba450
> ==18701==AddressSanitizer Init done
> PID: 18701
> ==18701== CovDump:
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal/coverage-direct.cc.tmp.18701.sancov:
> 1 PCs written
> ==18701== CovDump:
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal/libcoverage-
> direct.cc.tmp.dynamic.so.18701.sancov: 1 PCs written
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/lib/sanitizer_common/scripts/sancov.py:
> read 1 64-bit PCs from
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal/coverage-direct.cc.tmp.18701.sancov
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/lib/sanitizer_common/scripts/sancov.py:
> read 1 64-bit PCs from
> /home/seurer/llvm/build/llvm-test2/projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Posix/Output/coverage-direct/normal/libcoverage-
> direct.cc.tmp.dynamic.so.18701.sancov
> /home/seurer/llvm/llvm-test2/projects/compiler-rt/lib/sanitizer_common/scripts/sancov.py:
> 2 files merged; 2 PCs total
> ==19909==AddressSanitizer: failed to intercept '__isoc99_printf'
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170124/294fdde4/attachment-0001.html>