[llvm-dev] Building/Running LLVM Tests with Sanitizers

Fri Aug 31 12:34:13 PDT 2018

We already do this for Fuchsia. We've added support for building sanitized
runtimes (
https://github.com/llvm-mirror/llvm/blob/master/runtimes/CMakeLists.txt#L501)
and we use it to build ASan instrumented libc++/libc++abi/libunwind (
https://github.com/llvm-mirror/clang/blob/master/cmake/caches/Fuchsia-stage2.cmake#L115)
which end up in lib/clang/.../$triple/asan (if using the multiarch runtime
layout which is the default for us). We hope to use other sanitizers in the
future as well.

Since the instrumented and uninstrumented libc++/libc++abi/libunwind should
have the same ABI, there should be no need to change which library you link
against, the only thing  that's important is which library you load at
runtime. On Fuchsia this is handled by our dynamic linker. On Linux we'll
likely need to use -rpath or some other solution.

On Fri, Aug 31, 2018 at 11:30 AM Evgenii Stepanov via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> This would be nice.
> So, to be clear, there would be libc++.so / libc++.a somewhere under
> lib/clang/.../$triple/msan, and the driver would link that if libc++
> is not specified explicitly on the command line? Would it also add
> -rpath? This could be weird because toolchain may not be present when
> the binary is run.
>
> On Fri, Aug 31, 2018 at 11:14 AM, Vitaly Buka <vitalybuka at google.com>
> wrote:
> > +Evgenii Stepanov
> > sgtm
> >
> > On Fri, Aug 31, 2018 at 6:49 AM Brian Cain <brian.cain at gmail.com> wrote:
> >>
> >> Aside: would it be useful to execute a build of the libc++/libc++abi
> with
> >> msan normally during release, and change the driver to look for these
> >> msan-built C++ libs when "-fsanitize=memory"?  That would drastically
> cut
> >> down on the complexity of using msan.
> >>
> >> On Fri, Aug 31, 2018 at 5:43 AM Dean Michael Berris via llvm-dev
> >> <llvm-dev at lists.llvm.org> wrote:
> >>>
> >>> Thanks Vitaly and Peter,
> >>>
> >>> I went with a modified version of the instructions from
> >>>
> https://github.com/google/sanitizers/wiki/MemorySanitizerBootstrappingClang
> >>> — that’s a very helpful set of instructions!
> >>>
> >>> Cheers
> >>>
> >>> > On 31 Aug 2018, at 02:55, Vitaly Buka <vitalybuka at google.com> wrote:
> >>> >
> >>> > Another option is just to run corresponding script from
> >>> >
> https://llvm.org/svn/llvm-project/zorg/trunk/zorg/buildbot/builders/sanitizers/
> >>> > in empty directory.
> >>> >
> >>> > On Thu, Aug 30, 2018 at 5:00 AM Peter Smith via llvm-dev
> >>> > <llvm-dev at lists.llvm.org> wrote:
> >>> > Hello Dean,
> >>> >
> >>> > I've not done this myself for a while, I seem to remember it taking a
> >>> > while to get right. When I did I found the page:
> >>> >
> >>> >
> https://github.com/google/sanitizers/wiki/MemorySanitizerBootstrappingClang
> >>> > useful. The other thing that might work is following the individual
> >>> > cmake steps from
> >>> >
> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap-msan
> >>> > . These aren't in the same structure as the monorepo but they may be
> >>> > adaptable.
> >>> >
> >>> > My apologies if you've tried these already.
> >>> >
> >>> > Peter
> >>> >
> >>> > On 30 August 2018 at 12:42, Dean Michael Berris via llvm-dev
> >>> > <llvm-dev at lists.llvm.org> wrote:
> >>> > > Hi llvm-dev,
> >>> > >
> >>> > > I'm trying to reproduce an msan failure in one of the bots, but I
> >>> > > can't seem to get the right incantation of building LLVM with msan.
> >>> > > Here's what I've been doing:
> >>> > >
> >>> > > 1) Build the toolchain in one build directory, including
> >>> > > `compiler-rt`.
> >>> > >
> >>> > > 2) Build the toolchain again with the just built toolchain in step
> 1,
> >>> > > but this time with `-DLLVM_USE_SANITIZER=MemoryWithOrigins`.
> >>> > >
> >>> > > I get some msan errors, but of this kind instead of the one I'm
> >>> > > expecting:
> >>> > >
> >>> > > ====
> >>> > > $ ./unittests/XRay/XRayTests
> >>> > > ==236769==WARNING: MemorySanitizer: use-of-uninitialized-value
> >>> > >     #0 0x70eef8
> >>> > >
> >>> > >
> (/usr/local/google/home/dberris/xray/llvm-project-build-msan/unittests/XRay/XRayTests+0x70eef8)
> >>> > >     #1 0x70d2dc
> >>> > >
> >>> > >
> (/usr/local/google/home/dberris/xray/llvm-project-build-msan/unittests/XRay/XRayTests+0x70d2dc)
> >>> > >     #2 0x721044
> >>> > >
> >>> > >
> (/usr/local/google/home/dberris/xray/llvm-project-build-msan/unittests/XRay/XRayTests+0x721044)
> >>> > >     #3 0x6e2a37
> >>> > >
> >>> > >
> (/usr/local/google/home/dberris/xray/llvm-project-build-msan/unittests/XRay/XRayTests+0x6e2a37)
> >>> > >     #4 0x3722aa
> >>> > >
> >>> > >
> (/usr/local/google/home/dberris/xray/llvm-project-build-msan/unittests/XRay/XRayTests+0x3722aa)
> >>> > >     #5 0x372868
> >>> > >
> >>> > >
> (/usr/local/google/home/dberris/xray/llvm-project-build-msan/unittests/XRay/XRayTests+0x372868)
> >>> > >     #6 0x775a4c
> >>> > >
> >>> > >
> (/usr/local/google/home/dberris/xray/llvm-project-build-msan/unittests/XRay/XRayTests+0x775a4c)
> >>> > >     #7 0x7f85b6c7e23f  (/lib/x86_64-linux-gnu/libc.so.6+0x2023f)
> >>> > >     #8 0x2f4029
> >>> > >
> >>> > >
> (/usr/local/google/home/dberris/xray/llvm-project-build-msan/unittests/XRay/XRayTests+0x2f4029)
> >>> > >
> >>> > >   Uninitialized value was created by an allocation of 'ref.tmp' in
> >>> > > the
> >>> > > stack frame of function
> >>> > >
> >>> > >
> '_ZN7testing8internal12UnitTestImpl11AddTestInfoEPFvvES3_PNS_8TestInfoE'
> >>> > >     #0 0x720f10
> >>> > >
> >>> > >
> (/usr/local/google/home/dberris/xray/llvm-project-build-msan/unittests/XRay/XRayTests+0x720f10)
> >>> > >
> >>> > > SUMMARY: MemorySanitizer: use-of-uninitialized-value
> >>> > >
> >>> > >
> (/usr/local/google/home/dberris/xray/llvm-project-build-msan/unittests/XRay/XRayTests+0x70eef8)
> >>> > > Exiting
> >>> > >
> >>> > > ====
> >>> > >
> >>> > > Is there something else I'm missing here? I'm doing this on Linux.
> I
> >>> > > tried building with libc++ from the first toolchain, but somehow
> the
> >>> > > linker from the same toolchain in step 1 couldn't find the libc++
> >>> > > with
> >>> > > msan instrumentation.
> >>> > >
> >>> > > Here's the CMake command I'm using:
> >>> > >
> >>> > > ====
> >>> > > cmake -GNinja
> >>> > > -DCMAKE_C_COMPILER=$HOME/xray/llvm-project-build/bin/clang
> >>> > > -DCMAKE_CXX_COMPILER=$HOME/xray/llvm-project-build/bin/clang++
> >>> > > -DCMAKE_BUILD_TYPE=Debug -DLLVM_OPTIMIZED_TABLEGEN=On
> >>> > > -DLLVM_ENABLE_ASSERTIONS=On -DLLVM_USE_SANITIZER=MemoryWithOrigins
> >>> > > -DLLVM_ENABLE_LLD=On -DLLVM_INCLUDE_TESTS=On
> >>> > > -DCMAKE_CXX_FLAGS=-stdlib=libc++ -DCMAKE_LINK_FLAGS=-stdlib=libc++
> >>> > > ../llvm-project/llvm
> >>> > > ====
> >>> > >
> >>> > > And I'm running the tests like so:
> >>> > >
> >>> > > ====
> >>> > > ninja check-llvm
> >>> > > ====
> >>> > >
> >>> > > I'm doing this with the monorepo, if that's relevant.
> >>> > >
> >>> > > Help?
> >>> > >
> >>> > > --
> >>> > > Dean
> >>> > > _______________________________________________
> >>> > > LLVM Developers mailing list
> >>> > > llvm-dev at lists.llvm.org
> >>> > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>> > _______________________________________________
> >>> > LLVM Developers mailing list
> >>> > llvm-dev at lists.llvm.org
> >>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>>
> >>> -- Dean
> >>>
> >>> _______________________________________________
> >>> LLVM Developers mailing list
> >>> llvm-dev at lists.llvm.org
> >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>
> >>
> >>
> >> --
> >> -Brian
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180831/5feb8428/attachment.html>