[LLVMdev] Building sanitizers for Android

Alexey Samsonov samsonov at google.com
Thu Apr 3 07:31:59 PDT 2014 

On Thu, Apr 3, 2014 at 5:04 AM, Greg Fitzgerald <garious at gmail.com> wrote:

> > we would still want to use compiler-rt test-suite in a standalone mode,
> to test fully built/installed toolchains,
> and even GCC.
>
> Sounds good.
>
>
> > Clang driver links the static xsan runtimes from a hardcoded
> > paths in Clang resource directory, and doesn't add flags like
> > "-lasan -L/path/to/clang/resource/dir". I find this behavior reasonable.
>
> Can we change the flags to "-lasan -L/path/to/clang/resource/dir"?  If
> we make that change,


I don't think it's a good idea to let user hijack the driver and stuff in
custom version of
ASan runtime instead the one installed/built with compiler :)


> I can stop suggesting that the clang build should
> invoke the compiler-rt test suite directly.  :-)
>

I haven't yet fully understood your complaints. Note that if you
build/configure compiler-rt
in a standalone mode, then "make check-compiler-rt" in that tree wouldn't
rebuild runtimes,
and instead assume that they are provided by toolchain (that is, in
standalone mode runtimes and
test suites are effectively independent). What do you mean by "invoking the
test-suite directly"?


>
>
> > Do you want to give it a try?
>
> I'll take a crack it, sure.
>
>
> By the way, locally, I now have just over half the ASan test suite
> passing ARM-Linux via QEMU.  It's been really convenient to build
> compiler-rt in standalone mode.  The edit-compile-run cycle is much
> shorter this way.  Thanks for your help in making this work!
>
> -Greg
>
>
>
> On Wed, Apr 2, 2014 at 7:56 AM, Alexey Samsonov <samsonov at google.com>
> wrote:
> > Hi Greg,
> >
> > On Wed, Apr 2, 2014 at 2:50 AM, Greg Fitzgerald <garious at gmail.com>
> wrote:
> >>
> >> Alexey, Evgeniy,
> >>
> >> I propose the following steps to unify multi-arch support in
> compiler-rt:
> >>
> >> 1) The compiler-rt test suite adds "-L${CMAKE_CURRENT_BINARY_DIR}/lib"
> >> to its 'clang' variables.  This way we can test the sanitizers without
> >> installing any libs to the just-built-clang install directory.
> >
> >
> > This is possible, but please keep in mind that:
> > 1) we would still want to use compiler-rt test-suite in a standalone
> mode,
> > to test fully built/installed toolchains,
> > and even GCC.
> > 2) Adding -L... wouldn't work: Clang driver links the static xsan
> runtimes
> > from a hardcoded paths in Clang resource directory,
> > and doesn't add flags like "-lasan -L/path/to/clang/resource/dir". I find
> > this behavior reasonable.
> >
> > I don't see a problem with the current approach - we can make "run
> sanitizer
> > test suite" command in the
> > top-level build tree depend on "build/install compiler-rt
> ExternalProject"
> > steps (see how it's done currently).
> >
> >>
> >>
> >> 2) The "clang/runtime" build calls ExternalProject once for each arch
> >> it needs of compiler-rt.  So once to create x86_64 libs and once for
> >> i386 libs.
> >>
> >> 3) The compiler-rt build drops the ${arch} suffix from its libs, and
> >> the "clang/runtime" build uses CMAKE_INSTALL_PREFIX to control where
> >> the compiler-rt libs go.
> >>
> >>
> "CMAKE_INSTALL_PREFIX=${CMAKE_CURRENT_BINARY_DIR}/lib/clang/${CLANG_VERSION}/x86_64-linux"
> >>
> >>
> >> 4) Remove multi-arch support from the compiler-rt build.  Instead,
> >> declare compiler-rt as an "any-arch" build, configured with:
> >>     CMAKE_CXX_COMPILER (defaults to just-built-clang)
> >>     CMAKE_CXX_FLAGS (defaults to llvm's default target)
> >>     COMPILER_RT_TEST_COMPILER (defaults to CMAKE_CXX_COMPILER)
> >>     COMPILER_RT_TEST_FLAGS (defaults to CMAKE_CXX_FLAGS)
> >
> >
> > Unfortunately, this could be problematic for Mac - we use "universal
> > binaries" there - a single static/dynamic runtime
> > which contains runtimes for all the possible architectures. We might work
> > around this, however, by adding a custom
> > Mac-specific step that would merge all single-arch binaries into a
> > multi-arch one - I think this is fine as long as we
> > greatly simplify the rest of the ugly build system.
> >
> > Do you want to give it a try?
> >
> >>
> >>
> >> Thoughts?
> >>
> >> Thanks,
> >> Greg
> >>
> >> On Tue, Apr 1, 2014 at 12:58 AM, Evgeniy Stepanov
> >> <eugeni.stepanov at gmail.com> wrote:
> >> > It does sound like Android is better suited for "honest"
> >> > cross-compilation, rather than "build compiler-rt for all targets we
> >> > can find" model.
> >> >
> >> > I'm still not convinced that we must require the "ninja install" step.
> >> > Could we just "ninja clang" and then build the second stage against
> >> > the first stage build directory? Will this "find_package" thing not
> >> > work that way, or do you mean it as a way to copy target asan runtime
> >> > where the first stage compiler will find it (or probably both)?
> >> >
> >> > On Mon, Mar 31, 2014 at 6:46 PM, Alexey Samsonov <samsonov at google.com
> >
> >> > wrote:
> >> >>
> >> >> On Fri, Mar 28, 2014 at 10:56 PM, Greg Fitzgerald <garious at gmail.com
> >
> >> >> wrote:
> >> >>>
> >> >>> > Note that ASan tests on Android require llvm-symbolizer binary.
> >> >>>
> >> >>> That's a really good point.  And I see that llvm-symbolizer can't
> just
> >> >>> be pulled into compiler-rt because it has dependencies on DebugInfo,
> >> >>> Object, and Support libraries.
> >> >>>
> >> >>>
> >> >>> This throws a big wrench in Alexey's plan to have the native
> >> >>> compiler-rt build generate the cross-compiled binaries for all
> >> >>> supported targets.  We would need to do the same in the llvm repo.
> >> >>> The alternative is to run a separate cross-compiled build for each
> >> >>> architecture.  :-/
> >> >>
> >> >>
> >> >> Yes, this llvm-symbolizer part is frustrating... I though we can get
> >> >> away
> >> >> with it at first, as llvm-symbolizer is optional, and we generally
> >> >> won't be
> >> >> able
> >> >> to produce working *binaries* for architectures different from the
> host
> >> >> one
> >> >> (we might not have a linker, for instance).
> >> >> Android with its NDK looks like an exception, though.
> >> >>
> >> >>>
> >> >>>
> >> >>>
> >> >>> > Yes, so "ninja install" is optional.
> >> >>>
> >> >>> But the 'install' depends on the 'all' target.
> >> >>>
> >> >>> -Greg
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> On Fri, Mar 28, 2014 at 12:39 AM, Evgeniy Stepanov
> >> >>> <eugeni.stepanov at gmail.com> wrote:
> >> >>> > On Thu, Mar 27, 2014 at 10:18 PM, Greg Fitzgerald
> >> >>> > <garious at gmail.com>
> >> >>> > wrote:
> >> >>> >>> Alexey's approach with CMake sub-projects.
> >> >>> >>
> >> >>> >> I prefer that direction as well, but what I've proposed is a
> >> >>> >> solution
> >> >>> >> that works today.  To support cross-compilation, we'll need to
> loop
> >> >>> >> over each supported arch (llvm-config --targets-built), then loop
> >> >>> >> over
> >> >>> >> each supported triple for each arch (hard-coded map?), and then
> >> >>> >> pair
> >> >>> >> up each triple with a sysroot (system paths provided by the
> user).
> >> >>> >>
> >> >>> >>
> >> >>> >>> I think it already copies
> >> >>> >>> compiler-rt build products to the parent build directory, right?
> >> >>> >>
> >> >>> >> The Android compiler-rt build does this, but in an unnecessarily
> >> >>> >> complex way.  My proposal is a simplification of that process.
> >> >>> >> What
> >> >>> >> is described in "llvm/cmake/platforms/Android.cmake" is to build
> >> >>> >> llvm+clang for the host architecture and then run a second build
> to
> >> >>> >> cross-compile llvm+compiler-rt to get a the ASan shared object
> and
> >> >>> >> test suite.  Instead, I propose building only llvm for the host
> >> >>> >> architecture (for the purpose of building llvm-config) and then
> >> >>> >> only
> >> >>> >> compile compiler-rt for Android.  No need to build clang or to
> >> >>> >> cross-compile llvm.
> >> >>> >>
> >> >>> >>
> >> >>> >>> Requiring that llvm is installed complicates things.
> >> >>> >>
> >> >>> >> Sorry for the confusion.  By "llvm install directory", I don't
> mean
> >> >>> >> to
> >> >>> >> install llvm to the root directory.  I'm referring to the
> >> >>> >> intermediary
> >> >>> >> directory pointed to by CMAKE_INSTALL_PREFIX.  When you run
> "ninja
> >> >>> >> install", the build populates this directory with only the libs,
> >> >>> >> headers, docs, and executables that the LLVM build intends
> >> >>> >> customers
> >> >>> >> to use directly.  In the case of compiler-rt, we need LLVM's
> >> >>> >> "bin/llvm-config" to determine the build mode, flags, and targets
> >> >>> >> available.
> >> >>> >
> >> >>> > Yes, so "ninja install" is optional. This should work with LLVM
> >> >>> > build
> >> >>> > directory just as well.
> >> >>> >
> >> >>> >>
> >> >>> >>
> >> >>> >>> This should not be too hard to fix.
> >> >>> >>
> >> >>> >> Porting the LLVM build to Android?  That seems like a more
> >> >>> >> challenging
> >> >>> >> solution than to cut out the unused cross-compiled llvm build.
> >> >>> >
> >> >>> > Note that ASan tests on Android require llvm-symbolizer binary.
> >> >>> > Building just compiler-rt for the target would not be enough.
> >> >>> >
> >> >>> >>
> >> >>> >> -Greg
> >> >>> >>
> >> >>> >> On Thu, Mar 27, 2014 at 1:14 AM, Evgeniy Stepanov
> >> >>> >> <eugeni.stepanov at gmail.com> wrote:
> >> >>> >>> I'd prefer something based on Alexey's approach with CMake
> >> >>> >>> sub-projects. I think it already copies compiler-rt build
> products
> >> >>> >>> to
> >> >>> >>> the parent build directory, right?
> >> >>> >>>
> >> >>> >>> Requiring that llvm is installed complicates things.
> >> >>> >>>
> >> >>> >>> On Thu, Mar 27, 2014 at 5:00 AM, Greg Fitzgerald
> >> >>> >>> <garious at gmail.com>
> >> >>> >>> wrote:
> >> >>> >>>> The build for the Android sanitizers is unique in that it needs
> >> >>> >>>> to
> >> >>> >>>> link against the Android system libraries to create a shared
> >> >>> >>>> object
> >> >>> >>>> and its test suites.  The current solution to build ASan is to
> >> >>> >>>> drop
> >> >>> >>>> the compiler-rt repo into the llvm source tree and
> cross-compile
> >> >>> >>>> the
> >> >>> >>>> llvm build for Android.  This is a bit awkward for few reasons:
> >> >>> >>>>
> >> >>> >>>> 1) Not all of llvm can be cross-compiled for Android.  "ninja
> >> >>> >>>> all"
> >> >>> >>>> causes expected build errors.
> >> >>> >>>
> >> >>> >>> This should not be too hard to fix.
> >> >>> >>>
> >> >>> >>>> 2) The sanitizers don't depend on any LLVM libraries, only
> >> >>> >>>> llvm-config
> >> >>> >>>> for its build configuration.
> >> >>> >>>> 3) No "install" rule.  Instead, you cherry-pick files from the
> >> >>> >>>> build
> >> >>> >>>> directory.
> >> >>> >>>>
> >> >>> >>>> Building against the LLVM install directory cleans this up
> nicely
> >> >>> >>>> and
> >> >>> >>>> as it turns out, this mostly works today (see CMake
> configuration
> >> >>> >>>> below).  The only missing pieces are that the shared object is
> >> >>> >>>> not
> >> >>> >>>> added to the install directory and the test suites are not
> built.
> >> >>> >>>> Is
> >> >>> >>>> this a build configuration you'd consider using?
> >> >>> >>>>
> >> >>> >>>> $ cmake -G Ninja .. \
> >> >>> >>>>     -DCMAKE_INSTALL_PREFIX=ship \
> >> >>> >>>>     -DCMAKE_C_COMPILER=arm-linux-androideabi-gcc \
> >> >>> >>>>     -DCMAKE_CXX_COMPILER=arm-linux-androideabi-g++ \
> >> >>> >>>>     -DCMAKE_PREFIX_PATH=`pwd`/../../llvm/out/ship \
> >> >>> >>>>     -DANDROID=1 \
> >> >>> >>>>     -DCMAKE_SYSTEM_NAME=Linux \
> >> >>> >>>>
> >> >>> >>>>
> -DCMAKE_C_FLAGS=--sysroot=$(ndkDir)/platforms/android-19/arch-arm
> >> >>> >>>> \
> >> >>> >>>>
> >> >>> >>>>
> >> >>> >>>>
> -DCMAKE_CXX_FLAGS=--sysroot=$(ndkDir)/platforms/android-19/arch-arm
> >> >>> >>>> ...
> >> >>> >>>> $ ninja install
> >> >>> >>>> ...
> >> >>> >>>> [68/68]
> >> >>> >>>> -- Install configuration: "Release"
> >> >>> >>>> -- Installing: ship/include/sanitizer/asan_interface.h
> >> >>> >>>> -- Installing: ship/include/sanitizer/common_interface_defs.h
> >> >>> >>>> -- Installing: ship/include/sanitizer/dfsan_interface.h
> >> >>> >>>> -- Installing: ship/include/sanitizer/linux_syscall_hooks.h
> >> >>> >>>> -- Installing: ship/include/sanitizer/lsan_interface.h
> >> >>> >>>> -- Installing: ship/include/sanitizer/msan_interface.h
> >> >>> >>>> -- Installing: ship/include/sanitizer/tsan_interface_atomic.h
> >> >>> >>>> -- Installing: ship/asan_blacklist.txt
> >> >>> >>>> -- Installing: ship/bin/asan_device_setup
> >> >>> >>>>
> >> >>> >>>> Thanks,
> >> >>> >>>> Greg
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Alexey Samsonov, MSK
> >
> >
> >
> >
> > --
> > Alexey Samsonov, MSK
>



-- 
Alexey Samsonov, MSK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140403/cb1d86e9/attachment.html>

More information about the llvm-dev mailing list