[LLVMdev] Building sanitizers for Android

[Alexey Samsonov]

On Fri, Mar 28, 2014 at 10:56 PM, Greg Fitzgerald <garious at gmail.com> wrote:

> > Note that ASan tests on Android require llvm-symbolizer binary.
>
> That's a really good point.  And I see that llvm-symbolizer can't just
> be pulled into compiler-rt because it has dependencies on DebugInfo,
> Object, and Support libraries.


> This throws a big wrench in Alexey's plan to have the native
> compiler-rt build generate the cross-compiled binaries for all
> supported targets.  We would need to do the same in the llvm repo.
> The alternative is to run a separate cross-compiled build for each
> architecture.  :-/
>

Yes, this llvm-symbolizer part is frustrating... I though we can get away
with it at first, as llvm-symbolizer is optional, and we generally won't be
able
to produce working *binaries* for architectures different from the host one
(we might not have a linker, for instance).
Android with its NDK looks like an exception, though.


>
>
> > Yes, so "ninja install" is optional.
>
> But the 'install' depends on the 'all' target.
>
> -Greg
>
>
>
>
>
>
> On Fri, Mar 28, 2014 at 12:39 AM, Evgeniy Stepanov
> <eugeni.stepanov at gmail.com> wrote:
> > On Thu, Mar 27, 2014 at 10:18 PM, Greg Fitzgerald <garious at gmail.com>
> wrote:
> >>> Alexey's approach with CMake sub-projects.
> >>
> >> I prefer that direction as well, but what I've proposed is a solution
> >> that works today.  To support cross-compilation, we'll need to loop
> >> over each supported arch (llvm-config --targets-built), then loop over
> >> each supported triple for each arch (hard-coded map?), and then pair
> >> up each triple with a sysroot (system paths provided by the user).
> >>
> >>
> >>> I think it already copies
> >>> compiler-rt build products to the parent build directory, right?
> >>
> >> The Android compiler-rt build does this, but in an unnecessarily
> >> complex way.  My proposal is a simplification of that process.  What
> >> is described in "llvm/cmake/platforms/Android.cmake" is to build
> >> llvm+clang for the host architecture and then run a second build to
> >> cross-compile llvm+compiler-rt to get a the ASan shared object and
> >> test suite.  Instead, I propose building only llvm for the host
> >> architecture (for the purpose of building llvm-config) and then only
> >> compile compiler-rt for Android.  No need to build clang or to
> >> cross-compile llvm.
> >>
> >>
> >>> Requiring that llvm is installed complicates things.
> >>
> >> Sorry for the confusion.  By "llvm install directory", I don't mean to
> >> install llvm to the root directory.  I'm referring to the intermediary
> >> directory pointed to by CMAKE_INSTALL_PREFIX.  When you run "ninja
> >> install", the build populates this directory with only the libs,
> >> headers, docs, and executables that the LLVM build intends customers
> >> to use directly.  In the case of compiler-rt, we need LLVM's
> >> "bin/llvm-config" to determine the build mode, flags, and targets
> >> available.
> >
> > Yes, so "ninja install" is optional. This should work with LLVM build
> > directory just as well.
> >
> >>
> >>
> >>> This should not be too hard to fix.
> >>
> >> Porting the LLVM build to Android?  That seems like a more challenging
> >> solution than to cut out the unused cross-compiled llvm build.
> >
> > Note that ASan tests on Android require llvm-symbolizer binary.
> > Building just compiler-rt for the target would not be enough.
> >
> >>
> >> -Greg
> >>
> >> On Thu, Mar 27, 2014 at 1:14 AM, Evgeniy Stepanov
> >> <eugeni.stepanov at gmail.com> wrote:
> >>> I'd prefer something based on Alexey's approach with CMake
> >>> sub-projects. I think it already copies compiler-rt build products to
> >>> the parent build directory, right?
> >>>
> >>> Requiring that llvm is installed complicates things.
> >>>
> >>> On Thu, Mar 27, 2014 at 5:00 AM, Greg Fitzgerald <garious at gmail.com>
> wrote:
> >>>> The build for the Android sanitizers is unique in that it needs to
> >>>> link against the Android system libraries to create a shared object
> >>>> and its test suites.  The current solution to build ASan is to drop
> >>>> the compiler-rt repo into the llvm source tree and cross-compile the
> >>>> llvm build for Android.  This is a bit awkward for few reasons:
> >>>>
> >>>> 1) Not all of llvm can be cross-compiled for Android.  "ninja all"
> >>>> causes expected build errors.
> >>>
> >>> This should not be too hard to fix.
> >>>
> >>>> 2) The sanitizers don't depend on any LLVM libraries, only llvm-config
> >>>> for its build configuration.
> >>>> 3) No "install" rule.  Instead, you cherry-pick files from the build
> directory.
> >>>>
> >>>> Building against the LLVM install directory cleans this up nicely and
> >>>> as it turns out, this mostly works today (see CMake configuration
> >>>> below).  The only missing pieces are that the shared object is not
> >>>> added to the install directory and the test suites are not built.  Is
> >>>> this a build configuration you'd consider using?
> >>>>
> >>>> $ cmake -G Ninja .. \
> >>>>     -DCMAKE_INSTALL_PREFIX=ship \
> >>>>     -DCMAKE_C_COMPILER=arm-linux-androideabi-gcc \
> >>>>     -DCMAKE_CXX_COMPILER=arm-linux-androideabi-g++ \
> >>>>     -DCMAKE_PREFIX_PATH=`pwd`/../../llvm/out/ship \
> >>>>     -DANDROID=1 \
> >>>>     -DCMAKE_SYSTEM_NAME=Linux \
> >>>>     -DCMAKE_C_FLAGS=--sysroot=$(ndkDir)/platforms/android-19/arch-arm
> \
> >>>>
> -DCMAKE_CXX_FLAGS=--sysroot=$(ndkDir)/platforms/android-19/arch-arm
> >>>> ...
> >>>> $ ninja install
> >>>> ...
> >>>> [68/68]
> >>>> -- Install configuration: "Release"
> >>>> -- Installing: ship/include/sanitizer/asan_interface.h
> >>>> -- Installing: ship/include/sanitizer/common_interface_defs.h
> >>>> -- Installing: ship/include/sanitizer/dfsan_interface.h
> >>>> -- Installing: ship/include/sanitizer/linux_syscall_hooks.h
> >>>> -- Installing: ship/include/sanitizer/lsan_interface.h
> >>>> -- Installing: ship/include/sanitizer/msan_interface.h
> >>>> -- Installing: ship/include/sanitizer/tsan_interface_atomic.h
> >>>> -- Installing: ship/asan_blacklist.txt
> >>>> -- Installing: ship/bin/asan_device_setup
> >>>>
> >>>> Thanks,
> >>>> Greg
>



-- 
Alexey Samsonov, MSK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140331/4d07341e/attachment.html>