[LLVMdev] Building sanitizers for Android

[Greg Fitzgerald]

> Note that ASan tests on Android require llvm-symbolizer binary.

That's a really good point.  And I see that llvm-symbolizer can't just
be pulled into compiler-rt because it has dependencies on DebugInfo,
Object, and Support libraries.

This throws a big wrench in Alexey's plan to have the native
compiler-rt build generate the cross-compiled binaries for all
supported targets.  We would need to do the same in the llvm repo.
The alternative is to run a separate cross-compiled build for each
architecture.  :-/


> Yes, so "ninja install" is optional.

But the 'install' depends on the 'all' target.

-Greg






On Fri, Mar 28, 2014 at 12:39 AM, Evgeniy Stepanov
<eugeni.stepanov at gmail.com> wrote:
> On Thu, Mar 27, 2014 at 10:18 PM, Greg Fitzgerald <garious at gmail.com> wrote:
>>> Alexey's approach with CMake sub-projects.
>>
>> I prefer that direction as well, but what I've proposed is a solution
>> that works today.  To support cross-compilation, we'll need to loop
>> over each supported arch (llvm-config --targets-built), then loop over
>> each supported triple for each arch (hard-coded map?), and then pair
>> up each triple with a sysroot (system paths provided by the user).
>>
>>
>>> I think it already copies
>>> compiler-rt build products to the parent build directory, right?
>>
>> The Android compiler-rt build does this, but in an unnecessarily
>> complex way.  My proposal is a simplification of that process.  What
>> is described in "llvm/cmake/platforms/Android.cmake" is to build
>> llvm+clang for the host architecture and then run a second build to
>> cross-compile llvm+compiler-rt to get a the ASan shared object and
>> test suite.  Instead, I propose building only llvm for the host
>> architecture (for the purpose of building llvm-config) and then only
>> compile compiler-rt for Android.  No need to build clang or to
>> cross-compile llvm.
>>
>>
>>> Requiring that llvm is installed complicates things.
>>
>> Sorry for the confusion.  By "llvm install directory", I don't mean to
>> install llvm to the root directory.  I'm referring to the intermediary
>> directory pointed to by CMAKE_INSTALL_PREFIX.  When you run "ninja
>> install", the build populates this directory with only the libs,
>> headers, docs, and executables that the LLVM build intends customers
>> to use directly.  In the case of compiler-rt, we need LLVM's
>> "bin/llvm-config" to determine the build mode, flags, and targets
>> available.
>
> Yes, so "ninja install" is optional. This should work with LLVM build
> directory just as well.
>
>>
>>
>>> This should not be too hard to fix.
>>
>> Porting the LLVM build to Android?  That seems like a more challenging
>> solution than to cut out the unused cross-compiled llvm build.
>
> Note that ASan tests on Android require llvm-symbolizer binary.
> Building just compiler-rt for the target would not be enough.
>
>>
>> -Greg
>>
>> On Thu, Mar 27, 2014 at 1:14 AM, Evgeniy Stepanov
>> <eugeni.stepanov at gmail.com> wrote:
>>> I'd prefer something based on Alexey's approach with CMake
>>> sub-projects. I think it already copies compiler-rt build products to
>>> the parent build directory, right?
>>>
>>> Requiring that llvm is installed complicates things.
>>>
>>> On Thu, Mar 27, 2014 at 5:00 AM, Greg Fitzgerald <garious at gmail.com> wrote:
>>>> The build for the Android sanitizers is unique in that it needs to
>>>> link against the Android system libraries to create a shared object
>>>> and its test suites.  The current solution to build ASan is to drop
>>>> the compiler-rt repo into the llvm source tree and cross-compile the
>>>> llvm build for Android.  This is a bit awkward for few reasons:
>>>>
>>>> 1) Not all of llvm can be cross-compiled for Android.  "ninja all"
>>>> causes expected build errors.
>>>
>>> This should not be too hard to fix.
>>>
>>>> 2) The sanitizers don't depend on any LLVM libraries, only llvm-config
>>>> for its build configuration.
>>>> 3) No "install" rule.  Instead, you cherry-pick files from the build directory.
>>>>
>>>> Building against the LLVM install directory cleans this up nicely and
>>>> as it turns out, this mostly works today (see CMake configuration
>>>> below).  The only missing pieces are that the shared object is not
>>>> added to the install directory and the test suites are not built.  Is
>>>> this a build configuration you'd consider using?
>>>>
>>>> $ cmake -G Ninja .. \
>>>>     -DCMAKE_INSTALL_PREFIX=ship \
>>>>     -DCMAKE_C_COMPILER=arm-linux-androideabi-gcc \
>>>>     -DCMAKE_CXX_COMPILER=arm-linux-androideabi-g++ \
>>>>     -DCMAKE_PREFIX_PATH=`pwd`/../../llvm/out/ship \
>>>>     -DANDROID=1 \
>>>>     -DCMAKE_SYSTEM_NAME=Linux \
>>>>     -DCMAKE_C_FLAGS=--sysroot=$(ndkDir)/platforms/android-19/arch-arm \
>>>>     -DCMAKE_CXX_FLAGS=--sysroot=$(ndkDir)/platforms/android-19/arch-arm
>>>> ...
>>>> $ ninja install
>>>> ...
>>>> [68/68]
>>>> -- Install configuration: "Release"
>>>> -- Installing: ship/include/sanitizer/asan_interface.h
>>>> -- Installing: ship/include/sanitizer/common_interface_defs.h
>>>> -- Installing: ship/include/sanitizer/dfsan_interface.h
>>>> -- Installing: ship/include/sanitizer/linux_syscall_hooks.h
>>>> -- Installing: ship/include/sanitizer/lsan_interface.h
>>>> -- Installing: ship/include/sanitizer/msan_interface.h
>>>> -- Installing: ship/include/sanitizer/tsan_interface_atomic.h
>>>> -- Installing: ship/asan_blacklist.txt
>>>> -- Installing: ship/bin/asan_device_setup
>>>>
>>>> Thanks,
>>>> Greg