[PATCH] D64427: LLVM Test-Suite: Support Cross-Compilation and Cross-execution targeting arm64-linux-android

Tue Jul 9 10:31:52 PDT 2019

ziangwan created this revision.
ziangwan added reviewers: kongyi, srhines, pirama, MatzeB.
Herald added subscribers: llvm-commits, fedor.sergeev, kristof.beyls, javed.absar, mgorny.
Herald added a project: LLVM.

This patch allows users to cross-compile and cross-execute LLVM test-suite on an Android device for testing and benchmark purposes.

**Usage:**

1. Obtain llvm-test-suite and Android NDK for your host machine.  The test-suite also supports SPEC CPU2006 benchmark. (SPEC CPU2017 requires too much system memory and is not suitable for an actual android device.)

2. Get the build template inside cmake/caches/target-arm64-android-template.sh and create your own build script `build.sh`. Put `build.sh` inside the build directory you desire.
  - Special Requirement for build directory: Due to the current state of the build system, the tests are generated using the absolute path on the host machine. For example, if you build the test-suite at `/aa/bb/cc` on the host machine, all the build files should also be stored at `/aa/bb/cc` on the target android device for the tests to correctly execute. Therefore, we need to create a build directory, whose absolute path is also a writable location on the target Android device, on the host machine. For example, if you want the tests to run on `/data/local/tmp/build` on the Android device, you should build the test-suite inside `/data/local/tmp/build` as well. You specifically cannot use arbitrary path on the host machine since Android's SeLinux prevents creating directories at sysroot.

3. Run `sh build.sh` to setup the build. Then do `ninja` to build.

4. Run `ninja rsync_android` to push the tests onto an Android device, whose serial number is specified inside `build.sh`.

5. Run `llvm-lit -j1 -o result.json ./build` to cross-execute.

**Notes**

1. Throughout the build system of the test-suite, I set the cmake variable `TEST_SUITE_REMOTE_CLIENT` to `adb` to indicate we are doing an Android build. I also manually set sysroot and library load path for libgcc for cmake. The reason why I choose not to do `CMAKE_SYSTEM=Android` is that there is currently a compatibility issue between Android NDK and cmake. If you have a better solution please let me know.

2. I also manually exclude several tests and benchmarks for Android build by checking against `TEST_SUITE_REMOTE_CLIENT STREQUAL "adb"` in corresponding CMakeLists.txt files. All of them are excluded due to build failures. A list of excluded tests and benchmarks with concise build failure messages:
  - test-suite/External/SPEC/CINT2006/CMakeLists.txt:
    - Exclude cpu2006_subdir(400.perlbench)
      - incomplete definition of type 'struct __sFILE'
      - Many other errors
    - Exclude cpu2006_subdir(464.h264ref)
      - sys/timeb.h not found <- this header file is deprecated and no longer exists inside Android NDK.
    - Exclude cpu2006_subdir(483.xalancbmk)
      - undefined reference to `xercesc_2_5::XMLPlatformUtils::openFile(unsigned short const*, xercesc_2_5::MemoryManager*)'
      - Many other errors
  - test-suite/External/SPEC/CINT2006/462.libquantum/CMakeLists.txt
    - Use diff -w instead of diff --strip-trailing-cr
      - --strip-trailing-cr is not supported in Android's diff
  - test-suite/External/SPEC/CFP2006/CMakeLists.txt
    - Exclude cpu2006_subdir(446.dealII)
      - error: no template named 'pair' in namespace 'std'
  - test-suite/Multisource/Applications/CMakeLists.txt
    - Exclude add_subdirectory(JM)
    - Exclude add_subdirectory(hexxagon)
    - Exclude out add_subdirectory(SPASS)
      - sys/timeb.h not found
    - Exclude add_subdirectory(ClamAV)
      - undefined reference to `endprotoent'
    - Exclude add_subdirectory(siod)
      - undefined reference to `getpwent'
    - Exclude add_subdirectory(Burg)
      - error: conflicting types for 'atoi'
    - Exclude add_subdirectory(spiff)
      - parse.c:85: undefined reference to `index'
  - test-suite/Multisource/Benchmarks/MiBench/CMakeLists.txt
    - Exclude add_subdirectory(network-patricia)
      - undefined reference to `bcopy'
    - Exclude add_subdirectory(office-ispell)
      - fatal error: 'sgtty.h' file not found
  - test-suite/Multisource/Benchmarks/Prolangs-C/CMakeLists.txt
    - Exclude add_subdirectory(bison)
      - conflicts.c:266: undefined reference to `bcopy'
  - test-suite/Multisource/Benchmarks/DOE-ProxyApps-C++/CMakeLists.txt
    - Exclude add_subdirectory(CLAMR)
      - fatal error: 'sys/sysctl.h' file not found
  - test-suite/Multisource/Benchmarks/CMakeLists.txt
    - Exclude add_subdirectory(7zip)
      - 7zip/CPP/7zip/UI/Console/UserInputUtils.cpp:81:25: error: use of undeclared identifier 'getpass'
      - Many other errors
  - test-suite/SingleSource/UnitTests/
    - Exclude 2005-05-11-Popcount-ffs-fls.c
      - 110: undefined reference to `ffsl'
  - test-suite/Microbenchmarks
    - Exclude the whole folder because of unsupported test-format when cross-exec
      - An issue on the llvm-lit side
  - test-suite/MultiSource/Benchmarks/Prolangs-C/CMakeLists.txt
    - Exclude add_subdirectory(unix-small)
      - can't create /tmp/rmJKvjCz.
  - test-suite/MultiSource/Applications/CMakeLists.txt
    - Exclude out add_subdirectory(obsequi)
      - Unable to find library -lsupc++

3. Initially, I just want to use the test-suite to benchmark a compiler feature. Therefore, I only care about the benchmarks in the test-suite.

The patch works locally for me. If you have any suggestions /  comments / encounter bugs when targeting Android, please let me know.

Repository:
  rT test-suite

https://reviews.llvm.org/D64427

Files:
  CMakeLists.txt
  External/SPEC/CFP2006/CMakeLists.txt
  External/SPEC/CINT2006/462.libquantum/CMakeLists.txt
  External/SPEC/CINT2006/CMakeLists.txt
  MicroBenchmarks/CMakeLists.txt
  MultiSource/Applications/CMakeLists.txt
  MultiSource/Benchmarks/CMakeLists.txt
  MultiSource/Benchmarks/DOE-ProxyApps-C++/CMakeLists.txt
  MultiSource/Benchmarks/MiBench/CMakeLists.txt
  MultiSource/Benchmarks/Prolangs-C/CMakeLists.txt
  SingleSource/UnitTests/CMakeLists.txt
  cmake/caches/target-arm64-android-template.sh
  lit.cfg
  lit.site.cfg.in
  litsupport/modules/android.py
  utils/rsync_android.sh

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D64427.208723.patch
Type: text/x-patch
Size: 15180 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20190709/0b42497e/attachment.bin>