[Lldb-commits] [lldb] 18b9d49 - lldb: get lldb API tests working with newer Android NDKs

Tue Sep 24 20:00:37 PDT 2024

Author: Andrew Rogers
Date: 2024-09-24T20:00:34-07:00
New Revision: 18b9d49ce3370c012fdd04ec87d854d53293f6a6

URL: https://github.com/llvm/llvm-project/commit/18b9d49ce3370c012fdd04ec87d854d53293f6a6
DIFF: https://github.com/llvm/llvm-project/commit/18b9d49ce3370c012fdd04ec87d854d53293f6a6.diff

LOG: lldb: get lldb API tests working with newer Android NDKs

## Purpose
Running the LLDB API tests against a remote Android target with NDK
version r22 or later fails to compile the test inferiors. NDK r21 from
2021 is the most recent NDK that still works with the LLDB API tests.
This PR updates the Android make rules to support newer Android NDK
versions (r19 and later).

## Overview
* Updates and simplifies `Android.rules` to match the newer Android NDK
unified toolchain layout introduced in NDK r19
* Sets `OBJCOPY` and `ARCHIVER` env vars, required by a few test cases,
to their `llvm-` versions in the unified toolchain
* Drops support for pre-2019 Android NDK versions to keep the rules
simple
* Provides an error message if the tests are run using an incompatible
NDK layout

## Problem Details
Android introduced a unified tools layout in NDK r19 (2019) and removed
the old layout in r22 (2021). Releases r19, r20, and r21 support both
the old and new layout side-by-side. More details are in #106270.

## Validation
Ran a sub-set of the LLDB API tests against remote Android targets for
the four primary architectures i386, x86_64, arm, and aarch64. No
validation was done against riscv targets.

For each case, ran the copy of `lldb-server` from the Android NDK on the
device with the latest LLDB test cases in llvm-project

Ran tests with both r19 (the oldest supported) and r26 (more recent,
unified layout only) NDK versions.

Example test command for aarch64:
```
./build/bin/lldb-dotest --out-of-tree-debugserver --arch aarch64 --platform-name remote-android --platform-url connect://localhost:5432 --platform-working-dir /data/local/tmp --compiler=$ANDROID_NDK_ROOT/toolchains/llvm/prebuilt/linux-x86_64/bin/clang lldb/test/API/android/
```
**NOTE: there are a lot of test failures when running the full suite
(especially against 32-bit ARM target). These failures occur independent
of this change.**

Verified the expected error message appears when attempting to run using
NDK r18
```
Build Command Output:
make: Entering directory '/home/andrew/src/llvm/llvm-project/build/lldb-test-build.noindex/android/platform/TestDefaultCacheLineSize.test_cache_line_size'
/home/andrew/src/llvm/llvm-project/lldb/packages/Python/lldbsuite/test/make/Android.rules:16: *** "No unified toolchain sysroot found in /home/andrew/Android/Sdk/ndk/18.1.5063045/toolchains/llvm/prebuilt/linux-x86_64/bin/../../../../... NDK must be r19 or later.".  Stop.
make: Leaving directory '/home/andrew/src/llvm/llvm-project/build/lldb-test-build.noindex/android/platform/TestDefaultCacheLineSize.test_cache_line_size'
```

## Impact
**This change explicitly removes support for the pre-2019 NDK
structure.** Only NDK r19 (from 2019) and later can be used when running
the LLDB API tests. If the maintainers object, we can easily support
both the old and new NDK toolchain layouts side-by-side at the cost of
readability/maintainability. Since this change only impacts tests, I
don't see much value in supporting NDKs that are over 5 years old.

## Guidance to Reviewers
* I am not an expert on `clang` arguments so if anything looks off let
me know.
* While I personally thing supporting 5+ year old NDKs for testing seems
unnecessary, please chime-in if you are concerned with dropping that
support. I can easily revise to support both old and new layouts
side-by-side.
* If there are any specific tests you'd like me to run I will do my best
to accommodate. It doesn't look like there's much (any?) Android LLDB CI
coverage.

Added: 
    

Modified: 
    lldb/packages/Python/lldbsuite/test/make/Android.rules

Removed: 
    


################################################################################
diff  --git a/lldb/packages/Python/lldbsuite/test/make/Android.rules b/lldb/packages/Python/lldbsuite/test/make/Android.rules
index cd7d8ae74d6bf3..44aedf7248419e 100644

--- a/lldb/packages/Python/lldbsuite/test/make/Android.rules
+++ b/lldb/packages/Python/lldbsuite/test/make/Android.rules
@@ -1,81 +1,59 @@
 NDK_ROOT := $(shell dirname $(CC))/../../../../..
 
-ifeq "$(findstring 64, $(ARCH))" "64"
-	# lowest 64-bit API level
-	API_LEVEL := 21
-else ifeq "$(ARCH)" "i386"
-	# clone(2) declaration is present only since this api level
-	API_LEVEL := 17
+ifeq "$(HOST_OS)" "Linux"
+	HOST_TAG := linux-x86_64
+else ifeq "$(HOST_OS)" "Darwin"
+	HOST_TAG := darwin-x86_64
 else
-	# lowest supported 32-bit API level
-	API_LEVEL := 16
+	HOST_TAG := windows-x86_64
+endif
+
+TOOLCHAIN_ROOT := $(NDK_ROOT)/toolchains/llvm/prebuilt/$(HOST_TAG)
+TOOLCHAIN_SYSROOT := $(TOOLCHAIN_ROOT)/sysroot
+
+OBJCOPY ?= $(TOOLCHAIN_ROOT)/bin/llvm-objcopy
+ARCHIVER ?= $(TOOLCHAIN_ROOT)/bin/llvm-ar
+
+ifeq "$(wildcard $(TOOLCHAIN_SYSROOT)/.)" ""
+# Compiling test inferiors for Android requires an NDK with the unified
+# toolchain introduced in version r19.
+$(error "No unified toolchain sysroot found in $(NDK_ROOT). NDK must be r19 or later.")
 endif
 
 ifeq "$(ARCH)" "arm"
-	SYSROOT_ARCH := arm
-	STL_ARCH := armeabi-v7a
 	TRIPLE := armv7-none-linux-androideabi
 	ARCH_CFLAGS += -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -marm
 else ifeq "$(ARCH)" "aarch64"
-	SYSROOT_ARCH := arm64
-	STL_ARCH := arm64-v8a
 	TRIPLE := aarch64-none-linux-android
 else ifeq "$(ARCH)" "i386"
-	SYSROOT_ARCH := x86
-	STL_ARCH := x86
 	TRIPLE := i686-none-linux-android
 else
-	SYSROOT_ARCH := $(ARCH)
-	STL_ARCH := $(ARCH)
 	TRIPLE := $(ARCH)-none-linux-android
 endif
 
-ifeq "$(findstring 86,$(ARCH))" "86"
-	TOOLCHAIN_DIR := $(STL_ARCH)-4.9
-else ifeq "$(ARCH)" "arm"
-	TOOLCHAIN_DIR := arm-linux-androideabi-4.9
-else
-	TOOLCHAIN_DIR := $(subst -none,,$(TRIPLE))-4.9
-endif
+# lowest 64-bit API level
+API_LEVEL := 21
 
 ifeq "$(ARCH)" "arm"
-	TOOL_PREFIX := arm-linux-androideabi
+	ARCH_DIR := arm-linux-androideabi
 else
-	TOOL_PREFIX := $(subst -none,,$(TRIPLE))
+	ARCH_DIR := $(subst -none,,$(TRIPLE))
 endif
 
-ifeq "$(HOST_OS)" "Linux"
-	HOST_TAG := linux-x86_64
-else ifeq "$(HOST_OS)" "Darwin"
-	HOST_TAG := darwin-x86_64
-else
-	HOST_TAG := windows-x86_64
-endif
-
-GCC_TOOLCHAIN = $(NDK_ROOT)/toolchains/$(TOOLCHAIN_DIR)/prebuilt/$(HOST_TAG)
-
-OBJCOPY ?= $(GCC_TOOLCHAIN)/bin/$(TOOL_PREFIX)-objcopy
-ARCHIVER ?= $(GCC_TOOLCHAIN)/bin/$(TOOL_PREFIX)-ar
-
-ifeq "$(findstring clang,$(CC))" "clang"
-	ARCH_CFLAGS += -target $(TRIPLE) --gcc-toolchain=$(GCC_TOOLCHAIN)
-	ARCH_LDFLAGS += -target $(TRIPLE) --gcc-toolchain=$(GCC_TOOLCHAIN)
-endif
-
-ARCH_CFLAGS += --sysroot=$(NDK_ROOT)/sysroot \
-	-isystem $(NDK_ROOT)/sysroot/usr/include/$(TOOL_PREFIX) \
-	-D__ANDROID_API__=$(API_LEVEL) \
-	-isystem $(NDK_ROOT)/platforms/android-$(API_LEVEL)/arch-$(SYSROOT_ARCH)/usr/include
-
-ARCH_LDFLAGS += --sysroot=$(NDK_ROOT)/platforms/android-$(API_LEVEL)/arch-$(SYSROOT_ARCH) -lm
+ARCH_CFLAGS += \
+	--target=$(TRIPLE) \
+	--sysroot=$(TOOLCHAIN_SYSROOT) \
+	-D__ANDROID_API__=$(API_LEVEL)
 
 ARCH_CXXFLAGS += \
-	-isystem $(NDK_ROOT)/sources/cxx-stl/llvm-libc++/include \
-	-isystem $(NDK_ROOT)/sources/android/support/include \
-	-isystem $(NDK_ROOT)/sources/cxx-stl/llvm-libc++abi/include
+	-isystem $(TOOLCHAIN_SYSROOT)/usr/include/c++/v1
 
 ARCH_LDFLAGS += \
-	-L$(NDK_ROOT)/sources/cxx-stl/llvm-libc++/libs/$(STL_ARCH) \
-	$(NDK_ROOT)/sources/cxx-stl/llvm-libc++/libs/$(STL_ARCH)/libc++_static.a \
+	--target=$(TRIPLE) \
+	--sysroot=$(TOOLCHAIN_SYSROOT) \
+	--prefix=$(TOOLCHAIN_SYSROOT)/usr/lib/$(ARCH_DIR)/$(API_LEVEL) \
+	-L$(TOOLCHAIN_SYSROOT)/usr/lib/$(ARCH_DIR)/$(API_LEVEL) \
+	$(TOOLCHAIN_SYSROOT)/usr/lib/$(ARCH_DIR)/libc++_static.a \
+	-lm \
 	-lc++abi \
 	-nostdlib++


        
