Restructure the native module

Consolidate all code into the src folder
This commit is contained in:
topjohnwu 2022-07-23 13:51:56 -07:00
parent c7c9fb9576
commit b9e89a1a2d
198 changed files with 52 additions and 45 deletions

36
.gitmodules vendored
View File

@ -1,45 +1,45 @@
[submodule "selinux"]
path = native/jni/external/selinux
path = native/src/external/selinux
url = https://github.com/topjohnwu/selinux.git
[submodule "busybox"]
path = native/jni/external/busybox
path = native/src/external/busybox
url = https://github.com/topjohnwu/ndk-busybox.git
[submodule "dtc"]
path = native/jni/external/dtc
path = native/src/external/dtc
url = https://github.com/dgibson/dtc.git
[submodule "lz4"]
path = native/jni/external/lz4
path = native/src/external/lz4
url = https://github.com/lz4/lz4.git
[submodule "bzip2"]
path = native/jni/external/bzip2
path = native/src/external/bzip2
url = https://github.com/nemequ/bzip2.git
[submodule "xz"]
path = native/jni/external/xz
path = native/src/external/xz
url = https://github.com/xz-mirror/xz.git
[submodule "nanopb"]
path = native/jni/external/nanopb
path = native/src/external/nanopb
url = https://github.com/nanopb/nanopb.git
[submodule "mincrypt"]
path = native/jni/external/mincrypt
path = native/src/external/mincrypt
url = https://github.com/topjohnwu/mincrypt.git
[submodule "pcre"]
path = native/jni/external/pcre
path = native/src/external/pcre
url = https://android.googlesource.com/platform/external/pcre
[submodule "libcxx"]
path = native/jni/external/libcxx
path = native/src/external/libcxx
url = https://github.com/topjohnwu/libcxx.git
[submodule "zlib"]
path = native/jni/external/zlib
path = native/src/external/zlib
url = https://android.googlesource.com/platform/external/zlib
[submodule "parallel-hashmap"]
path = native/jni/external/parallel-hashmap
path = native/src/external/parallel-hashmap
url = https://github.com/greg7mdp/parallel-hashmap.git
[submodule "zopfli"]
path = native/src/external/zopfli
url = https://github.com/google/zopfli.git
[submodule "cxx-rs"]
path = native/src/external/cxx-rs
url = https://github.com/topjohnwu/cxx.git
[submodule "termux-elf-cleaner"]
path = tools/termux-elf-cleaner
url = https://github.com/termux/termux-elf-cleaner.git
[submodule "zopfli"]
path = native/jni/external/zopfli
url = https://github.com/google/zopfli.git
[submodule "cxx-rs"]
path = native/jni/external/cxx-rs
url = https://github.com/topjohnwu/cxx.git

View File

@ -211,6 +211,7 @@ def binary_dump(src, var_name):
def run_ndk_build(flags):
os.chdir('native')
flags = 'NDK_PROJECT_PATH=. NDK_APPLICATION_MK=src/Application.mk ' + flags
proc = system(f'{ndk_build} {flags} -j{cpu_count}')
if proc.returncode != 0:
error('Build binary failed!')
@ -223,7 +224,7 @@ def run_ndk_build(flags):
def run_cargo_build(args):
os.chdir(op.join('native', 'rust'))
os.chdir(op.join('native', 'src'))
targets = set(args.target) & set(rust_targets)
env = os.environ.copy()
@ -242,9 +243,9 @@ def run_cargo_build(args):
cxxbridge = op.join(local_cargo_root, 'bin', 'cxxbridge' + EXE_EXT)
mkdir(native_gen_path)
for p in ['base', 'boot', 'core', 'init', 'sepolicy']:
text = cmd_out([cxxbridge, op.join(p, 'src', 'lib.rs')])
text = cmd_out([cxxbridge, op.join(p, 'lib.rs')])
write_if_diff(op.join(native_gen_path, f'{p}-rs.cpp'), text)
text = cmd_out([cxxbridge, '--header', op.join(p, 'src', 'lib.rs')])
text = cmd_out([cxxbridge, '--header', op.join(p, 'lib.rs')])
write_if_diff(op.join(native_gen_path, f'{p}-rs.hpp'), text)
# Start building the actual build commands

View File

@ -4,24 +4,17 @@
Install the NDK required to build and develop Magisk with `./build.py ndk`. The NDK will be installed to `$ANDROID_SDK_ROOT/ndk/magisk`. You don't need to manually install a Rust toolchain with `rustup`, as the NDK installed already has a Rust toolchain bundled.
## Code Paths
- `jni`: Magisk's code in C++
- `jni/external`: external dependencies, mostly submodules
- `rust`: Magisk's code in Rust
- `src`: irrelevant, only exists to setup a native Android Studio project
## Build Configs
All C/C++ code and its dependencies are built with [`ndk-build`](https://developer.android.com/ndk/guides/ndk-build) and configured with several `*.mk` files scatterred in many places.
All C/C++ code and its dependencies are built with [`ndk-build`](https://developer.android.com/ndk/guides/ndk-build) and configured with several `*.mk` files scattered in many places.
The `rust` folder is a proper Cargo workspace, and all Rust code is built with `cargo` just like any other Rust projects.
The `src` folder is also a proper Cargo workspace, and all Rust code is built with `cargo` just like normal Rust projects.
## Rust + C/C++
To reduce complexity involved in linking, all Rust code is built as `staticlib` and linked to C++ targets to ensure our final product is built with an officially supported NDK build system. Each C++ target can at most link to **one** Rust `staticlib` or else multiple definitions error will occur.
We use the [`cxx`](https://cxx.rs) project for interop between Rust and C++. Although cxx supports interop in both directions, for Magisk, it is strongly advised to avoid calling C++ functions in Rust; if some functionality required in Rust is already implemented in C++, the desired solution is to port the C++ implementation into Rust and export the migrated function back to C++.
We use the [`cxx`](https://cxx.rs) project for Rust and C++ interop.
## Development / IDE
@ -29,6 +22,6 @@ All C++ code should be recognized and properly indexed by Android Studio out of
- Install the [Rust plugin](https://www.jetbrains.com/rust/) in Android Studio
- In Preferences > Languages & Frameworks > Rust, set `$ANDROID_SDK_ROOT/ndk/magisk/toolchains/rust/bin` as the toolchain location
- Open `native/rust/Cargo.toml`, and select "Attach" in the "No Cargo projects found" banner
- Open `native/src/Cargo.toml`, and select "Attach" in the "No Cargo projects found" banner
Note: run `./build.py binary` before developing to make sure generated code is created.

View File

@ -9,10 +9,14 @@ android {
externalNativeBuild {
ndkBuild {
path("jni/Android.mk")
path("src/Android.mk")
}
}
sourceSets.getByName("main") {
manifest.srcFile("src/AndroidManifest.xml")
}
defaultConfig {
externalNativeBuild {
ndkBuild {

View File

@ -1 +0,0 @@
target/

View File

@ -1 +1,2 @@
test.cpp
target/

View File

@ -153,7 +153,7 @@ include $(BUILD_EXECUTABLE)
endif
ifdef B_TEST
ifneq (,$(wildcard jni/test.cpp))
ifneq (,$(wildcard src/test.cpp))
include $(CLEAR_VARS)
LOCAL_MODULE := test
@ -176,7 +176,7 @@ LOCAL_MODULE := libpolicy
LOCAL_STATIC_LIBRARIES := \
libbase \
libsepol
LOCAL_C_INCLUDES := jni/sepolicy/include
LOCAL_C_INCLUDES := src/sepolicy/include
LOCAL_EXPORT_C_INCLUDES := $(LOCAL_C_INCLUDES)
LOCAL_SRC_FILES := \
sepolicy/api.cpp \
@ -186,12 +186,12 @@ LOCAL_SRC_FILES := \
sepolicy/statement.cpp
include $(BUILD_STATIC_LIBRARY)
include jni/Android-rs.mk
include jni/base/Android.mk
include jni/external/Android.mk
include src/Android-rs.mk
include src/base/Android.mk
include src/external/Android.mk
ifdef B_BB
include jni/external/busybox/Android.mk
include src/external/busybox/Android.mk
endif

View File

@ -0,0 +1,2 @@
<!-- This file exists only to make Android Studio happy -->
<manifest/>

View File

@ -1,3 +1,4 @@
APP_BUILD_SCRIPT := src/Android.mk
APP_ABI := armeabi-v7a arm64-v8a x86 x86_64
APP_CFLAGS := -Wall -Oz -fomit-frame-pointer -flto
APP_LDFLAGS := -flto
@ -10,7 +11,7 @@ APP_STRIP_MODE := --strip-all
# Busybox should use stock libc.a
ifdef B_BB
APP_PLATFORM := android-24
APP_LDFLAGS += -T jni/lto_fix.lds
APP_LDFLAGS += -T src/lto_fix.lds
ifeq ($(OS),Windows_NT)
APP_SHORT_COMMANDS := true
endif

View File

@ -23,4 +23,4 @@ panic = "abort"
strip = true
[patch.crates-io]
cxx = { path = "../jni/external/cxx-rs" }
cxx = { path = "../src/external/cxx-rs" }

View File

@ -4,7 +4,7 @@ LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := libbase
LOCAL_C_INCLUDES := jni/include $(LOCAL_PATH)/include out/generated
LOCAL_C_INCLUDES := src/include $(LOCAL_PATH)/include out/generated
LOCAL_EXPORT_C_INCLUDES := $(LOCAL_C_INCLUDES)
LOCAL_EXPORT_STATIC_LIBRARIES := libcxx
LOCAL_STATIC_LIBRARIES := libcxx
@ -28,5 +28,5 @@ LOCAL_MODULE := libcompat
LOCAL_SRC_FILES := compat/compat.cpp
# Fix static variables' ctor/dtor when using LTO
# See: https://github.com/android/ndk/issues/1461
LOCAL_EXPORT_LDLIBS := -static -T jni/lto_fix.lds
LOCAL_EXPORT_LDLIBS := -static -T src/lto_fix.lds
include $(BUILD_STATIC_LIBRARY)

View File

@ -3,5 +3,8 @@ name = "base"
version = "0.1.0"
edition = "2021"
[lib]
path = "lib.rs"
[dependencies]
cxx = "1.0.69"

View File

@ -5,6 +5,7 @@ edition = "2021"
[lib]
crate-type = ["staticlib"]
path = "lib.rs"
[dependencies]
base = { path = "../base" }

View File

@ -5,6 +5,7 @@ edition = "2021"
[lib]
crate-type = ["staticlib"]
path = "lib.rs"
[dependencies]
base = { path = "../base" }

Some files were not shown because too many files have changed in this diff Show More