Skip to content

Commit

Permalink
Fix Android docs and add gradlew helper (pytorch#2751)
Browse files Browse the repository at this point in the history
Summary: Pull Request resolved: pytorch#2751

Test Plan: https://docs-preview.pytorch.org/pytorch/executorch/2751/llm/llama-demo-android.html

Reviewed By: shoumikhin

Differential Revision: D55492241

Pulled By: kirklandsign

fbshipit-source-id: 84a2aad3474ba0c8dcfbae515221cef8506f1648
  • Loading branch information
kirklandsign authored and facebook-github-bot committed Mar 28, 2024
1 parent 45c2557 commit 95b0daa
Show file tree
Hide file tree
Showing 6 changed files with 238 additions and 29 deletions.
2 changes: 2 additions & 0 deletions docs/source/llm/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,5 @@ In some cases, it is necessary to write custom kernels or import them from anoth
## How to build Mobile Apps

Here's how to finally build a mobile app on Android and iOS. TODO

For Android demo app build, please see [this tutorial](./llama-demo-android.md).
2 changes: 2 additions & 0 deletions docs/source/llm/llama-demo-android.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
```{include} ../../../examples/demo-apps/android/LlamaDemo/README.md
```
72 changes: 48 additions & 24 deletions examples/demo-apps/android/LlamaDemo/README.md
Original file line number Diff line number Diff line change
@@ -1,68 +1,83 @@
# Welcome to the ExecuTorch LLaMA Android Demo App

This app demonstrates the use of the LLaMA local inference use case with ExecuTorch.
This app demonstrates the use of the LLaMA chat app demonstrating local inference use case with ExecuTorch.

::::{grid} 2
:::{grid-item-card} What you will learn
:class-card: card-prerequisites
* How to set up a build target for Android arm64-v8a
* How to build the required ExecuTorch runtime, LLaMA runner, and JNI wrapper for Android
* How to build the app with required JNI library
:::
:::{grid-item-card} Prerequisites
:class-card: card-prerequisites
* Refer to [Setting up ExecuTorch](https://pytorch.org/executorch/stable/getting-started-setup) to set up the repo and dev environment.
* Download and install [Android Studio and SDK](https://developer.android.com/studio).
* Supported Host OS: CentOS, macOS Sonoma on Apple Silicon.
:::
::::

## Getting models
```{note}
This demo app and tutorial has only been validated with arm64-v8a [ABI](https://developer.android.com/ndk/guides/abis), with NDK 25.
```

Please refer to the LLaMA tutorial to generate the model.
## Getting models
Please refer to the LLaMA tutorial to export the model.

Once you have the model, you can push it to your device:
```
Once you have the model, you need to push it to your device:
```bash
adb shell mkdir -p /data/local/tmp/llama
adb push model.pte /data/local/tmp/llama
adb push tokenizer.bin /data/local/tmp/llama
```

```{note}
The demo app searches in `/data/local/tmp/llama` for .pte and .bin files as LLAMA model and tokenizer.
```

## Build JNI library
1. Open a terminal window and navigate to the root directory of the `executorch`.
2. Set the following environment variables:
```
export BUCK=<path_to_buck>
export ANDROID_NDK=<path_to_android_ndk>
export PYTHON_EXECUTABLE=<path_to_python_executable>
export ANDROID_ABI=<abi_type> # ex: arm64-v8a
export FLATC=<path_to_flatc_executable>
export ANDROID_ABI=arm64-v8a
```
3. Create a new directory for the CMake build output:
```
mkdir cmake-out
pushd cmake-out
```
4. Run the following command to configure the CMake build:
```
cmake .. -DCMAKE_INSTALL_PREFIX=cmake-out \
# Build the core ExecuTorch runtime library
cmake . -DCMAKE_INSTALL_PREFIX=cmake-out \
-DCMAKE_TOOLCHAIN_FILE="${ANDROID_NDK}/build/cmake/android.toolchain.cmake" \
-DANDROID_ABI="${ANDROID_ABI}" \
-DBUCK2="${BUCK2}" \
-DEXECUTORCH_BUILD_XNNPACK=ON \
-DEXECUTORCH_BUILD_FLATC=OFF \
-DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
-DFLATC_EXECUTABLE="${FLATC}" \
-DEXECUTORCH_BUILD_EXTENSION_MODULE=ON
-DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
-DEXECUTORCH_BUILD_OPTIMIZED=ON \
-Bcmake-out
cmake --build . -j16 --target install
cmake --build cmake-out -j16 --target install
cmake ../examples/models/llama2 -DBUCK2="$BUCK" \
# Build the llama2 runner library and custom ops
cmake examples/models/llama2 \
-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
-DANDROID_ABI="$ANDROID_ABI" \
-DCMAKE_INSTALL_PREFIX=cmake-out \
-Bexamples/models/llama2
-Bcmake-out/examples/models/llama2
cmake --build examples/models/llama2 -j16
cmake --build cmake-out/examples/models/llama2 -j16
cmake ../extension/android -DBUCK2="${BUCK2}" \
# Build the Android JNI library
cmake extension/android \
-DCMAKE_TOOLCHAIN_FILE=${ANDROID_NDK}/build/cmake/android.toolchain.cmake \
-DANDROID_ABI="${ANDROID_ABI}" \
-DCMAKE_INSTALL_PREFIX=cmake-out \
-DEXECUTORCH_BUILD_LLAMA_JNI=ON \
-Bextension/android
-Bcmake-out/extension/android
cmake --build extension/android -j16
popd
cmake --build cmake-out/extension/android -j16
```

5. Copy the built library to your app:
Expand All @@ -75,3 +90,12 @@ cp cmake-out/extension/android/libexecutorch_llama_jni.so "${JNI_LIBS_PATH}/${AN
## Build Java app
1. Open Android Studio and select "Open an existing Android Studio project" to open examples/demo-apps/android/LlamaDemo.
2. Run the app (^R).

On the phone or emulator, you can try running the model:
<img src="../_static/img/android_llama_app.png" alt="Android LLaMA App" /><br>

## Takeaways
Through this tutorial we've learnt how to build the ExecuTorch LLAMA library with XNNPACK backend, and expose it to JNI layer to build the Android app.

## Reporting Issues
If you encountered any bugs or issues following this tutorial please file a bug/issue here on [Github](https://github.com/pytorch/executorch/issues/new).
185 changes: 185 additions & 0 deletions examples/demo-apps/android/LlamaDemo/gradlew
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
#!/usr/bin/env sh

#
# Copyright 2015 the original author or authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

##############################################################################
##
## Gradle start up script for UN*X
##
##############################################################################

# Attempt to set APP_HOME
# Resolve links: $0 may be a link
PRG="$0"
# Need this for relative symlinks.
while [ -h "$PRG" ] ; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> \(.*\)$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG=`dirname "$PRG"`"/$link"
fi
done
SAVED="`pwd`"
cd "`dirname \"$PRG\"`/" >/dev/null
APP_HOME="`pwd -P`"
cd "$SAVED" >/dev/null

APP_NAME="Gradle"
APP_BASE_NAME=`basename "$0"`

# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'

# Use the maximum available, or set MAX_FD != -1 to use that value.
MAX_FD="maximum"

warn () {
echo "$*"
}

die () {
echo
echo "$*"
echo
exit 1
}

# OS specific support (must be 'true' or 'false').
cygwin=false
msys=false
darwin=false
nonstop=false
case "`uname`" in
CYGWIN* )
cygwin=true
;;
Darwin* )
darwin=true
;;
MINGW* )
msys=true
;;
NONSTOP* )
nonstop=true
;;
esac

CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar


# Determine the Java command to use to start the JVM.
if [ -n "$JAVA_HOME" ] ; then
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
# IBM's JDK on AIX uses strange locations for the executables
JAVACMD="$JAVA_HOME/jre/sh/java"
else
JAVACMD="$JAVA_HOME/bin/java"
fi
if [ ! -x "$JAVACMD" ] ; then
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
else
JAVACMD="java"
which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi

# Increase the maximum file descriptors if we can.
if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then
MAX_FD_LIMIT=`ulimit -H -n`
if [ $? -eq 0 ] ; then
if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then
MAX_FD="$MAX_FD_LIMIT"
fi
ulimit -n $MAX_FD
if [ $? -ne 0 ] ; then
warn "Could not set maximum file descriptor limit: $MAX_FD"
fi
else
warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT"
fi
fi

# For Darwin, add options to specify how the application appears in the dock
if $darwin; then
GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""
fi

# For Cygwin or MSYS, switch paths to Windows format before running java
if [ "$cygwin" = "true" -o "$msys" = "true" ] ; then
APP_HOME=`cygpath --path --mixed "$APP_HOME"`
CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`

JAVACMD=`cygpath --unix "$JAVACMD"`

# We build the pattern for arguments to be converted via cygpath
ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null`
SEP=""
for dir in $ROOTDIRSRAW ; do
ROOTDIRS="$ROOTDIRS$SEP$dir"
SEP="|"
done
OURCYGPATTERN="(^($ROOTDIRS))"
# Add a user-defined pattern to the cygpath arguments
if [ "$GRADLE_CYGPATTERN" != "" ] ; then
OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)"
fi
# Now convert the arguments - kludge to limit ourselves to /bin/sh
i=0
for arg in "$@" ; do
CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -`
CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option

if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition
eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"`
else
eval `echo args$i`="\"$arg\""
fi
i=`expr $i + 1`
done
case $i in
0) set -- ;;
1) set -- "$args0" ;;
2) set -- "$args0" "$args1" ;;
3) set -- "$args0" "$args1" "$args2" ;;
4) set -- "$args0" "$args1" "$args2" "$args3" ;;
5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;
6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;
7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;
8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;
9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;
esac
fi

# Escape application args
save () {
for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done
echo " "
}
APP_ARGS=`save "$@"`

# Collect all arguments for the java command, following the shell quoting and substitution rules
eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS"

exec "$JAVACMD" "$@"
1 change: 1 addition & 0 deletions examples/demo-apps/android/LlamaDemo/setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ cmake . -DCMAKE_INSTALL_PREFIX="${CMAKE_OUT}" \
-DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
-DFLATC_EXECUTABLE="${FLATC}" \
-DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
-DEXECUTORCH_BUILD_OPTIMIZED=ON \
-B"${CMAKE_OUT}"

if [ "$(uname)" == "Darwin" ]; then
Expand Down
5 changes: 0 additions & 5 deletions extension/android/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,9 @@ repositories {
mavenCentral()
}


task makeJar(type: Jar) {
compileJava {
excludes = ['**/TensorImageUtils.java']
}
dependencies {
implementation 'com.facebook.fbjni:fbjni-java-only:0.2.2'
implementation 'com.facebook.soloader:nativeloader:0.10.5'
}
}

0 comments on commit 95b0daa

Please sign in to comment.