[ARM plugin] GSoC 2022: Android demo application for ARM CPUs #382

IRONICBo · 2022-06-25T22:58:25Z

This is the Android demo application using arm_plugin to reach inference engine.

Using object detection model and recognition model to locate and recognize attributes of detected vehicles.

IRONICBo · 2022-07-15T07:48:29Z

Running result.

allnes · 2022-07-16T17:48:41Z

@IRONICBo hello.
If you use arm plugin as base plugin in your example. Please move the example to arm plugin directory.

IRONICBo · 2022-07-18T01:45:07Z

@IRONICBo hello. If you use arm plugin as base plugin in your example. Please move the example to arm plugin directory.

OK, I have changed them. :)

likholat · 2022-08-18T09:45:39Z

@IRONICBo please upgrade source code and dependencies to OpenVINO 2022.1 and use OpenVINO Java API 2.0

allnes · 2022-08-23T16:27:29Z

Please find demo for java api 2.0 this

IRONICBo · 2022-08-30T01:51:22Z

OK，I will do that :)

IRONICBo · 2022-09-06T15:44:54Z

New result.

IRONICBo · 2022-09-06T15:48:07Z

Refer to #420 and #422, API update process is still in progress.

dkurt · 2022-09-07T10:57:41Z

modules/arm_plugin/demos/coco_detection_android_demo/README.md

+| --- | --- | --- |
+| ssdlite_mobilenet_v2 | 120ms | 120ms |
+| efficientdet-d0-tf | 800ms | 800ms |
+| pelee-coco | 220ms | X |


There is no difference between FP16 and FP32 for CPU - the plugin will convert it automatically to FP32. So no need for two columns.

Thanks, I will modify it.

dkurt · 2022-09-07T10:58:21Z

modules/arm_plugin/demos/coco_detection_android_demo/README.md

+cd "$WORK_DIR/open_model_zoo/tools/downloader"
+python3 -m pip install -r requirements.in
+omz_downloader  --name ssdlite_mobilenet_v2 --output_dir $WORK_DIR/open_model_zoo/tools/downloader
+omz_converter --name ssdlite_mobilenet_v2 --download_dir $WORK_DIR/open_model_zoo/tools/downloader --precision FP32


I recommend download only FP16 model because it's just smaller but gives same accuracy and performance.

Thanks, I have replace FP16 to FP32.

dkurt · 2022-09-07T10:59:19Z

modules/arm_plugin/demos/coco_detection_android_demo/README.md

+  1. Clone `"$WORK_DIR/openvino/bin/aarch64/Release/lib/inference_engine_java_api.jar"` to `app/libs` folder.
+  2. Clone `"$WORK_DIR/openvino/bin/aarch64/Release/lib/*.so"` and `"$WORK_DIR/android-ndk-r20/sources/cxx-stl/llvm-libc++/libs/arm64-v8a/libc++_shared.so"` to `"app/src/main/jniLibs/arm64-v8a"`
+  3. Clone `"$WORK_DIR/openvino/bin/aarch64/Release/lib/plugins.xml"` to `"app/src/main/assets"`
+  4. Download and convert model "ssdlite_mobilenet_v2" [or pelee-coco, efficientdet-d0-tf] with Open Model Zoo in following steps and copy `"$WORK_DIR/open_model_zoo/tools/downloader/intel/ssdlite_mobilenet_v2/FP32/ssdlite_mobilenet_v2.xml"`, `"$WORK_DIR/open_model_zoo/tools/downloader/intel/ssdlite_mobilenet_v2/FP32/ssdlite_mobilenet_v2.bin"` to `"app/src/main/assets"`


Model downloading step is required before.

Thanks, I will adjust order.

dkurt · 2022-09-07T11:00:30Z

modules/arm_plugin/demos/coco_detection_android_demo/README.md

+| --- | --- | --- |
+| ssdlite_mobilenet_v2 | 120ms | 120ms |
+| efficientdet-d0-tf | 800ms | 800ms |
+| pelee-coco | 220ms | X |


Any performance benchmark is a quite sensitive point. It requires an info about hardware/software setup at least. So we recommend not to provide the numbers.

Thanks, I will remove it.

IRONICBo · 2022-09-08T13:07:00Z

Update to Java API 2.0

allnes · 2022-09-08T13:23:03Z

Update to Java API 2.0

Could you share demo's video :)

IRONICBo · 2022-09-08T16:34:27Z

Update to Java API 2.0

Could you share demo's video :)

OK, I have upload a GIF.

IRONICBo · 2022-09-10T23:58:26Z

Hello, this check pipeline openvino_contrib-android-arm64 failed, the error message is not clear, how can I solve it?

...gin/demos/coco_detection_android_demo/app/src/main/java/org/intel/openvino/MainActivity.java

dkurt · 2022-09-13T12:16:16Z

@IRONICBo, please edit PR title to something like "[ARM plugin] GSoC 2022: Android demo application for ARM CPUs".

IRONICBo · 2022-09-13T15:24:36Z

@IRONICBo, please edit PR title to something like "[ARM plugin] GSoC 2022: Android demo application for ARM CPUs".

OK, got it! :)

dkurt · 2022-09-14T07:17:59Z

Please fix code style warning: https://github.com/openvinotoolkit/openvino_contrib/actions/runs/3037037332 (download patch from artifacts section)

IRONICBo · 2022-09-14T07:55:20Z

Please fix code style warning: https://github.com/openvinotoolkit/openvino_contrib/actions/runs/3037037332 (download patch from artifacts section)

Ok, I didn't find a log from the previous code style check that solves this problem. I've done the changes in the new commit.

IRONICBo · 2022-09-14T08:42:31Z

Thanks, now that the code style check has been passed. :)

...gin/demos/coco_detection_android_demo/app/src/main/java/org/intel/openvino/MainActivity.java

dkurt · 2022-09-14T14:18:49Z

@alvoron, @ilya-lavrenov, please prioritize this PR as this is a part of student's Google Summer Of Code challenge.

IRONICBo · 2022-09-15T10:12:04Z

Thanks! :)

ilya-lavrenov · 2022-09-15T20:14:52Z

may I know what is ARM specific in this demo?
Why is it not a Java API demo?

IRONICBo · 2022-09-16T00:47:48Z

may I know what is ARM specific in this demo? Why is it not a Java API demo?

This is an demo of running OpenVINO API 2.0 with ARM plugin on ARM CPUs Android(arm64-v8a). On Android, I need to use Java API 2.0 wrappers to get the inference engine.

And the suggestions are from allnes #382 (comment) and Adrian

IRONICBo · 2022-09-16T03:21:29Z

blog link: https://medium.com/openvino-toolkit/object-detection-demo-on-coco-dataset-using-openvino-toolkit-java-api-2-0-with-arm-plugin-28ce1f44bb80

IRONICBo · 2022-09-28T01:04:14Z

Hello, I can not reach these links, what should I do to solve these checks?

alvoron · 2022-09-28T08:21:38Z

Hi @IRONICBo
The pre-commit check build_ubuntu20_release has been failed because of internal infrastructure issue.
I've already contacted with dev team, they will fix the issue shortly and restart the check.

I also see that your branch is out of date. Could you please update it and merge the latest changes from master?

IRONICBo · 2022-09-28T08:30:41Z

Hi @IRONICBo The pre-commit check build_ubuntu20_release has been failed because of internal infrastructure issue. I've already contacted with dev team, they will fix the issue shortly and restart the check.

I also see that your branch is out of date. Could you please update it and merge the latest changes from master?

Thanks! I've updated my branch.

Using a single photo

5f27274

IRONICBo marked this pull request as draft June 26, 2022 13:37

IRONICBo added 2 commits July 15, 2022 15:50

Fix bugs and update README

92a43b5

Fix links

d3849d8

Move to arm plugin directory

93ea90b

Update to coco demo

8f075d4

Update demo result

110daec

IRONICBo marked this pull request as ready for review September 6, 2022 15:48

IRONICBo requested a review from a team as a code owner September 6, 2022 15:48

dkurt reviewed Sep 7, 2022

View reviewed changes

IRONICBo and others added 2 commits September 9, 2022 00:17

Update to Java API 2.0 and fix issues

30d3de5

Merge branch 'openvinotoolkit:master' into master

c8575ac

IRONICBo added 2 commits September 9, 2022 01:05

Update README for jar package

bae434d

Add different color for COCO classes

ffd01fe

IRONICBo changed the title ~~[java_api] Android demo application for ARM CPUs~~ [ARM plugin] Android demo application for ARM CPUs Sep 10, 2022

likholat reviewed Sep 12, 2022

View reviewed changes

...gin/demos/coco_detection_android_demo/app/src/main/java/org/intel/openvino/MainActivity.java Show resolved Hide resolved

...gin/demos/coco_detection_android_demo/app/src/main/java/org/intel/openvino/MainActivity.java Show resolved Hide resolved

IRONICBo added 2 commits September 12, 2022 19:49

Fix log and replace loadLibrary function

a94e577

Fix format

107f70e

IRONICBo requested a review from likholat September 12, 2022 11:53

IRONICBo changed the title ~~[ARM plugin] Android demo application for ARM CPUs~~ [ARM plugin] GSoC 2022: Android demo application for ARM CPUs Sep 13, 2022

Fix code style

a51ce0a

IRONICBo and others added 4 commits September 14, 2022 16:27

Fix package

ce53a51

Fix style

ba1a6e1

Fix return style

d52b9e3

Merge branch 'master' into master

4f82bba

likholat approved these changes Sep 14, 2022

View reviewed changes

...gin/demos/coco_detection_android_demo/app/src/main/java/org/intel/openvino/MainActivity.java Outdated Show resolved Hide resolved

...gin/demos/coco_detection_android_demo/app/src/main/java/org/intel/openvino/MainActivity.java Show resolved Hide resolved

Update to loadLibrary

1aad354

ilya-lavrenov enabled auto-merge (squash) September 15, 2022 20:13

ilya-lavrenov disabled auto-merge September 15, 2022 20:13

Merge branch 'openvinotoolkit:master' into master

782ffac

allnes approved these changes Sep 28, 2022

View reviewed changes

ilya-lavrenov merged commit ce51398 into openvinotoolkit:master Sep 28, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ARM plugin] GSoC 2022: Android demo application for ARM CPUs #382

[ARM plugin] GSoC 2022: Android demo application for ARM CPUs #382

IRONICBo commented Jun 25, 2022

IRONICBo commented Jul 15, 2022

allnes commented Jul 16, 2022

IRONICBo commented Jul 18, 2022

likholat commented Aug 18, 2022

allnes commented Aug 23, 2022

IRONICBo commented Aug 30, 2022

IRONICBo commented Sep 6, 2022

IRONICBo commented Sep 6, 2022

dkurt Sep 7, 2022

IRONICBo Sep 8, 2022

dkurt Sep 7, 2022

IRONICBo Sep 8, 2022

dkurt Sep 7, 2022

IRONICBo Sep 8, 2022

dkurt Sep 7, 2022

IRONICBo Sep 8, 2022

IRONICBo commented Sep 8, 2022

allnes commented Sep 8, 2022

IRONICBo commented Sep 8, 2022 •

edited

Loading

IRONICBo commented Sep 10, 2022 •

edited

Loading

dkurt commented Sep 13, 2022

IRONICBo commented Sep 13, 2022

dkurt commented Sep 14, 2022

IRONICBo commented Sep 14, 2022

IRONICBo commented Sep 14, 2022 •

edited

Loading

dkurt commented Sep 14, 2022

IRONICBo commented Sep 15, 2022

ilya-lavrenov commented Sep 15, 2022

IRONICBo commented Sep 16, 2022 •

edited

Loading

IRONICBo commented Sep 16, 2022

IRONICBo commented Sep 28, 2022

alvoron commented Sep 28, 2022

IRONICBo commented Sep 28, 2022 •

edited

Loading

[ARM plugin] GSoC 2022: Android demo application for ARM CPUs #382

[ARM plugin] GSoC 2022: Android demo application for ARM CPUs #382

Conversation

IRONICBo commented Jun 25, 2022

IRONICBo commented Jul 15, 2022

allnes commented Jul 16, 2022

IRONICBo commented Jul 18, 2022

likholat commented Aug 18, 2022

allnes commented Aug 23, 2022

IRONICBo commented Aug 30, 2022

IRONICBo commented Sep 6, 2022

IRONICBo commented Sep 6, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

IRONICBo commented Sep 8, 2022

allnes commented Sep 8, 2022

IRONICBo commented Sep 8, 2022 • edited Loading

IRONICBo commented Sep 10, 2022 • edited Loading

dkurt commented Sep 13, 2022

IRONICBo commented Sep 13, 2022

dkurt commented Sep 14, 2022

IRONICBo commented Sep 14, 2022

IRONICBo commented Sep 14, 2022 • edited Loading

dkurt commented Sep 14, 2022

IRONICBo commented Sep 15, 2022

ilya-lavrenov commented Sep 15, 2022

IRONICBo commented Sep 16, 2022 • edited Loading

IRONICBo commented Sep 16, 2022

IRONICBo commented Sep 28, 2022

alvoron commented Sep 28, 2022

IRONICBo commented Sep 28, 2022 • edited Loading

IRONICBo commented Sep 8, 2022 •

edited

Loading

IRONICBo commented Sep 10, 2022 •

edited

Loading

IRONICBo commented Sep 14, 2022 •

edited

Loading

IRONICBo commented Sep 16, 2022 •

edited

Loading

IRONICBo commented Sep 28, 2022 •

edited

Loading