Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PythonActivityUtil helper for unpacking data #2189

Merged

Conversation

AndreMiras
Copy link
Member

This helper class shares code to be reused between the PythonActivity
classes of the different bootstraps.
Also adds few minor adjustments to existing classes.

@opacam
Copy link
Member

opacam commented May 11, 2020

I tested the artifacts for arm64-v8a and armeabi-v7a and both apps crashes before unpacking files:

adb logcat
05-11 11:20:54.197 18372 18372 V PythonActivity: PythonActivity onCreate running
05-11 11:20:54.197 18372 18372 V PythonActivity: About to do super onCreate
05-11 11:20:54.198 18372 18372 V SDL     : Device: hydrogen
05-11 11:20:54.198 18372 18372 V SDL     : Model: Mi Max
05-11 11:20:54.198 18372 18372 V SDL     : onCreate()
05-11 11:20:54.198 18372 18372 V PythonActivity: Did super onCreate
05-11 11:20:54.198 18372 18372 V SDL     : getting identifier
05-11 11:20:54.198 18372 18372 V SDL     : kind is drawable and name presplash
05-11 11:20:54.198 18372 18372 V SDL     : result is 2130771970
05-11 11:20:54.236 18372 18372 V SDL     : asked to get string presplash_color
05-11 11:20:54.236 18372 18372 V SDL     : getting identifier
05-11 11:20:54.236 18372 18372 V SDL     : kind is string and name presplash_color
05-11 11:20:54.236 18372 18372 V SDL     : result is 2130968577
05-11 11:20:54.262 18372 18396 V PythonActivity: Ready to unpack
05-11 11:20:54.276 18372 18372 V PythonActivity: onResume()
05-11 11:20:54.276 18372 18372 V SDL     : onResume()
05-11 11:20:54.281 18372 18396 E AndroidRuntime: FATAL EXCEPTION: AsyncTask #1
05-11 11:20:54.281 18372 18396 E AndroidRuntime: Process: org.test.unit_tests_app, PID: 18372
05-11 11:20:54.281 18372 18396 E AndroidRuntime: java.lang.RuntimeException: An error occurred while executing doInBackground()
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at android.os.AsyncTask$4.done(AsyncTask.java:399)
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:383)
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at java.util.concurrent.FutureTask.setException(FutureTask.java:252)
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at java.util.concurrent.FutureTask.run(FutureTask.java:271)
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:289)
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at java.lang.Thread.run(Thread.java:919)
05-11 11:20:54.281 18372 18396 E AndroidRuntime: Caused by: java.lang.RuntimeException: Can't create handler inside thread Thread[AsyncTask #1,5,main] that has not called Looper.prepare()
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at android.os.Handler.<init>(Handler.java:207)
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at android.os.Handler.<init>(Handler.java:119)
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at android.app.Activity.<init>(Activity.java:880)
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at org.kivy.android.PythonActivityUtil.<init>(PythonActivityUtil.java:22)
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at org.kivy.android.PythonActivity$UnpackFilesTask.doInBackground(PythonActivity.java:104)
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at org.kivy.android.PythonActivity$UnpackFilesTask.doInBackground(PythonActivity.java:99)
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at android.os.AsyncTask$3.call(AsyncTask.java:378)
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
05-11 11:20:54.281 18372 18396 E AndroidRuntime:        ... 4 more

@AndreMiras
Copy link
Member Author

Thank you for checking @opacam so I could eventually break it 🎉 😄
From my local tests it was working perfect with the tests apps, but maybe then I haven't cleaned things up properly before building again.
I'll take a look later. Thanks again for the heads up

@AndreMiras
Copy link
Member Author

Incredible I've tested both artifacts and they are working on my device 🤔
For instance one output:

05-11 20:10:27.657 24939 25018 I python  : Initializing Python for Android
05-11 20:10:27.657 24939 25018 I python  : Setting additional env vars from p4a_env_vars.txt
05-11 20:10:27.658 24939 25018 I python  : Changing directory to the one provided by ANDROID_ARGUMENT
05-11 20:10:27.658 24939 25018 I python  : /data/user/0/org.test.unit_tests_app/files/app
05-11 20:10:27.701 24939 25018 I python  : Preparing to initialize python
05-11 20:10:27.701 24939 25018 I python  : _python_bundle dir exists
05-11 20:10:27.701 24939 25018 I python  : calculated paths to be...
05-11 20:10:27.701 24939 25018 I python  : /data/user/0/org.test.unit_tests_app/files/app/_python_bundle/stdlib.zip:/data/user/0/org.test.unit_tests_app/files/app/_python_bundle/modules
05-11 20:10:27.738 24939 25018 I python  : set wchar paths...
05-11 20:10:27.960 24939 25018 I python  : Initialized python
05-11 20:10:27.960 24939 25018 I python  : AND: Init threads
05-11 20:10:27.963 24939 25018 I python  : testing python print redirection
05-11 20:10:27.965 24939 25018 I python  : Android path ['.', '/data/user/0/org.test.unit_tests_app/files/app/_python_bundle/stdlib.zip', '/data/user/0/org.test.unit_tests_app/files/app/_python_bundle/modules', '/data/user/0/org.test.unit_tests_app/files/app/_python_bundle/site-packages']
05-11 20:10:27.966 24939 25018 I python  : os.environ is environ({'PATH': '/sbin:/vendor/bin:/system/sbin:/system/bin:/system/xbin:/system/vendor/bin:/vendor/xbin:/system/vendor/xbin:/product/bin:/product/xbin', 'ANDROID_BOOTLOGO': '1', 'ANDROID_ROOT': '/system', 'ANDROID_ASSETS': '/system/app', 'ANDROID_DATA': '/data', 'ANDROID_STORAGE': '', 'EXTERNAL_STORAGE': '/sdcard', 'ASEC_MOUNTPOINT': '/mnt/asec', 'BOOTCLASSPATH': '/system/framework/core-oj.jar:/system/framework/core-libart.jar:/system/framework/conscrypt.jar:/system/framework/okhttp.jar:/system/framework/core-junit.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/ims-common.jar:/system/framework/apache-xml.jar:/system/framework/org.apache.http.legacy.boot.jar:/system/framework/hwEmui.jar:/system/framework/hwTelephony-common.jar:/system/framework/hwframework.jar:/system/framework/org.simalliance.openmobileapi.jar:/system/framework/org.ifaa.android.manager.jar:/system/framework/hwaps.jar:/system/framework/hwcustEmui.jar:/system/framework/hwcustTelephony-common.jar:/system/framework/hwcustframework.jar', 'SYSTEMSERVERCLASSPATH': '/system/framework/services.jar:/system/framework/ethernet-service.jar:/system/framework/wifi-service.jar:/system/framework/hwServices.jar:/system/framework/hwWifi-service.jar:/system/framework/hwcustServices.jar:/system/framework/hwcustwifi-service.jar', 'OEM_ROOT': '/hw_oem', 'CUST_POLICY_DIRS': '/system/emui/base:/system/emui/oversea:/system/emui/oversea_honor:/system/emui/lite:/system/global:/vendor/etc:/product/etc:/hw_oem:/cust/hw/normal:/cust_comm:/version/region_comm/oversea:/cust_spec', 'ANDROID_SOCKET_zygote': '14', 'ANDROID_ENTRYPOINT': 'main.pyc', 'ANDROID_ARGUMENT': '/data/user/0/org.test.unit_tests_app/files/app', 'ANDROID_APP_PATH': '/data/user/0/org.test.unit_tests_app/files/app', 'ANDROID_PRIVATE': '/data/user/0/org.test.unit_tests_app/files', 'ANDROID_UNPACK': '/data/user/0/org.test.unit_tests_app/files/app', 'PYTHONHOME': '/data/user/0/org.test.unit_tests_app/files/app', 'PYTHONPATH': '/data/user/0/org.test.unit_tests_app/files/app:/data/user/0/org.test.unit_tests_app/files/app/lib', 'PYTHONOPTIMIZE': '2', 'P4A_BOOTSTRAP': 'SDL2', 'PYTHON_NAME': 'python', 'P4A_IS_WINDOWED': 'False', 'P4A_ORIENTATION': 'sensor', 'P4A_NUMERIC_VERSION': 'None', 'P4A_MINSDK': '21', 'LC_CTYPE': 'C.UTF-8'})
05-11 20:10:27.966 24939 25018 I python  : Android kivy bootstrap done. __name__ is __main__
05-11 20:10:27.966 24939 25018 I python  : AND: Ran string
05-11 20:10:27.966 24939 25018 I python  : Run user program, change dir and execute entrypoint

05-11 20:10:28.409 24939 25018 I python  : Imported unittest
05-11 20:10:28.411 24939 25018 I python  : App requirements are:  {'numpy', 'python3', 'libffi', 'pyjnius', 'requests', 'sqlite3', 'kivy', 'openssl', 'setuptools', 'sdl2'}
05-11 20:10:28.414 24939 25018 I python  : Defined test case
05-11 20:10:28.416 24939 25018 I python  : Adding Testcase:  tests.test_requirements.NumpyTestCase
05-11 20:10:28.419 24939 25018 I python  : Adding Testcase:  tests.test_requirements.LibffiTestCase
05-11 20:10:28.420 24939 25018 I python  : Adding Testcase:  tests.test_requirements.PyjniusTestCase
05-11 20:10:28.421 24939 25018 I python  : Adding Testcase:  tests.test_requirements.RequestsTestCase
05-11 20:10:28.422 24939 25018 I python  : Adding Testcase:  tests.test_requirements.Sqlite3TestCase
05-11 20:10:28.423 24939 25018 I python  : Adding Testcase:  tests.test_requirements.KivyTestCase
05-11 20:10:28.424 24939 25018 I python  : Adding Testcase:  tests.test_requirements.OpensslTestCase
05-11 20:10:28.426 24939 25018 I python  : Tests to perform are:  {'numpy': 'tests.test_requirements.NumpyTestCase', 'libffi': 'tests.test_requirements.LibffiTestCase', 'pyjnius': 'tests.test_requirements.PyjniusTestCase', 'requests': 'tests.test_requirements.RequestsTestCase', 'sqlite3': 'tests.test_requirements.Sqlite3TestCase', 'kivy': 'tests.test_requirements.KivyTestCase', 'openssl': 'tests.test_requirements.OpensslTestCase'}
05-11 20:10:28.506 24939 25018 I python  : [WARNING] [Config      ] Older configuration version detected (0 instead of 21)
05-11 20:10:28.506 24939 25018 I python  : [WARNING] [Config      ] Upgrading configuration in progress.
05-11 20:10:28.513 24939 25018 I python  : [INFO   ] [Logger      ] Record log in /data/user/0/org.test.unit_tests_app/files/app/.kivy/logs/kivy_20-05-11_0.txt
05-11 20:10:28.513 24939 25018 I python  : [INFO   ] [Kivy        ] v1.11.1
05-11 20:10:28.513 24939 25018 I python  : [INFO   ] [Kivy        ] Installed at "/data/user/0/org.test.unit_tests_app/files/app/_python_bundle/site-packages/kivy/__init__.pyc"
05-11 20:10:28.514 24939 25018 I python  : [INFO   ] [Python      ] v3.8.1 (default, May 10 2020, 22:11:20) 
05-11 20:10:28.514 24939 25018 I python  : [Clang 8.0.2 (https://android.googlesource.com/toolchain/clang 40173bab62ec7462
05-11 20:10:28.514 24939 25018 I python  : [INFO   ] [Python      ] Interpreter at ""

05-11 20:10:29.427 24939 25018 I python  : [INFO   ] [Factory     ] 184 symbols loaded

05-11 20:10:29.935 24939 25018 I python  : [INFO   ] [Image       ] Providers: img_tex, img_dds, img_sdl2, img_gif (img_pil, img_ffpyplayer ignored)
05-11 20:10:30.004 24939 25018 I python  : [INFO   ] [Text        ] Provider: sdl2
05-11 20:10:30.110 24939 25018 I python  : [INFO   ] [Window      ] Provider: sdl2

05-11 20:10:30.912 24939 25018 I python  : [INFO   ] [GL          ] Using the "OpenGL ES 2" graphics system
05-11 20:10:30.917 24939 25018 I python  : [INFO   ] [GL          ] Backend used <sdl2>
05-11 20:10:30.920 24939 25018 I python  : [INFO   ] [GL          ] OpenGL version <b'OpenGL ES 3.2 v1.r12p1-04bet0.1bb9662be2ebee934dcbd7265c794a91'>
05-11 20:10:30.921 24939 25018 I python  : [INFO   ] [GL          ] OpenGL vendor <b'ARM'>
05-11 20:10:30.923 24939 25018 I python  : [INFO   ] [GL          ] OpenGL renderer <b'Mali-T830'>
05-11 20:10:30.924 24939 25018 I python  : [INFO   ] [GL          ] OpenGL parsed version: 3, 2
05-11 20:10:30.926 24939 25018 I python  : [INFO   ] [GL          ] Texture max size <8192>
05-11 20:10:30.928 24939 25018 I python  : [INFO   ] [GL          ] Texture max units <16>
05-11 20:10:31.037 24939 25018 I python  : [INFO   ] [Window      ] auto add sdl2 input provider
05-11 20:10:31.047 24939 25018 I python  : [INFO   ] [Window      ] virtual keyboard not allowed, single mode, not docked
05-11 20:10:31.105 24939 25018 I python  : on_tests_to_perform:  dict_keys(['numpy', 'libffi', 'pyjnius', 'requests', 'sqlite3', 'kivy', 'openssl'])
05-11 20:10:31.390 24939 25018 I python  : [INFO   ] [GL          ] NPOT texture support is available

05-11 20:10:31.649 24939 25018 I python  : [WARNING] [Base        ] Unknown <android> provider
05-11 20:10:31.650 24939 25018 I python  : [INFO   ] [Base        ] Start application main loop

05-11 20:10:32.432 24939 25018 I python  : Imported unittest
05-11 20:10:32.432 24939 25018 I python  : loading tests...
05-11 20:10:32.453 24939 25018 I python  : running unittest...


05-11 20:10:35.678 24939 25018 I python  : unittest result is:
05-11 20:10:35.678 24939 25018 I python  : ..............
05-11 20:10:35.678 24939 25018 I python  : ----------------------------------------------------------------------
05-11 20:10:35.678 24939 25018 I python  : Ran 14 tests in 3.223s
05-11 20:10:35.678 24939 25018 I python  : 
05-11 20:10:35.678 24939 25018 I python  : OK
05-11 20:10:35.678 24939 25018 I python  : 
05-11 20:10:35.678 24939 25018 I python  : Ran tests

I'll try to understand what's going on by looking at your stacktrace.

@opacam
Copy link
Member

opacam commented May 11, 2020

I will try again, it could be a transfer issue with my phone

@AndreMiras
Copy link
Member Author

That seems very unlikely to be a transfer issue to me. The stacktrace you shared is close to things I've touched so it could really be a regression on certain device. If you can reproduce on this PR, could you also check if current develop testapps are also crashing or if it's just this pull request?
In the meantime I'll try to see if I can reproduce with other devices. I may have idea how to fix, but first I want to be able to reproduce

@opacam
Copy link
Member

opacam commented May 11, 2020

Yea, I will try with latest artifact of develop branch...

BTW, the same error:

05-11 21:44:57.881 13760 13783 V PythonActivity: Ready to unpack
05-11 21:44:57.886 13760 13760 V PythonActivity: onResume()
05-11 21:44:57.886 13760 13760 V SDL     : onResume()
05-11 21:44:57.889 13760 13783 E AndroidRuntime: FATAL EXCEPTION: AsyncTask #1
05-11 21:44:57.889 13760 13783 E AndroidRuntime: Process: org.test.unit_tests_app, PID: 13760
05-11 21:44:57.889 13760 13783 E AndroidRuntime: java.lang.RuntimeException: An error occurred while executing doInBackground()
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at android.os.AsyncTask$4.done(AsyncTask.java:399)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:383)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at java.util.concurrent.FutureTask.setException(FutureTask.java:252)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at java.util.concurrent.FutureTask.run(FutureTask.java:271)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:289)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at java.lang.Thread.run(Thread.java:919)
05-11 21:44:57.889 13760 13783 E AndroidRuntime: Caused by: java.lang.RuntimeException: Can't create handler inside thread Thread[AsyncTask #1,5,main] that has not called Looper.prepare()
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at android.os.Handler.<init>(Handler.java:207)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at android.os.Handler.<init>(Handler.java:119)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at android.app.Activity.<init>(Activity.java:880)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at org.kivy.android.PythonActivityUtil.<init>(PythonActivityUtil.java:22)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at org.kivy.android.PythonActivity$UnpackFilesTask.doInBackground(PythonActivity.java:104)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at org.kivy.android.PythonActivity$UnpackFilesTask.doInBackground(PythonActivity.java:99)

For the record: I tested with Android 10 device

@opacam
Copy link
Member

opacam commented May 11, 2020

Develop is working, tested arm64-v8a artifact

@AndreMiras
Copy link
Member Author

OK thank you good to know it's only this PR and develop is still OK.
I'll try to find a device where to reproduce, there's no rush to merge this.
Thanks for the help 🍻

@opacam
Copy link
Member

opacam commented May 11, 2020

No problem, ping me when you need me 😉
Btw, I think that is a very good refactoring, thanks!!

@AndreMiras
Copy link
Member Author

Great I could reproduce on another device!
Will try to work on the fix now

@AndreMiras AndreMiras force-pushed the feature/share_unpack_data_java_code branch from e486766 to dfea735 Compare May 11, 2020 20:50
@AndreMiras
Copy link
Member Author

Got a working fix I think, but linting errors started popping.
So let's first merge #2191 and rebase

This helper class shares code to be reused between the `PythonActivity`
classes of the different bootstraps.
Also adds few minor adjustments to existing classes.
@AndreMiras AndreMiras force-pushed the feature/share_unpack_data_java_code branch from dfea735 to 5312595 Compare May 12, 2020 07:20
Copy link
Member

@opacam opacam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work!!, thanks!!

Note: tested the latest artifact for arm64-v8a without any issue 😄

@AndreMiras AndreMiras merged commit 0a76c4a into kivy:develop May 12, 2020
@AndreMiras AndreMiras deleted the feature/share_unpack_data_java_code branch May 12, 2020 11:32
lerela added a commit to chronolife-rd/python-for-android that referenced this pull request Jun 23, 2020
🔖 v2020.06.02

* Adds missing requests sub dependencies (kivy#2221)
* Bumps to Gradle 6.4.1 (kivy#2222)
* Bumps to Cython==0.29.19 (kivy#2220)
* Updates install and troubleshooting docs (kivy#2219)
* Bumps to Ubuntu 20.04 (kivy#2218)
* Attempt to improve the issue template (kivy#2217)
* Add `opencv_extras` recipe (kivy#2209)
* Split logic for build modes & debug symbols (kivy#2213)
* Troubleshoot SSL error (kivy#2205)
* Remove superfluous recipes fixes (kivy#2202)
* Add tests for hostpython3 recipe (kivy#2196)
* Fix for 'cannot find setuptools module' (kivy#2195)
* Rename `Hostpython3Recipe` class to camel case (kivy#2194)
* Fix `test_should_build` (kivy#2193)
* Add initial tests for python3 recipe (kivy#2192)
* PythonActivityUtil helper for unpacking data (kivy#2189)
* Fixes flake8 errors post update (kivy#2191)
* Share PythonUtil.java between bootstraps (kivy#2188)
* Java code linting using PMD 6.23.0 (kivy#2187)
* Deletes deprecated renpy Python{Activity,Service}.java (kivy#2186)
* Removes java concurrency/ folder (kivy#2185)
* Reuse common AssetExtract.java (kivy#2182)
* Use common Hardware.java (kivy#2183)
* Moves kamranzafar/ java directory to common/ (kivy#2184)
* Updates release documentation (kivy#2177)
* Fixes service only unittest loading (kivy#2181)
* Narrows some context manager scopes (kivy#2179)
* Downgrades to SDL2 2.0.9 (kivy#2180)
* Bump to SDL2 2.0.10 & extract .java from SDL2 tarball (kivy#2113)
* Adds pygame recipe (kivy#2164)
* Adds macOS install instructions (2165)
* Removed python2 support mention from README (kivy#2162)
* Adding more assets (kivy#2132)
* Get --add-source working for dirs in Gradle builds (kivy#2156)
* Fixes python build with macOS venv (kivy#2159)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants