Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buildozer is choosing odd venv, and using a different python? #1080

Closed
jaggzh opened this issue Apr 29, 2020 · 12 comments
Closed

buildozer is choosing odd venv, and using a different python? #1080

jaggzh opened this issue Apr 29, 2020 · 12 comments

Comments

@jaggzh
Copy link

jaggzh commented Apr 29, 2020

Okay, so on Debian Buster (Stable). I've tested a ton of things, and used different versions of buildozer, including directly from github master.zip.

My virtualenv is at ~/venv-3.6/ (but it has python 3.7 installed in it...)
It's active, and "which python" shows it there, and python --version shows 3.7.3.
(I don't personally care for this version; it's just what was at /usr/bin/python3 on this debian system).

So:

  1. What causes buildozer to run: virtualenv --python=python3 venv
  2. What is it doing when it then sources "venv/bin/activate"? Where did that come from? I went ahead and symlinked ~/venv to ~/venv-3.6 just for kicks, but it didn't change anything.
  3. Why is it then accessing the system /usr/lib/python3.7/runpy.py ?

I have no env vars for $PYTHONPATH or anything set. I have no clue what's going on here.

[INFO]:    <- directory context /home/jaggz/kivy/tests/splash/.buildozer/android/pla
tform/python-for-android                                                              
[INFO]:    Postbuilding android for armeabi-v7a                                       
[INFO]:    Postbuilding kivy for armeabi-v7a                                          
[INFO]:    # Installing pure Python modules                                           
[INFO]:    *** PYTHON PACKAGE / PROJECT INSTALL STAGE ***                             
[INFO]:    The requirements (certifi) don't have recipes, attempting to install them w
ith pip
[INFO]:    If this fails, it may mean that the module has compiled components and need
s a recipe.
[INFO]:    -> directory context /home/jaggz/kivy/tests/splash/.buildozer/android/pla
tform/build-armeabi-v7a/build
[INFO]:    -> running virtualenv --python=python3 venv                                
[INFO]:    Upgrade pip to latest version                                              
[INFO]:    -> running bash -c source venv/bin/activate && pip install -U pip          
Traceback (most recent call last):                                                    
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main                
    "__main__", mod_spec)                                                             
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code                           
    exec(code, run_globals)                                                           
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/toolchain.py", line 1199, in <module>                               
    main()                                                                            
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/entrypoints.py", line 18, in main                                   
    ToolchainCL()                                                                     
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/toolchain.py", line 671, in __init__                                
    getattr(self, args.subparser_name.replace('-', '_'))(args)                        
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/toolchain.py", line 155, in wrapper_func                            
    build_dist_from_args(ctx, dist, args)                                             
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/toolchain.py", line 209, in build_dist_from_args                    
    args, "ignore_setup_py", False                                                    
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/build.py", line 605, in build_recipes                               
    ignore_setup_py=ignore_project_setup_py                                           
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/build.py", line 791, in run_pymodules_install                       
    ), _env=copy.copy(base_env))                                                      
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/logger.py", line 179, in shprint                                    
    output = command(*args, **kwargs)                                                 
  File "/home/jaggz/venv-3.6/lib/python3.7/site-packages/sh.py", line 1474, in __ca$
l__                                                                                   
    extracted_call_args, kwargs = self._extract_call_args(kwargs)                     
  File "/home/jaggz/venv-3.6/lib/python3.7/site-packages/sh.py", line 1377, in _ext$
act_call_args                                                                         
    raise TypeError("Invalid special arguments:\n\n%s\n" % exc_msg)                   
TypeError: Invalid special arguments:                                                 
                                                                                      
  'env': env must be a dict. Got environ({'PROFILEHOME': '',
@AndreMiras
Copy link
Member

Yes this just got fixed upstream via kivy/python-for-android#2160
Please buildozer android update

@jaggzh
Copy link
Author

jaggzh commented Apr 29, 2020

$ buildozer android update
Results in the following. Separate issue?

[INFO]:    Postbuilding android for armeabi-v7a
[INFO]:    Postbuilding kivy for armeabi-v7a
[INFO]:    # Installing pure Python modules
[INFO]:    *** PYTHON PACKAGE / PROJECT INSTALL STAGE ***
[INFO]:    The requirements (certifi) don't have recipes, attempting to install them with pip
[INFO]:    If this fails, it may mean that the module has compiled components and needs a recipe.
[INFO]:    -> directory context /home/jaggz/kivy/tests/splash/.buildozer/android/platform/build-armeabi-v7a/build
[INFO]:    -> running virtualenv --python=python3 venv

           working: Already using interpreter /home/jaggz/venv-3.6/bin/python3
           working: Using base prefix '/usr'
           working: New python executable in /home/jaggz/kivy/tes...(and 78 more)
           working: Not overwriting existing python script /home/br...(and 207 more)
           working: Installing setuptools, pip, wheel...
           working: done.
[INFO]:    Upgrade pip to latest version
[INFO]:    -> running bash -c source venv/bin/activate && pip install -U pip
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 1199, in <module>
    main()
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/entrypoints.py", line 18, in main
    ToolchainCL()
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 671, in __init__
    getattr(self, args.subparser_name.replace('-', '_'))(args)
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 155, in wrapper_func
    build_dist_from_args(ctx, dist, args)
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 209, in build_dist_from_args
    args, "ignore_setup_py", False
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/build.py", line 605, in build_recipes
    ignore_setup_py=ignore_project_setup_py
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/build.py", line 791, in run_pymodules_install
    ), _env=copy.copy(base_env))
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/logger.py", line 179, in shprint
    output = command(*args, **kwargs)
  File "/home/jaggz/venv-3.6/lib/python3.7/site-packages/sh.py", line 1474, in __call__
    extracted_call_args, kwargs = self._extract_call_args(kwargs)
  File "/home/jaggz/venv-3.6/lib/python3.7/site-packages/sh.py", line 1377, in _extract_call_args
    raise TypeError("Invalid special arguments:\n\n%s\n" % exc_msg)
TypeError: Invalid special arguments:
{p...}
# Command failed: /home/jaggz/venv-3.6/bin/python3 -m pythonforandroid.toolchain create --dist_name=myapp --bootstrap=sdl2 --requirements=python3,kivy --arch armeabi-v7a --copy-libs --color=always --storage-dir="/home/jaggz/kivy/tests/splash/.buildozer/android/platform/build-armeabi-v7a" --ndk-api=21
# ENVIRONMENT:
#     SHELL = '/bin/bash'

@AndreMiras
Copy link
Member

Yes if p4a was installed from pip, try removing it first. Also maybe manually delete .buildozer/ directory from your project dir rm -rf ~/kivy/tests/splash/.buildozer to be sure to be sure and try again

@jaggzh
Copy link
Author

jaggzh commented Apr 29, 2020

Okay, still getting those errors:

[INFO]:    Postbuilding android for armeabi-v7a
[INFO]:    Postbuilding kivy for armeabi-v7a
[INFO]:    # Installing pure Python modules
[INFO]:    *** PYTHON PACKAGE / PROJECT INSTALL STAGE ***
[INFO]:    The requirements (certifi) don't have recipes, attempting to install them with pip
[INFO]:    If this fails, it may mean that the module has compiled components and needs a recipe.
[INFO]:    -> directory context /home/jaggz/kivy/tests/splash/.buildozer/android/platform/build-armeabi-v7a/build
[INFO]:    -> running virtualenv --python=python3 venv

           working: Already using interpreter /home/jaggz/venv-3.6/bin/python3
           working: Using base prefix '/usr'
           working: New python executable in /home/jaggz/kivy/tes...(and 78 more)
           working: Also creating executable in /home/jaggz/kivy/...(and 80 more)
           working: Installing setuptools, pip, wheel...
           working: done.

[INFO]:    Upgrade pip to latest version
[INFO]:    -> running bash -c source venv/bin/activate && pip install -U pip
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 1199, in <module>
    main()
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/entrypoints.py", line 18, in main
    ToolchainCL()
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 671, in __init__
    getattr(self, args.subparser_name.replace('-', '_'))(args)
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 155, in wrapper_func
    build_dist_from_args(ctx, dist, args)
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 209, in build_dist_from_args
    args, "ignore_setup_py", False
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/build.py", line 605, in build_recipes
    ignore_setup_py=ignore_project_setup_py
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/build.py", line 791, in run_pymodules_install
    ), _env=copy.copy(base_env))
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/logger.py", line 179, in shprint
    output = command(*args, **kwargs)
  File "/home/jaggz/venv-3.6/lib/python3.7/site-packages/sh.py", line 1474, in __call__
    extracted_call_args, kwargs = self._extract_call_args(kwargs)
  File "/home/jaggz/venv-3.6/lib/python3.7/site-packages/sh.py", line 1377, in _extract_call_args
    raise TypeError("Invalid special arguments:\n\n%s\n" % exc_msg)
TypeError: Invalid special arguments:

{p...}

# Command failed: /home/jaggz/venv-3.6/bin/python3 -m pythonforandroid.toolchain create --dist_name=myapp --bootstrap=sdl2 --requirements=python3,kivy --arch armeabi-v7a --copy-libs --color=always --storage-dir="/home/jaggz/kivy/tests/splash/.buildozer/android/platform/build-armeabi-v7a" --ndk-api=21
# ENVIRONMENT:
#     SHELL = '/bin/bash'

@AndreMiras
Copy link
Member

Which p4a.branch are you in? You should usually share your buildozer.spec file too

@jaggzh
Copy link
Author

jaggzh commented Apr 29, 2020

[Edit]
"which python3.7" shows /usr/bin/python3.7

[Original post]
After pip removing python-for-android, I did not install it manually again. I just went right to buildozer android update.
Note that "which python3" shows:
/home/brandon/venv-3.6/bin/python3
"which pip":
/home/brandon/venv-3.6/bin/pip
In that log the error shows it's using:
/usr/lib/python3.7/runpy.py

My buildozer.spec:

[app]

# (str) Title of your application
title = My Application

# (str) Package name
package.name = myapp

# (str) Package domain (needed for android/ios packaging)
package.domain = org.test

# (str) Source code where the main.py live
source.dir = .

# (list) Source files to include (let empty to include all the files)
source.include_exts = py,png,jpg,kv,atlas,gif,mp4,mkv

# (list) List of inclusions using pattern matching
#source.include_patterns = assets/*,images/*.png

# (list) Source files to exclude (let empty to not exclude anything)
#source.exclude_exts = spec

# (list) List of directory to exclude (let empty to not exclude anything)
#source.exclude_dirs = tests, bin

# (list) List of exclusions using pattern matching
#source.exclude_patterns = license,images/*/*.jpg

# (str) Application versioning (method 1)
version = 0.1

# (str) Application versioning (method 2)
# version.regex = __version__ = ['"](.*)['"]
# version.filename = %(source.dir)s/main.py

# (list) Application requirements
# comma separated e.g. requirements = sqlite3,kivy
#requirements = python3,kivy,ffpyplayer,ffpyplayer_codecs
requirements = python3,kivy
#requirements = python3,kivy==1.11.0

# (str) Custom source folders for requirements
# Sets custom source for any requirements with recipes
# requirements.source.kivy = ../../kivy

# (list) Garden requirements
#garden_requirements =

# (str) Presplash of the application
#presplash.filename = %(source.dir)s/data/presplash.png

# (str) Icon of the application
#icon.filename = %(source.dir)s/data/icon.png

# (str) Supported orientation (one of landscape, sensorLandscape, portrait or all)
orientation = portrait

# (list) List of service to declare
#services = NAME:ENTRYPOINT_TO_PY,NAME2:ENTRYPOINT2_TO_PY

#
# OSX Specific
#

#
# author = © Copyright Info

# change the major version of python used by the app
osx.python_version = 3

# Kivy version to use
osx.kivy_version = 1.9.1

#
# Android specific
#

# (bool) Indicate if the application should be fullscreen or not
fullscreen = 0

# (string) Presplash background color (for new android toolchain)
# Supported formats are: #RRGGBB #AARRGGBB or one of the following names:
# red, blue, green, black, white, gray, cyan, magenta, yellow, lightgray,
# darkgray, grey, lightgrey, darkgrey, aqua, fuchsia, lime, maroon, navy,
# olive, purple, silver, teal.
#android.presplash_color = #FFFFFF

# (list) Permissions
#android.permissions = INTERNET

# (int) Target Android API, should be as high as possible.
#android.api = 27

# (int) Minimum API your APK will support.
#android.minapi = 21

# (int) Android SDK version to use
#android.sdk = 20

# (str) Android NDK version to use
#android.ndk = 17c

# (int) Android NDK API to use. This is the minimum API your app will support, it should usually match android.minapi.
#android.ndk_api = 21

# (bool) Use --private data storage (True) or --dir public storage (False)
#android.private_storage = True

# (str) Android NDK directory (if empty, it will be automatically downloaded.)
#android.ndk_path = /home/jaggz/Android/Sdk/ndk-bundle

# (str) Android SDK directory (if empty, it will be automatically downloaded.)
#android.sdk_path = /home/jaggz/Android/Sdk/

# (str) ANT directory (if empty, it will be automatically downloaded.)
#android.ant_path = /home/jaggz/projects/android-studio/lib/ant/

# (bool) If True, then skip trying to update the Android sdk
# This can be useful to avoid excess Internet downloads or save time
# when an update is due and you just want to test/build your package
# android.skip_update = False

# (bool) If True, then automatically accept SDK license
# agreements. This is intended for automation only. If set to False,
# the default, you will be shown the license when first running
# buildozer.
# android.accept_sdk_license = False

# (str) Android entry point, default is ok for Kivy-based app
#android.entrypoint = org.renpy.android.PythonActivity

# (str) Android app theme, default is ok for Kivy-based app
# android.apptheme = "@android:style/Theme.NoTitleBar"

# (list) Pattern to whitelist for the whole project
#android.whitelist =

# (str) Path to a custom whitelist file
#android.whitelist_src =

# (str) Path to a custom blacklist file
#android.blacklist_src =

# (list) List of Java .jar files to add to the libs so that pyjnius can access
# their classes. Don't add jars that you do not need, since extra jars can slow
# down the build process. Allows wildcards matching, for example:
# OUYA-ODK/libs/*.jar
#android.add_jars = foo.jar,bar.jar,path/to/more/*.jar

# (list) List of Java files to add to the android project (can be java or a
# directory containing the files)
#android.add_src =

# (list) Android AAR archives to add (currently works only with sdl2_gradle
# bootstrap)
#android.add_aars =

# (list) Gradle dependencies to add (currently works only with sdl2_gradle
# bootstrap)
#android.gradle_dependencies =

# (list) add java compile options
# this can for example be necessary when importing certain java libraries using the 'android.gradle_dependencies' option
# see https://developer.android.com/studio/write/java8-support for further information
# android.add_compile_options = "sourceCompatibility = 1.8", "targetCompatibility = 1.8"

# (list) Gradle repositories to add {can be necessary for some android.gradle_dependencies}
# please enclose in double quotes 
# e.g. android.gradle_repositories = "maven { url 'https://kotlin.bintray.com/ktor' }"
#android.add_gradle_repositories =

# (list) packaging options to add 
# see https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.PackagingOptions.html
# can be necessary to solve conflicts in gradle_dependencies
# please enclose in double quotes 
# e.g. android.add_packaging_options = "exclude 'META-INF/common.kotlin_module'", "exclude 'META-INF/*.kotlin_module'"
#android.add_gradle_repositories =

# (list) Java classes to add as activities to the manifest.
#android.add_activites = com.example.ExampleActivity

# (str) OUYA Console category. Should be one of GAME or APP
# If you leave this blank, OUYA support will not be enabled
#android.ouya.category = GAME

# (str) Filename of OUYA Console icon. It must be a 732x412 png image.
#android.ouya.icon.filename = %(source.dir)s/data/ouya_icon.png

# (str) XML file to include as an intent filters in <activity> tag
#android.manifest.intent_filters =

# (str) launchMode to set for the main activity
#android.manifest.launch_mode = standard

# (list) Android additional libraries to copy into libs/armeabi
#android.add_libs_armeabi = libs/android/*.so
#android.add_libs_armeabi_v7a = libs/android-v7/*.so
#android.add_libs_arm64_v8a = libs/android-v8/*.so
#android.add_libs_x86 = libs/android-x86/*.so
#android.add_libs_mips = libs/android-mips/*.so

# (bool) Indicate whether the screen should stay on
# Don't forget to add the WAKE_LOCK permission if you set this to True
#android.wakelock = False

# (list) Android application meta-data to set (key=value format)
#android.meta_data =

# (list) Android library project to add (will be added in the
# project.properties automatically.)
#android.library_references =

# (list) Android shared libraries which will be added to AndroidManifest.xml using <uses-library> tag
#android.uses_library =

# (str) Android logcat filters to use
#android.logcat_filters = *:S python:D

# (bool) Copy library instead of making a libpymodules.so
#android.copy_libs = 1

# (str) The Android arch to build for, choices: armeabi-v7a, arm64-v8a, x86, x86_64
android.arch = armeabi-v7a

#
# Python for android (p4a) specific
#

# (str) python-for-android fork to use, defaults to upstream (kivy)
#p4a.fork = kivy

# (str) python-for-android branch to use, defaults to master
#p4a.branch = master

# (str) python-for-android git clone directory (if empty, it will be automatically cloned from github)
#p4a.source_dir =

# (str) The directory in which python-for-android should look for your own build recipes (if any)
#p4a.local_recipes =

# (str) Filename to the hook for p4a
#p4a.hook =

# (str) Bootstrap to use for android builds
# p4a.bootstrap = sdl2

# (int) port number to specify an explicit --port= p4a argument (eg for bootstrap flask)
#p4a.port =


#
# iOS specific
#

# (str) Path to a custom kivy-ios folder
#ios.kivy_ios_dir = ../kivy-ios
# Alternately, specify the URL and branch of a git checkout:
ios.kivy_ios_url = https://github.com/kivy/kivy-ios
ios.kivy_ios_branch = master

# Another platform dependency: ios-deploy
# Uncomment to use a custom checkout
#ios.ios_deploy_dir = ../ios_deploy
# Or specify URL and branch
ios.ios_deploy_url = https://github.com/phonegap/ios-deploy
ios.ios_deploy_branch = 1.7.0

# (str) Name of the certificate to use for signing the debug version
# Get a list of available identities: buildozer ios list_identities
#ios.codesign.debug = "iPhone Developer: <lastname> <firstname> (<hexstring>)"

# (str) Name of the certificate to use for signing the release version
#ios.codesign.release = %(ios.codesign.debug)s


[buildozer]

# (int) Log level (0 = error only, 1 = info, 2 = debug (with command output))
log_level = 2

# (int) Display warning if buildozer is run as root (0 = False, 1 = True)
warn_on_root = 1

# (str) Path to build artifact storage, absolute or relative to spec file
# build_dir = ./.buildozer

# (str) Path to build output (i.e. .apk, .ipa) storage
# bin_dir = ./bin

#    -----------------------------------------------------------------------------
#    List as sections
#
#    You can define all the "list" as [section:key].
#    Each line will be considered as a option to the list.
#    Let's take [app] / source.exclude_patterns.
#    Instead of doing:
#
#[app]
#source.exclude_patterns = license,data/audio/*.wav,data/images/original/*
#
#    This can be translated into:
#
#[app:source.exclude_patterns]
#license
#data/audio/*.wav
#data/images/original/*
#


#    -----------------------------------------------------------------------------
#    Profiles
#
#    You can extend section / key with a profile
#    For example, you want to deploy a demo version of your application without
#    HD content. You could first change the title to add "(demo)" in the name
#    and extend the excluded directories to remove the HD content.
#
#[app@demo]
#title = My Application (demo)
#
#[app:source.exclude_patterns@demo]
#images/hd/*
#
#    Then, invoke the command line with the "demo" profile:
#
#buildozer --profile demo android debug

@AndreMiras
Copy link
Member

Try with p4a.branch = develop because the fix is on p4a develop

@jaggzh
Copy link
Author

jaggzh commented Apr 29, 2020

Okay, I did:
$ pip install git+git://github.com/kivy/python-for-android.git@develop
$ rm -rf .buildozer
$ buildozer android update

It actually gives me the same TypeError again, and the env must be a dict error returned.

  File "/home/jaggz/venv-3.6/lib/python3.7/site-packages/sh.py", line 1377, in _extr
act_call_args                                                                         
    raise TypeError("Invalid special arguments:\n\n%s\n" % exc_msg)
TypeError: Invalid special arguments:

  'env': env must be a dict.

@AndreMiras
Copy link
Member

That's not what I meant. I mean from your buildozer.spec file. Use p4a.branch = develop

@nlhnt
Copy link

nlhnt commented May 1, 2020

Changing p4a.branch = develop in buildozer.spec helped me today with exactly same issue.

@needyamin
Copy link

Thanks @AndreMiras

@dcvcv
Copy link

dcvcv commented Feb 3, 2022

How to remove the packaging in venv, that venv is completely unnecessary, there are many mistakes! The unit cannot be used at all. please solve

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants