Installation of pyobjc-Core makes watchdog unusable on non MacOS hardware

[jperkelens]

Collecting pyobjc-core==6.1

  Downloading https://artifactory.us-east-1.bamgrid.net/api/pypi/dataengineering-pypi/packages/packages/3a/80/0cdb2130b86a984e4765beb93135aa9c2120fd64be1a27e1c7cb31c719bc/pyobjc-core-6.1.tar.gz (791 kB)

    ERROR: Command errored out with exit status 1:

     command: /databricks/python3/bin/python3.7 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-09k75p2e/pyobjc-core/setup.py'"'"'; __file__='"'"'/tmp/pip-install-09k75p2e/pyobjc-core/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' --no-user-cfg egg_info --egg-base /tmp/pip-install-09k75p2e/pyobjc-core/pip-egg-info

         cwd: /tmp/pip-install-09k75p2e/pyobjc-core/

    Complete output (2 lines):

    running egg_info

    error: PyObjC requires macOS to build

    ----------------------------------------

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

script returned exit code 1

[BoboTiG]

Unfortunately, I cannot reproduce yet.
Could you tell me what returns:

import os
import sys
from distutils.util import get_platform

print(sys.platform)
print(os.getenv("_PYTHON_HOST_PLATFORM"))
print(get_platform())
print(os.name)
print(os.uname())

And what is your OS and OS version?

[BoboTiG]

Could you try to install the version from that branch: https://github.com/gorakhargosh/watchdog/tree/fix-use-sys-platform?

[jperkelens]

The output of the commands is:

Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import sys
>>> from distutils.util import get_platform
>>>
>>> print(sys.platform)
linux2
>>> print(os.getenv("_PYTHON_HOST_PLATFORM"))
None
>>> print(get_platform())
linux-x86_64
>>> print(os.name)
posix
>>> print(os.uname())
('Linux', '4969f8496763', '4.9.184-linuxkit', '#1 SMP Tue Jul 2 22:58:16 UTC 2019', 'x86_64')
>>>

The contents of the /etc/os-release file are:

# cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.4 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.4 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

[jperkelens]

installing from the branch you mentioned seems to have worked:

# git clone https://github.com/gorakhargosh/watchdog.git
Cloning into 'watchdog'...
remote: Enumerating objects: 35, done.
remote: Counting objects: 100% (35/35), done.
remote: Compressing objects: 100% (28/28), done.
remote: Total 6532 (delta 16), reused 16 (delta 7), pack-reused 6497
Receiving objects: 100% (6532/6532), 1.49 MiB | 1.47 MiB/s, done.
Resolving deltas: 100% (4296/4296), done.
Checking connectivity... done.
# cd watchdog
# git co fix-use-sys-platform
git: 'co' is not a git command. See 'git --help'.

Did you mean one of these?
        commit
        clone
        log
# git checkout fix-use-sys-platform
Branch fix-use-sys-platform set up to track remote branch fix-use-sys-platform from origin.
Switched to a new branch 'fix-use-sys-platform'
# pip install .
Processing /proj/watchdog
Requirement already satisfied: pathtools>=0.1.1 in /databricks/python3/lib/python3.7/site-packages (from watchdog==0.10.1) (0.1.2)
Building wheels for collected packages: watchdog
  Building wheel for watchdog (setup.py) ... done
  Created wheel for watchdog: filename=watchdog-0.10.1-cp37-none-any.whl size=73440 sha256=33fccd6797d042e4cae828d49264ba86ae17b36ccbb9c704740b72c8088995e7
  Stored in directory: /tmp/pip-ephem-wheel-cache-xdb4ajfs/wheels/81/44/af/0aa0091701a91f0b9c476c731521faee5ea94f30fdede49318
Successfully built watchdog
Installing collected packages: watchdog
  Found existing installation: watchdog 0.9.0
    Uninstalling watchdog-0.9.0:
      Successfully uninstalled watchdog-0.9.0
Successfully installed watchdog-0.10.1

[BoboTiG]

Great news! Even if I do not understand how it happened :D

[bhargavrpatel]

@BoboTiG We are seeing this issue right not as well, so wondering when 0.10.1 will be released.
Edit: Looks like this version will be released tomorrow.

[robvdl]

I've got something similar to this but I am not on a Mac.

I am working on a project right now that still uses Python 3.4 (yes I know, beyond my control) and ever since the release of watchdog 0.10.0 on January 27 our Travis CI jobs have started failing.

That is because for some weird reason it's started thinking Travis CI is a Mac and it tries to install PyLibObjC which then blows up because this file is never goingo exist on Linux:

FileNotFoundError: [Errno 2] No such file or directory: '/System/Library/CoreServices/SystemVersion.plist'

I believe however it shouldn't be installing PyLibObjC at all on Linux, only on Mac.

For now I am pinning watchdog to the previous version until this is resolved. It does not fail on Python 3.6 as we run CI on various Python versions, only 3.4 is actually doing this.

[bhargavrpatel]

@robvdl Looks like 0.10.1 will be released tomorrow. So you can probably unpin your dependencies after that.

Addendum: An alternative would be to pn the pyobjc requirements in your Pipfile specificing the version marker. This is similar to Constraints files.

# This is being installed specific for drawin and a patch around the incorrect
# requirements declared in `watchdog`. This is resolved in watchdog==0.10.1
# which is currently not released.
pyobjc-core = {version="*", sys_platform = "== 'darwin'"}
pyobjc-framework-cocoa = {version="*", sys_platform = "== 'darwin'"}
pyobjc-framework-fsevents = {version="*", sys_platform = "== 'darwin'"}

[robvdl]

Ah good, I was just re-reading over the previous comments and realised it was the same issue and a fix was already made thanks. I'll probably just wait it out, no point adding a pin for 1 day, it's weekend anyway.

[BoboTiG]

The release 0.10.1 is out :)

[robvdl]

Hate to be the bearer of bad news but unfortunately this issue still exists when installing watchdog==0.10.1 on Travis CI using Trusty/python 3.4 (though 3.6 doesn't do it).

Collecting pyobjc-framework-Cocoa>=4.2.2 (from watchdog==0.10.1)
  Downloading https://files.pythonhosted.org/packages/ff/86/acf828be23970164440c9a3591ae4ace703f707a348f544941272ace52ff/pyobjc-framework-Cocoa-5.3.tar.gz (3.8MB)
    ERROR: Complete output from command python setup.py egg_info:
    ERROR: Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-wheel-c8o6gsxs/pyobjc-framework-Cocoa/setup.py", line 24, in <module>
        extra_link_args=['-framework', 'CoreFoundation']),
      File "/tmp/pip-wheel-c8o6gsxs/pyobjc-framework-Cocoa/pyobjc_setup.py", line 392, in Extension
        os_level = get_os_level()
      File "/tmp/pip-wheel-c8o6gsxs/pyobjc-framework-Cocoa/pyobjc_setup.py", line 206, in get_os_level
        pl = plistlib.readPlist('/System/Library/CoreServices/SystemVersion.plist')
      File "/opt/python/3.4.6/lib/python3.4/plistlib.py", line 162, in readPlist
        with _maybe_open(pathOrFile, 'rb') as fp:
      File "/opt/python/3.4.6/lib/python3.4/contextlib.py", line 59, in __enter__
        return next(self.gen)
      File "/opt/python/3.4.6/lib/python3.4/plistlib.py", line 120, in _maybe_open
        with open(pathOrFile, mode) as fp:
    FileNotFoundError: [Errno 2] No such file or directory: '/System/Library/CoreServices/SystemVersion.plist'

I still am finding that installing watchdog==0.10.1 on Travis (trusty) and python 3.4 is thinking it's a Mac and trying to install PyObjC.

I can understand you may want to drop support for Trusty, if that is the case I can pin the old version of watchdog temporarily until we can finalise moving our client to Bionic, it's in progress anyway just slower than I would hope.

The weird thing is in a Trusty vagrant VM it won't do it, tried rebuilding the Vagrant VM and it's absolutely fine. So it seems to be isolated to Travis CI and Python 3.4?

[somewacko]

Another issue related to this is that because the system platform is determined at install-time, this makes pinning dependencies (e.g with pip freeze or pip-tools) across platforms difficult.

For example if I'm on macOS and do:

pip install watchdog ...
pip freeze > requirements.txt

requirements.txt would have pyobjc-core, which would break installs for non-macOS users.

Our solution was to add ; sys_platform == 'darwin' to the requirements.txt manually:

pyobjc-core; sys_platform == 'darwin'                # via watchdog
pyobjc-framework-cocoa; sys_platform == 'darwin'     # via watchdog
pyobjc-framework-fsevents; sys_platform == 'darwin'  # via watchdog

(perhaps this is pyobjc's fault though, I feel like it should fail at run-time, not install-time)

[somewacko]

Also, would #623 fix this?

[BoboTiG]

Also, would #623 fix this?

Not at all. As PyObjC will always required.

[bhargavrpatel]

We are using Pipfile, and have opted to pin pyobjc dependencies with the sys_platform constraint. This resolves the symptom, not the cause. I suggest you do the same while a long term solution is rolled out.

…

On Sat, Feb 1, 2020 at 5:23 AM Mickaël Schoentgen ***@***.***> wrote: Also, would #623 <#623> fix this? Not at all. As PyObjC will always required. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#618?email_source=notifications&email_token=AAG6QNJ3REUZN2MUA65XNJ3RAVERPA5CNFSM4KMTIEKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKQZRUI#issuecomment-581015761>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG6QNIAX7HRUNOEKF7NUP3RAVERPANCNFSM4KMTIEKA> .

[BoboTiG]

This is already what we do (since the beginning), see:

watchdog/setup.cfg

Lines 4 to 5 in 8f7e051

	pyobjc-framework-Cocoa >= 4.2.2 ; sys_platform == "darwin"
	pyobjc-framework-FSEvents >= 4.2.2 ; sys_platform == "darwin"

[somewacko]

Yeah thinking through this more I don't see many solutions that watchdog could implement.

I think pyobjc shouldn't fail installs on non-macOS platforms. I opened an issue there if anyone wants to follow/contribute: https://bitbucket.org/ronaldoussoren/pyobjc/issues/291/install-fails-on-non-macos-platforms

[robvdl]

Also @zo7 in my particular case I'm pretty sure it could be failing to parse this on python 3.4 because of the version of setuptools and pip in use:

pyobjc-framework-Cocoa >= 4.2.2 ; sys_platform == "darwin" 
pyobjc-framework-FSEvents >= 4.2.2 ; sys_platform == "darwin" 

Python 3.4 will only take upto a particular version of setuptools and pip and it's quite possible that those versions are not parsing this syntax yet?

I am just thinking because this is working fine on Python 3.6, we have a CI job with a build matrix and only 3.4 is actually failing here but 3.6 is passing.

There for I was suggesting an option could be to say you no longer support python 3.4 for the 0.10.x version, it's no longer in support anyway and many projects have done this already.

People still using Python 3.4 are likely already transitioning of it anyway, we can just pin 0.9.0 right now and it's fine, I wouldn't worry if Python 3.4 support was actually dropped officially in watchdog.