pip install fails on macOS Big Sur 11.2 (M1 2020): No matching distribution found for ttfautohint-py

[eliheuer]

On macOS Big Sur 11.2 (M1 2020) in a Python3 v3.9.2 virtual environment:

$ pip install ttfautohint-py
ERROR: Could not find a version that satisfies the requirement ttfautohint-py
ERROR: No matching distribution found for ttfautohint-py

This issue prevents installation of gftools.

[chrissimpkins]

I think that this is not supported on macOS 11.0+ yet?

#6
googlefonts/gftools#298 (comment)

[eliheuer]

Yes, thanks! As a workaround I installed ttfautohint manually, then brew install libgit2, and it works on macOS 11.2 for me.

[eliheuer]

I'm still having an issue with this when installing gftools in --editable mode though: googlefonts/gftools#352

[anthrotype]

reminder to self: check out how Pillow managed to build arm64 wheels for Apple M1 silicon:
https://github.com/python-pillow/pillow-wheels/pull/185/files

[felipesanches]

we noticed that FontBakery had zero installs form PyPI during the past week and then figured out that this ttfautohint-py is what is breaking the package installation.

source: https://pypistats.org/packages/fontbakery

[chrissimpkins]

It looks like there may have been a data collection hiccup during the past week:

https://pypistats.org/packages/requests

[felipesanches]

oh! That's interesting... so there could be 2 problems at the same time and it is harder to know which is which :-P

[felipesanches]

The fontbakery download stats are updated now and I do not see that dip anymore. It seems that PyPI itself was having problems collections (or displaying) the data during that week.

[felipesanches]

@eliheuer, are you still unable to install fontbakery?

[felipesanches]

As I mentioned before, the fontbakery installation issue may be twofold. I reckon that ttfa-py may still need fixing.

[eliheuer]

@felipesanches Results from testing this today:

On a 2020 MacBook Air (Intel, not M1 Apple Silicon) running macOS Big Sur v11.2, in a new venv, pip install fontbakery gives me this error:

ERROR: Could not find a version that satisfies the requirement ttfautohint-py (from fontbakery) (from versions: none)
ERROR: No matching distribution found for ttfautohint-py (from fontbakery)

On a 2020 M1 Apple Silicon Mac Mini, running Big Sur v11.2, it installs, but I get version 0.5.0,

On a 2013 Air running macOS Mojave v10.14, it installed version 0.7.34, which is what I want.

I'm interested in finding out why I get an old outdated version when installing on the Mac Mini M1.

[m4rc1e]

I'm willing to dedicate some time to this. I haven't updated to Big Sur yet but fixing this should give me an incentive.

[chrissimpkins]

I'm willing to dedicate some time to this. I haven't updated to Big Sur yet but fixing this should give me an incentive.

I think that this may be the current state of the problem on Big Sur @m4rc1e

[chrissimpkins]

sorry, link: #6

[eliheuer]

Thank you Marc, I'm happy to test this.

[anthrotype]

It looks like recent versions of cibuildwheel support building wheels for Apple Silicon architecture, see https://cibuildwheel.readthedocs.io/en/stable/faq/#apple-silicon

[anthrotype]

for reference, we already use cibuildwheel (instead of multibuild) in https://github.com/harfbuzz/uharfbuzz/blob/master/.github/workflows/ci.yml

[m4rc1e]

Cool! I unfortunately I cannot reproduce this since I have an Intel mac. @eliheuer we can attempt this on you mac if you'd like at some point this week.

[anthrotype]

the issue is that not only that we don't build wheels for Apple Silicon, but also we currently don't publish the source distribution to PyPI, thus if one attempts to install on one of the platforms for which we don't publish a pre-compiled wheel, pip can't fetch a source distribution and one can't even attempt to build from source with a direct pip install ttfautohint-py.

This is because our current build setup relies on git submodules to fetch ttfautohint, freetype and harfbuzz:

ttfautohint-py/.travis.yml

Lines 71 to 73 in 3cd84b5

	# To build from source, the cloned git repository is required since
	# we are using git submodules. Thus do not upload a sdist package, only wheels.
	# TODO uncomment this once we can build from sdist.

We should be able to use setuptools_git_ls_files (like I did for skia-pathops, for example) such that we include all the files tracked by all the git submodules in the source distribution's manifest. This way we can create a zip file containg all the sources in this and all the submodules' repositories, upload that to PyPI, and then one can pip install ttfautohint-py from soruce if a wheel for one's platform/architecture is missing (provided all other required build tools are also present).

While we are at it, we also need to drop Travis CI in favor of Github Actions.

[m4rc1e]

SGTM, I've finally just encountered this error on my system so I'm happy to go ahead and fix this.

[eliheuer]

Marc and I were able to get this to install on a Mac mini (M1, 2020), Big Sur 11.2, there is a step-by-step here: https://gist.github.com/eliheuer/7aebbf72e58aca48c32592574a4f6fe0

[eliheuer]

But there is still an issue when I try to import ttfautohint from a python repl, Marc is working on it and might get a M1 to do so:

Python 3.9.5 (default, May  3 2021, 19:12:05)
[Clang 12.0.5 (clang-1205.0.22.9)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ttfautohint
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/eli/Py/venvs/ttfautohint-py-test/lib/python3.9/site-packages/ttfautohint/__init__.py", line 141, in <module>
    libttfautohint = TALibrary()
  File "/Users/eli/Py/venvs/ttfautohint-py-test/lib/python3.9/site-packages/ttfautohint/__init__.py", line 46, in __init__
    self.lib = lib = cdll.LoadLibrary(path, **kwargs)
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py", line 452, in LoadLibrary
    return self._dlltype(name)
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(/Users/eli/Py/venvs/ttfautohint-py-test/lib/python3.9/site-packages/ttfautohint/libttfautohint.dylib, 6): no suitable image found.  Did find:
	/Users/eli/Py/venvs/ttfautohint-py-test/lib/python3.9/site-packages/ttfautohint/libttfautohint.dylib: mach-o, but wrong architecture
	/Users/eli/Py/venvs/ttfautohint-py-test/lib/python3.9/site-packages/ttfautohint/libttfautohint.dylib: mach-o, but wrong architecture

[anthrotype]

maybe try ask @folengo as well

[m4rc1e]

Just a quick update.

I have a wip branch which can build ttfa from source for everything other than M1 Silicon. I plan to use Scaleway's M1 as a service once an instance is free. This means I don't need to hassle Eli to test my builds. Will keep you all posted.

[anthrotype]

@eliheuer @m4rc1e since you guys have an M1 mac at hand, can you please try to download and install the following wheel (built using our current Github Actions workflow) and see if you can, not only install it, but also if you can import and run the module?

ttfautohint_py-0.4.3.post1-py2.py3-none-macosx_10_9_universal2.whl.zip

The CI currently only cross-compiles for arm64 on an intel x86_64 machine, so it cannot run the test suite. Until Github adds a runner that supports arm64 platform, without trying it on a real mac we don't know if this will actually work or not.

I suggest you make a new venv, pip install the above wheel file, then install the fontTools and pytest, and run the pytest command from the root of the ttfautohint-py repository, and confirm that the test suite passes.

Make sure to use the latest python 3.9.5 (which should fix #6), or even Python 3.10rc1. And make sure that python -c "import platform; print(platform.machine())" gives you arm64, and not x86_64.

Thanks!

[anthrotype]

actually, I believe @simoncozens already tested (or had Eli test) the new wheels on a local M1 machine ( see #8), so I think we're fine

I can see that the libttfautohint.dylib file contains binaries for both x86_64 and arm64, so it should work.

$ file .venv/lib/python3.9/site-packages/ttfautohint/libttfautohint.dylib
.venv/lib/python3.9/site-packages/ttfautohint/libttfautohint.dylib: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit dynamically linked shared library x86_64] [arm64]
.venv/lib/python3.9/site-packages/ttfautohint/libttfautohint.dylib (for architecture x86_64):	Mach-O 64-bit dynamically linked shared library x86_64
.venv/lib/python3.9/site-packages/ttfautohint/libttfautohint.dylib (for architecture arm64):	Mach-O 64-bit dynamically linked shared library arm64

[anthrotype]

universal2 wheels are on PyPI now https://pypi.org/project/ttfautohint-py/0.4.3.post1/#files
Reopen if there's still issues

[eliheuer]

Wonderful! Thank you, will test this as soon as I can.