Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Anaconda, mod_wsgi and arm64 architecture #900

Closed
genghisken opened this issue Aug 7, 2024 · 17 comments
Closed

Anaconda, mod_wsgi and arm64 architecture #900

genghisken opened this issue Aug 7, 2024 · 17 comments

Comments

@genghisken
Copy link

genghisken commented Aug 7, 2024 •
edited
Loading

Hi Graham

I fear this is a very frequently asked question and with some repetition of #824. I have to switch to the M3 Arm-64 platform (Sonoma) and I cannot get mod_wsgi installed and running. The install looks like it worked but I get the dreaded message below. If this is in an FAQ, please feel free to direct me towards it!

[Thu Aug 08 00:03:08.987260 2024] [so:error] [pid 290:tid 8508444480] AH06665: No code signing authority for module at /Users/kws/anaconda3/envs/psat_panstarrs38/lib/libpython3.8.dylib specified in LoadModule directive.
httpd: Syntax error on line 160 of /tmp/panstarrs/httpd.conf: Code signing absent - not loading module at: /Users/kws/anaconda3/envs/psat_panstarrs38/lib/libpython3.8.dylib

I've tried signing the code (which complains that it's already signed - hence removed the existing signature and added my own), but specifying the certificate after the LoadModule statement in the generated httpd.conf achieves nothing. It still complains that the module is unsigned.

So, then I uninstalled it and tried to installed mod_wsgi-httpd or mod_wsgi-standalone. Now I get the same compilation errors. If only I could see where the code lives, I could probably fix the cast error for pid_t, etc.

      In file included from encoding/apr_encode.c:28:
      In file included from ./include/apr_encode.h:24:
      ./include/apr.h:561:2: error: Can not determine the proper size for pid_t
      #error Can not determine the proper size for pid_t
       ^
      In file included from encoding/apr_encode.c:30:
      In file included from ./include/apr_strings.h:52:
      ./include/apr_want.h:94:8: error: redefinition of 'iovec'
      struct iovec
             ^
      /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/sys/_types/_iovec_t.h:31:8: note: previous definition is here
      struct iovec {
             ^
      2 errors generated.
      make[1]: *** [encoding/apr_encode.lo] Error 1
      make: *** [all-recursive] Error 1
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/b5/cnlc_xr1187_pb1q0xrg5qxc0000gp/T/pip-install-ejrobamv/mod-wsgi-httpd_c6473afbb1af42769b3f92f2d2719b44/setup.py", line 75, in <module>
          raise RuntimeError('Failed to build APR.')
      RuntimeError: Failed to build APR.
      [end of output]

So I'm stuck from here. Is mod_wsgi simply now not compatible with Mac / Arm-64? Do I need to move my development platform off Sonoma?
@GrahamDumpleton
Copy link
Owner

On macOS you cannot use the system supplied Apache httpd, or even build Apache httpd yourself from ASF (Apache Software Foundation) source code, as the APR libraries distributed by the ASF don't support the last few macOS versions and have not done so for quite a while.

Your only option on macOS is to use Apache httpd from HomeBrew and use gcc from HomeBrew as well, ignoring the system macOS versions.

@GrahamDumpleton
Copy link
Owner

GrahamDumpleton commented Aug 8, 2024 •
edited
Loading

On top of that, you also can't use macOS supplied version of Apache httpd due to the code signing issues where they require Apache modules to be signed. I don't know what is required to sign the module once compiled and no one has been able to explain it to me either.

@genghisken
Copy link
Author

Yeah, many thanks. I've gone through several tutorials re: signing modules, but modifying the generated httpd.conf to add the certificate name after the module in the LoadModule command is simply not recognised. (I get the same not signed error.)

I'm also curious to know where my original mod_wsgi arm64 python wheel came from in the first place. If I try to reinstall with --no-cache-dir I get the following...

(psat_panstarrs38) kws::MacBook-Pro { /tmp/panstarrs }-> pip install mod_wsgi --no-cache-dir
Collecting mod_wsgi
  Downloading mod_wsgi-5.0.0.tar.gz (497 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 497.7/497.7 kB 7.9 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [57 lines of output]
      
      WARNING: The Python installation you are using does not appear to have
      been installed with a shared library, or in the case of MacOS X, as a
      framework. Where these are not present, the compilation of mod_wsgi may
      fail, or if it does succeed, will result in extra memory being used by
      all processes at run time as a result of the static library needing to
      be loaded in its entirety to every process. It is highly recommended
      that you reinstall the Python installation being used from source code,
      supplying the '--enable-shared' option to the 'configure' script when
      configuring the source code prior to building and installing it.
      
      running egg_info
      creating /private/var/folders/b5/cnlc_xr1187_pb1q0xrg5qxc0000gp/T/pip-pip-egg-info-l_xq431h/mod_wsgi.egg-info
      writing /private/var/folders/b5/cnlc_xr1187_pb1q0xrg5qxc0000gp/T/pip-pip-egg-info-l_xq431h/mod_wsgi.egg-info/PKG-INFO
      writing dependency_links to /private/var/folders/b5/cnlc_xr1187_pb1q0xrg5qxc0000gp/T/pip-pip-egg-info-l_xq431h/mod_wsgi.egg-info/dependency_links.txt
      writing entry points to /private/var/folders/b5/cnlc_xr1187_pb1q0xrg5qxc0000gp/T/pip-pip-egg-info-l_xq431h/mod_wsgi.egg-info/entry_points.txt
      writing top-level names to /private/var/folders/b5/cnlc_xr1187_pb1q0xrg5qxc0000gp/T/pip-pip-egg-info-l_xq431h/mod_wsgi.egg-info/top_level.txt
      writing manifest file '/private/var/folders/b5/cnlc_xr1187_pb1q0xrg5qxc0000gp/T/pip-pip-egg-info-l_xq431h/mod_wsgi.egg-info/SOURCES.txt'
      reading manifest file '/private/var/folders/b5/cnlc_xr1187_pb1q0xrg5qxc0000gp/T/pip-pip-egg-info-l_xq431h/mod_wsgi.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no files found matching 'docs/_build/html/_static/*'
      warning: no files found matching 'docs/_build/html/_static/css/*'
      warning: no files found matching 'docs/_build/html/_static/font/*'
      warning: no files found matching 'docs/_build/html/_static/js/*'
      warning: no files found matching 'docs/_build/html/_sources/*'
      warning: no files found matching 'pyproject.toml'
      adding license file 'LICENSE'
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/b5/cnlc_xr1187_pb1q0xrg5qxc0000gp/T/pip-install-tano55ib/mod-wsgi_4f4f421b6c4c4f5eacefc609d691fae2/setup.py", line 416, in <module>
          setup(name = package_name,
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/__init__.py", line 108, in setup
          return distutils.core.setup(**attrs)
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 184, in setup
          return run_commands(dist)
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
          dist.run_commands()
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 970, in run_commands
          self.run_command(cmd)
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/dist.py", line 945, in run_command
          super().run_command(command)
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
          cmd_obj.run()
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 310, in run
          self.find_sources()
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 318, in find_sources
          mm.run()
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 544, in run
          self.prune_file_list()
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 610, in prune_file_list
          base_dir = self.distribution.get_fullname()
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/_core_metadata.py", line 266, in get_fullname
          return _distribution_fullname(self.get_name(), self.get_version())
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/_core_metadata.py", line 284, in _distribution_fullname
          canonicalize_version(version, strip_trailing_zero=False),
      TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero'

@GrahamDumpleton
Copy link
Owner

If you can, avoid using Anaconda Python. They keep breaking the ability to embed Python in other applications and even if you can get it to build, not guaranteed it will work. Thus also suggest using Python from HomeBrew.

Am not really sure why getting the error with Anaconda Python, but do be aware you are using a quite old version of it, ie., Python 3.8. Are you able to at least update to newer Anaconda Python?

@genghisken
Copy link
Author

Incidentally mod_wsgi works fine on my old Intel Mac, which is also running MacOS Sonoma!

Is it possible to get my Mac to defer to using my homebrew install of httpd and gcc when installing mod_wsgi-httpd or mod-wsgi-standalone. Or do I have to revert to a manual config? Do I need to set CFLAGS or LDFLAGS, etc to force the issue?

@genghisken
Copy link
Author

Yes, I can update - I realise it's a very old python. I was trying to replicate as near as possible my Mac intel dev environment first, but I'll try upgrading. Everything else works - mod_wsgi is my only remaining module to fix. (I use python 3.11 everywhere else, so I'll try that.)

Sorry - Anaconda is our default dev python. I'll not be switching unless there's a major showstopper in the near future. I'll switch to using nginx if I have to!

Many thanks for the speedy reponses!

@GrahamDumpleton
Copy link
Owner

When using mod_wsgi-httpd or mod-wsgi-standalone it it is relying on building from Apache httpd source code, which as already mentioned will fail on macOS due to APR not being updated by the ASF. Although that said, it seems there was a release of APR back at April 16, 2023. It has been so long since I have done anything with mod_wsgi and checked whether they had fixed it is possible they may have as mod_wsgi-httpd is still using api-1.7.0, where as latest is now 1.7.4. Let me see if I can get it working with the later version.

@genghisken
Copy link
Author

Thanks again for your engagement with this. mod_wsgi has been the solid backbone of our webserver code and I'll be gutted if I need to move my dev platform to a remote Ubuntu server without at least making an effort to resolve the ongoing issues. Not sure what time zone you're in, but I need to go offline for about 6 hours now! (I'm on UK time.)

@GrahamDumpleton
Copy link
Owner

Am in Australia, so morning, but also got a lot of other work have to do. :-(

Isn't a straight forward change to mod_wsgi-httpd due to demise of setuptools. So have to update setup.py file to whatever supposed to use now. Will see if can still sort something out.

@GrahamDumpleton
Copy link
Owner

Okay, from Python 3.12 you have to install setuptools yourself.

@GrahamDumpleton
Copy link
Owner

Seems later versions do work and can get mod_wsgi-httpd working again on macOS. Only took them a couple of years to fix APR. 🤷‍♂️

@GrahamDumpleton
Copy link
Owner

My internet connection had a lot of unexpected problems today due to a botched service transfer, so haven't managed to update mod_wsgi-standalone package yet as need to also roll some other mod_wsgi changes in as well. What I have been able to do though is update mod_wsgi-httpd, so try the following:

pip uninstall mod_wsgi-standalone
pip uninstall mod_wsgi
pip uninstall mod_wsgi-httpd

Keep running until it says none of the packages are installed.

Then (preferably using a virtual environment), run:

pip install mod_wsgi-httpd

Then run:

pip install mod_wsgi

When installing mod_wsgi package it should detect that mod_wsgi-httpd was installed and use it.

This is the long handed way that works instead of installing mod_wsgi-standalone package.

@genghisken
Copy link
Author

Still persisting with my ancient python 3.8 install the pip install mod_wsgi-httpd step fell over at this stage:

      adding license file 'LICENSE'
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/b5/cnlc_xr1187_pb1q0xrg5qxc0000gp/T/pip-install-798v6fxw/mod-wsgi-httpd_cdf813f943834c4198bf97a325c479ed/setup.py", line 148, in <module>
          setup(name = 'mod_wsgi-httpd',
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/__init__.py", line 108, in setup
          return distutils.core.setup(**attrs)
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 184, in setup
          return run_commands(dist)
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
          dist.run_commands()
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 970, in run_commands
          self.run_command(cmd)
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/dist.py", line 945, in run_command
          super().run_command(command)
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
          cmd_obj.run()
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 310, in run
          self.find_sources()
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 318, in find_sources
          mm.run()
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 544, in run
          self.prune_file_list()
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 610, in prune_file_list
          base_dir = self.distribution.get_fullname()
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/_core_metadata.py", line 266, in get_fullname
          return _distribution_fullname(self.get_name(), self.get_version())
        File "/Users/kws/anaconda3/envs/psat_panstarrs38/lib/python3.8/site-packages/setuptools/_core_metadata.py", line 284, in _distribution_fullname
          canonicalize_version(version, strip_trailing_zero=False),
      TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero'

Hang on while I upgrade my whole environment to python 3.11, which I should have done in the first place. (Installation of mod_wsgi seemed to work, though I haven't tested it yet.) I have some reliance on setuptools for my own package, so I'll delay upgrading to 3.12 until I've got that fixed!

@GrahamDumpleton
Copy link
Owner

Need to just search around to try and find a solution.

What version of setuptools are you using? Is it the latest version or an older version?

@GrahamDumpleton
Copy link
Owner

Can you pin setuptools to be <71.

@genghisken
Copy link
Author

Amazing. Pinning setuptools to 70 works fine! Many thanks for your help!! My python 3.8 environment is now functional.

And also - many apologies for wasting your time. It turns out that upgrading to python 3.11, mod_wsgi (no standalone or httpd required) and Anaconda work together out of the box. No code signing, etc. All a good exercise for me, but not sure it's been useful for you.

@GrahamDumpleton
Copy link
Owner

Not a waste of time as I have been sitting on needing to make some mod_wsgi changes for a while. Wasn't the best day for me to decide to do something about it with some big internet access issues and also needing to finish a release of a different project at same time but has worked out okay so far. :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@GrahamDumpleton @genghisken