Installing Python
Skip this if you have Python and your paths are all set up correctly.
It is October 2022, the latest macOS is 12.6. There are so many ways to install Python on macOS. And it is all very confusing. Here's an attempt to explain it a little.
Starting with macOS 12.3 Monterey, macOS no longer includes Python. However, macOS does come with /usr/bin/python3, which is a stub that asks if you want to install the command line developer tools. Executing /usr/bin/python3 and proceeding with the prompt is one way to get Python. Basically /usr/bin/python3 will be replaced with a real version of Python. Starting in macOS 12.5 Apple updated /usr/bin/python3 to 3.9.6, which isn't that old (macOS 12.4 had 3.8.9).
You can also install Python using the official installer. As of October 2022, it installs Python 3.10.7. If you do it this way, it will be located at /Library/Frameworks/Python.framework/Versions/3.10/bin/python3.
You can also install Python by installing Anaconda (or one of its derivative projects). If you do it this way, it will be located either in /opt/anaconda/bin/ or in the user home directory in ~/opt/anaconda/bin. Anaconda 2022.05 installs Python 3.9.12.
If you have already installed Homebrew, you can install it with that (brew install python). In this case, it will be located in /opt/homebrew/bin/python3 on an Apple Silicon Mac, or /usr/local/bin/python3 if it's Intel. As of October 2022, Homebrew installs Python 3.10.6.
Are you confused yet? If not, then let's talk about the difference between the python and python3 binaries. Up until macOS 12.3, Apple shipped /usr/bin/python and it was Python 2. Starting in macOS 12.3, Apple removed it. There's a /usr/bin/python3 but not a /usr/bin/python.
Apple's Command Line tools, the official Python, and Homebrew only install python3. But Anaconda installs both python3 and python. So if you have Anaconda installed, you'll also have a python, pip, python3, and a pip3 (located wherever Anaconda is installed). But unlike macOS 12.2 and earlier, python and pip point to python3 and pip3, respectively.
If you're still not confused, let's talk about the PATH environment variable. When I first learned Unix and heard the 2 words, "environment variable", I wondered what the climate and outside weather had to do with the computer. It took me a while to figure out it simply meant a variable in the Terminal window that only lasted as long as the Terminal window (each window is a different "environment").
The PATH environment variable tells the shell where to look for binaries. By default, the PATH is set to "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin" (this is set in /etc/paths). So if you type python3 in the Terminal, it will look in each of these locations, in order, for an executable named python3. The first one it finds is the one it will use.
/usr/local/bin/usr/bin/bin/usr/sbin/sbin
You can find out which python3 is going to be executed by typing which python3. You can view the PATH variable by typing echo $PATH.
If you install the official Python, there is an installer option, "Shell profile updater", which will add "/Library/Frameworks/Python.framework/Versions/3.10/bin/" to the PATH by modifying the file ~/.zprofile.
If you install Homebrew it will instruct you to modify your PATH yourself (by modifying ~/.zprofile).
If you install Anaconda, there is an installer option to also fix the PATH, in this case by modifying ~/.zshrc. Because ~/.zshrc runs after ~/.zprofile, Anaconda will modify the PATH last, meaning its python will take precedence (tricky them) over all other pythons that are installed. If you want to change this, you need to modify the PATH variable yourself.
The problem with all of these is that it only fixes the PATH for the currently logged-in user.
Apple has this cool mechanism that if you put a custom path in a file in /etc/paths.d then it will be automatically added to every user and every shell by calling /usr/libexec/path_helper (which is called by the shell startup scripts).
However, the default paths are always first. That means if you modify the PATH using /etc/paths.d (instead of ~/.zshrc and ~/.zprofile) Apple's /usr/bin/python3 will be found before the official Python or Anaconda's python3. Homebrew on Intel installs /usr/local/bin/python3, which comes before Apple's.
Basically, you can't use /etc/paths.d to set the PATH. The python3 installed by Homebrew on Intel or Apple will always win. We could vilify Homebrew but Apple is also guilty. If you want a different python3, you have to specify it in /etc/zprofile, /etc/zshrc, ~/.zprofile, or ~/.zshrc.
The real solution is that Apple needs to change path_helper to put /etc/paths.d before the default PATH. I don't have a perfect memory but I'm pretty sure that everything I've read about modifying the PATH variable looks like this: export PATH="/custom/location:$PATH", which puts the customizations first. path_helper is basically doing this: export PATH="$PATH:/custom/location", which kind of defeats the whole purpose.
Sad. I really liked /etc/paths.d and there are several tools that use it (like XQuartz). But hey, I'm a Systems Administrator, I can fix this. Put this in your shell startup scripts (like /etc/zprofile)! LOL
export PATH=`echo $PATH | sed -e 's#\(/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin\):\(.*\)#\2:\1#g'`
The next problem with /etc/paths.d is that they can't be ordered (because of APFS?). So you have no idea what will have precedence.
As a side note, there is an open-source replacement.
Why does this all matter?
Each version of Python has a library folder associated with it. Apple's Command Line Tools python has a library folder (/Library/Python/3.9/site-packages). The official Python has its library folder (/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages). Anaconda has a library folder (/opt/ancaonda3/lib/python3.9/site-packages or ~/opt/ancaonda3/lib/python3.9/site-packages). The same with Homebrew's Python.
If you execute the wrong Python, you'll get this error.
Traceback (most recent call last):
File "/Users/james/jctl", line 27, in <module>
import python-jamf
ModuleNotFoundError: No module named 'python-jamf'
You can find which python-jamf you are using by executing pip3 show python-jamf.
jctl calls /usr/bin/env python3.
/usr/bin/env basically checks the PATH variable for python3 and runs that. So your PATH being correct is crucial for jctl to work.
However, I just noticed that if you install jctl with pip3, the #!/usr/bin/env python3 at the top of the jctl file is changed! There is so much going on here. I think it's a huge mess. I've been bumping into Python installer problems for years and I always solved them by removing every Python on my computer I could find and starting over.
Using a Python virtual environment solves all of these problems for you. It's now March 2023 and we switched our instructions to use pipenv, which manages virtual environments. So this page is still here because the pain from years of struggling and having to figure all of this out is still fresh and we're not ready to forget yet.