Recommend pipx as the installation method of choice

[kwigley]

Describe the feature

Recommend using pipx to install dbt as a cli app for all OS.

Describe alternatives you've considered

The alternatives are to continue suggesting pip and Homebrew as recommended installation methods. pip is certainly the best way to install dbt. Homebrew is starting to become difficult to maintain because it wants to install all dependencies from source without using any accessible features of pip. We should continue trying our best to maintain Homebrew formulas for dbt, but some external dependencies are very challenging or impossible to install from source without jumping through a bunch of hoops. Homebrew is supposed to make the installation process of macOS easier and more accessible, it is evident that this is not the case at the moment. pipx can be a drop in replacement for Homebrew and for installing dbt globally. IMO this better than trying to manage your own virtualenv and installing via pip, it takes care creating an isolated environment for you.

Additional context

pipx addresses some shortcomings of Homebrew as well. With pipx, users can install any adapter (dbt-postgres, dbt-spark, etc.).

Who will this benefit?

I think this will help with new users especially. They will enjoy the more stable pip installation of dbt without dealing with any virtualenv management. Also, this will benefit the sanity of maintainers. There are less variables at play when using pipx, so debugging and helping users install software should be easier.

Are you interested in contributing this feature?

Of course! The only change involved is updating https://docs.getdbt.com/dbt-cli/installation I believe. I can move this issue over to the documentation repo once this gets some input.

[kwigley]

I've tested this out in a Windows VM, my macbook, and a docker container running ubuntu and works great! The only caveat is that you must pass the --include-deps arg when installing because of how dbt dependencies are organized (pipx install dbt --include-deps, pipx install dbt-postgres --include-deps, etc.) If nothing else, I propose we remove "(recommended for MacOS)" for Homebrew 😄

[jtcohen6]

I just had a generally positive experience trying out pipx, doing things like:

pipx install --python python3.8 dbt --include-deps
pipx install --python python3.8 dbt-spark --include-deps
pipx install --python python3.9 dbt-postgres==0.19.0-rc2 --include-deps

Each of these is installed in a separate venv:

~/.local/pipx/venvs/dbt/bin/dbt
~/.local/pipx/venvs/dbt-postgres/bin/dbt
~/.local/pipx/venvs/dbt-postgres/bin/dbt

Only the first one is directly symlinked to dbt (via ~/.local/bin/dbt).

What seems best here for users?

• Explicit python version at installation (unlike brew)
• Implicit venv-ing, does not overwrite existing installations in the same env like pip does
• Support across systems/OS
• Legible, human-friendlier info and errors

For us?

• Uses same installation process and dependency resolver as latest versions of pip (unlike brew)
• pipx is dedicated to supporting programs like dbt: written in python, executable from CLI. I found this page instructive.

What feels less good?

• More involved CLI syntax than just brew install dbt. That simplicity has been masking a lot of complexity for a long time, though!
• Recommended install on macOS is brew install pipx—who forestalls the installers? I guess I do trust the pipx team to better handle the brew/python/macOS challenges we've faced, and pip install pipx works, too.

One thing we may want is to add anonymous usage tracking for how folks are installing dbt (i.e. which dbt). We currently track information about python versions and operating systems. If we were to de-emphasize brew, or stop supporting it entirely, we don't have a good sense of how many people that might affect.

[jtcohen6]

To summarize where we ended up on this: The granular specificity of Homebrew, and the fact that we occasionally need to override some dependencies (e.g. dbt-labs/homebrew-dbt#10), can be as much a feature as a bug. It will be an asset for us if/when we want to release dbt with Rust components.

I'm going to close this one for now, but we may revisit later on!