Use 'pip install' in the virtual environment created by conda

[CicoZhang]

Hi, guys:

I have some confusion about the scenario where I use 'pip install' in the virtual environment create by conda. If I did so, is the python package installed by 'pip' global or local to this conda virtual environment?

Thank you in advance.

Best,

Cico

[anton-petrov]

Command 'pip install' installs packages globally to ~/Users/user/anaconda/lib/python3.6/site-packages

[bryevdv]

This is not true. pip install will install into whatever environment pip itself is installed into and running from.

$ conda create -n pipenv numpy pip                                                  (mattpap/5810_typedarray_map)
Fetching package metadata ...............
Solving package specifications: .

Package plan for installation in environment /Users/bryan/anaconda/envs/pipenv:

The following NEW packages will be INSTALLED:

    mkl:        2017.0.1-0
    numpy:      1.11.3-py36_0
    openssl:    1.0.2k-0
    pip:        9.0.1-py36_1
    python:     3.6.0-0
    readline:   6.2-2
    setuptools: 27.2.0-py36_0
    sqlite:     3.13.0-0
    tk:         8.5.18-0
    wheel:      0.29.0-py36_0
    xz:         5.2.2-1
    zlib:       1.2.8-3

Proceed ([y]/n)?

#
# To activate this environment, use:
# > source activate pipenv
#
# To deactivate this environment, use:
# > source deactivate pipenv
#

[bryan:~/work/bokeh/examples/models]$ source activate pipenv                                                            (mattpap/5810_typedarray_map)
(pipenv) [bryan:~/work/bokeh/examples/models]$ pip install scipy                                                        (mattpap/5810_typedarray_map)
Collecting scipy
  Downloading scipy-0.18.1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (22.2MB)
    100% |████████████████████████████████| 22.2MB 36kB/s
Requirement already satisfied: numpy>=1.7.1 in /Users/bryan/anaconda/envs/pipenv/lib/python3.6/site-packages (from scipy)
Installing collected packages: scipy
Successfully installed scipy-0.18.1
(pipenv) [bryan:~/work/bokeh/examples/models]$ python                                                                   (mattpap/5810_typedarray_map)
Python 3.6.0 |Continuum Analytics, Inc.| (default, Dec 23 2016, 13:19:00)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import scipy
>>> scipy
<module 'scipy' from '/Users/bryan/anaconda/envs/pipenv/lib/python3.6/site-packages/scipy/__init__.py'>
>>>

Note that SciPy was installed into the "pipenv" conda environment, because that is where pip was run from.

If you only happen to have pip installed in your root env, and your root env is on your path, then yes a pip install will always (by coincidence) install into the root env.

[CicoZhang]

@bryevdv When I run the command conda create -n pipenv, it is a clear environment without pip installed. When I install pip in this environment, pip is local to it; therefore the tools installed by pip is, transitively, local to this environment. Cool. Thank you for the explanation.

[ccordoba12]

Closing then, as the question is solved now.

[kronos29296]

This works only if pip is added when the conda env is created.

If I add pip later using conda install pip then use pip to install packages it installs to global directory.

[makmanalp]

@kronos29296 huh - any idea why?

[Koriolisah]

@kronos29296 @makmanalp

Kronos, when you come back later and do 'conda install pip' as you said, did you specify which environment to install pip into? Perhaps its installing to the global directory as you say because its installing to the root environment.

[kronos29296]

The problem occurs when i create an empty conda env first and activate it. Then I conda install pip. In the same activated env, if i use pip install, it installs to global directory. The env is activated and conda install works properly though.

This doesn't occur if I add pip to env when creating it. This is the problem.

[billism]

@kronos29296
I guess both the root env and the virtual env have pip installed. When you ''pip install' something in the virtual env, the global pip is used by default. You could try to specify the directory of the virual env pip, like ''/miniconda3/envs/myenv/bin/pip install numpy''.

[andrewsanchez]

@billism That worked for me, but was not necessary if I installed pip when creating the env, i.e. conda create -n env pip

[thammegowda]

@billism
If global pip is being used the which pip command should show the path right?
For my surprise, the which pip pointed to the one inside virtual env.

So I 👍 to @kronos29296 report of pip's behavior

[ashander]

I've investigated a bit. The scenario reported by @kronos29296 occurs when you

1. create an env without pip
2. install pip later
3. have not yet deactivated and reactivated the env

It can be fixed by deactivating and reactivating the env.

An example:

conda create -n pipenv
source activate pipenv
pip freeze # shows packages installed to global
conda install pip

at this point, the behavior reported occurs

which pip # shows local pip 
pip freeze # STILL shows global packages!
pip install  # installs global packages!

to fix:

source deactivate
source activate pipenv
which pip # shows local pip
pip freeze # now shows local packages!
pip install  # now installs a local package

[h2oworker]

I also have confusion about the environments.

I tried the @ashander 's suggestion but it is not working. So far I am unable to isolate the environments (and install packages only in environment of my choice using pip)?

My scenario:

• Windows 10 with system wide python 3.6 and 2.7 - the paths to these are in the environment variable
• Installed anaconda (python 3.6.2 / 32 bit) - path is not on the environment variable
• Created an environment - test-anaconda - using the anaconda navigator

I opened up 3 terminal windows, activated the environments and issued the pip3 list command:

• Window 1 - system wide python 3.6 installation
  • Expect to see the packages on my global installation - pip 9.0.1, setuptools 28.8.0
  • Result -- same as expected
• Window 2 - Anaconda root (@ ..\anaconda3)
  • Expect to see all the packages that were installed with the default anaconda installation. These are visible / installed as per the anaconda navigator (e.g. jupyter, matplotlib...)
  • Result -- lists the packages on the system wide python installation ( pip 9.0.1, setuptools 28.8.0) nothing from the anaconda root installation
• Window 3 - isolated environment test-anaconda (@ ..\anaconda3\envs\test-anaconda)
  • Expect to see the pip and setuptools packages that were installed during the environment installation. These are visible / installed as per the anaconda navigator (e.g. pip 9.0.1, setuptools 36.4.0...)
  • Result -- lists the packages on the system wide python installation ( pip 9.0.1, setuptools 28.8.0) nothing from the environment.

I also added the python interpreter from test-environment to pycharm. Pycharm does show the packages on the test-environment environment. However, when I open the terminal within pycharm it again shows the packages on the system-wide python installation and not on the environment.

[kalefranz]

It can be fixed by deactivating and reactivating the env.

Conda 4.4 will resolve this issue.

[hoangcuong2011]

Thank you all for a very good question and excellent answers. I learned a lot from this topic!

[lydhr]

Try 'type pip' instead of 'which pip' to check the real current version of your 'pip'. And trying 'hash -r' to clean the cache might help you sometimes.

[egayer]

Thanks guys, now I understand why I got my global env totally messed up after creating and installing what I thought to be a clean fresh env...

https://groups.google.com/a/continuum.io/forum/#!topic/conda/yC5xVTbJfwA

It looks like it is not a good idea to mix conda packages with pip ones since it breaks conda's hard links (learned it the hard way). However, is it safe to create a pipenv (like the exemple in this discussion) and then install only pip packages ? should it break any links ?

[heyhenryz]

Thank you, this thread helped me a lot.

[joaoferrao]

I've read through this thread and understood the workaround proposed, but I still have an issue.
If I do pip freeze inside the activated env, I don't get the full list of packages installed. Instead, I get a single package certifi==2018.1.18.
Otherwise, if I do conda list I do get what I expect to be the packages I installed in the virtual env including the ones with pip install <package>.

Is someone getting the same issue?

[JoshuaC3]

The easiest work around I have found, on Linux, is sudo $(which pip) install package_name.

NB: This is done after activating the desired conda environment.

[xguse]

@JoshuaC3 this not a good idea in my opinion. There is no reason to invoke root user to install a package. Depending one which pip $(which pip) points to, you may be installing into your system's python environment. This is not optimal. And in the case where that IS what you intend, it is MUCH more rational to do that through your system's package manager like yum, apt, pacman, etc. If you just do it through pip, your system is not made aware of it can cannot use this information when it does its system maintenance etc. It will likely not happen TOO often but you could end up borking your system if your system expects the versions of python packages to be different than what actually exist.

[JoshuaC3]

@xguse thanks for your explanation. I do this once I have used conda activate xxxx or source activate xxxx for the xxxx conda environment.

It has not (so far) affected my root python environment, I guess because of what is described in thammegowda's post above.

[SevenZhan]

@ashander I tried your advices and the problem got resolved

[FBosler]

Apologies for reviving this thread, but I am experiencing some very unexpected behaviour.

I created a Conda environment through Pycharm (when setting up a new project)
I then activated the environment in the terminal
I then run

pip install -r requirements.txt

This runs as expected and tells me that it installed all the packages.

Running pip freeze shows none of the packages.

I tried deactivating and reactivating, I tried explicitly calling the environment specific pip, I checked type pip and which pip (both pointing to the environment specific pip) but without success.

Any pointers?

[joaoferrao]

@FBosler with the environment activated, run echo $(which pip) and echo $(which python)

Maybe something like $(which python) -m pip install -r requirements.txt should ensure you are using the python's activated executable

[Yossarian0916]

in ~/.bashrc instead of

# added by Anaconda3 4.3.0 installer
export PATH="/home/user/anaconda3/bin:$PATH"

one would use

export PATH="$PATH:/home/user/anaconda3/bin"

so the bash will find system default pip and pip3 first instead of pip within anaconda

[ytobi]

This is a very useful post!

[wangg12]

After conda 4.4 we should add . /data/wanggu/anaconda3/etc/profile.d/conda.sh into .bashrc.

[aabid0193]

A good way to also do this imo is by creating the conda environment and installing a python version in it: conda create -n yourenvname python=x.x
This way if you pip install xx it will be installed in the environment path and not globally.
It can also be helpful this way because you can have specific python versions in different environments as needed.

[GeekyShacklebolt]

The thread is helpful. I am using Ubuntu 18.04.2 LTS and Conda==4.5.12. A quick workaround for me was creating alias pip=$(which pip). Now, if I am in conda environment which pip points to local pip otherwise, global pip.

[scottcode]

Deactivating and reactivating worked for me :)

I've investigated a bit. The scenario reported by @kronos29296 occurs when you

1. create an env without pip
2. install pip later
3. have not yet deactivated and reactivated the env

It can be fixed by deactivating and reactivating the env.

An example:

conda create -n pipenv
source activate pipenv
pip freeze # shows packages installed to global
conda install pip

at this point, the behavior reported occurs

which pip # shows local pip 
pip freeze # STILL shows global packages!
pip install  # installs global packages!

to fix:

source deactivate
source activate pipenv
which pip # shows local pip
pip freeze # now shows local packages!
pip install  # now installs a local package

[stripedpumpkin]

Until now I was managing my Python envs with conda and

pip freeze

was giving local packages.

Then I

pip install some_package

(I had not conda install pip beforehand)

Now for any conda environment I am activating

> which pip
=> /home/pumpkin/miniconda3/bin/pip
> pip freeze
=> global packages

And getting still the same after conda install pip locally and deactivating/activating.

Is there a way I can restore the situation, meaning having pip freeze correctly describe the conda env that is currently active?

> conda --version
=> 4.6.14

[stripedpumpkin]

Ok the reason was that I was using conda deactivate and this was falling back to a previous conda env. Using source deactivate or conda deactivate until out of a conda env does work.

[computersarecool]

There is a lot to read here. What is the simplest way to install packages into a preexisting conda environment using pip when:

• The existing conda environment does not have pip installed

Thanks

[3yakuya]

conda install pip
which pip -> if it points to your conda env, you are good. If not:
source deactivate or conda deactivate until you are our of conda envs (including base)
conda activate name_of_your_env
which pip should now point to conda env
pip list should show a fresh, clean list.

[marcglobality]

had a similar problem

which pip giving /home/ubuntu/.local/bin/pip

. My solution was to do

conda deactivate to get to (base)
conda install pip
conda deactivate
conda activate
which pip should give /home/ubuntu/miniconda3/bin/pip
conda activate my_env
which pip should give /home/ubuntu/miniconda3/envs/my_env/bin/pip

[leibowitzdaniel]

I'm way late to this thread, but I have experienced lots of trouble in the past with getting the pip command to use the version of pip installed within a conda environment. As others in this thread have noted, even if you use conda install pip in a new empty conda environment, the pip command in the command line tends to keep pointing to the global install.

Before I proceed, I would like to note that the solution proposed by several people above (installing pip at the time of creation of a new conda environment using conda create -n my_env pip) does indeed work! However, my solution below allows the problem to be resolved in an existing conda environment.

After running conda install pip in example_env, the surefire way to force the existing conda environment to use the environment's own local pip installation is to run the following command within the activated environment:

python -m ensurepip --upgrade

After that the pip --version command should show the pip installation sitting within example_env.

NOTE: The conda install pip command does need to be run first.

I hope that this helps!

[alexisthual]

Surprisingly enough, when I activate my envs in tmux panes, which pip points to a global instance of pip.
Outside of tmux, it points to the env's pip version.

[marcglobality]

Surprisingly enough, when I activate my envs in tmux panes, which pip points to a global instance of pip. Outside of tmux, it points to the env's pip version.

conda deactivate (maybe multiple times) + conda activate my_env should work. I use tmux as well and have to do this all the time. I actually have an alias to do this :)

[makcedward]

Surprisingly enough, when I activate my envs in tmux panes, which pip points to a global instance of pip. Outside of tmux, it points to the env's pip version.

conda deactivate (maybe multiple times) + conda activate my_env should work. I use tmux as well and have to do this all the time. I actually have an alias to do this :)

It works. Need to deactivate twice. Here are steps:

# Change the virutal env to base
conda deactivate
# Deactivate conda
conda deactivate
# Activate conda
conda activate
# change to your env
conda source {your_env}

[ahdeanlau]

Thanks @marcglobality @makcedward , that's what I need, just wondering why this is not solved entirely, when I activate an venv, it should automatically set the path to the venv's python and pip