PIP Installation problem - ModuleNotFoundError: No module named 'numpy' <resolved>

[Silas-Asamoah]

I tried installing earthpy using
pip3 install --upgrade git+https://github.com/earthlab/earthpy.git

And I get the following error
`Collecting git+https://github.com/earthlab/earthpy.git
Cloning https://github.com/earthlab/earthpy.git to /tmp/pip-req-build-oi4iyxo6
Installing build dependencies ... done
Getting requirements to build wheel ... error
Complete output from command /usr/bin/python3 /usr/local/lib/python3.6/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmp0kktck3c:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pep517/_in_process.py", line 207, in
main()
File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pep517/_in_process.py", line 197, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pep517/_in_process.py", line 54, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/pip-build-env-lny3v0hx/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 115, in get_requires_for_build_wheel
return _get_build_requires(config_settings, requirements=['wheel'])
File "/tmp/pip-build-env-lny3v0hx/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 101, in _get_build_requires
_run_setup()
File "/tmp/pip-build-env-lny3v0hx/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 85, in _run_setup
exec(compile(code, file, 'exec'), locals())
File "setup.py", line 3, in
from numpy.distutils.core import setup
ModuleNotFoundError: No module named 'numpy'

---

Command "/usr/bin/python3 /usr/local/lib/python3.6/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmp0kktck3c" failed with error code 1 in /tmp/pip-req-build-oi4iyxo6`

I did not understand why I got this error and figured it might be an error with my setuptools so I did
pip3 install --upgrade setuptools

And the error still persisted, meanwhile numpy is correctly installed and i can import it in any program I run. Could someone kindly help me out with this error?

[lwasser]

Hi there! Can you tell us 1. What operating system you are running 2. What version of python are you running? 3. What happens when you try pip instead of pip3? It’s interesting it’s saying it can’t find numpy on the install. I’m not able to recreate this error locally right now but the above information will help us troubleshoot. I actually also see something that could be causing this. Please also tell us what version of NUMPY you have installed.

[Silas-Asamoah]

I'm running
Ubuntu 18.04
Python 3.6.7
numpy 1.13.3

I still caught the same error when I used pip

[maykef]

go back to pip 9:

pip install --upgrade pip==9.0.3

And reinstall using pip3 install --upgrade git+https://github.com/earthlab/earthpy.git

[lwasser]

hey @maykef THANK YOU. we've been running into this issue as well and i've tried many things without it working. Hey @jlpalomino @nkorinek @katysill Try to do this

1. open up bash and run pip install --upgrade pip==9.0.3 (i have version 9.0.1)
2. Then try to install earthpy again.

this actually worked for me well.
Let me know if it works for others. i was starting to dig into rasterio and numpy updates.

[maykef]

Nice to hear it worked for you too.

[lwasser]

@maykef i'm just curious. do you happen to know what the issue is with certain versions of pip that impact an install?

[maykef]

pypa/pip#6163

[maykef]

Sorry, that wasn't the right link.

pypa/pip#5240

[maykef]

I wrote a short tutorial to run jupyter notebooks on Google Cloud Platform.
It corrects most of the errors on the installation.
I hope it helps:
https://medium.com/@maykef/how-to-create-a-vm-in-gcp-for-geospatial-processing-in-python-3dd00ad5f3e7

[Silas-Asamoah]

After running what you suggested this is the error, I run into, any reason why this is happening?

`Collecting git+https://github.com/earthlab/earthpy.git
Cloning https://github.com/earthlab/earthpy.git to /tmp/pip-36qhaiyh-build
Collecting tqdm (from earthpy===VERSION-0.6.0-)
Using cached https://files.pythonhosted.org/packages/6c/4b/c38b5144cf167c4f52288517436ccafefe9dc01b8d1c190e18a6b154cd4a/tqdm-4.31.1-py2.py3-none-any.whl
Collecting pandas (from earthpy===VERSION-0.6.0-)
Using cached https://files.pythonhosted.org/packages/e6/de/a0d3defd8f338eaf53ef716e40ef6d6c277c35d50e09b586e170169cdf0d/pandas-0.24.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting numpy (from earthpy===VERSION-0.6.0-)
Cache entry deserialization failed, entry ignored
Downloading https://files.pythonhosted.org/packages/f5/bf/4981bcbee43934f0adb8f764a1e70ab0ee5a448f6505bd04a87a2fda2a8b/numpy-1.16.1-cp36-cp36m-manylinux1_x86_64.whl (17.3MB)
51% |████████████████▋ | 9.0MB 6.8kB/s eta 0:20:30Exception:
Traceback (most recent call last):
File "/home/silas/.local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 302, in _error_catcher
yield
File "/home/silas/.local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 384, in read
data = self._fp.read(amt)
File "/home/silas/.local/lib/python3.6/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 60, in read
data = self.__fp.read(amt)
File "/usr/lib/python3.6/http/client.py", line 449, in read
n = self.readinto(b)
File "/usr/lib/python3.6/http/client.py", line 493, in readinto
n = self.fp.readinto(b)
File "/usr/lib/python3.6/socket.py", line 586, in readinto
return self._sock.recv_into(b)
File "/usr/lib/python3.6/ssl.py", line 1012, in recv_into
return self.read(nbytes, buffer)
File "/usr/lib/python3.6/ssl.py", line 874, in read
return self._sslobj.read(len, buffer)
File "/usr/lib/python3.6/ssl.py", line 631, in read
v = self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/silas/.local/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/home/silas/.local/lib/python3.6/site-packages/pip/commands/install.py", line 335, in run
wb.build(autobuilding=True)
File "/home/silas/.local/lib/python3.6/site-packages/pip/wheel.py", line 749, in build
self.requirement_set.prepare_files(self.finder)
File "/home/silas/.local/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "/home/silas/.local/lib/python3.6/site-packages/pip/req/req_set.py", line 620, in _prepare_file
session=self.session, hashes=hashes)
File "/home/silas/.local/lib/python3.6/site-packages/pip/download.py", line 821, in unpack_url
hashes=hashes
File "/home/silas/.local/lib/python3.6/site-packages/pip/download.py", line 659, in unpack_http_url
hashes)
File "/home/silas/.local/lib/python3.6/site-packages/pip/download.py", line 882, in _download_http_url
_download_url(resp, link, content_file, hashes)
File "/home/silas/.local/lib/python3.6/site-packages/pip/download.py", line 603, in _download_url
hashes.check_against_chunks(downloaded_chunks)
File "/home/silas/.local/lib/python3.6/site-packages/pip/utils/hashes.py", line 46, in check_against_chunks
for chunk in chunks:
File "/home/silas/.local/lib/python3.6/site-packages/pip/download.py", line 571, in written_chunks
for chunk in chunks:
File "/home/silas/.local/lib/python3.6/site-packages/pip/utils/ui.py", line 139, in iter
for x in it:
File "/home/silas/.local/lib/python3.6/site-packages/pip/download.py", line 560, in resp_read
decode_content=False):
File "/home/silas/.local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 436, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "/home/silas/.local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 401, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
File "/usr/lib/python3.6/contextlib.py", line 99, in exit
self.gen.throw(type, value, traceback)
File "/home/silas/.local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 307, in _error_catcher
raise ReadTimeoutError(self._pool, None, 'Read timed out.')
pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.
You are using pip version 9.0.3, however version 19.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.`

Downgrading pip does not work from my end

[Silas-Asamoah]

Going back to pip 9 didn't work but pip 10 did, after going through @maykef tutorial earthpy is now correctly installed. Thanks soo much

[mbjoseph]

I can confirm I have this issue too. I am running Ubuntu 18.04.2 LTS (Bionic Beaver), python version 3.6.7, conda version 4.6.3, in the earthpy-dev conda environment.

Steps to reproduce the issue

git clone git@github.com:earthlab/earthpy.git
cd earthpy/
conda env create -f environment.yml
source activate earthpy-dev
pip install -e .

Which gives the following error:

Obtaining file:///home/max/Documents/earthpy
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  Complete output from command /home/max/anaconda3/envs/earthpy-dev/bin/python /home/max/anaconda3/envs/earthpy-dev/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmp8ivum59x:
  Traceback (most recent call last):
    File "/home/max/anaconda3/envs/earthpy-dev/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 207, in <module>
      main()
    File "/home/max/anaconda3/envs/earthpy-dev/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 197, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/max/anaconda3/envs/earthpy-dev/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 54, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/pip-build-env-4nayy5gl/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 130, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "/tmp/pip-build-env-4nayy5gl/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 112, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-4nayy5gl/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 211, in run_setup
      self).run_setup(setup_script=setup_script)
    File "/tmp/pip-build-env-4nayy5gl/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 126, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 3, in <module>
      from numpy.distutils.core import setup
  ModuleNotFoundError: No module named 'numpy'
  
  ----------------------------------------
Command "/home/max/anaconda3/envs/earthpy-dev/bin/python /home/max/anaconda3/envs/earthpy-dev/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmp8ivum59x" failed with error code 1 in /home/max/Documents/earthpy

[lwasser]

@mbjoseph what version of pip do you have?

[mbjoseph]

@lwasser looks like pip 19.0.2

$ pip --version
pip 19.0.2 from /home/max/anaconda3/envs/earthpy-dev/lib/python3.6/site-packages/pip (python 3.6)

[lwasser]

and i have

(earthpy-dev) cu-biot-11-10:earthpy leah-su$ pip --version
pip 9.0.3 from /Users/leah-su/anaconda3/envs/earthpy-dev/lib/python3.6/site-packages (python 3.6)
(earthpy-dev) cu-biot-11-10:earthpy leah-su$ 

yup -- so if you DOWNGRADE to 0.9.3
pip install --upgrade pip==9.0.3 it should work. that worked for me. when i just upgraded pip i get the numpy error. that version however is months old so it's not ideal!!

UPDATE: so @mbjoseph version 18.0 also works for me. But version 19.0 of pip breaks the earthpy install. Curious if
pip install --upgrade pip==18.0 works for linux users.

[lwasser]

All -- it looks like @mbjoseph has a solution to this issue
#231

it's in the works. if someone can test this on windows and we see it's stable we will merge it and a lot this particular install issue will (hopefully) go away!!

[lwasser]

All... we have implemented a change to our setup.py file that we HOPE might address this install issue regardless of what version you have of pip running. Please try to install again and let us know how it goes!

[katysill]

hey @lwasser, I just tried to install, but received the error below. I am using pip 9.0.3 and my environment is activated. I am on a windows machine

$ pip install --upgrade git+https://github.com/earthlab/earthpy.git
Collecting git+https://github.com/earthlab/earthpy.git
Cloning https://github.com/earthlab/earthpy.git to c:\users\katy\appdata\local\temp\pip-u_vjq2cr-build
Collecting tqdm (from earthpy===VERSION-0.6.0-)
Using cached https://files.pythonhosted.org/packages/6c/4b/c38b5144cf167c4f52288517436ccafefe9dc01b8d1c190e18a6b154cd4a/tqdm-4.31.1-py2.py3-none-any.whl
Requirement already up-to-date: pandas in c:\users\katy\anaconda3\envs\earth-analytics-python\lib\site-packages (from earthpy===VERSION-0.6.0-)
Collecting numpy (from earthpy===VERSION-0.6.0-)
Using cached https://files.pythonhosted.org/packages/b5/11/82916e23836a37c0d76babf74a7ca6f7b4fedd0814eaa166aacc2318b87c/numpy-1.16.1-cp36-cp36m-win_amd64.whl
Requirement already up-to-date: geopandas in c:\users\katy\anaconda3\envs\earth-analytics-python\lib\site-packages (from earthpy===VERSION-0.6.0-)
Requirement already up-to-date: matplotlib>=2.0.0 in c:\users\katy\anaconda3\envs\earth-analytics-python\lib\site-packages (from earthpy===VERSION-0.6.0-)
Collecting rasterio (from earthpy===VERSION-0.6.0-)
Using cached https://files.pythonhosted.org/packages/4e/79/b3e2d1783cde76d31eb87fa859eadf6855743d18c52942fbaa5f638e98a9/rasterio-1.0.18.tar.gz
Complete output from command python setup.py egg_info:
INFO:root:Building on Windows requires extra options to setup.py to locate needed GDAL files. More information is available in the README.
ERROR: A GDAL API version must be specified. Provide a path to gdal-config using a GDAL_CONFIG environment variable or use a GDAL_VERSION environment variable.

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in C:\Users\katy\AppData\Local\Temp\pip-build-dt8kcov1\rasterio\

[lwasser]

@katysill oops i see it is activated... hmmmm @mbjoseph NOTE: this is the same issue i had on my windows VM. interestingly this is an exception from rasterio ... https://github.com/mapbox/rasterio/blob/9f128bc0da3fdaf04b7f23b94eabf1d7c02fe99c/setup.py#L114

[lwasser]

what version of rasterio do you have @katysill ?

[katysill]

@lwasser - I have rasterio 1.0.15

[lwasser]

did you update your envt? i have 1.0.18 on my windows VM.

[katysill]

I just ran an update on my earth-analytics environment, and the earthpy install worked perfectly!

[lwasser]

oh good. @katysill i noticed some issues with the older version of rasterio... i'm glad that worked for you. we will have many install kinks to work out!!

[lwasser]

ok i am going to close this issue given this pr
#231
if it comes up again we can re-open it!!