XC7 install problem: ".../symbiflow-examples/xc7/condaenv.asvir9n0.requirements.txt"

[tcal-x]

I encountered this today, and made a new clone of the repository on a different laptop, and ran into the same issue.

In the conda env create -f $FPGA_FAM/environment.yml step, I get this error (note the garbled path to requirements.txt in the python/pip command about 8 lines down; you probably need to scroll right to see it):

...
make-4.3             | 507 KB    | ############################################################################# | 100% 
pcre2-10.37          | 1.1 MB    | ############################################################################# | 100% 
_openmp_mutex-4.5    | 22 KB     | ############################################################################# | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: \ Ran pip subprocess with arguments:
['/home/tim/symbiflow/install/xc7/conda/envs/xc7/bin/python', '-m', 'pip', 'install', '-U', '-r', '/media/tim/GIT/tmp/symbiflow-examples/xc7/condaenv.asvir9n0.requirements.txt']
Pip subprocess output:

Pip subprocess error:
ERROR: Exception:
Traceback (most recent call last):
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 173, in _main
    status = self.run(options, args)
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 203, in wrapper
    return func(self, options, args)
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 286, in run
    reqs = self.get_requirements(args, options, finder, session)
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 385, in get_requirements
    filename, finder=finder, options=options, session=session
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_internal/req/req_file.py", line 135, in parse_requirements
    for parsed_line in parser.parse(filename, constraint):
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_internal/req/req_file.py", line 319, in parse
    yield from self._parse_and_recurse(filename, constraint)
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_internal/req/req_file.py", line 348, in _parse_and_recurse
    yield from self._parse_and_recurse(req_path, nested_constraint)
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_internal/req/req_file.py", line 324, in _parse_and_recurse
    for line in self._parse_file(filename, constraint):
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_internal/req/req_file.py", line 353, in _parse_file
    _, content = get_file_content(filename, self._session)
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_internal/req/req_file.py", line 518, in get_file_content
    resp = session.get(url)
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_internal/network/session.py", line 454, in request
    return super().request(method, url, *args, **kwargs)
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 697, in send
    r.content
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_vendor/requests/models.py", line 836, in content
    self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
  File "/home/tim/symbiflow/install/xc7/conda/envs/xc7/lib/python3.7/site-packages/pip/_vendor/requests/models.py", line 769, in generate
    chunk = self.raw.read(chunk_size)
AttributeError: 'FileNotFoundError' object has no attribute 'read'

failed

CondaEnvException: Pip failed

[mithro]

FYI - @kgugala

[tcal-x]

The Miniconda 'latest' pointer was recently updated, and is now a py39 version. I don't know what it was before. I can try using a py37 version.

From https://repo.anaconda.com/miniconda/ :

Miniconda3-latest-Linux-x86_64.sh | 63.6M | 2021-07-21 11:05:08

Index of /

[tcal-x]

I tried Miniconda3-py37_4.9.2-Linux-x86_64.sh and encountered the same error.

[tcal-x]

I noticed the identical issue in Fomu Sphinx build tests (garbled path to requirements.txt): https://github.com/im-tomu/fomu-workshop/runs/3194022998?check_suite_focus=true

Installing pip dependencies: ...working... Ran pip subprocess with arguments:
['/home/runner/work/fomu-workshop/fomu-workshop/docs/env/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/runner/work/fomu-workshop/fomu-workshop/docs/condaenv.mqhkfsw0.requirements.txt']

GitHub

Build software better, together

GitHub is where people build software. More than 65 million people use GitHub to discover, fork, and contribute to over 200 million projects.

[kgugala]

this does seem to be some kind of conda issue. I could reproduce it when running installation native on my machine, but could not when running it in Debian 10 Docker.

@tcal-x can you try removing file: prefix here https://github.com/SymbiFlow/symbiflow-examples/blob/master/xc7/environment.yml#L20

GitHub

symbiflow-examples/environment.yml at master · SymbiFlow/symbiflow-examples

Example designs showing different ways to use SymbiFlow toolchains. - symbiflow-examples/environment.yml at master · SymbiFlow/symbiflow-examples

[tcal-x]

@kgugala , yes, if I remove file: where you suggest, it then works correctly for me, and I'm able to build counter_test.

[kgugala]

@tcal-x @mithro it seems this commit in PIP pypa/pip@f6b184c breaks this. It adds special handling for links starting with file here https://github.com/pypa/pip/blame/main/src/pip/_internal/req/req_file.py#L514, so instead of going with simple f.read() path we were going through session.get().
I suppose this is a conda issue - Conda env resolver should either strip the file: prefix or change it to file://.

See comment in pip issue pypa/pip#10237

GitHub

pip/src/pip/_internal/req/req_file.py at main · pypa/pip

The Python package installer. Contribute to pypa/pip development by creating an account on GitHub.

[tcal-x]

Thanks @kgugala for tracking it down!

[mithro]

@kgugala - It seems like this should be reported upstream somewhere too?

[kgugala]

@kgugala - It seems like this should be reported upstream somewhere too?

it already is conda/conda#10815. I commented there with what I found.