PDK Not Building

[donn]

Describe the bug
The PDK is not building properly, apparently because of a conda issue with https://github.com/google/skywater-pdk.

Issue is currently affecting CI, myself and a number of users.

To Reproduce
make pdk

Expected behavior
PDK is made

Desktop (please complete the following information):

• OS: Ubuntu 20.04/macOS 11.x
• Openlane Version: HEAD
• open_pdks Version: de95d784746c0a727a3e66765c76c605c5512179
• skywater-pdk Version: 00bdbcf4a3aa922cc1f4a0d0cd8b80dbd73149d3

Additional context

cd /usr/local/pdk/skywater-pdk && \
		git checkout main && git submodule init && git pull --no-recurse-submodules && \
		git checkout -qf 00bdbcf4a3aa922cc1f4a0d0cd8b80dbd73149d3
Switched to branch 'main'
Your branch is up to date with 'origin/main'.
Already up to date.
cd /usr/local/pdk/skywater-pdk && \
		git submodule update --init libraries/sky130_fd_sc_hd/latest && \
		git submodule update --init libraries/sky130_fd_io/latest && \
		git submodule update --init libraries/sky130_fd_sc_hvl/latest && \
		/Applications/Xcode.app/Contents/Developer/usr/bin/make timing
source /usr/local/pdk/skywater-pdk/env/conda/bin/activate && conda env update --name skywater-pdk-scripts --file environment.yml
Collecting package metadata (repodata.json): done
Solving environment: done
Installing pip dependencies: | Ran pip subprocess with arguments:
['/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/bin/python', '-m', 'pip', 'install', '-U', '-r', '/usr/local/pdk/skywater-pdk/condaenv.zkyq4omm.requirements.txt']
Pip subprocess output:

Pip subprocess error:
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 173, in _main
    status = self.run(options, args)
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 203, in wrapper
    return func(self, options, args)
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 286, in run
    reqs = self.get_requirements(args, options, finder, session)
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 384, in get_requirements
    for parsed_req in parse_requirements(
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal/req/req_file.py", line 135, in parse_requirements
    for parsed_line in parser.parse(filename, constraint):
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal/req/req_file.py", line 319, in parse
    yield from self._parse_and_recurse(filename, constraint)
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal/req/req_file.py", line 348, in _parse_and_recurse
    yield from self._parse_and_recurse(req_path, nested_constraint)
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal/req/req_file.py", line 324, in _parse_and_recurse
    for line in self._parse_file(filename, constraint):
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal/req/req_file.py", line 353, in _parse_file
    _, content = get_file_content(filename, self._session)
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal/req/req_file.py", line 518, in get_file_content
    resp = session.get(url)
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal/network/session.py", line 454, in request
    return super().request(method, url, *args, **kwargs)
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 697, in send
    r.content
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_vendor/requests/models.py", line 836, in content
    self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
  File "/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_vendor/requests/models.py", line 769, in generate
    chunk = self.raw.read(chunk_size)
AttributeError: 'FileNotFoundError' object has no attribute 'read'

failed

CondaEnvException: Pip failed

make[1]: *** [/usr/local/pdk/skywater-pdk/env/conda/envs/skywater-pdk-scripts/bin/python] Error 1
make: *** [skywater-library] Error 2

[donn]

Workaround from Eugene Stolbikov :

Looks like I found a workaround:

1. Run 'make pdk' and got error during pip dependancy installation.
2. Go to $PDK_ROOT/skywater-pdk
3. Get pip dependancy installation command line from screen log and run it with correct name of requirements.txt file: ...skywater-pdk/env/conda/envs/skywater-pdk-scripts/bin/python -m pip install -U -r $PDK_ROOT/skywater-pdk/requirements.txt
4. Go back to openlane dir
5. Re-run 'make pdk'
6. 'make test' in openlane show that PDK was installed successfully

[mithro]

I think this is another case of chipsalliance/f4pga-examples#169 / conda/conda#10815

From @kgugala @ chipsalliance/f4pga-examples#169 (comment)

@tcal-x @mithro it seems this commit in PIP pypa/pip@f6b184c breaks this. It adds special handling for links starting with file here https://github.com/pypa/pip/blame/main/src/pip/_internal/req/req_file.py#L514, so instead of going with simple f.read() path we were going through session.get().
I suppose this is a conda issue - Conda env resolver should either strip the file: prefix or change it to file://.

See comment in pip issue pypa/pip#10237

[donn]

Fixed by #486

[dralabeing]

I ran into this problem again today:
['/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/bin/python', '-m', 'pip', 'install', '-U', '-r' , '/home/indir/openlane/openlane/pdks/skywater-pdk/condaenv.3eigdwon.requirements.txt']
Pip subprocess output:

Pip subprocess error:
ERROR: Exception:
Traceback (most recent call last):
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal /cli/base_command.py", line 173, in _main
status = self.run(options, args)
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal /cli/req_command.py", line 203, in wrapper
return func(self, options, args)
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal /commands/install.py", line 286, in run
reqs = self.get_requirements(args, options, finder, session)
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal /cli/req_command.py", line 384, in get_requirements
for parsed_req in parse_requirements(
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal /req/req_file.py", line 135, in parse_requirements
for parsed_line in parser.parse(filename, constraint):
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal /req/req_file.py", line 319, in parse
yield from self._parse_and_recurse(filename, constraint)
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal /req/req_file.py", line 348, in _parse_and_recurse
yield from self._parse_and_recurse(req_path, nested_constraint)
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal /req/req_file.py", line 324, in _parse_and_recurse
for line in self._parse_file(filename, constraint):
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal /req/req_file.py", line 353, in _parse_file
_, content = get_file_content(filename, self._session)
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal /req/req_file.py", line 518, in get_file_content
resp = session.get(url)
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_vendor/r equests/sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_internal /network/session.py", line 454, in request
return super().request(method, url, *args, **kwargs)
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_vendor/r equests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_vendor/r equests/sessions.py", line 697, in send
r.content
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_vendor/r equests/models.py", line 836, in content
self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
File "/home/indir/openlane/openlane/pdks/skywater-pdk/env/conda/envs/skywater-pdk-scripts/lib/python3.8/site-packages/pip/_vendor/r equests/models.py", line 769, in generate
chunk = self.raw.read(chunk_size)
AttributeError: 'FileNotFoundError' object has no attribute 'read'
fa iled

CondaEnvException: Pip failed

make[1]: *** [/home/indir/openlane/openlane/pdks/skywater-pdk/third_party/make-env/conda.mk:145: /home/indir/openlane/openlane/pdks/s kywater-pdk/env/conda/envs/skywater-pdk-scripts/bin/python] Error 1
make[1]: Leaving directory '/home/indir/openlane/openlane/pdks/skywater-pdk'
make: *** [Makefile:88: skywater-library] Error 2

[dralabeing]

See problem report posted above

[d-m-bailey]

@dralabeing Thanks for posting.

Could you check your $PDK_ROOT/skywater-pdk/environment.yml file?

If the last line is
- -r file:requirements.txt
then your repo is out of date. The file: is causing the problem.

This may be caused by the open_pdks repo being out of date, which in turn may be caused by the openlane repo being out of date.

Here's a quick fix if you don't want to update your local openlane.

https://skywater-pdk.slack.com/archives/C016UL7AQ73/p1628617713103000?thread_ts=1628586609.100100&cid=C016UL7AQ73

[dralabeing]

@d-m-bailey Thanks for the work around..noted. I updated openLane and this fixed the problem barring an error in import yaml:
make pdk
Traceback (most recent call last):
File "./dependencies/tool.py", line 15, in
import yaml <---
ModuleNotFoundError: No module named 'yaml'

...This was fixed by explicitly installing the correct pip3 library.

Suggestions:

1. Update documentation with the correct images to update
2. provide info to install dependencies

[d-m-bailey]

@dralabeing See #544. Looks like yaml is documented as a prerequisite that needs to be installed before openlane. You can close this issue if your satisfied.

[dralabeing]

ok thanks @d-m-bailey