tljh-config ContextualVersionConflict - chardet 4 vs jupyterhub-traefik-proxy

[ixcat]

Bug description

tljh-config failing to run after a fresh container rebuild of integration-tests/Dockerfile . Was working for me last install round.

as best I can tell - something in the tljh install is installing chardet 4, which conflicts with the pinned aiohttp==3.6.2 in jupyterhub-traefik-proxy; tried to pip upgrade chardet/aiohttp, but this triggered errors. Not sure if it would be better to file against jupyterhub-traefik-proxy or here; filing here since this is where I encountered the error, and concievably the aiohttp pin is deliberate in jupyterhub-traefik-proxy

was able to resolve via:

. /opt/tljh/hub/bin/activate
pip uninstall chardet
pip install chardet==3.0.4

Expected behaviour

tljh-config should run

Actual behaviour

root@b65e93e47512:/srv/src# tljh-config set auth.GenericOAuthenticator.client_id 'client1'
Traceback (most recent call last):
  File "/opt/tljh/hub/lib/python3.6/site-packages/pkg_resources/__init__.py", line 574, in _build_master
    ws.require(__requires__)
  File "/opt/tljh/hub/lib/python3.6/site-packages/pkg_resources/__init__.py", line 892, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/tljh/hub/lib/python3.6/site-packages/pkg_resources/__init__.py", line 783, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/opt/tljh/hub/lib/python3.6/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/tljh-config", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/opt/tljh/hub/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3088, in <module>
    @_call_aside
  File "/opt/tljh/hub/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3072, in _call_aside
    f(*args, **kwargs)
  File "/opt/tljh/hub/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3101, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/opt/tljh/hub/lib/python3.6/site-packages/pkg_resources/__init__.py", line 576, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/opt/tljh/hub/lib/python3.6/site-packages/pkg_resources/__init__.py", line 589, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/opt/tljh/hub/lib/python3.6/site-packages/pkg_resources/__init__.py", line 783, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (chardet 4.0.0 (/opt/tljh/hub/lib/python3.6/site-packages), Requirement.parse('chardet<4.0,>=2.0'), {'aiohttp'})

How to reproduce

build using integration-tests/Dockerfile , try to run tljh-config

Your personal set up

Ubuntu 20.04
Docker version 19.03.14, build 5eb3275d40
docker-compose version 1.26.2, build eefe0d31

Also, as a tweak, I added a systemd oneshot to the docker which runs the tljh-install script on container up - if interested, can PR:

 diff -urw the-littlest-jupyterhub/integration-tests/Dockerfile  Dockerfile.tljh
--- the-littlest-jupyterhub/integration-tests/Dockerfile	2020-12-14 15:11:17.565877744 -0600
+++ Dockerfile.tljh	2020-12-17 18:55:16.663907039 -0600
@@ -1,6 +1,14 @@
 # Systemd inside a Docker container, for CI only
+# via:
+#
+# https://tljh.jupyter.org/en/latest/contributing/dev-setup.html#contributing-dev-setup
+# https://github.com/jupyterhub/the-littlest-jupyterhub/blob/master/integration-tests/Dockerfile
+
 FROM ubuntu:18.04
 
+ARG TLJH_ADMIN_USER=admin
+ARG TLJH_ADMIN_PASSWORD=admin
+
 RUN apt-get update --yes
 
 RUN apt-get install --yes systemd curl git sudo
@@ -16,7 +24,18 @@
 
 RUN mkdir -p /etc/sudoers.d
 
-RUN systemctl set-default multi-user.target
+COPY ./tljh-install.service /etc/systemd/system/tljh-install.service
+
+# set default systemd target, enable tljh-install via symlink
+
+RUN systemctl set-default multi-user.target \
+	&& ln -s /etc/systemd/system/tljh-install.service \
+		/etc/systemd/system/multi-user.target.wants/ \
+	&& echo "TLJH_ADMIN_USER=${TLJH_ADMIN_USER}" > \
+		/etc/default/tljh-install \
+	&& echo "TLJH_ADMIN_PASSWORD=${TLJH_ADMIN_PASSWORD}" >> \
+		/etc/default/tljh-install
+
 
 STOPSIGNAL SIGRTMIN+3
 

and:

$ cat /dev/null tljh-install.service 
[Unit]
Description=Install TLJH

[Service]
Type=oneshot
EnvironmentFile=/etc/default/tljh-install
ExecStart=/usr/bin/python3 /srv/src/bootstrap/bootstrap.py --admin ${TLJH_ADMIN_USER}:${TLJH_ADMIN_PASSWORD}

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[6palace]

+1, thanks for finding the fix!

[eizanendoso]

+1, thank you so much!

[wallytutor]

Facing the same issue here but straight from my personal Ubuntu 20.04 server (no container, used the standard curl -L https://tljh.jupyter.org/bootstrap.py | sudo -E python3 - --admin <admin-user-name> to deploy the server). Already tried uninstalling/installing the packages with the environment provided by tljh active and no success. One thing that bothers me is that the /opt/tljh/user/bin/python is 3.7.3 but the environment site-packages is in a folder /opt/tljh/hub/lib/python3.8/. Any suggestions? Thanks.

[onwilliam]

I have the same issues, when i try to do pip uninstall chardet I get PermissionError: [Errno 13]. Any fix please?

[GeorgianaElena]

Thanks a lot for reporting this. I have opened #643 to pin the chardet version as per @ixcat's suggestion (thanks a lot @ixcat) until we figure out what caused this in the first place.

One thing that bothers me is that the /opt/tljh/user/bin/python is 3.7.3 but the environment site-packages is in a folder /opt/tljh/hub/lib/python3.8/.

@waltermateriais, the Python that's used in the hub env is the same with the system's default version (which for Ubuntu 18.04 is 3.6 and now for Ubuntu 20.04 is 3.8). We should update the Python version for the user environment too. There is an open issue about this (#618) and this should happen soon.

Thank you! ☀️

[wallytutor]

Thanks @GeorgianaElena!! I tried for a good moment but couldn't find evidences of order of precedence errors between the system python (3.8 in my case) and the virtual environment (3.7). Initially thought that could be a track to solve this issue but apparently it comes from elsewhere after I read the tljh scripts in detail.

[meeseeksmachine]

This issue has been mentioned on Jupyter Community Forum. There might be relevant details there:

https://discourse.jupyter.org/t/running-juptyerlab-with-jupyterhub-tljh-on-own-server/7249/5

[jkmackie]

I have the same issues, when i try to do pip uninstall chardet I get PermissionError: [Errno 13]. Any fix please?

@onwilliam First, activate the hub environment as per ixcat's instruction at the top. I did this in the JupyterHub terminal.

. /opt/tljh/hub/bin/activate

Then try this command:

sudo -H pip uninstall chardet

[meeseeksmachine]

This issue has been mentioned on Jupyter Community Forum. There might be relevant details there:

https://discourse.jupyter.org/t/running-juptyerlab-with-jupyterhub-tljh-on-own-server/7249/7

[GeorgianaElena]

Fixed in #643