Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for installing TLJH on Arm64 systems and bump traefik (1.7.18 -> 1.7.33) #679

Merged
merged 3 commits into from
Nov 3, 2021

Conversation

cdibble
Copy link
Contributor

@cdibble cdibble commented Apr 4, 2021

Fixes #62
Fixes #727

  • [X ] Add / update documentation
  • This PR shouldn't change anything from the user's point of view, but I added a couple of lines to state explicitly that this supports ARM 64 and AMD64 architectures and that it uses a miniforge conda environment.
  • [ X] Add tests
  • Updated tests/test_conda.py to use the miniforge installer instead of miniconda.

@welcome
Copy link

welcome bot commented Apr 4, 2021

Thanks for submitting your first pull request! You are awesome! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please make sure you followed the pull request template, as this will help us review your contribution more quickly.
welcome
You can meet the other Jovyans by joining our Discourse forum. There is also a intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

@meeseeksmachine
Copy link

This pull request has been mentioned on Jupyter Community Forum. There might be relevant details there:

https://discourse.jupyter.org/t/tljh-on-m1-mac-arm-docker-installer-is-x86-specific/10894/2

@consideRatio
Copy link
Member

consideRatio commented Oct 22, 2021

Thanks for your work on this @cdibble!!

In a recent maintenance push of TLJH, we are now installing mambaforge (#697). Would you be interested in reworking this PR to just support Arm64?


@yuvipanda @GeorgianaElena @manics - do you think it's important that we keep checking checksums against for example mambaforge or traefik when we install them? It adds notable maintenance efforts that goes hand in hand with strictly pinning versions.

We opted to remove that from jupyter/docker-stacks as part of no longer strictly pinning everything but instead clearly providing reports on what is installed in each given release.

@manics
Copy link
Member

manics commented Oct 22, 2021

I can think of two main uses for checksums:

  • protection against tampering: downloads are over https so this requires a compromise of a GitHub account to modify the binary. There's probably loads of other packages we don't check though so I think it's fine to drop the checksum for this purpose
  • checking for corrupted/partial downloads: does the installer have a built-in checksum/self-check, if so we can definitely drop our check. Otherwise it would be useful to know how many tljh installations are in areas with poor networking where keeping the checksum could be beneficial.

@cdibble
Copy link
Contributor Author

cdibble commented Oct 25, 2021

Thanks for your work on this @cdibble!!

In a recent maintenance push of TLJH, we are now installing mambaforge (#697). Would you be interested in reworking this PR to just support Arm64?

Hey @consideRatio - Yeah I could rework this a bit. It has been a minute, so I'll have to refresh a bit.

I saw the mambaforge PR (nice!)- It looks like mambaforge can handle x86, but not arm64. Is that right? So you'd like to see miniforge used for arm64 builds? Seems pretty straightforward. LMK if that is what you had in mind and I'll re-work this PR.

@manics
Copy link
Member

manics commented Oct 25, 2021

There are Linux mambaforge packages for amd64 and arm64:
https://github.com/conda-forge/miniforge#mambaforge

@consideRatio
Copy link
Member

consideRatio commented Oct 25, 2021

@cdibble wieee nice, thanks for sticking with this project! ❤️ 🎉

I saw the mambaforge PR (nice!)- It looks like mambaforge can handle x86, but not arm64. Is that right? So you'd like to see miniforge used for arm64 builds? Seems pretty straightforward. LMK if that is what you had in mind and I'll re-work this PR.

Yepp this is what I had in mind! I suggest you go for adding checksums etc as well at this point so we don't have to discuss if we should stop using checksums at this point in time. We can always create a PR later to remove them.

@cdibble
Copy link
Contributor Author

cdibble commented Oct 26, 2021

@consideRatio , @manics Do either of you have access to an M1 Mac so that you could what the result of python3 -c 'import os; print(os.uname().machine)' looks like on that platform? I am unsure whether that will show arm64, which is necessary to get the right installer for that platform.

Edit: It looks we can actually just skip this, since only Linux seems to be supported by TLJH! Nevermind!

@cdibble cdibble changed the title Supporting ARM64 (and still AMD64) architectures. Traefik binaries adjusted. Swapped miniforge in for miniconda. Supporting ARM64 (and still AMD64) architectures with mambaforge Oct 26, 2021
@cdibble
Copy link
Contributor Author

cdibble commented Oct 28, 2021

@consideRatio @manics Alrighty so I think this works now- i tested it on Ubuntu 20.x on both ARM and x86 machines and was able to successfully install and run TLJH. However, it's slightly tricky to test definitively because I needed to point the installer to my fork/branch in bootstrap.py. Those changes are reverted now on this branch, so the PR should be ready to go and the install should pull from jupyterhub/the-littlest-jupyterhub. If you notice any issues please LMK and I will happily revise.

Copy link
Member

@consideRatio consideRatio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your work on this @cdibble! I've noticed a few issues related to merging the changes made into the main branch, and that a few artifacts should be cleaned up as well.

@cdibble are you okay with me cleaning up the git history of this PR a bit and applying the changes suggested in my comments?

If you are, then I figure I'll:

  • apply the suggestions
  • squash a lot of commits to cleanup the git history that has a lot of back and forth changes
  • force-push a revised git history where you remain listed as the author of the commits
  • ping you for review of the changes

tljh/utils.py Outdated Show resolved Hide resolved
tljh/traefik.py Outdated Show resolved Hide resolved
tljh/traefik.py Outdated Show resolved Hide resolved
tljh/installer.py Outdated Show resolved Hide resolved
tljh/conda.py Outdated Show resolved Hide resolved
setup.py Outdated Show resolved Hide resolved
docs/topic/installer-actions.rst Outdated Show resolved Hide resolved
docs/index.rst Outdated Show resolved Hide resolved
bootstrap/bootstrap.py Outdated Show resolved Hide resolved
.circleci/config.yml Show resolved Hide resolved
@consideRatio consideRatio changed the title Supporting ARM64 (and still AMD64) architectures with mambaforge Add support for installing TLJH on Arm64 systems Oct 31, 2021
@cdibble
Copy link
Contributor Author

cdibble commented Nov 1, 2021

Hey @consideRatio - yes thank you for picking up the slack. I did bumble the git history badly didn't I. My apologies- a result of having rolled back a bunch of the changes in this original PR and then add some new changes from there. My own attempt to squash was ugly and I was too distracted with work projects to re-do it last week. So my apologies for that and thank you for making those changes.

Would you like me to go ahead and commit those suggestions to the PR?

@consideRatio consideRatio force-pushed the dev_connor branch 2 times, most recently from a24910f to fe37e2d Compare November 3, 2021 21:51
@consideRatio
Copy link
Member

@cdibble I've squashed this PR into a few commits now, does this look okay to you?

I refactored some indentation etc as well in circleci config. I have a backup available of the previous PR state, and you probably have it as well on your local computer.

@cdibble
Copy link
Contributor Author

cdibble commented Nov 3, 2021

@cdibble I've squashed this PR into a few commits now, does this look okay to you?

I refactored some indentation etc as well in circleci config. I have a backup available of the previous PR state, and you probably have it as well on your local computer.

LGTM. Thanks for doing that @consideRatio. You could drop that commit 08d937b and go back to the TLJH default of tries=20 -- I had only bumped that up to help out with some dev and I didn't really mean for it to sneak into the PR. Of course, it doesn't really hurt anything, but in most cases where things actually work, tries=20 is plenty.

@consideRatio
Copy link
Member

but in most cases where things actually work, tries=20 is plenty.

Ah I wondered about that! Okay thanks for clarifying this @cdibble, and thanks for your work with this PR, thanks for sticking with it even though we were very low on maintenance capacity for a while in this repo! ❤️ 🎉

@cdibble
Copy link
Contributor Author

cdibble commented Nov 3, 2021

Awesome @consideRatio - I am glad to be of some help and I much appreciate this project and all of its contributors!

My apologies for the added overhead here- my own need to refresh my knowledge of what was going on led to some fumbling.

@consideRatio consideRatio merged commit e75de14 into jupyterhub:main Nov 3, 2021
@welcome
Copy link

welcome bot commented Nov 3, 2021

Congrats on your first merged pull request in this project! 🎉
congrats
Thank you for contributing, we are very proud of you! ❤️

@consideRatio
Copy link
Member

Wieee! Thanks again @cdibble!!!! ❤️ 🎉

@consideRatio consideRatio added dependencies Pull requests that update a dependency file enhancement New feature or request labels Nov 3, 2021
@consideRatio consideRatio changed the title Add support for installing TLJH on Arm64 systems Add support for installing TLJH on Arm64 systems and bump traefik (1.7.18 -> 1.7.33) Nov 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bump traefik version Supporting ARM architectures
4 participants