Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update cookiecutter to use GitHub Actions #5

Merged
merged 16 commits into from
Apr 12, 2021

Conversation

trallard
Copy link
Member

@trallard trallard commented Feb 18, 2021

This PR relates to jupyter/docker-stacks#1139.

I am working on swapping from Travis to GH Actions, however there are a couple of things I wanted to check before moving forward:

  • The current version of the cookie-cutter uses Python 3.6. To keep in sync with docker-stacks I am upgrading to 3.8

For the actual {{cookiecutter}}

  • I made this initial draft using the official docker action to login to Docker Hub. I have been using this on other repos and has been working well so far.
  • I noticed there is a post_push hook which uses the sha to tag the image with the SHA and push to Docker. Though I do not think this currently in use within the Travis CI. I recently did some work on Docker for SciPy and do the build + push directly from within GH Actions (see https://github.com/scipy/scipy/blob/master/.github/workflows/docker.yml for reference) I could follow a similar approach here but do not want to impose any changes without discussing first

@trallard trallard marked this pull request as draft February 18, 2021 13:30
@parente
Copy link
Member

parente commented Feb 27, 2021

Thanks for working on this @trallard. I totally forgot this cookiecutter was in a separate repo from jupyter/docker-stacks. (I've been living in monorepos lately.) I'll add the other maintainers of the docker-stacks here so pull requests like this one don't go unnoticed in my GitHub notifications alone for so long.

The current version of the cookie-cutter uses Python 3.6. To keep in sync with docker-stacks I am upgrading to 3.8

+1 Feel free to move to the latest stable Python release if it's easy as well.

I made this initial draft using the official docker action to login to Docker Hub. I have been using this on other repos and has been working well so far.

+1 Makes sense to use the official actions when available. Ideally this cookiecutter would give users the ability to push images to any registry, but that's definitely out of scope of this PR. (Noting only as a potential future project for anyone following along.)

I noticed there is a post_push hook which uses the sha to tag the image with the SHA and push to Docker. Though I do not think this currently in use within the Travis CI. I recently did some work on Docker for SciPy and do the build + push directly from within GH Actions (see https://github.com/scipy/scipy/blob/master/.github/workflows/docker.yml for reference) I could follow a similar approach here but do not want to impose any changes without discussing first

The original CI/CD implementation in jupyter/docker-stacks built images on Travis for pull requests only, but then used Docker Hub CI to build merges to the main branch and push to the Docker Hub registry. The post_push hook is a script honored by builds done on Docker Hub and was what we used to apply the git SHA tags to images built there. This entire design was a giant workaround for build time limits on Travis and is ripe for replacement.

I think it'd be best to move the cookiecutter to building and pushing PR and main branch images from GitHub actions like we now do in jupyter/docker-stacks. I think it's fine to use whatever technique you think works best to apply the git SHA as a tag given that we'd be replacing the entire CI/CD implementation anyway.

@trallard trallard marked this pull request as ready for review March 31, 2021 14:19
@trallard trallard changed the title [WIP] - Update cookiecutter to use GitHub Actions Update cookiecutter to use GitHub Actions Mar 31, 2021
@trallard
Copy link
Member Author

I have finally made time to finish this off. I went ahead with the changes discussed above. However, there are some outstanding actions / decisions:

  • I wrote the GH action so that it pushes a latest and a SHA tagged image
  • With this action then the contents in hooks becomes irrelevant and can be removed
  • The travis.yml files can now be deleted but I wanted to check if you all are all happy with this implementation before going ahead

@romainx romainx self-requested a review April 11, 2021 06:40
romainx added 5 commits April 11, 2021 11:18
- push cookiecutter variables where not converted
- fix line break introduced by raw blocks
- Add main branch along with master to trigger the build / push on the main branch
- Two steps builld and push to be able to test
- Use `driver: docker` to be able to test locally
- Fix dev target
- Add run target to run a shell (for debuging purpose)
Copy link
Collaborator

@romainx romainx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @trallard,

Many thanks for this PR. I've done a couple of fixes / addition.

  • docker.yml
    • Create two distinct steps build and push to be able to test in between.
    • Use driver: docker to be able to test locally.
    • Fix cookiecutter variables in push where not converted.
    • Fix line break introduced by raw blocks.
    • Add main branch along with master to trigger the build / push on the main branch.
  • Makefile
    • Fix dev target
    • Add run target to run a shell (for debugging purpose)
  • Add default .dockerignore

I've tested and successfully built an image from this template.

I think you can now safely remove

  • hooks: they are useless and moreover this system will be replaced in docker stack. If users need to add additional tags, they will be able to add them in the GitHub docker.yml file.
  • .travis.yml files (in template and main repo) since they will be replaced by corresponding GitHub actions.

After that I think we will be good for the merge. If we need to refine or improve, we will do it in a follow-on PR.

Thanks.
Best

Copy link
Member

@mathbunnyru mathbunnyru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job @trallard and @romainx as well. I'm not fully available and equipped at the moment, but I wrote few minor things. Feel free to ignore them.

.github/workflows/CI.yml Show resolved Hide resolved
.github/workflows/CI.yml Outdated Show resolved Hide resolved
@trallard
Copy link
Member Author

Thanks for your reviews @romainx and @mathbunnyru I went ahead and deleted the travis.yaml files and the hooks

Unless there is anything else outstanding this should be good to merge

@romainx
Copy link
Collaborator

romainx commented Apr 12, 2021

Hello Ok let's merge it. We will fix issues -- if any -- in follow-on PR.
@trallard & @mathbunnyru thanks for this work.

@romainx romainx merged commit 35208be into jupyter:master Apr 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants