Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to GitHub Actions for CI/CD #1132

Merged
merged 11 commits into from
Aug 8, 2020
Merged

Conversation

parente
Copy link
Member

@parente parente commented Jul 30, 2020

This pull request re-implements the docker-stacks build pipeline for Docker images, Sphinx documentation, and Transifex translation templates in GitHub Actions instead of a mix of Travis CI and Docker Hub automated builds.

I'll leave this PR open for review and feedback for the next few days. I'll plan to merge after that if there's no strong push back or everyone is too busy to review as the pros outweigh the cons.

Pros

  • One continuous integration and deployment technology
  • Publicly visible build logs for images pushed to Docker Hub (Logs for current Docker Hub builds are private and cannot be made public as far as I can tell)
  • Longer build runtime allowance: 6 hour per job, 72 hours per workflow (Runtime limits on Travis have caused trouble in the past)
  • Single build job for all images triggered for a git SHA instead of an independent chain of Docker Hub jobs where downstream jobs may pick up a different master HEAD (e.g., someone merges to master while builds are in progress)
  • Ability to make the build hook scripts more readable in other languages in a future PR

Cons

  • More reliance on GitHub alone?

Testing

I manually tested steps that run on PR push in my fork. I did not test steps that run on merge to master. We'll need to iterate after merging to work out any issues with those final steps.

Other Notes

  • I added wait and sleep to a couple tests that lacked them and failed intermittently.
  • We'll need to disable the base-notebook image automated build in Docker Hub before merging this PR to avoid a double build.
  • Once we're comfy with this new approach, we should disable the chain of build jobs on Docker Hub and remove the SSH deploy key used to push to the wiki.

@parente parente requested a review from romainx July 31, 2020 12:30
Copy link
Collaborator

@romainx romainx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest it's not easy to figure out by reviewing a CI PR if everything will work properly 😅.
However it's worth switching to GitHub CI since it will simplify the whole process. It will definitely make our lives easier.

My only remark concerns the hooks that contain a lot of duplication. Do you think it would be possible to define a base hook common to all images (for example part of the tags) and specific hooks when needed?

Thanks for this work. Let me know if I can help further.

Best.

@parente
Copy link
Member Author

parente commented Aug 7, 2020

My only remark concerns the hooks that contain a lot of duplication. Do you think it would be possible to define a base hook common to all images (for example part of the tags) and specific hooks when needed?

Yes. I'd really like the post-build scripts to be replaced with a single Python script with templates instead of duplicative bash scripts and heredoc magic. However, I'd like to delay replacing them until after we get GitHub Actions functioning properly which, as you said, will probably take some trial and error after merging.

@parente
Copy link
Member Author

parente commented Aug 7, 2020

Thanks for taking a look, @romainx. I've got time today to iterate on making this work, so I'm going to go ahead and merge. If some unexpected blocker arises, I can revert everything and get us back to using Docker Hub while we regroup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants