Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Travis] Added logging into Docker Hub #644

Merged
merged 1 commit into from
Jan 8, 2021
Merged

[Travis] Added logging into Docker Hub #644

merged 1 commit into from
Jan 8, 2021

Conversation

mnocon
Copy link
Member

@mnocon mnocon commented Jan 7, 2021

JIRA: https://issues.ibexa.co/browse/EZP-32270

Introduction

Docker Hub introduces rate limiting when downloading images. From https://www.docker.com/blog/docker-hub-image-retention-policy-delayed-and-subscription-updates/ :

 Anonymous free users will be limited to 100 pulls per six hours, and authenticated free users will be limited to 200 pulls per six hours. 

This makes our build fail from time to time with:

(selenium/standalone-chrome-debug:3.141.59-20200326)...
3.141.59-20200326: Pulling from selenium/standalone-chrome-debug
stable: Pulling from library/nginx
ERROR: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

(from https://travis-ci.com/github/ezsystems/ezplatform-http-cache/jobs/469283098 )

Travis also has a blogpost about it: https://blog.travis-ci.com/docker-rate-limits
with the key line being Travis CI has a built in registry cache for Docker builds .

I've received confirmation from Travis support that the registry cache is active only when we're authenticated to Docker:

Our Registry is designed to work as follows:

* Following Docker's rate limiting of anonymous pulls, we require builds to authenticate to Docker to ensure a consistent result of successful pulls unaffected by rate limits.
* After a successful, authenticated pull from Docker, new builds can take advantage of the Travis Registry for new builds.
* We recommend running builds while authenticated to avoid issues that may arise die to lack of authentication.

This comes with an obvious downside - env variables are not available when PRs are made from forks, which means PRs from forks won't be using the Travis registry cache and might encounter Docker hub rate limiting.
There are two solutions to that:

  1. make our Docker Hub credentials public - not a good idea as they could be abused
  2. explore caching images using Travis build cache - also not an elegant solution, at least in the ways I've explored it

This PR aims to solve the issue for our internal use, I think we can spend more time later to think about the best solution that takes our contributors into account (one of the above or something else completely).

What also needs to be done, if this is accepted: configure the DOCKER_USERNAME and DOCKER_PASSWORD variables for each repository.

Results

Build passing without Docker credentials configured:
https://travis-ci.com/github/ezsystems/ezplatform/builds/212349699

Build passing with Docker credentials configured:
https://travis-ci.com/github/ezsystems/ezplatform/jobs/469590356

@mnocon mnocon marked this pull request as ready for review January 8, 2021 12:59
Copy link
Member

@micszo micszo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's go for it.

@micszo micszo merged commit 5563489 into 2.5 Jan 8, 2021
@micszo micszo deleted the add-docker-token branch January 8, 2021 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

3 participants