Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download hadoop-2.8.4.tar.gz during release building or third-parties image building for unit tests #1164

Closed
2 tasks done
empiredan opened this issue Sep 16, 2022 · 0 comments · Fixed by #1167 or #1168
Closed
2 tasks done
Labels
type/enhancement Indicates new feature requests

Comments

@empiredan
Copy link
Contributor

empiredan commented Sep 16, 2022

hadoop-2.8.4.tar.gz failed to be downloaded very frequently for workflows as follows:

image

image

image

The reason could be that during a short period of time there are several tens of test cases started, all of which request hadoop-2.8.4.tar.gz from remote repository. This might trigger throttling for the downloads.

To solve this problem, hadoop-2.8.4.tar.gz can be downloaded during:

  • release building: once third-parties have changed, it should be rebuilt and image would not be used, thus download hadoop-2.8.4.tar.gz; or otherwise
  • third-parties image building: none of third-parties has ever changed, image would be used, thus third-parties image should include hadoop-2.8.4.tar.gz

Thus there will be 2 pull requests for both conditions:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement Indicates new feature requests
Projects
None yet
2 participants