Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Various CI fixes and cleanup #8289

Merged
merged 21 commits into from
May 5, 2020
Merged

Conversation

mehrdadn
Copy link
Contributor

@mehrdadn mehrdadn commented May 2, 2020

Why are these changes needed?

Addresses various CI issues.

Related issue number

#631

Checks

  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/latest/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failure rates at https://ray-travis-tracker.herokuapp.com/.
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested (please justify below)

@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@mehrdadn mehrdadn force-pushed the ci-cleanup branch 29 times, most recently from d40a97a to 88eaa1f Compare May 2, 2020 23:30
@ray-project ray-project deleted a comment from AmplabJenkins May 4, 2020
@ray-project ray-project deleted a comment from AmplabJenkins May 4, 2020
@ray-project ray-project deleted a comment from AmplabJenkins May 4, 2020
@ray-project ray-project deleted a comment from AmplabJenkins May 4, 2020
@ray-project ray-project deleted a comment from AmplabJenkins May 4, 2020
@ray-project ray-project deleted a comment from AmplabJenkins May 4, 2020
@ray-project ray-project deleted a comment from AmplabJenkins May 4, 2020
@ray-project ray-project deleted a comment from AmplabJenkins May 4, 2020
@ray-project ray-project deleted a comment from AmplabJenkins May 4, 2020
@ray-project ray-project deleted a comment from AmplabJenkins May 4, 2020
@ray-project ray-project deleted a comment from AmplabJenkins May 4, 2020
@ray-project ray-project deleted a comment from AmplabJenkins May 4, 2020
@ray-project ray-project deleted a comment from AmplabJenkins May 4, 2020
@mehrdadn mehrdadn marked this pull request as ready for review May 4, 2020 20:25
@@ -0,0 +1,49 @@
diff --git deps/jemalloc/configure deps/jemalloc/configure
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what happen when redis update and this patch break?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the question is more of "how likely will that happen"

Copy link
Contributor Author

@mehrdadn mehrdadn May 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question! I think fixing it in the event of a breakage shouldn't need more than 5 minutes—it's just a matter of copy/pasting a few changed lines.

Note that we don't just update Redis randomly. We have much more difficult patches to take care of for Windows compatibility and such (see the existing patches). So I think the overhead is negligible. And it's also a really significant gain in terms of how much noise it cuts down on in the builds (that itself wastes a lot of time). And in any case this patch is trivial enough to comment out if it somehow becomes a problem.

@@ -60,7 +60,7 @@ install_miniconda() {
;;
esac
reload_env
if [ -n "${PYTHON-}" ]; then
if [ -n "${PYTHON-}" ] && [ "${PYTHON}" != "$(python -s -c "import sys; sys.stdout.write('.'.join(map(str, sys.version_info[:2])))")" ]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to function

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh this line already changed in a subsequent commit—I think it's a bit clearer there, let me know if it still needs changing?

ci/travis/install-bazel.sh Outdated Show resolved Hide resolved
ci/travis/install-dependencies.sh Outdated Show resolved Hide resolved
ci/travis/install-dependencies.sh Outdated Show resolved Hide resolved
@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/25538/
Test PASSed.

Copy link
Contributor

@simon-mo simon-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few minor nits, great work

Comment on lines +6 to +9
GITHUB_BASE_SHA: ${{ github.event.pull_request.base.sha }}
GITHUB_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
GITHUB_PULL_REQUEST: ${{ github.event.number }}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment saying where these are used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, they're used for translating Travis environment variables, I'll update them.

.travis.yml Outdated
- . ./ci/travis/ci.sh build
script:
- . ./ci/travis/ci.sh lint
- true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we are just skipping, we can just remove the script block right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sadly not :( if you do that then it falls back to the default outermost block. I think I can technically put script: [] but it's not clear to me why that works, it's kind of confusing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Add a comment so people maintaining this in the future knows?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

ci/travis/ci.sh Outdated
if [ -z "${TRAVIS_BRANCH-}" ] && [ -n "${GITHUB_WORKFLOW-}" ]; then
TRAVIS_BRANCH="${GITHUB_BASE_REF:-${GITHUB_REF}}"
TRAVIS_BRANCH="${TRAVIS_BRANCH#refs/heads/}"
TRAVIS_BRANCH="${GITHUB_HEAD_SHA:-${TRAVIS_BRANCH}}" # Need a hash on GitHub Actions when possible (even though on Travis it's always a name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this comment doesn't say much, can we explain more?
also can you explain does this block do in the comment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

assert ("python" in process or "ray" in process
or "travis" in process)
assert ("python" in process or "conda" in process
or "travis" in process or "runner" in process
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runner? why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think travis was meant to refer to the username on Travis, so I added runner, which is the username on GitHub Actions.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/25552/
Test PASSed.

@simon-mo simon-mo merged commit 4bdef78 into ray-project:master May 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants