Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(storage): bump opendal to v0.49.0 #18266

Merged
merged 12 commits into from
Sep 23, 2024
Merged

Conversation

chenzl25
Copy link
Contributor

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

  • As title

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added test labels as necessary. See details.
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
  • All checks passed in ./risedev check (or alias, ./risedev c)
  • My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)
  • My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

  • My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

@wcy-fdu
Copy link
Contributor

wcy-fdu commented Aug 27, 2024

Is there any special reason to bump? I think every time we bump OpenDAL we need to do a lot of testing in aws, gcp and azure environments.

@chenzl25
Copy link
Contributor Author

chenzl25 commented Aug 27, 2024

Is there any special reason to bump? I think every time we bump OpenDAL we need to do a lot of testing in aws, gcp and azure environments.

No special purpose 🥵 . Previously I suspected some bug in v0.47 opendal, but actually not. However, I already have finished the coding part, so just create a PR by the way.

@wcy-fdu
Copy link
Contributor

wcy-fdu commented Aug 27, 2024

Is there any special reason to bump? I think every time we bump OpenDAL we need to do a lot of testing in aws, gcp and azure environments.

No special purpose 🥵 . Previously I suspected some bug in the low-level opendal, but actually not. However, I already have finished the coding part, so just create a PR by the way.

Thanks for the efforts first🙌.
As we already do some test on OpenDAL 0.47 for realsed 2.0, Patrick do you think we should bump to 0.49 as well? Or just hold this pr for a while and merge it when needed. I have no preference, I can help with testing if needed. It's up to you to decide cc @hzxa21

@chenzl25 chenzl25 requested a review from hzxa21 August 27, 2024 07:56
@hzxa21
Copy link
Collaborator

hzxa21 commented Aug 27, 2024

Is there any special reason to bump? I think every time we bump OpenDAL we need to do a lot of testing in aws, gcp and azure environments.

No special purpose 🥵 . Previously I suspected some bug in the low-level opendal, but actually not. However, I already have finished the coding part, so just create a PR by the way.

Thanks for the efforts first🙌. As we already do some test on OpenDAL 0.47 for realsed 2.0, Patrick do you think we should bump to 0.49 as well? Or just hold this pr for a while and merge it when needed. I have no preference, I can help with testing if needed. It's up to you to decide cc @hzxa21

There is no technical blocker of upgrading OpenDAL to the latest version if we think the latest version is also stable. We just need to run all the tests needed to make sure we are confident about this version before merging.

To reduce risks, I suggest we don't bump OpenDAL to latest in 2.0 given that 0.47 is the version we have proved working with several rounds of bug fixing and testing. I think we can certify OpenDAL 0.49 with the same tests we did for 0.47 before merging this PR to main, without blocking release 2.0. @wcy-fdu can decide the schedule because you are more familiar with OpenDAL than us.

@wcy-fdu
Copy link
Contributor

wcy-fdu commented Aug 27, 2024

can decide the schedule because you are more familiar with OpenDAL than us.

Let's first run all the tests based on this pr, and merge this pr into main if all pass.
And let’s not cherry pick version 2.0 yet. We can continue testing during the release test of version 2.1.

I will run test based on this PR later, so just hold for a while.

@graphite-app graphite-app bot requested a review from a team September 9, 2024 07:18
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 5396 files.

Valid Invalid Ignored Fixed
2296 1 3099 0
Click to see the invalid file list
  • src/connector/src/parser/protobuf/recursive.rs

src/connector/src/parser/protobuf/recursive.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@wcy-fdu wcy-fdu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I run longevity test and perf test based on this branch, and both look good to me(detailed in #18590). So I think this upgrade is safe.
I will resolve the conflict.
cc @hzxa21 @Li0k for another review.

Copy link
Collaborator

@hzxa21 hzxa21 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Cargo.lock Outdated Show resolved Hide resolved
@wcy-fdu wcy-fdu added this pull request to the merge queue Sep 23, 2024
Merged via the queue into main with commit 27458d8 Sep 23, 2024
29 of 30 checks passed
@wcy-fdu wcy-fdu deleted the dylan/bump_opendal_into_v0_49 branch September 23, 2024 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants