Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decouple ObjectStore from Reqwest #7183

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

tustvold
Copy link
Contributor

Which issue does this PR close?

Closes #6056
Related to #6818
Related to #7171

Rationale for this change

See tickets

What changes are included in this PR?

Are there any user-facing changes?

@tustvold tustvold changed the title Http client abstraction Decouple ObjectStore from Reqwest Feb 23, 2025
@github-actions github-actions bot added the object-store Object Store Interface label Feb 23, 2025
Copy link
Contributor Author

@tustvold tustvold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The vast majority of this PR is fairly mechanical, but I've highlighted some key areas


/// The [`Body`] of an [`HttpRequest`]
#[derive(Debug, Clone)]
pub struct HttpRequestBody(Inner);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We define fixed body types as we need HttpClient to be dyn-compatible (it also makes things slightly easier to follow)

}
}

pub(crate) struct HttpRequestBuilder {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a crate-private builder that provides an interface similar to reqwest but which can then be used with the HttpClient. Unfortunately reqwest::Request is largely opaque and I couldn't see an obvious way to make use of it.

}
}

pub(crate) fn add_query_pairs<I, K, V>(uri: &mut Uri, query_pairs: I)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a shame that http::Request uses http::Uri which lacks the ergonomic niceities of Url, so we require some additional shenanigans


#[derive(Debug, Copy, Clone, PartialEq, Eq)]
#[non_exhaustive]
pub enum HttpErrorKind {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This allows the retry logic to work on top of arbitrary HttpClients

inner: RequestError,
}

impl std::fmt::Display for RetryError {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This somewhat lays the ground work for #6377

}

impl RetryExt for reqwest::RequestBuilder {
impl RetryExt for HttpRequestBuilder {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could theoretically remove the extension trait, as HttpRequestBuilder, is not a foreign type. I opted to keep it for now though.

};

// Reqwest error variants aren't great, attempt to refine them
let mut source = e.source();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic is moved out of retry.rs and reframed in terms of HttpErrorKind

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
object-store Object Store Interface
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Decouple ObjectStore from Reqwest / Generic HTTP Client Support
1 participant