Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Name trait + Any encoding support #896

Merged
merged 7 commits into from
Sep 1, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
155 changes: 155 additions & 0 deletions prost-types/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,11 @@ use core::i64;
use core::str::FromStr;
use core::time;

use prost::alloc::format;
use prost::alloc::string::String;
use prost::alloc::vec::Vec;
use prost::{DecodeError, EncodeError, Message, Name};

pub use protobuf::*;

// The Protobuf `Duration` and `Timestamp` types can't delegate to the standard library equivalents
Expand All @@ -33,6 +38,58 @@ pub use protobuf::*;
const NANOS_PER_SECOND: i32 = 1_000_000_000;
const NANOS_MAX: i32 = NANOS_PER_SECOND - 1;

const PACKAGE: &str = "google.protobuf";

impl Any {
/// Serialize the given message type `M` as [`Any`].
pub fn from_msg<M>(msg: &M) -> Result<Self, EncodeError>
where
M: Name,
{
let type_url = M::type_url();
let mut value = Vec::new();
Message::encode(msg, &mut value)?;
Ok(Any { type_url, value })
}

/// Decode the given message type `M` from [`Any`], validating that it has
/// the expected type URL.
pub fn to_msg<M>(&self) -> Result<M, DecodeError>
where
M: Default + Name + Sized,
{
let expected_type_url = M::type_url();

match (
TypeUrl::new(&expected_type_url),
TypeUrl::new(&self.type_url),
) {
(Some(expected), Some(actual)) => {
if expected == actual {
return Ok(M::decode(&*self.value)?);
}
}
_ => (),
}

let mut err = DecodeError::new(format!(
"expected type URL: \"{}\" (got: \"{}\")",
expected_type_url, &self.type_url
));
err.push("unexpected type URL", "type_url");
Err(err)
}
}

impl Name for Any {
const PACKAGE: &'static str = PACKAGE;
const NAME: &'static str = "Any";

fn type_url() -> String {
type_url_for::<Self>()
}
}

impl Duration {
/// Normalizes the duration to a canonical format.
///
Expand Down Expand Up @@ -85,6 +142,15 @@ impl Duration {
}
}

impl Name for Duration {
const PACKAGE: &'static str = PACKAGE;
const NAME: &'static str = "Duration";

fn type_url() -> String {
type_url_for::<Self>()
}
}

impl TryFrom<time::Duration> for Duration {
type Error = DurationError;

Expand Down Expand Up @@ -298,6 +364,15 @@ impl Timestamp {
}
}

impl Name for Timestamp {
const PACKAGE: &'static str = PACKAGE;
const NAME: &'static str = "Timestamp";

fn type_url() -> String {
type_url_for::<Self>()
}
}

/// Implements the unstable/naive version of `Eq`: a basic equality check on the internal fields of the `Timestamp`.
/// This implies that `normalized_ts != non_normalized_ts` even if `normalized_ts == non_normalized_ts.normalized()`.
#[cfg(feature = "std")]
Expand Down Expand Up @@ -421,6 +496,49 @@ impl fmt::Display for Timestamp {
}
}

/// URL/resource name that uniquely identifies the type of the serialized protocol buffer message,
/// e.g. `type.googleapis.com/google.protobuf.Duration`.
///
/// This string must contain at least one "/" character.
///
/// The last segment of the URL's path must represent the fully qualified name of the type (as in
/// `path/google.protobuf.Duration`). The name should be in a canonical form (e.g., leading "." is
/// not accepted).
///
/// If no scheme is provided, `https` is assumed.
///
/// Schemes other than `http`, `https` (or the empty scheme) might be used with implementation
/// specific semantics.
#[derive(Debug, Eq, PartialEq)]
struct TypeUrl<'a> {
/// Fully qualified name of the type, e.g. `google.protobuf.Duration`
full_name: &'a str,
}

impl<'a> TypeUrl<'a> {
fn new(s: &'a str) -> core::option::Option<Self> {
// Must contain at least one "/" character.
let slash_pos = s.rfind('/')?;

// The last segment of the URL's path must represent the fully qualified name
// of the type (as in `path/google.protobuf.Duration`)
let full_name = s.get((slash_pos + 1)..)?;

// The name should be in a canonical form (e.g., leading "." is not accepted).
if full_name.starts_with('.') {
return None;
}

Some(Self { full_name })
}
}

/// Compute the type URL for the given `google.protobuf` type, using `type.googleapis.com` as the
/// authority for the URL.
fn type_url_for<T: Name>() -> String {
format!("type.googleapis.com/{}.{}", T::PACKAGE, T::NAME)
}

#[cfg(test)]
mod tests {
use super::*;
Expand Down Expand Up @@ -744,4 +862,41 @@ mod tests {
);
}
}

#[test]
fn check_any_serialization() {
let message = Timestamp::date(2000, 01, 01).unwrap();
let any = Any::from_msg(&message).unwrap();
assert_eq!(
&any.type_url,
"type.googleapis.com/google.protobuf.Timestamp"
);

let message2 = any.to_msg::<Timestamp>().unwrap();
assert_eq!(message, message2);

// Wrong type URL
assert!(any.to_msg::<Duration>().is_err());
}

#[test]
fn check_type_url_parsing() {
let example_type_name = "google.protobuf.Duration";

let url = TypeUrl::new("type.googleapis.com/google.protobuf.Duration").unwrap();
assert_eq!(url.full_name, example_type_name);

let full_url =
TypeUrl::new("https://type.googleapis.com/google.protobuf.Duration").unwrap();
assert_eq!(full_url.full_name, example_type_name);

let relative_url = TypeUrl::new("/google.protobuf.Duration").unwrap();
assert_eq!(relative_url.full_name, example_type_name);

// The name should be in a canonical form (e.g., leading "." is not accepted).
assert_eq!(TypeUrl::new("/.google.protobuf.Duration"), None);

// Must contain at least one "/" character.
assert_eq!(TypeUrl::new("google.protobuf.Duration"), None);
}
}
2 changes: 2 additions & 0 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,15 @@ pub use bytes;

mod error;
mod message;
mod name;
mod types;

#[doc(hidden)]
pub mod encoding;

pub use crate::error::{DecodeError, EncodeError};
pub use crate::message::Message;
pub use crate::name::Name;

use bytes::{Buf, BufMut};

Expand Down
28 changes: 28 additions & 0 deletions src/name.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
//! Support for associating type name information with a [`Message`].

use crate::Message;
use alloc::{format, string::String};

/// Associate a type name with a [`Message`] type.
pub trait Name: Message {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this just live in prost_types, why does it need to live in the core crate?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was hoping it could eventually be autogenerated by prost-build, at least optionally. We're hand annotating them right now and it's getting quite cumbersome.

It seems like prost-build currently only refers to types/traits in prost. If it were optional and it could refer to types/traits in prost-types, I'd be fine with the Name trait living there.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see your point, do we just want to go ahead and do the work to also add the prost-build parts? The next release will be breaking and I would be ok if we add something like that.

/// Type name for this [`Message`]. This is the camel case name,
/// e.g. `TypeName`.
const NAME: &'static str;

/// Package name this message type is contained in. They are domain-like
/// and delimited by `.`, e.g. `google.protobuf`.
const PACKAGE: &'static str;

/// Full name of this message type containing both the package name and
/// type name, e.g. `google.protobuf.TypeName`.
fn full_name() -> String {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these be &'static str, is there a case where these are not statically defined with the generated proto?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If prost-build filled them in they potentially could be all computed at build-time.

Otherwise it's not really possible to construct a &'static str as a concatenation of Self::PACKAGE + '.' + Self::NAME AFAIK.

format!("{}.{}", Self::NAME, Self::PACKAGE)
}

/// Type URL for this message, which by default is the full name with a
/// leading slash, but may also include a leading domain name, e.g.
/// `type.googleapis.com/google.profile.Person`.
Comment on lines +22 to +24
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this could include some of the documentation I added to the internal TypeUrl type.

fn type_url() -> String {
format!("/{}", Self::full_name())
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could potentially benefit from an easier way to customize the URL authority (i.e. hostname+port). It was a bit tricky to do this for the "well known" types:

https://github.com/tokio-rs/prost/pull/896/files#diff-fe4eba36fffb306e25d27bf29ddea9cd1634ea4ad86705045c1cb13a3aa27346R536-R540

}
}