Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmark for DynamoDB serialization/deserialization #507

Merged
merged 6 commits into from
Jun 17, 2021
Merged

Add benchmark for DynamoDB serialization/deserialization #507

merged 6 commits into from
Jun 17, 2021

Conversation

jdisanti
Copy link
Collaborator

More work towards #161. This adds a benchmark for JSON serialization/deserialization for real-world DynamoDB calls.

Sample output:

$ cargo bench
ser_deser_bench         time:   [224.39 us 226.10 us 228.14 us]
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Copy link
Collaborator

@rcoh rcoh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if intentionally or unintentionally, you're actually testing a lot more than just the serializers. If you just want to benchmark the serializer, I would write something much closer to what the protocol tests generate and use make_operation and ParseResponse directly. (Note that in the case of dynamo, you can use ParseStrictResponse since none of the responses are streaming)

@jdisanti
Copy link
Collaborator Author

Definitely was unintentionally testing more than I wanted to, but I wasn't aware of how to do the parsing directly. Thanks for the pointers! I updated the PR with a much nicer implementation that should give us better information.

@jdisanti jdisanti requested a review from rcoh June 17, 2021 01:32
Copy link
Collaborator

@rcoh rcoh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, removing async will also simplify things.

We should also add the benches folder to the copy step in build.gradle so that CI will at least compile these

use smithy_http::response::ParseHttpResponse;
use tokio::runtime::Builder as RuntimeBuilder;

async fn do_bench() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function actually doesn't need to be async

};
}

async fn do_bench(config: &Config, input: &PutItemInput) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also doesn't need to be async anymore

Copy link
Collaborator

@rcoh rcoh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@@ -240,7 +245,7 @@ tasks.register<Exec>("cargoCheck") {
workingDir(sdkOutputDir)
// disallow warnings
environment("RUSTFLAGS", "-D warnings")
commandLine("cargo", "check")
commandLine("cargo", "check", "--lib", "--tests", "--benches")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since CI doesn't actually hit this, we should also update the CI job

@jdisanti jdisanti merged commit 1fd6e97 into smithy-lang:main Jun 17, 2021
@jdisanti jdisanti deleted the dynamo-bench branch June 18, 2021 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants