Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metric callback improve #390

Merged
merged 18 commits into from
Dec 21, 2023
Merged

Metric callback improve #390

merged 18 commits into from
Dec 21, 2023

Conversation

TingDaoK
Copy link
Contributor

@TingDaoK TingDaoK commented Dec 4, 2023

Move the telemetry callback to be invoked from the meta request event loop

  • Previously, we invoke the telemetry callback once the send_data gets clean up, which happens when the s3 request gone, or the send_data get cleaned up.
  • Now, we schedule the callback to be invoked from the events on the meta request
    • When the send_data get set up, which will happen if the request prepared the http message (before the signing).
    • When the finished_request invoked, which is the request will not be retried anymore and finished.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@TingDaoK TingDaoK marked this pull request as ready for review December 4, 2023 21:23
@codecov-commenter
Copy link

codecov-commenter commented Dec 4, 2023

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (2c19abf) 88.74% compared to head (44382a1) 88.74%.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #390   +/-   ##
=======================================
  Coverage   88.74%   88.74%           
=======================================
  Files          21       21           
  Lines        5986     6024   +38     
=======================================
+ Hits         5312     5346   +34     
- Misses        674      678    +4     
Files Coverage Δ
source/s3_auto_ranged_get.c 98.28% <100.00%> (+0.02%) ⬆️
source/s3_copy_object.c 83.33% <100.00%> (+0.27%) ⬆️
source/s3_default_meta_request.c 95.70% <100.00%> (-0.53%) ⬇️
source/s3_request.c 95.20% <100.00%> (+0.07%) ⬆️
source/s3_util.c 99.62% <100.00%> (+0.01%) ⬆️
source/s3_auto_ranged_put.c 92.58% <98.64%> (-0.11%) ⬇️
source/s3_meta_request.c 90.62% <83.33%> (-0.15%) ⬇️

source/s3_auto_ranged_put.c Outdated Show resolved Hide resolved
source/s3_util.c Outdated
metric->crt_info_metrics.error_code = error_code;
if (record_end) {
aws_high_res_clock_get_ticks((uint64_t *)&metric->time_metrics.end_timestamp_ns);
metric->time_metrics.total_duration_ns =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the deal with record_end? The docs on end_timestamp and total_duration say they're always available. If record_end=false we're going to end up queuing the telemetry event for delivery without these values being set yet?

If we're trying to delay until the body is delivered, should we just wait until its delivered before we call aws_s3_request_finish_up_metrics_synced() ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah ... I think you're double-delivering this telemetry event for downloads.

You probably want to just NOT call aws_s3_request_finish_up_metrics_synced() if you're going to call it instead after the body is delivered

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is when we have body to be delivered, we delay the end_timestamp until the callback is scheduled to be invoked, from here

source/s3_auto_ranged_put.c Outdated Show resolved Hide resolved
source/s3_copy_object.c Show resolved Hide resolved
source/s3_util.c Outdated
metric->crt_info_metrics.error_code = error_code;
if (record_end) {
aws_high_res_clock_get_ticks((uint64_t *)&metric->time_metrics.end_timestamp_ns);
metric->time_metrics.total_duration_ns =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah ... I think you're double-delivering this telemetry event for downloads.

You probably want to just NOT call aws_s3_request_finish_up_metrics_synced() if you're going to call it instead after the body is delivered

Copy link
Contributor

@graebm graebm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix & ship

if (request->send_data.metrics != NULL) {
/* Request is done streaming the body, complete the metrics for the request now. */
struct aws_s3_request_metrics *metrics = request->send_data.metrics;
metrics->crt_info_metrics.error_code = error_code;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are you altering the metric's error_code? the error_code on the right-hand-side is whether the meta-request has failed. I would think the metrics callback would want to report whether that individual HTTP request failed.

@TingDaoK TingDaoK merged commit c91f4c3 into main Dec 21, 2023
30 checks passed
@TingDaoK TingDaoK deleted the metric-callback-improve branch December 21, 2023 00:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants