Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NH-24786 fix xtraceoptions response calc #88

Merged
merged 23 commits into from
Dec 14, 2022

Conversation

tammy-baylis-swi
Copy link
Contributor

@tammy-baylis-swi tammy-baylis-swi commented Dec 6, 2022

Updated Dec 8.

Summary of changes

  1. Fixes bug where x-trace-options-response header was not being added to responses if x-trace-options request header was present but missing trigger-trace -- it should be added if any x-trace-options. Addressed in b75c652 with other commits for tests.
  2. Fixes bug where xtrace_options_response was included in injected/outgoing tracestate request header -- it should only be used for stashing before calculation of x-trace-options-response response header. Addressed in 38e74ef.
  3. Related refactor: W3CTransformer helper is now responsible for removing xtrace_options_response from tracestate, instead of Sampler (7b8a993).
  4. Also: xtrace_options_response is an APM constant, instead of class method of W3CTransformer helper (ec3cf3a).
  5. Add/update some unit tests and lots of integration tests (nearly all the other commits!)

Base automatically changed from NH-24786-fix-and-test-input-validation to main December 6, 2022 01:22
Copy link
Contributor

@cheempz cheempz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there I think @tammy-baylis-swi! There is still one point of misunderstanding(?) about when the x-trace-options-response should be set, and I think it would be worth adding explicit checks of the injected response header itself rather than the indirect check of what's stashed in SpanContext.

entry span (mocked). Unsigned trigger trace header is ignored. This is
not the root span because it continues an existing OTel context.
2. traceparent and tracestate headers in the request to the test app are
injected into the outgoing request (done by OTel TraceContextTextMapPropagator).
3. The injected x-trace-options header is still also propagated.
3. The valid injected x-trace-options header is still also propagated.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very Nit: "header is still also propagated" might give the impression that we inject something into outbound calls. Might be clearer to says something like "valid x-trace-options is still handled and an x-trace-options-response header is injected into the response".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one addressed in c872217

2. traceparent and tracestate headers in the request to the test app are
injected into the outgoing request (done by OTel TraceContextTextMapPropagator).
3. The valid injected x-trace-options header is not propagated because not TT.
4. No xtraceoptions response header because not TT.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hehe. Please don't go nuts, but this is why we all did go a bit crazy sussing out the TT logic edge cases. This is a case where if a valid x-trace-options header is given, we should set an x-trace-options-response response header... it's not talked about in the W3C spec but you can see an example in the Trigger Trace request/response matrix row 21 (note X-Trace-Options-Response: trigger-trace=not-requested).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok right yes. So the src code fix for this is addressed in b75c652. The tests and docstrings/comments around them have been updated in several other commits. Let me know if anything still amiss!

# though it will have different span ID because of Flask
# app's outgoing request
assert "x-trace" in resp.headers
assert new_trace_id in resp.headers["x-trace"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: should also assert that response header has x-trace-options-response

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in e60aa96

expected_trace_state = trace_api.TraceState([
("sw", "0000000000000000-01"),
("xtrace_options_response", "auth####ok;trigger-trace####not-requested;ignored####this-will-be-ignored"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's also worth checking the actual resp.headers for x-trace-options-response (maybe close to the x-trace response header check), because that would be the actual expected and customer visible behaviour as part of the TT feature.

It's a side benefit that you are able to check the response header "ingredients" are being stashed in the SpanContext, but it's an implementation detail and should not be used as the only verification that "yes, we're injecting the response header" :)

This applies to several of the subsequent tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great call! Added in e60aa96!

@tammy-baylis-swi tammy-baylis-swi force-pushed the NH-24786-fix-xtraceoptions-response-calc branch from 1126455 to e8d147c Compare December 8, 2022 21:40
@tammy-baylis-swi
Copy link
Contributor Author

Thanks @cheempz for spotting the issues! I've addressed them in commits as commented, and updated the PR description for clarity. As suggested in your absence I've also tagged @molafson on this PR for x-trace-options expertise!

Please have a look when you have a moment and let me know if you've spotted any other deviations from spec, unexpected values, or even docstrings/comments that might still be conflicting. 😃

@tammy-baylis-swi
Copy link
Contributor Author

tammy-baylis-swi commented Dec 13, 2022

Invalid value cases 15,16 in Request/Response matrix should be fixed in 59bec51 so that ignored should no longer be in x-trace-options response header if bad auth! Thank you @molafson for pointing this out.

Copy link

@molafson molafson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍 Lots of test cases!

@tammy-baylis-swi tammy-baylis-swi merged commit 3b4a68c into main Dec 14, 2022
@tammy-baylis-swi tammy-baylis-swi deleted the NH-24786-fix-xtraceoptions-response-calc branch December 14, 2022 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants