Add instrumentation for Async/socketry gems

[tannalynn]

Create instrumentation using traces to create a traces-backend-newrelic.

It isn't possible to insert any arbitrary headers into the request headers via traces, so our http instrumentation will need to remain separate to maintain that functionality. Because of this, we should make sure things look ok when making async http requests after adding instrumentation via traces.

related comments:
#2272 (comment)
#2272 (comment)

[workato-integration]

https://new-relic.atlassian.net/browse/NR-172683

[ioquatix]

So, I'm okay with considering how to inject headers. However I'd like us to consider how we can do this in the least invasive/most predictable/robust way possible.

[tannalynn]

I'm starting work on this ticket now, so I'm going to get started on creating a trace-backend-newrelic.

For injecting headers, it would be great if that was possible since it would allow us to replace our async-http instrumentation. For the trace context, it looks like we're supposed to create a Context object and return it in the trace_context method, perhaps a similar thing could be done for other headers? What were you thinking about the best way to accomplish this?
@ioquatix

[ioquatix]

Let me take a look and get back to you. I'm not against adding an extra hook but the goal was to be generic as possible.

[tannalynn]

For sure, that totally makes sense. I know the new relic agent needs might not be a common use case, so I appreciate wanting it to be as generic is possible.

[ioquatix]

Can you clarify the headers you need to add, with perhaps some examples?

[tannalynn]

Other than the trace_context headers, there are 3 sets of headers that we might need to insert. Two of them are just older versions of distributed tracing that New Relic implemented prior to the w3c trace context, and the third is a set of headers to support the New Relic Synthetics product.

The New Relics Synthetics headers are X-NewRelic-Synthetics and X-NewRelic-Synthetics-Info.
example:

"X-NewRelic-Synthetics": "agMfAAECGxpLQkNAQUZHG0VKS0IcAwEHARtFSktCHEBBRkdERUpLQkNAQRYZFF1SU1pbWFkZX1xdUhQBAwEHGV9cXVIUWltYWV5fXF1SU1pbEB8WWFtaVVRdXB9eWVhbGgkLAwUfXllYWxpVVF1cX15ZWFtaVVQSbA=="

"X-NewRelic-Synthetics-Info": {
  "version": "1",
  "type": "scheduled",
  "initiator": "cli",
  "attributes": {
    "exampleAttribute": "value"
  }
}

The older types of distributed tracing, Cross Application Tracing and NR Distributed Tracing, are deprecated and not used often, but they are still supported by the agent for now (though they will be removed at some point in the future).

Cross Application Tracing example:

"X-NewRelic-App-Data": ["123#456", "WebTransaction/MVC/Controller/Action", 1.2, 0.3, 50, "EB5061D47E2217E1", false]

"X-NewRelic-ID": "123#456"

"X-NewRelic-Transaction": "EB5061D47E2217E1", false, "D7780A703DE3752C", "0f2c2491"]

NR Distributed Tracing example:

"newrelic": {
  "v": [0,1],
  "d": {
    "ty": "App",
    "ac": "212311",
    "ap": "51424",
    "id": "0996097a36a1cd29"
    "tr": "d6b4ba1c3a712ca",
    "pr": 0.421,
    "ti": 1482959525577,
    "tx": "27856f70d3d314b7"
  }
}

Hopefully that helps, let me know if there is anything else I can do to help!

[tannalynn]

hello @ioquatix
I've created the new relic backend implementation for traces, and created a PR with my initial code for easier review: newrelic/traces-backend-newrelic#1

For the headers, the trace context is implemented, but since we aren't able to insert other headers, if someone needs them implemented, like the new relic synthetics headers, we'll recommend they use the in agent instrumentation for async-http instead. If in the future there is a way for the agent to insert those headers, please let me know and we can get this updated though!

Please let me know if there are any concerns you have with the implementation. I was able to test it with Async http and everything seems to be working well as far as I could tell.

[tannalynn]

Closed by newrelic/traces-backend-newrelic#1