[extension/headerssetter] Add more options for setting headers from the context

[TylerHelmuth]

Is your feature request related to a problem? Please describe.
There are situations where the incoming telemetry needs multi-tenant support but the incoming request does not contain the headers necessary to correctly route traffic. Instead, it is possible that the incoming telemetry itself or an external source accessed via a processor contain the information necessary to form the correct headers.

In these instances, the current headerssetter extension cannot be used to correct set the exporter request headers.

Describe the solution you'd like
As discussed in the SIG meeting today (2022-08-31) there could be a solution where the headerssetter extension nows how to pull headers not only from the incoming request headers, but from a new section added to the Collector's context. This would allow any component to set the required multi-tenant values for the specific context and propagate those changes to the exporters outgoing requests.

This change solution would require changes to Collector Core in order to be able to set and retrieve this new context section. I will add a proposed API change to Client/Something new in this issue in the future.

Describe alternatives you've considered
A processor in theory could change with the Context's headers, but that feels a little hacky.

Another option would be to update the extension to be able to pull data from an attribute or baggage, but these solutions require putting potential secret information in the telemetry payload, which is risky, and would require a deletion before exporting.

[github-actions]

Pinging code owners: @jpkrohling @kovrus. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[TylerHelmuth]

/cc @jpkrohling @kovrus @bogdandrutu

[jpkrohling]

As @bogdandrutu mentioned in the call, we have precedence about this with the client package. We use that for auth data already, and we could extend this to allow processors, and potentially other components, to set values to the context.

In a previous conversation around a similar topic, I remember @tigrannajaryan saying that this could lead to a bad user experience, as it would force users to know that moving things around could potentially break their pipelines.

There's also the caveat that components like the batch processor discard the context, so we need a way to make this information survive somehow. I think we even have an issue tracking this aspect already.

[tigrannajaryan]

I had a PoC that I can't find anymore where I patched generated Protobuf messages to allow carrying PdataContext in the pipeline transparently such that it works with the batch processor correctly. I don't think regular context.Context can be made to work nicely.

[jpkrohling]

I found a PR that references your PoC: open-telemetry/opentelemetry-collector#3745

[tomasfreund]

Hi, could you also consider kafka headers in your implementation? Thanks
#13899

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• extension/headerssetter: @jpkrohling @kovrus

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• extension/headerssetter: @jpkrohling @kovrus

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[fatsheep9146]

Any update? @jpkrohling

[jpkrohling]

No, this is not part of my queue at the moment. Perhaps @kovrus is interested in this one? Or @TylerHelmuth?

[TylerHelmuth]

This is no longer something I need, but maybe someone else does.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.