Multi tenancy support proposal

[pmatyjasek-sumo]

Description: <Describe what has changed.
Multi tenancy support proposal:

• Extend client struct to add ability to store client token.
• Extract token from incoming request and associate it with resource (traces, metrics).
• Batch data by token and send those batches after defined time

Testing: < Describe what testing was performed and which tests were added.>
Unit tests

[pmatyjasek-sumo]

@jpkrohling we've prepared some proposal related to multi tenant support

[jpkrohling]

Before reviewing this, I would like to ask whether you are familiar with the configauth helper, which will extract and validate bearer tokens, placing the subject and membership information in the context. Looks like some code here overlaps with that one. There's also the routing exporter, which could be used to route each tenant to its own backend if necessary. If you believe that a different multi-tenancy support is needed after reviewing the existing building blocks, please do share the use cases.

[pmatyjasek-sumo]

@jpkrohling sure I'll check those and verify if it covers my use case. This one is only a draft proposal.

[bogdandrutu]

@pmatyjasek-sumo an alternative approach that I think would be good to investigate, is to add these data as ClientData (or a better name) in the pdata messages we propagate on the pipelines. As en entry to the top level pdata.Traces (pdata.Metrics, etc.) OR to every ResourceX (ResourceSpans, ResourceMetrics, etc.). The reason for this is because in the Context there are other entries that we don't necessary want to preserve, example Deadline.

This is just to brainstorm :)

[jpkrohling]

I like @bogdandrutu's proposal, and I think a good first step is to convert what we have in the configauth to place the incoming auth data in there. Candidate values:

• token, in case a downstream component needs to obtain an extra field from the JWT
• subject, with the username
• membership, with the groups the users belongs to
• IP, with the client's IP address

[tigrannajaryan]

@pmatyjasek-sumo an alternative approach that I think would be good to investigate, is to add these data as ClientData (or a better name) in the pdata messages we propagate on the pipelines. As en entry to the top level pdata.Traces (pdata.Metrics, etc.) OR to every ResourceX (ResourceSpans, ResourceMetrics, etc.). The reason for this is because in the Context there are other entries that we don't necessary want to preserve, example Deadline.

This is just to brainstorm :)

I remember we briefly discussed this in the past. I believe the best place is in ResourceX. This way it can travel through most processors without any special effort (except the very special ones like groupbytrace which disassembles the span lists and assembles them in a different combination). If we place it in top-level pdata it will not survive batch processor.

[jpkrohling]

except the very special ones like groupbytrace which disassembles the span lists and assembles them in a different combination

Great point. We'll need to add a warning to the readme for that processor. If I remember correctly, we create a new resource entirely, which means that we might not need to explicitly remove the client data from the resulting resource.

[pmm-sumo]

except the very special ones like groupbytrace which disassembles the span lists and assembles them in a different combination

Great point. We'll need to add a warning to the readme for that processor. If I remember correctly, we create a new resource entirely, which means that we might not need to explicitly remove the client data from the resulting resource.

Or, the batch processor can be made aware of that and group it accordingly (might be handled by an option, as it will get computing extensive and would be required only for specific cases)

[tigrannajaryan]

@pmatyjasek-sumo Given all the discussions above I think this proposal needs a more detailed design document that shows how it should work end to end. I think we need to see the design and discuss it before we can move forward with the implementation.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[github-actions]

Closed as inactive. Feel free to reopen if this PR is still being worked on.

[jean]

Is this issue really closed, or is it just stale-bot noise?

[jpkrohling]

The first and second parts of the original request should be part of #2733. I don't think we have a tracking issue for the last point.