[extension/memorylimiter] add memory limiter extension, a copy of memorylimiter processor

[timannguyen]

Description:

following #8632, this change is to move memory limiter to being an extension. This allows us to place the component to reject incoming connections due to limited memory, providing better protection to the otel collector.

missing feature: receiver fairness. issue where a receiver hogs all the resource can happen.

Link to tracking Issue:
#8632

Testing:
unit test

Documentation:

README.md

[splunkericl]

• Can you outline how the extension can be used in http server? I didn't see a specific interface to use it.
• A lot of codes are just copying. Will the refactor come in a different MR?

[timannguyen]

@mx-psi will you take a look?

[timannguyen]

@splunkericl impl of httlp interceptor a9f24a1

[splunkericl]

LGTM. waiting for maintainers to take a look

[timannguyen]

going to follow up with PRs after this:

• adding the extension to http and grpc
• support balancing between multiple receivers if they were to use the same extension
• post some notice in memorylimiterprocessor that this is an alternative?

[mx-psi]

@timannguyen Thanks for raising this PR and apologies for not replying sooner. Since this is a new component, would you mind bringing this up to the Collector SIG weekly meeting? It happens on Wednesdays at 09:00 PT

[codecov]

Codecov Report

Attention: 40 lines in your changes are missing coverage. Please review.

Comparison is base (34b6544) 90.35% compared to head (3dd4816) 90.22%.

Files	Patch %	Lines
internal/memorylimiter/memorylimiter.go	76.59%	26 Missing and 7 partials ⚠️
...terextension/internal/metadata/generated_status.go	0.00%	4 Missing ⚠️
extension/memorylimiterextension/memorylimiter.go	75.00%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8964      +/-   ##
==========================================
- Coverage   90.35%   90.22%   -0.13%     
==========================================
  Files         339      343       +4     
  Lines       17965    18003      +38     
==========================================
+ Hits        16232    16244      +12     
- Misses       1409     1430      +21     
- Partials      324      329       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[timannguyen]

@mx-psi is there a link to the meeting? I can make it.

[mx-psi]

@mx-psi is there a link to the meeting? I can make it.

@timannguyen Sure! Here are the meeting notes that include the Zoom link https://docs.google.com/document/d/1r2JC5MB7GupCE7N32EwGEXs9V_YIsPgoFiLP4VWVMkE/edit Please add this as an item for the agenda :)

[timannguyen]

updated impl of http and grpc timannguyen@e84fa98

[timannguyen]

@dmitryax will you take a look whenever you have a chance?

[dmitryax]

@timannguyen can you please move the shared code to internal/memorylimiter and reuse it instead of copying?

[timannguyen]

@dmitryax i did the refactoring. let me know how it is

[timannguyen]

cant quite replicate make genotelcorecol to what github has. mirrored what is in main instead.

[dmitryax]

LGTM. @mx-psi PTAL as well

[jpkrohling]

Can you add what's the strategy for the future? How would this extension interact with the pipeline? Would receivers need to do anything at all to take advantage of this?

[dmitryax]

@jpkrohling, this is not well-defined yet. I have some ideas. I can put them in a doc or an issue and bring it for discussion. I believe this can be done as a follow-up and we can merge this PR to make it available for playing. The new component is in development stability and is not expected to be used in prod anyway. WDYT?

[jpkrohling]

Most of the information I'm looking for is already part of the github issue, but I think the design should be clear by looking at the component alone (readme and examples). That said, it's certainly OK to have it done in a follow-up PR.