Add limit on number of processors in Ingest pipelines

[anandkrrai]

Description

Add limit on number of processors in Ingest pipelines

Related Issues

Resolves #[15464]
#15464

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for a108941: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[saratvemulapalli]

@anandkrrai thanks for the PR.
Could you help me understand whats your motivation ? In other words, what problem are you trying to solve?
I see you are putting a limit to processors within a pipeline, is your goal to limit on config stored in cluster state ?
The down side is users could still have a) one processor and inner processors registered b) Configure a processor with lots of data.

[anandkrrai]

@anandkrrai thanks for the PR. Could you help me understand whats your motivation ? In other words, what problem are you trying to solve? I see you are putting a limit to processors within a pipeline, is your goal to limit on config stored in cluster state ? The down side is users could still have a) one processor and inner processors registered b) Configure a processor with lots of data.

The idea is to add the limit on number of processors because there processors are applied recurrsively and if some clined has very large number of processors then it can result in the Stack overflow errror and that node would crash. Rigt now we are just adding the limit to Integer.MAV_VAL, in the later major version upgrade we can have some upper hard limit. As this

[github-actions]

✅ Gradle check result for a108941: SUCCESS

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.69%. Comparing base (834b7c1) to head (a108941).
Report is 52 commits behind head on 2.x.

Additional details and impacted files

@@             Coverage Diff              @@
##                2.x   #15585      +/-   ##
============================================
- Coverage     71.73%   71.69%   -0.04%     
- Complexity    63966    63976      +10     
============================================
  Files          5241     5241              
  Lines        299111   299124      +13     
  Branches      43539    43540       +1     
============================================
- Hits         214569   214461     -108     
- Misses        66626    66802     +176     
+ Partials      17916    17861      -55     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.