Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance DataCarrier#MultipleChannelsConsumer to add priority #8664

Merged
merged 3 commits into from
Mar 11, 2022

Conversation

wu-sheng
Copy link
Member

  • If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.
  • Update the CHANGES log.
  • Enhance DataCarrier#MultipleChannelsConsumer to add priority for the channels, which makes OAP server has a better
    performance to activate all analyzers on default.
  • Activate receiver-otel#enabledOcRules receiver with k8s-cluster,k8s-node,k8s-service,oap,vm rules on default.
  • Activate satellite,spring-sleuth for agent-analyzer#meterAnalyzerActiveFiles on default.
  • Activate receiver-zabbix receiver with agent rule on default.

@wankai123 @mrproliu @kezhenxu94 As a result, many rules are being activated on default, also for the latest UI, all metrics would be visible on the configuration page(FYI @Fine0830).
For the end-users, they don't need to change deployment, reboot the OAP server, or face confused status.

* Priority determines the consuming strategy. On default every period consumer thread loops all groups trying
* to fetch the data from queue, if the queue only contains few elements, it is too expensive to consume every
* time.
*
* if 'size of last fetched data' > 0
*
* priority = 'size of last fetched data' * 100 / {@link Channels#size()} * {@link Channels#getChannelSize()}
*
* else
*
* priority = priority / 2
*
* Meaning, priority is the load factor of {@link #channels}
*
* After consuming loop, priority = (priority of current loop + priority of last loop) / 2.
*
* If priority > 50, consuming happens in next loop, otherwise, priority += 10, and wait until priority > 50. In
* worth case, for a low traffic group, consuming happens in 1/10.

…e channels, which makes OAP server has a better

  performance to activate all analyzers on default.
* Activate `receiver-otel#enabledOcRules` receiver with `k8s-cluster,k8s-node,k8s-service,oap,vm` rules on default.
* Activate `satellite,spring-sleuth` for `agent-analyzer#meterAnalyzerActiveFiles`  on default.
* Activate `receiver-zabbix` receiver with `agent` rule on default.
@wu-sheng wu-sheng added core feature Core and important feature. Sometimes, break backwards compatibility. backend OAP backend related. labels Mar 11, 2022
@wu-sheng wu-sheng added this to the 9.0.0 milestone Mar 11, 2022
Copy link
Member

@wankai123 wankai123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend OAP backend related. core feature Core and important feature. Sometimes, break backwards compatibility.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants