Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(core): Drop use of num_cpus crate #13197

Merged
merged 1 commit into from
Jun 16, 2022
Merged

Conversation

bruceg
Copy link
Member

@bruceg bruceg commented Jun 16, 2022

As of the release of Rust 1.61, std::thread::available_parallelism now
takes cgroup quotas into account
.
This puts available_parallelism into feature parity with num_cpus
for our purposes, allowing us to drop our use of this crate.

Question: Would it be worth using a once cell to cache this?

As of the release of Rust 1.61, [`std::thread::available_parallelism` now
takes cgroup quotas into account](rust-lang/rust#92697).
This puts `available_parallelism` into feature parity with `num_cpus`
for our purposes, allowing us to drop our use of this crate.
@bruceg bruceg added type: tech debt A code change that does not add user value. domain: deps Anything related to Vector's dependencies labels Jun 16, 2022
@bruceg bruceg requested review from blt, tobz and lukesteensen June 16, 2022 20:49
@bruceg bruceg self-assigned this Jun 16, 2022
@github-actions github-actions bot added domain: sources Anything related to the Vector's sources domain: topology Anything related to Vector's topology code labels Jun 16, 2022
@netlify
Copy link

netlify bot commented Jun 16, 2022

Deploy Preview for vector-project ready!

Name Link
🔨 Latest commit 27bec73
🔍 Latest deploy log https://app.netlify.com/sites/vector-project/deploys/62ab9766d7e42c0009370bf1
😎 Deploy Preview https://deploy-preview-13197--vector-project.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

Copy link
Member

@jszwedko jszwedko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet!

@bruceg bruceg enabled auto-merge (squash) June 16, 2022 21:24
@bruceg bruceg merged commit e2850b6 into master Jun 16, 2022
@bruceg bruceg deleted the available_parallelism branch June 16, 2022 22:14
@github-actions
Copy link

Soak Test Results

Baseline: 2fddd2b
Comparison: 27bec73
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_humio_logs 293.66KiB 1.88 100.00% 15.28MiB 654.44KiB 13.41KiB 0 0.0418159 15.57MiB 759.06KiB 15.54KiB 0 0.0476078 False False
syslog_log2metric_humio_metrics 165.82KiB 1.17 100.00% 13.84MiB 344.72KiB 7.03KiB 0 0.0243132 14.0MiB 193.76KiB 3.97KiB 0 0.0135078 False False
http_to_http_acks 181.32KiB 1.01 57.20% 17.54MiB 7.66MiB 160.07KiB 0 0.436456 17.72MiB 7.82MiB 163.42KiB 0 0.44113 True True
syslog_log2metric_splunk_hec_metrics 154.14KiB 0.85 100.00% 17.63MiB 728.76KiB 14.83KiB 0 0.0403607 17.78MiB 599.78KiB 12.22KiB 0 0.0329364 False False
datadog_agent_remap_blackhole_acks 520.03KiB 0.82 99.98% 61.91MiB 5.09MiB 106.13KiB 0 0.0821294 62.42MiB 4.44MiB 92.67KiB 0 0.0710408 False False
syslog_splunk_hec_logs 104.79KiB 0.63 100.00% 16.2MiB 301.4KiB 6.15KiB 0 0.0181604 16.31MiB 301.42KiB 6.16KiB 0 0.0180476 False False
syslog_regex_logs2metric_ddmetrics 66.14KiB 0.53 99.98% 12.23MiB 581.84KiB 11.85KiB 0 0.0464669 12.29MiB 629.98KiB 12.82KiB 0 0.0500473 False False
http_to_http_noack 78.14KiB 0.32 99.99% 23.77MiB 973.81KiB 19.84KiB 0 0.0399989 23.85MiB 244.9KiB 5.0KiB 0 0.0100271 False False
http_pipelines_blackhole 12.92KiB 0.3 50.68% 4.23MiB 649.93KiB 13.31KiB 0 0.150145 4.24MiB 655.12KiB 13.36KiB 0 0.150895 False False
syslog_loki 27.3KiB 0.18 97.12% 15.06MiB 554.56KiB 11.3KiB 0 0.0359601 15.08MiB 259.37KiB 5.3KiB 0 0.0167891 False False
http_to_http_json 1.42KiB 0.01 11.75% 23.85MiB 333.16KiB 6.8KiB 0 0.0136414 23.85MiB 331.53KiB 6.77KiB 0 0.0135738 False False
splunk_hec_to_splunk_hec_logs_noack -189.08B -0 1.56% 23.84MiB 323.3KiB 6.62KiB 0 0.0132432 23.84MiB 328.41KiB 6.72KiB 0 0.0134526 False False
splunk_hec_to_splunk_hec_logs_acks -478.03B -0 1.60% 23.75MiB 810.45KiB 16.48KiB 0 0.0333118 23.75MiB 810.6KiB 16.48KiB 0 0.0333188 False False
fluent_elasticsearch 456.6B 0 23.36% 79.47MiB 51.81KiB 1.04KiB 0 0.000636471 79.47MiB 53.53KiB 1.08KiB 0 0.000657609 False False
splunk_hec_indexer_ack_blackhole -28.89KiB -0.12 79.52% 23.77MiB 732.37KiB 14.91KiB 0 0.0300801 23.74MiB 847.3KiB 17.22KiB 0 0.034842 False False
datadog_agent_remap_blackhole -110.59KiB -0.17 99.68% 65.3MiB 1.3MiB 27.29KiB 0 0.0199369 65.19MiB 1.23MiB 25.66KiB 0 0.0187892 False False
file_to_blackhole -253.25KiB -0.26 98.03% 95.37MiB 3.55MiB 73.83KiB 0 0.0371756 95.12MiB 3.85MiB 79.59KiB 0 0.0405176 False False
http_pipelines_blackhole_acks -19.17KiB -0.43 76.26% 4.39MiB 575.61KiB 11.77KiB 0 0.127976 4.37MiB 546.98KiB 11.17KiB 0 0.12213 False False
splunk_hec_route_s3 -117.55KiB -0.58 94.25% 19.66MiB 2.1MiB 43.99KiB 0 0.107014 19.55MiB 2.08MiB 43.52KiB 0 0.106472 False False
datadog_agent_remap_datadog_logs -863.87KiB -1.26 100.00% 67.02MiB 2.71MiB 56.71KiB 0 0.040496 66.18MiB 3.87MiB 80.85KiB 0 0.0584362 False False
http_pipelines_no_grok_blackhole -263.5KiB -1.47 100.00% 17.46MiB 1.16MiB 24.2KiB 0 0.0661891 17.2MiB 1.37MiB 28.47KiB 0 0.0793385 False False
datadog_agent_remap_datadog_logs_acks -1.01MiB -1.52 100.00% 66.35MiB 1.73MiB 36.16KiB 0 0.0260487 65.34MiB 3.33MiB 69.61KiB 0 0.0509504 False False
socket_to_socket_blackhole -364.24KiB -2.6 100.00% 13.7MiB 727.56KiB 14.82KiB 0 0.0518565 13.34MiB 647.55KiB 13.21KiB 0 0.0473842 False False

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: deps Anything related to Vector's dependencies domain: sources Anything related to the Vector's sources domain: topology Anything related to Vector's topology code type: tech debt A code change that does not add user value.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants