[CONTP-369] extract kubernetes resource parsers (pod, deployment and metadata) to a separate package #28479
Conversation
adel121
added
qa/done
adel121
force-pushed
the
adelhajhassan/extract_kubernetes_resource_parsers_to_their_own_package
branch
3 times, most recently
from
887cde8 to
3a0150b
Compare
Go Package Import DifferencesBaseline: 6156158
|
adel121
changed the title
adel121
force-pushed
the
adelhajhassan/extract_kubernetes_resource_parsers_to_their_own_package
branch
from
3a0150b to
8b4179d
Compare
Test changes on VMUse this command from test-infra-definitions to manually test this PR changes on a VM: inv create-vm --pipeline-id=43245444 --os-family=ubuntuNote: This applies to commit f816a83 |
Regression DetectorRegression Detector ResultsRun ID: 7eb17a07-ba67-4498-8264-149ba2a32e30 Metrics dashboard Target profiles Baseline: 6156158 Performance changes are noted in the perf column of each table:
No significant changes in experiment optimization goalsConfidence level: 90.00% There were no significant changes in experiment optimization goals at this confidence level and effect size tolerance.
|
| perf | experiment | goal | Δ mean % | Δ mean % CI | links |
|---|---|---|---|---|---|
| ➖ | tcp_syslog_to_blackhole | ingress throughput | +3.17 | [-10.08, +16.42] | Logs |
| ➖ | basic_py_check | % cpu utilization | +2.58 | [-0.15, +5.30] | Logs |
| ➖ | pycheck_lots_of_tags | % cpu utilization | +1.05 | [-1.42, +3.51] | Logs |
| ➖ | uds_dogstatsd_to_api_cpu | % cpu utilization | +1.00 | [+0.05, +1.96] | Logs |
| ➖ | idle | memory utilization | +0.11 | [+0.07, +0.14] | Logs |
| ➖ | tcp_dd_logs_filter_exclude | ingress throughput | +0.00 | [-0.01, +0.01] | Logs |
| ➖ | uds_dogstatsd_to_api | ingress throughput | -0.00 | [-0.00, +0.00] | Logs |
| ➖ | otel_to_otel_logs | ingress throughput | -0.08 | [-0.89, +0.73] | Logs |
| ➖ | file_tree | memory utilization | -0.70 | [-0.76, -0.63] | Logs |
Bounds Checks
| perf | experiment | bounds_check_name | replicates_passed |
|---|---|---|---|
| ✅ | idle | memory_usage | 10/10 |
Explanation
A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".
For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:
-
Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
-
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
-
Its configuration does not mark it "erratic".
|
/merge |
|
🚂 MergeQueue: pull request added to the queue The median merge time in Use |
|
/merge |
|
🚂 MergeQueue: waiting for PR to be ready This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals. Use |
|
This merge request was unqueued If you need support, contact us on Slack #devflow! |
adel121
force-pushed
the
adelhajhassan/extract_kubernetes_resource_parsers_to_their_own_package
branch
from
8b4179d to
c406bc2
Compare
adel121
force-pushed
the
adelhajhassan/extract_kubernetes_resource_parsers_to_their_own_package
branch
from
c406bc2 to
21e4c4a
Compare
| @@ -0,0 +1,74 @@ | |||
| // Unless explicitly stated otherwise all files in this repository are licensed | |||
There was a problem hiding this comment.
Were this file and filters.go meant to be included in this PR? The title and the PR description don't seem to imply it.
There was a problem hiding this comment.
Nope they weren't, thanks for pointing it out. 🙇
adel121
force-pushed
the
adelhajhassan/extract_kubernetes_resource_parsers_to_their_own_package
branch
from
21e4c4a to
b60c7ed
Compare
… a separate package
adel121
force-pushed
the
adelhajhassan/extract_kubernetes_resource_parsers_to_their_own_package
branch
from
b60c7ed to
f816a83
Compare
|
/merge |
|
🚂 MergeQueue: pull request added to the queue The median merge time in Use |
|
❌ MergeQueue: The build pipeline contains failing jobs for this merge request Build pipeline has failing jobs for 68ba8d4: What to do next?
DetailsSince those jobs are not marked as being allowed to fail, the pipeline will most likely fail. If you need support, contact us on Slack #devflow with those details! |
|
/merge |
|
🚂 MergeQueue: pull request added to the queue The median merge time in Use |
dd-mergequeue
bot
deleted the
adelhajhassan/extract_kubernetes_resource_parsers_to_their_own_package
branch
What does this PR do?
Extracts kubernetes resource parsers (pods, deployments and metadata) to their own package.
Motivation
This PR doesn't change behaviour, but it is a preparatory step to allow other components to parse kubernetes resources and convert them to workloadmeta entities.
The end goal is to be able to refactor some components (such as the orchestrator cluster checks) to compute resource tags on the fly instead of storing all tags for all resources in the tagger.
As a concrete example, the cluster checks are usually interested only in unscheduled pods (pods that are not assigned to a node yet). However, by depending on the tagger, we will necessarily need to populate the tagger with tags for all pods in the cluster because there is no way for the tagger to distinguish between scheduled and unscheduled pods, which consumes a lot of memory, cpu and network.
The plan is to extract parsers to their own package, then expose some public utility methods in the tagger package that can help extract tags from a workloadmeta entity without having to populate tags in the tagger.
Additional Notes
Possible Drawbacks / Trade-offs
None
Describe how to test/QA your changes
No need for qa, E2E tests are enough.