Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[release-4.17] OCPBUGS-43820: UPSTREAM: <carry>: kubelet/cm: fix bug where kubelet restarts from missing cpuset cgroup #2126

Merged
merged 4 commits into from
Dec 9, 2024
Merged

[release-4.17] OCPBUGS-43820: UPSTREAM: <carry>: kubelet/cm: fix bug where kubelet restarts from missing cpuset cgroup #2126

merged 4 commits into from
Dec 9, 2024

Conversation

haircommander
Copy link
Member

This is a manual cherry-pick of #2031

fixes verify test from #2123

none

@haircommander
UPSTREAM: <carry>: kubelet/cm: fix bug where kubelet restarts from mi…
65330c1 
…ssing cpuset cgroup

on None cpumanager policy, cgroupv2, and systemd cgroup manager, kubelet
could get into a situation where it believes the cpuset cgroup was created
(by libcontainer in the cgroupfs) but systemd has deleted it, as it wasn't requested
to create it. This causes one unnecessary restart, as kubelet fails with

`failed to initialize top level QOS containers: root container [kubepods] doesn't exist.`

This only causes one restart because the kubelet skips recreating the cgroup
if it already exists, but it's still a bother and is fixed this way

adapted version of kubernetes@c51195d, avoid merge conflict
this can be dropped in the 4.19 rebase

Signed-off-by: Peter Hunt <pehunt@redhat.com>
@haircommander
UPSTREAM: <carry>: kubelet/cm: move CPU reading from cm to cm/cpumanager
40151eb 
Adapted from kubernetes@77d03e4 to avoid merge conflict
This can be dropped in the 4.19 rebase

Authored-by: Francesco Romani <fromani@redhat.com>
Signed-off-by: Peter Hunt <pehunt@redhat.com>
@openshift-ci-robot openshift-ci-robot added jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. labels Nov 1, 2024
@openshift-ci-robot
Copy link

@haircommander: This pull request references Jira Issue OCPBUGS-43820, which is invalid:

  • expected the bug to target either version "4.18." or "openshift-4.18.", but it targets "4.17.z" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

This is a manual cherry-pick of #2031

fixes verify test from #2123

none

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Nov 1, 2024
@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 1, 2024
@haircommander haircommander changed the base branch from master to release-4.17 November 1, 2024 20:31
@openshift-ci-robot
Copy link

@haircommander: the contents of this pull request could not be automatically validated.

The following commits are valid:

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

@openshift-ci-robot
Copy link

@haircommander: This pull request references Jira Issue OCPBUGS-43820, which is invalid:

  • release note text must be set and not match the template OR release note type must be set to "Release Note Not Required". For more information you can reference the OpenShift Bug Process.
  • expected dependent Jira Issue OCPBUGS-28812 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

This is a manual cherry-pick of #2031

fixes verify test from #2123

none

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 1, 2024
@openshift-ci openshift-ci bot requested review from deads2k and mrunalp November 1, 2024 20:32
ffromani and others added 2 commits November 1, 2024 16:35
@ffromani @haircommander
UPSTREAM: 125923: e2e_node: add a test to verify the kubelet starts
54e92e6 
with systemd cgroup driver and cpumanager none policy.

This was originally planned to be a correctness check for
https://issues.k8s.io/125923, but it was difficult to reproduce the bug,
so it's now a regression test against it.

Signed-off-by: Francesco Romani <fromani@redhat.com>
Signed-off-by: Peter Hunt <pehunt@redhat.com>
@haircommander
UPSTREAM: 125923: e2e_node: use restart instead of start stop
777f620 
Signed-off-by: Peter Hunt <pehunt@redhat.com>
@haircommander haircommander force-pushed the cherry-pick-2031-to-release-4.17 branch from 4cb7d5c to 777f620 Compare November 1, 2024 20:35
@openshift-ci-robot
Copy link

@haircommander: the contents of this pull request could not be automatically validated.

The following commits are valid:

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

@haircommander haircommander mentioned this pull request Nov 1, 2024
Closed
@rphillips
Copy link

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Nov 1, 2024
@mrunalp mrunalp added approved Indicates a PR has been approved by an approver from all required OWNERS files. and removed backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. labels Nov 1, 2024
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Nov 1, 2024

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: haircommander, mrunalp, rphillips

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@rphillips
Copy link

/label backport-risk-assessed

@openshift-ci openshift-ci bot added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label Nov 1, 2024
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Nov 2, 2024 •
edited
Loading

@haircommander: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-downgrade 4cb7d5c link true /test e2e-aws-ovn-downgrade

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@haircommander
Copy link
Member Author

/retest

@sreber84
Copy link

sreber84 commented Dec 9, 2024

/jira refresh

@openshift-ci-robot
Copy link

@sreber84: This pull request references Jira Issue OCPBUGS-43820, which is invalid:

  • release note text must be set and not match the template OR release note type must be set to "Release Note Not Required". For more information you can reference the OpenShift Bug Process.

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@haircommander
Copy link
Member Author

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Dec 9, 2024
@openshift-ci-robot
Copy link

@haircommander: This pull request references Jira Issue OCPBUGS-43820, which is valid. The bug has been moved to the POST state.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.17.z) matches configured target version for branch (4.17.z)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note type set to "Release Note Not Required"
  • dependent bug Jira Issue OCPBUGS-28812 is in the state Verified, which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
  • dependent Jira Issue OCPBUGS-28812 targets the "4.18.0" version, which is one of the valid target versions: 4.18.0
  • bug has dependents

No GitHub users were found matching the public email listed for the QA contact in Jira (schoudha@redhat.com), skipping review request.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@cpmeadors
Copy link

/label cherry-pick-approved

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Dec 9, 2024

@cpmeadors: Can not set label cherry-pick-approved: Must be member in one of these teams: [openshift-staff-engineers]

In response to this:

/label cherry-pick-approved

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@mrunalp mrunalp added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Dec 9, 2024
@openshift-merge-bot openshift-merge-bot bot merged commit a5b43ec into openshift:release-4.17 Dec 9, 2024
21 checks passed
@openshift-ci-robot
Copy link

@haircommander: Jira Issue OCPBUGS-43820: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-43820 has been moved to the MODIFIED state.

In response to this:

This is a manual cherry-pick of #2031

fixes verify test from #2123

none

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@haircommander
Copy link
Member Author

/cherry-pick release-4.16

@openshift-cherrypick-robot
Copy link

@haircommander: new pull request created: #2158

In response to this:

/cherry-pick release-4.16

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-cherrypick-robot openshift-cherrypick-robot mentioned this pull request Dec 9, 2024
Closed
@openshift-bot
Copy link

[ART PR BUILD NOTIFIER]

Distgit: openshift-enterprise-pod
This PR has been included in build openshift-enterprise-pod-container-v4.17.0-202412092205.p0.ga5b43ec.assembly.stream.el9.
All builds following this will include this PR.

@openshift-bot
Copy link

[ART PR BUILD NOTIFIER]

Distgit: openshift-enterprise-hyperkube
This PR has been included in build openshift-enterprise-hyperkube-container-v4.17.0-202412092205.p0.ga5b43ec.assembly.stream.el9.
All builds following this will include this PR.

@openshift-bot
Copy link

[ART PR BUILD NOTIFIER]

Distgit: ose-installer-kube-apiserver-artifacts
This PR has been included in build ose-installer-kube-apiserver-artifacts-container-v4.17.0-202412092205.p0.ga5b43ec.assembly.stream.el9.
All builds following this will include this PR.

@haircommander haircommander mentioned this pull request Feb 6, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mrunalp mrunalp mrunalp approved these changes

@deads2k deads2k Awaiting requested review from deads2k

Assignees

@rphillips rphillips

@kasturinarra kasturinarra

@zhouying7780 zhouying7780

@xingxingxia xingxingxia

@sunilcio sunilcio

@duanwei33 duanwei33

@wangke19 wangke19

@gangwgr gangwgr

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.