Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added log collection support for vanilla Kubeadm clusters #211

Merged
merged 2 commits into from
Feb 1, 2023
Merged

Added log collection support for vanilla Kubeadm clusters #211

merged 2 commits into from
Feb 1, 2023

Conversation

gnuninu
Copy link

@gnuninu gnuninu commented Jan 31, 2023

This PR addresses issue/request #209

New functions created:

    kubeadm-k8s
    kubeadm-certs
    kubeadm-etcd

the switch -r now also includes kubeadm as a selector from the list

KUBEADM_DIR="/etc/kubernetes/"
KUBEADM_STATIC_DIR="/etc/kubernetes/manifests/"
techo "Collecting k8s kubeadm cluster logs"
if [ -f /$USER/.kube/config ]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible that a user doesn't store the kubeconfig in this location. Can we make this configurable?

KUBECONFIG=${KUBECONFIG:-"$USER/.kube/config"}

if [ -f /$USER/.kube/config ]; then
mkdir -p $TMPDIR/kubeadm/kubectl
KUBECONFIG=/$USER/.kube/config
/usr/bin/kubectl --kubeconfig=$KUBECONFIG get nodes -o wide > $TMPDIR/kubeadm/kubectl/nodes 2>&1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make kubectl binary location also configurable.

KUBECTL_CMD=${KUBECTL_CMD:-"/usr/bin/kubectl"}

/usr/bin/kubectl --kubeconfig=$KUBECONFIG cluster-info dump > $TMPDIR/kubeadm/kubectl/cluster-info_dump 2>&1
fi

if [ -f /$USER/.kube/config ]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious, why do we have two "if" blocks?


mkdir -p $TMPDIR/kubeadm/podlogs
techo "Collecting k8s kubeadm system pod logs"
if [ -f /$USER/.kube/config ]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these if checks can be done once at the top of the function, not needed for every section.

kubeadm-etcd() {

KUBEADM_ETCD_DIR="/var/lib/etcd/"
KUBEADM_ETCD_CERTS="/etc/kubernetes/pki/etcd/"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add ETCD_CMD checks. This binary might not exist on the host.

@gnuninu
Copy link
Author

gnuninu commented Feb 1, 2023

thanks @leodotcloud for reviewing and for the good suggestions.
My attempt was to reuse parts of the already existent rancher code and not altering things much by keeping the same style/structure.
This PR it's just an attempt to introduce a "fallback" collection that should be sufficient for the most common scenario described in the k8s docs (kubeadm and kubectl binary location, etcd dirs and so on...).
No doubt it can be improved with the help of the wider community to support more environments and i would be willing to help by sending future PRs.
However, may i suggest to accept the code as it is for the time being so at least a log collection from kubeadm is in place? i have tested from a couple of brand new test installs i have and it works.
This PR leaves rancher collection untouched and everything will continue to work as expected.
Perhaps can we create a new issue using your points as a basis for an improvement, hence a new PR?

@gnuninu gnuninu requested a review from leodotcloud February 1, 2023 12:37
@leodotcloud
Copy link
Contributor

I appreciate your PR and am glad you could reuse some parts, but the existing code makes some assumptions and those are not applicable to your changes.

may i suggest to accept the code as it is for the time being

That's not how code reviews work anywhere :)

Anyways, I quickly made the changes that I suggested.

@leodotcloud leodotcloud merged commit 626d032 into rancherlabs:master Feb 1, 2023
@gnuninu
Copy link
Author

gnuninu commented Feb 1, 2023

Anyways, I quickly made the changes that I suggested.

Thanks appreciated :)

may i suggest to accept the code as it is for the time being

That's not how code reviews work anywhere :)

I welcome your criticism but have you actually tested your modification ? :)
I have tried to fetch it and run it from my kubeadm env, looks like the automatic detection is not working.

wget https://raw.githubusercontent.com/rancherlabs/support-tools/master/collection/rancher/v2.x/logs-collector/rancher2_logs_collector.sh

sudo ./rancher2_logs_collector.sh 
2023-02-01 15:30:33: Created ./tmp.jchaD0DGbD
2023-02-01 15:30:33: Detecting available commands... renice ionoice
2023-02-01 15:30:33: Detecting OS... opensuse-leap 15.4
2023-02-01 15:30:33: Detecting k8s distribution... 
2023-02-01 15:30:33: couldn't detect k8s distro

It's missing this part in the sherlock function from my original PR:

      if $(command -v kubeadm >/dev/null 2>&1)
        then
          if $(kubeadm version >/dev/null 2>&1)
            then
              DISTRO=kubeadm
              echo "kubeadm"
            else
              FOUND+="kubeadm"
          fi
      fi

I am more than happy to send a new PR to fix this

@leodotcloud
Copy link
Contributor

leodotcloud commented Feb 1, 2023

  1. Yes, I tested the script.
  2. I removed that piece of logic. We don't want any unintended side effects as the new code has not been tested thoroughly by our team.

For your use-case, you have to invoke the script with -r kubeadm.

@gnuninu
Copy link
Author

gnuninu commented Feb 1, 2023

2. I removed that piece of logic. We don't want any unintended side effects as the new code has not been tested thoroughly by our team.

An automatic detection would be more appropriate for my case because the idea was to embed this script into other projects where we have mixed environments, Rancher and Kubeadm.
This is what led me to create this PR in the first place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants