[JENKINS-74992] Print relevant pod status changes in build logs

[amuniz]

JENKINS-74992

Testing done

Submitter checklist

• Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
• Ensure that the pull request title represents the desired changelog entry
• Please describe what you did
• Link to relevant issues in GitHub or Jira
• Link to relevant pull requests, esp. upstream and downstream changes
• Ensure you have provided tests - that demonstrates feature works or fixes the issue

[Vlatombe]

Spotless issues

[Vlatombe]

This is moving nicely!

[Vlatombe]

Need to be careful on what would happen to these informers if/when a KubernetesCloud gets removed.

[amuniz]

AFAICT the informers map would be unreachable and it would be garbage collected, so the DefaultSharedIndexInformer would be collected too and with it the informerExecutor (each instance has an executor), so the resync would stop happening (I've checked it actually stops reporting events).

[Vlatombe]

What would happen on controller restart ? Do we care about re-attaching existing nodes or do we only care about new nodes? I guess events are mostly relevant on agent initialization but just to be sure...

[amuniz]

Good point. I think once the pod is running and containers ready, these pod events are less interesting (actually there are usually no events until the pod is deleted).
IMO we can live without this, until it proves to be needed.

[Vlatombe]

Worth backlinking #1167 & #1192

[Vlatombe]

Interactive testing:

Created Pod: kubernetes kubernetes-plugin-test/pp-46-hxh1j-ntnm0-j1f39
[PodInfo] kubernetes-plugin-test/pp-46-hxh1j-ntnm0-j1f39
	Container [maven] waiting [PodInitializing] No message
	Pod [Pending][ContainersNotInitialized] containers with incomplete status: [set-up-jenkins-agent]
	Pod [Pending][ContainersNotReady] containers with unready status: [maven]
	Pod [Pending][ContainersNotReady] containers with unready status: [maven]
[PodInfo] kubernetes-plugin-test/pp-46-hxh1j-ntnm0-j1f39
	Container [maven] waiting [PodInitializing] No message
	Pod [Pending][ContainersNotReady] containers with unready status: [maven]
	Pod [Pending][ContainersNotReady] containers with unready status: [maven]

I see same events multiple times, can we avoid them?

[amuniz]

Those are pod conditions as they come from K8s, not really duplicated. I can try to implement some sort of equals and not print the repeated ones. I'll have a look.

[amuniz]

The type is what's making them different. Output printing the condition.type:

[PodInfo] kubernetes-plugin-test/test-12-m8mct-w2wf0-hfd9m
	Container [jnlp] waiting [ContainerCreating] No message
	Container [shell] waiting [ContainerCreating] No message
	Pod [Pending][ContainersNotReady][Ready] containers with unready status: [shell jnlp]
	Pod [Pending][ContainersNotReady][ContainersReady] containers with unready status: [shell jnlp]

This is what the client library is giving us. I can try to make it up on our side by ignoring the type and not printing "duplicates".

[amuniz]

Workaround applied. Output now:

Created Pod: kubernetes kubernetes-plugin-test/test-13-62tdj-khn15-ftdm3
[PodInfo] kubernetes-plugin-test/test-13-62tdj-khn15-ftdm3
	Container [jnlp] waiting [ContainerCreating] No message
	Container [shell] waiting [ContainerCreating] No message
	Pod [Pending][ContainersNotReady] containers with unready status: [shell jnlp]

[jglick]

The PR title is potentially misleading. This PR is only listing changes to values to some .status subfields. It is not printing “pod events” in the sense of

kubectl events --for pod/$name --watch

which would potentially print interesting things in various unusual configurations, for example if there are special controllers running that interact with the agent pod to do something, such as provision ephemeral volumes.

[amuniz]

OK. Title updated (here, in released notes and Jira).