Skip to content
This repository has been archived by the owner on Feb 24, 2020. It is now read-only.

tests: tentative fixes for sporadic host and kvm failures #3434

Merged
merged 2 commits into from
Dec 2, 2016

Conversation

lucab
Copy link
Member

@lucab lucab commented Dec 1, 2016

This PR tentatively fixes the following flakes:

  • multiple jenkins-master failures, possibly due to debug output leading to a children blocked on write()
  • kvm stop failures, possibly due to a too short timeout (or kvm: too low thread limit #3382, to be determined)

@lucab lucab added this to the v1.21.0 milestone Dec 1, 2016
Copy link
Contributor

@s-urbaniak s-urbaniak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just two nits.

func waitOrFail(t *testing.T, child *gexpect.ExpectSubprocess, expectedStatus int) {
bufOut := []string{}
ttyIn, ttyOut := child.AsyncInteractChannels()
close(ttyIn)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Can we leave a TODO to change gexpect.AsyncInteractChannels to accept channels rather than returning them?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean opening a ticket to gexpect or a just a reminder for us here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest just a TODO here, I'll set up a PR in gexpect and some documentation around the usage patterns for these channels.

bufOut := []string{}
ttyIn, ttyOut := child.AsyncInteractChannels()
close(ttyIn)
for line := range ttyOut {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I guess) this relies on the fact that the ttyOut chan will be closed when a real err happens or io.EOF in https://github.com/ThomasRooney/gexpect/blob/8e96656e8fb794d716ee75b5a8cecbc15cde4393/gexpect.go#L167. I think we should document this here and in gexpect.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does, I'll note this down.

@lucab lucab changed the title tests: teantative fixes for sporadic host and kvm flavors tests: tentative fixes for sporadic host and kvm flavors Dec 1, 2016
@lucab lucab changed the title tests: tentative fixes for sporadic host and kvm flavors tests: tentative fixes for sporadic host and kvm failures Dec 1, 2016
This commit fixes waitOrFail to drain output from the child, in order
to avoid a deadlock situation where the children is blocked on a write
toward the parent, while the parent is blocked waiting for the children.
This raises the pod-stop timeout from 5 to 15 seconds.
Sporadic test failures on KVM flavor may be due to a
too strict limit, so we just 3x here to make sure failures
are not due to bad timing.
@lucab
Copy link
Member Author

lucab commented Dec 2, 2016

Rebased, PTAL.

Copy link
Contributor

@s-urbaniak s-urbaniak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@s-urbaniak s-urbaniak merged commit a4706c1 into rkt:master Dec 2, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants