Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix race condition in test and other test bugs #162

Merged
merged 1 commit into from
Jan 11, 2022
Merged

Conversation

kentquirk
Copy link
Contributor

Which problem is this PR solving?

This fixes #161 and a couple of other issues discovered while debugging it.

The immediate problem is that running the test under -race worked on my machine, while running without it hung forever.

Short description of the changes

  • There was a race condition in the assumptions made by the test code -- the test code needed Add() to run before Flush() but on a fast machine, it might not finish in time. (It was confusing to debug because the test code is trying to induce potential race conditions!)
  • There was an assertion in a test comment that a write to a nil channel would panic, but in fact, a write to a nil channel blocks forever. I changed the test to use a closed channel instead, which will panic if it occurs.
  • Two successive statements attempted to log the same value (the length of a queue), but under concurrency they might not be the same value, so I added a temp var.

@kentquirk kentquirk requested a review from a team January 10, 2022 21:05
w.Add(ev)
time.Sleep(50 * time.Millisecond)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can use metrics to wait for the Add deterministically (wait for messages_queued to be the right number), but it's probably not worth the hassle of setting up metrics, eh?

Copy link
Contributor Author

@kentquirk kentquirk Jan 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually tried that with a testMetrics implementation, but it failed because the metrics are updated before the item is added to the queue. So you can still see deadlocks sometimes. This way isn't pretty but I haven't had it fail yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rubbin' some sleep on it.

tenor-243394000

@kentquirk kentquirk added type: maintenance The necessary chores to keep the dust off. version: no bump A PR with maintenance or doc changes that aren't included in a release. labels Jan 11, 2022
@kentquirk kentquirk changed the title Fix some test bugs Fix race condition in test and other test bugs Jan 11, 2022
@kentquirk kentquirk merged commit 73f9fed into main Jan 11, 2022
@kentquirk kentquirk deleted the kent.xmission-race branch January 11, 2022 21:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: maintenance The necessary chores to keep the dust off. version: no bump A PR with maintenance or doc changes that aren't included in a release.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tests won't complete unless -race is used
3 participants