[trace] fix race when child and parent send is called at the same time #40
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`CreateChild` adds a new span to the current span's list of children. `CreateAsyncChild` calls `CreateChild`, then sets the isAsync bool on the span. This can lead to a race caused by simultaneous read of the child span's `isAsync` bool by `Send` as it's being written by `CreateAsyncChild`. To fix this, I refactored the code a bit to allow us to set `isAsync` before the span is added to the list of child spans. This doesn't seem to be the same race reported in #38 but it should be fixed either way.
Fixes #38 - if `Send` is called for a span and its parent at the same time, a race can occur when accesing "isSent". This adds a lock to `Send`, which sets `isSent` and protects reads with read locks. This also effectively prevents `Send` from executing multiple times for the same span.
martin308
approved these changes
Nov 26, 2018
| child.sendLock.RLock() | ||
| isChildSent := child.isSent | ||
| child.sendLock.RUnlock() | ||
| if !isChildSent { | ||
| child.AddField("meta.sent_by_parent", true) |
There was a problem hiding this comment.
it seems like it could still be possible for this field to be set on a child which was not sent by a parent? I'm not sure how important that is if it possible though 🤔
There was a problem hiding this comment.
Yeah this is not perfect, but holding the lock here will deadlock the Send method. I think it might require a new method that sets the field while under the send lock.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #38 - if
Sendis called for a span and its parent at the same time, a race can occur when accesing "isSent". This adds a lock toSend, which setsisSentand protects reads with read locks. This also effectively preventsSendfrom executing multiple times for the same span.