Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix space accounting in the receive buffer #33

Merged
merged 4 commits into from
Aug 29, 2020
Merged

Conversation

Stebalien
Copy link
Member

First, I've removed the atomics because they don't really give us much and are a bit misleading. We only use them to access the length taking a lock, but we can't actually set them without holding the lock.

Second, the receive buffer logic wasn't quite correct. The window size is len+cap, not len+cap+pending. When we copy data from the network, we add it to len, but then we remove the same amount from cap, so copying data from the network does not affect the window size.

The receive buffer looks like:

|            receive window (10)    |
|     data      | empty space       |
|< len (5)     >|< cap (5)         >|
                |< pending (4) ->|

As data is read, the buffer gets updated like so:

    |         receive window (8)     |
    |     data   | empty space       |
    |< len (3)  >|< cap (5)         >|
                |< pending (4) ->|

It can then grow as follows (given a "max" of 10):

    |         receive window (10)       |
    |     data   | empty space          |
    |< len (3)  >|< cap (7)            >|
                |< pending (4) ->|

By counting "pending" towards the window size, we wouldn't send a window update if:

  1. The window was below half the maximum (should send an update).
  2. We were about to read some data from the network (reserved).

This was causing the partial window update test to flake.

First, I've removed the atomics because they don't really give us much and are a
bit misleading. We only use them to access the length taking a lock, but we
can't actually _set_ them without holding the lock.

Second, the receive buffer logic wasn't quite correct. The window size is
`len+cap`, not `len+cap+pending`.

The receive buffer looks like:

    |            receive window (10)    |
    |     data      | empty space       |
    |< len (5)     >|< cap (5)         >|
                    |< pending (4) ->|

As data is read, the buffer gets updated like so:

       |         receive window (8)     |
       |     data   | empty space       |
       |< len (3)  >|< cap (5)         >|
                    |< pending (4) ->|

It can then grow as follows (given a "max" of 10):

       |         receive window (10)       |
       |     data   | empty space          |
       |< len (3)  >|< cap (7)            >|
                    |< pending (4) ->|
util.go Outdated Show resolved Hide resolved
}

// Cap is the remaining capacity in the receive buffer.
func (s *segmentedBuffer) Cap() uint32 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we didn't do great on naming this, since Cap in go terminology gets used for the allocated space (len+cap) of the object. the comment helps though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. I tried making it work like go, but it wasn't much better.

@Stebalien Stebalien merged commit fd43d7f into master Aug 29, 2020
@Stebalien Stebalien deleted the fix/recv-buf branch August 29, 2020 02:52
@aschmahmann aschmahmann mentioned this pull request Feb 18, 2021
73 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants