Skip to content

Commit

Permalink
storage: return byte batches in ScanResponse
Browse files Browse the repository at this point in the history
Previously, the MVCCScan API call completely deserialized the batch it
got from C++ into a slice of roachpb.KeyValue, and sent that slice of
roachpb.KeyValue over gRPC via ScanResponse. This is needlessly
expensive for several reasons.

- gRPC must re-serialize what we sent it to a flat byte stream. But, we
already had a flat byte stream to begin with, before inflating it into
KeyValues. In effect, we're doing pointless deserialization and
reserialization.
- We needed to dynamically allocate a slice of roachpb.KeyValue on every
scan request, in buildScanResponse. This was the second largest cause of
allocations in our system, beside the first copy from C++ to Go. But,
it's pointless, since we're just going to throw that slice away again
when we either serialize to the network or iterate over it and inflate
the KeyValues into rows later down the pipe.

Now, MVCCScan can optionally skip this inflation and return the raw
write batch that it got from C++. The txnKVFetcher and rowFetcher are
modified to use this option. They now deserialize keys from the write
batch as necessary.

This results in a large decrease in the number of allocations performed
per scan. When going over the network, only 1 object has to be
marshalled and demarshalled (the batch) instead of the number of
returned keys. Also, we don't have to allocate the initial slice of
[]KeyValue, or any of the slices within Key or Value, to return data.

Release note: None
  • Loading branch information
jordanlewis committed Jul 12, 2018
1 parent 9f30f01 commit 14f0009
Show file tree
Hide file tree
Showing 11 changed files with 885 additions and 442 deletions.
2 changes: 2 additions & 0 deletions pkg/roachpb/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,7 @@ func (sr *ScanResponse) combine(c combinable) error {
if sr != nil {
sr.Rows = append(sr.Rows, otherSR.Rows...)
sr.IntentRows = append(sr.IntentRows, otherSR.IntentRows...)
sr.BatchResponse = append(sr.BatchResponse, otherSR.BatchResponse...)
if err := sr.ResponseHeader.combine(otherSR.Header()); err != nil {
return err
}
Expand All @@ -265,6 +266,7 @@ func (sr *ReverseScanResponse) combine(c combinable) error {
if sr != nil {
sr.Rows = append(sr.Rows, otherSR.Rows...)
sr.IntentRows = append(sr.IntentRows, otherSR.IntentRows...)
sr.BatchResponse = append(sr.BatchResponse, otherSR.BatchResponse...)
if err := sr.ResponseHeader.combine(otherSR.Header()); err != nil {
return err
}
Expand Down
Loading

0 comments on commit 14f0009

Please sign in to comment.