Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge frames in deserialize_bytes #3639

Merged
merged 9 commits into from
Jun 5, 2020
Merged

Commits on Jun 4, 2020

  1. Test byte serialization in frame splitting case

    In some cases where the frames are particularly large, we may opt to
    split them into smaller frames. This may be due to performance reasons
    when transmitting data or it may be due to limitations like those of
    compressors used to compact frames. So include a test case that we know
    will get split to make sure it is handled correctly. Or at least make
    sure we are catching errors that would cause it to be mishandled.
    jakirkham committed Jun 4, 2020
    Configuration menu
    Copy the full SHA
    33f197f View commit details
    Browse the repository at this point in the history
  2. Merge frames in deserialize_bytes

    It appears that we are splitting frames in `serialize_byteslist` so that
    we can compress them. However we are not merging them back together
    afterwards during deserialization. This can cause an exception to be
    raised by serializers that expected their frames to be structured in a
    particular way.
    
    To fix this, we make sure to call `merge_frames` after `decompress` in
    `deserialize_bytes`. Further we make sure to pack the `lengths` of the
    original `frames` in the `header` (if not already present) in
    `serialize_byteslist`. This should ensure deserializers get the original
    frame structuring back when they operate on them.
    jakirkham committed Jun 4, 2020
    Configuration menu
    Copy the full SHA
    33594d3 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    59afbde View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    9ab510f View commit details
    Browse the repository at this point in the history
  5. Avoid copying bytes

    As using the `bytes` constructor will cause a copy (even if it is
    provided a `bytes` object), special case handling a `bytes` object and
    just return.
    jakirkham committed Jun 4, 2020
    Configuration menu
    Copy the full SHA
    f688b21 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    198f81f View commit details
    Browse the repository at this point in the history

Commits on Jun 5, 2020

  1. Drop fast-path for single frame

    This is already handled in the check above.
    jakirkham committed Jun 5, 2020
    Configuration menu
    Copy the full SHA
    3b14b99 View commit details
    Browse the repository at this point in the history
  2. Drop zero frame case

    This is already handled by the fast path for more than zero frames.
    jakirkham committed Jun 5, 2020
    Configuration menu
    Copy the full SHA
    eafbf77 View commit details
    Browse the repository at this point in the history
  3. Wrap ** with with int

    jakirkham committed Jun 5, 2020
    Configuration menu
    Copy the full SHA
    63a2086 View commit details
    Browse the repository at this point in the history