Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature(stdlib): add snappy codec #543

Merged
merged 1 commit into from
Nov 8, 2023

Conversation

MichaHoffmann
Copy link
Contributor

@MichaHoffmann MichaHoffmann commented Nov 3, 2023

  • add 'encode_snappy' and 'decode_snappy'

I have to deal with snappy encoded messages and this function would help me a lot!

@pront pront requested review from fuchsnj and pront November 3, 2023 18:11
Copy link
Member

@pront pront left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @MichaHoffmann,

Thank you for your contribution!

This looks good overall. Note that if there are any bugs with this function, they will likely not be prioritized over other work.

A few things before we merge:

  • Please add a CHANGELOG.md entry
  • Please make a new PR here to document these two new fucntions.

Cargo.toml Outdated Show resolved Hide resolved
@MichaHoffmann MichaHoffmann force-pushed the mhoffm-add-snappy-codec branch from 3bc8a6a to 779ff2f Compare November 7, 2023 20:07
@MichaHoffmann MichaHoffmann requested a review from pront November 7, 2023 20:09
Copy link
Member

@pront pront left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Please run dd-rust-license-tool write
  2. Ensure ./scripts/checks.sh passes
  3. Please make a new PR here to document these two new fucntions.

@MichaHoffmann MichaHoffmann force-pushed the mhoffm-add-snappy-codec branch from 779ff2f to d030c29 Compare November 7, 2023 20:28
@MichaHoffmann
Copy link
Contributor Author

  1. Please run dd-rust-license-tool write

    1. Ensure ./scripts/checks.sh passes

    2. Please make a new PR here to document these two new fucntions.

Oh damn, i dont think i can get "wasm32-unknown-unknown" to work; i repaired the other checks though. Do you have an idea what to do about wasm?

@pront
Copy link
Member

pront commented Nov 7, 2023

  1. Please run dd-rust-license-tool write

    1. Ensure ./scripts/checks.sh passes
    2. Please make a new PR here to document these two new fucntions.

Oh damn, i dont think i can get "wasm32-unknown-unknown" to work; i repaired the other checks though. Do you have an idea what to do about wasm?

You can use #[cfg(not(target_arch = "wasm32"))].

@MichaHoffmann
Copy link
Contributor Author

#[cfg(not(target_arch = "wasm32"))].

Ah! I see there is precedent for it with other functions. Thats awesome, ill fix it tomorrow, thank you!

@jszwedko
Copy link
Member

jszwedko commented Nov 7, 2023

Had we considered using https://github.com/BurntSushi/rust-snappy instead? That seems to be a native Rust implementation and so may be easier to get working under WASM. In general, we try to avoid having functions that cannot be run in the playground (WASM) since it introduces some confusion / friction. Some are unavoidable, but this one seems like there may be an option that allows for it.

@pront
Copy link
Member

pront commented Nov 7, 2023

Indeed, https://crates.io/crates/snap is more mature.

https://crates.io/crates/snappy looks abandoned.

* add 'encode_snappy' and 'decode_snappy'

Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
@MichaHoffmann MichaHoffmann force-pushed the mhoffm-add-snappy-codec branch from d030c29 to 23b769f Compare November 8, 2023 06:56
@MichaHoffmann MichaHoffmann requested a review from pront November 8, 2023 06:57
@MichaHoffmann
Copy link
Contributor Author

Indeed, https://crates.io/crates/snap is more mature.

https://crates.io/crates/snappy looks abandoned.

Converted to snap! Thanks for pointing it out!

decode_snappy => DecodeSnappy;

right_snappy {
args: func_args![value: value!(decode_base64("LKxUaGUgcXVpY2sgYnJvd24gZm94IGp1bXBzIG92ZXIgMTMgbGF6eSBkb2dzLg==").as_bytes())],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to go through base64?

For example see tests here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont know if its needed, but having it in decode and encode shows the duality nicely.

Copy link
Member

@pront pront left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@pront pront added this pull request to the merge queue Nov 8, 2023
Merged via the queue into vectordotdev:main with commit 6f859fc Nov 8, 2023
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants