Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

impl Read and Write for VecDeque<u8> #95632

Merged
merged 1 commit into from
Jun 9, 2022
Merged

Conversation

evanrichter
Copy link
Contributor

Implementing Read and Write for VecDeque<u8> fills in the VecDeque api surface where Vec<u8> and Cursor<Vec<u8>> already impl Read and Write. Not only for completeness, but VecDeque in particular is a very handy mock interface for a TCP echo service, if only it supported Read/Write.

Since this PR is just an impl trait, I don't think there is a way to limit it behind a feature flag, so it's "insta-stable". Please correct me if I'm wrong here, not trying to rush stability.

@rust-highfive
Copy link
Collaborator

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @kennytm (or someone else) soon.

Please see the contribution instructions for more information.

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 4, 2022
@scottmcm scottmcm added the needs-fcp This change is insta-stable, so needs a completed FCP to proceed. label Apr 4, 2022
@paolobarbolini
Copy link
Contributor

Shouldn't this also have a Read::read_buf implementation?

@evanrichter
Copy link
Contributor Author

Shouldn't this also have a Read::read_buf implementation?

Just pushed up another commit to include this, thanks! The codegen looks decent at least to me, still compiling down to a number of slice checks and a single memcpy

@rustbot rustbot added the T-libs Relevant to the library team, which will review and decide on the PR/issue. label Apr 15, 2022
@evanrichter
Copy link
Contributor Author

Just squashed the commits together and rewrote the commit message.

And I think I should have highfived the libs team?

r? rust-lang/libs

@rust-highfive rust-highfive assigned thomcc and unassigned kennytm Apr 15, 2022
@thomcc
Copy link
Member

thomcc commented Apr 15, 2022

r? rust-lang/libs-api @rustbot label +T-libs-api

@rustbot rustbot added the T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. label Apr 15, 2022
bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 28, 2022
…=the8472

Add VecDeque::extend from vec::IntoIter and slice::Iter specializations

Inspired from the [`Vec` `SpecExtend` implementation](https://github.com/rust-lang/rust/blob/027a232755fa9728e9699337267f6675dfd0a8ba/library/alloc/src/vec/spec_extend.rs), but without the specialization for `TrustedLen` which I'll look into in the future.

Should help rust-lang#95632 and KillingSpark/zstd-rs#17

## Benchmarks

Before

```
test vec_deque::bench_extend_bytes    ... bench:         862 ns/iter (+/- 10)
test vec_deque::bench_extend_vec      ... bench:         883 ns/iter (+/- 19)
```

After

```
test vec_deque::bench_extend_bytes    ... bench:           8 ns/iter (+/- 0)
test vec_deque::bench_extend_vec      ... bench:          24 ns/iter (+/- 1)

```
@paolobarbolini
Copy link
Contributor

Now that I think of it shouldn't there also be a write_vectored implementation?

This line shouldn't be necessary anymore after #95904 lands

nice! I suppose whichever PR gets merged second can update this line?

If nothing goes wrong with CI #95904 should be merged today 🥳

@m-ou-se m-ou-se removed the T-libs Relevant to the library team, which will review and decide on the PR/issue. label Apr 28, 2022
@m-ou-se
Copy link
Member

m-ou-se commented Apr 28, 2022

@rfcbot merge

@rfcbot
Copy link

rfcbot commented Apr 28, 2022

Team member @m-ou-se has proposed to merge this. The next step is review by the rest of the tagged team members:

Concerns:

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

@rfcbot rfcbot added proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. labels Apr 28, 2022
@rfcbot
Copy link

rfcbot commented Apr 29, 2022

🔔 This is now entering its final comment period, as per the review above. 🔔

@rfcbot rfcbot removed the proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. label Apr 29, 2022
@joshtriplett
Copy link
Member

I love this idea, and I really look forward to using it! I also really look forward to when we have async read/write traits in std, and we can add async versions of this, to get a byte-oriented pipe in std.

I do have one concern:

@rfcbot concern write-vs-write_all

I'd like to ask if we should really make write do partial writes up to capacity, or if we should make write and write_all behave identically. Currently, the Write impl for Vec always writes all the data. Is there an advantage and use case for doing partial writes to VecDeque?

I am extremely open to hearing about such a use case, and I'd be happy to drop this concern if so.

@evanrichter
Copy link
Contributor Author

@rfcbot concern trait-impl-behavior

This is my first code contribution and I'm a little nervous about getting this wrong 😅

Am I right that how a type implements a trait is guaranteed for the future? i.e. if rust 1.62.0 ships and VecDeque does partial reads, then it's decided it would be better to make read behave the same as read_exact, can we even do that? It's possible crates would have failing tests due to that change, even though VecDeque still implements Read either way.

Also: I will soon push an update making write behave the same as write_all

@evanrichter
Copy link
Contributor Author

Oh no I can't raise a concern for my PR 🙃

@joshtriplett @m-ou-se Am I correct above?

@m-ou-se
Copy link
Member

m-ou-se commented May 18, 2022

This is my first code contribution and I'm a little nervous about getting this wrong

No worries, there are many eyes on this and we won't merge this if we think there are any issues, nor will we blame you if we miss something during review. :)

Am I right that how a type implements a trait is guaranteed for the future?

That's a complicated question that has no general answer. We've regularly changed (trait) implementations of all kind of types to do something better or more efficient, as long as we think it doesn't break any code that was making reasonable assumptions. For read and write specifically, I think it's perfectly fine for the exact details of how many bytes they can produce/consume at once to change over time, as those are designed to be used in a loop (like write_all) anyway.

Also: I will soon push an update making write behave the same as write_all

👍

No need to block the FCP on that with an rfcbot concern. Rfcbot is just counting down to the decision for 'do we want VecDeque to implement Read and Write?', not for 'is this the exact code we should merge?'. Reviewing and merging this PR is a separate (manual) step, after the FCP finishes.

(If you want to make clear a PR not ready for merging yet, you can always mark a PR as a draft. (Top right in the GitHub web interface, right under the reviewers.))

@evanrichter
Copy link
Contributor Author

Thanks a bunch for clarifying 🙂

I'm happy with the PR as it is now, knowing those details are considered somewhat flexible.

* For read and read_buf, only the front slice of a discontiguous
VecDeque is copied. The VecDeque is advanced after reading, making any
back slice available for reading with a second call to Read::read(_buf).

* For write, the VecDeque always appends the entire slice to the end,
growing its allocation when necessary.
@evanrichter
Copy link
Contributor Author

bumped the version on the "stable since" attribute to 1.63.0 after today's stable release of 1.61.0

@rfcbot rfcbot added finished-final-comment-period The final comment period is finished for this PR / Issue. to-announce Announce this issue on triage meeting and removed final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. labels May 26, 2022
@rfcbot
Copy link

rfcbot commented May 26, 2022

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

@apiraino apiraino removed the to-announce Announce this issue on triage meeting label May 27, 2022
@joshtriplett
Copy link
Member

@bors r+

@bors
Copy link
Contributor

bors commented Jun 8, 2022

📌 Commit 8b7a3f4 has been approved by joshtriplett

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 8, 2022
JohnTitor added a commit to JohnTitor/rust that referenced this pull request Jun 9, 2022
impl Read and Write for VecDeque<u8>

Implementing `Read` and `Write` for `VecDeque<u8>` fills in the VecDeque api surface where `Vec<u8>` and `Cursor<Vec<u8>>` already impl Read and Write. Not only for completeness, but VecDeque in particular is a very handy mock interface for a TCP echo service, if only it supported Read/Write.

Since this PR is just an impl trait, I don't think there is a way to limit it behind a feature flag, so it's "insta-stable". Please correct me if I'm wrong here, not trying to rush stability.
bors added a commit to rust-lang-ci/rust that referenced this pull request Jun 9, 2022
Rollup of 5 pull requests

Successful merges:

 - rust-lang#95632 (impl Read and Write for VecDeque<u8>)
 - rust-lang#95860 (Stabilize `$$` in Rust 1.63.0)
 - rust-lang#97838 (hexagon: adapt test for upstream output changes)
 - rust-lang#97843 (Relax mipsel-sony-psp's linker script)
 - rust-lang#97874 (rewrite combine doc comment)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit f14ccdb into rust-lang:master Jun 9, 2022
@rustbot rustbot added this to the 1.63.0 milestone Jun 9, 2022
compiler-errors added a commit to compiler-errors/rust that referenced this pull request Jun 9, 2022
…serve, r=the8472

Remove redundant calls to reserve in impl Write for VecDeque

Removes the reserve calls made redundant by rust-lang#95904 (as discussed in rust-lang#95632 (comment))
Dylan-DPC added a commit to Dylan-DPC/rust that referenced this pull request Jun 10, 2022
…serve, r=the8472

Remove redundant calls to reserve in impl Write for VecDeque

Removes the reserve calls made redundant by rust-lang#95904 (as discussed in rust-lang#95632 (comment))
JohnTitor added a commit to JohnTitor/rust that referenced this pull request Jun 10, 2022
…serve, r=the8472

Remove redundant calls to reserve in impl Write for VecDeque

Removes the reserve calls made redundant by rust-lang#95904 (as discussed in rust-lang#95632 (comment))
workingjubilee pushed a commit to tcdi/postgrestd that referenced this pull request Sep 15, 2022
…the8472

Remove redundant calls to reserve in impl Write for VecDeque

Removes the reserve calls made redundant by #95904 (as discussed in rust-lang/rust#95632 (comment))
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Oct 10, 2022
Pkgsrc changes:
 * Adjust patches as needed & checksum updates.

Upstream changes:

Version 1.63.0 (2022-08-11)
==========================

Language
--------
- [Remove migrate borrowck mode for pre-NLL errors.][95565]
- [Modify MIR building to drop repeat expressions with length zero.][95953]
- [Remove label/lifetime shadowing warnings.][96296]
- [Allow explicit generic arguments in the presence of `impl Trait` args.]
  [96868]
- [Make `cenum_impl_drop_cast` warnings deny-by-default.][97652]
- [Prevent unwinding when `-C panic=abort` is used regardless of
  declared ABI.][96959]
- [lub: don't bail out due to empty binders.][97867]

Compiler
--------
- [Stabilize the `bundle` native library modifier,][95818] also removing the
  deprecated `static-nobundle` linking kind.
- [Add Apple WatchOS compile targets\*.][95243]
- [Add a Windows application manifest to rustc-main.][96737]

\* Refer to Rust's [platform support page][platform-support-doc] for more
   information on Rust's tiered platform support.

Libraries
---------
- [Implement `Copy`, `Clone`, `PartialEq` and `Eq` for
  `core::fmt::Alignment`.][94530]
- [Extend `ptr::null` and `null_mut` to all thin (including extern)
  types.][94954]
- [`impl Read and Write for VecDeque<u8>`.][95632]
- [STD support for the Nintendo 3DS.][95897]
- [Make write/print macros eagerly drop temporaries.][96455]
- [Implement internal traits that enable `[OsStr]::join`.][96881]
- [Implement `Hash` for `core::alloc::Layout`.][97034]
- [Add capacity documentation for `OsString`.][97202]
- [Put a bound on collection misbehavior.][97316]
- [Make `std::mem::needs_drop` accept `?Sized`.][97675]
- [`impl Termination for Infallible` and then make the `Result` impls
  of `Termination` more generic.][97803]
- [Document Rust's stance on `/proc/self/mem`.][97837]

Stabilized APIs
---------------

- [`array::from_fn`]
- [`Box::into_pin`]
- [`BinaryHeap::try_reserve`]
- [`BinaryHeap::try_reserve_exact`]
- [`OsString::try_reserve`]
- [`OsString::try_reserve_exact`]
- [`PathBuf::try_reserve`]
- [`PathBuf::try_reserve_exact`]
- [`Path::try_exists`]
- [`Ref::filter_map`]
- [`RefMut::filter_map`]
- [`NonNull::<[T]>::len`][`NonNull::<slice>::len`]
- [`ToOwned::clone_into`]
- [`Ipv6Addr::to_ipv4_mapped`]
- [`unix::io::AsFd`]
- [`unix::io::BorrowedFd<'fd>`]
- [`unix::io::OwnedFd`]
- [`windows::io::AsHandle`]
- [`windows::io::BorrowedHandle<'handle>`]
- [`windows::io::OwnedHandle`]
- [`windows::io::HandleOrInvalid`]
- [`windows::io::HandleOrNull`]
- [`windows::io::InvalidHandleError`]
- [`windows::io::NullHandleError`]
- [`windows::io::AsSocket`]
- [`windows::io::BorrowedSocket<'handle>`]
- [`windows::io::OwnedSocket`]
- [`thread::scope`]
- [`thread::Scope`]
- [`thread::ScopedJoinHandle`]

These APIs are now usable in const contexts:

- [`array::from_ref`]
- [`slice::from_ref`]
- [`intrinsics::copy`]
- [`intrinsics::copy_nonoverlapping`]
- [`<*const T>::copy_to`]
- [`<*const T>::copy_to_nonoverlapping`]
- [`<*mut T>::copy_to`]
- [`<*mut T>::copy_to_nonoverlapping`]
- [`<*mut T>::copy_from`]
- [`<*mut T>::copy_from_nonoverlapping`]
- [`str::from_utf8`]
- [`Utf8Error::error_len`]
- [`Utf8Error::valid_up_to`]
- [`Condvar::new`]
- [`Mutex::new`]
- [`RwLock::new`]

Cargo
-----
- [Stabilize the `--config path` command-line argument.][cargo/10755]
- [Expose rust-version in the environment as
  `CARGO_PKG_RUST_VERSION`.][cargo/10713]

Compatibility Notes
-------------------

- [`#[link]` attributes are now checked more strictly,][96885]
  which may introduce errors for invalid attribute arguments that
  were previously ignored.

Internal Changes
----------------

These changes provide no direct user facing benefits, but represent
significant improvements to the internals and overall performance
of rustc and related tools.

- [Prepare Rust for LLVM opaque pointers.][94214]

[94214]: rust-lang/rust#94214
[94530]: rust-lang/rust#94530
[94954]: rust-lang/rust#94954
[95243]: rust-lang/rust#95243
[95565]: rust-lang/rust#95565
[95632]: rust-lang/rust#95632
[95818]: rust-lang/rust#95818
[95897]: rust-lang/rust#95897
[95953]: rust-lang/rust#95953
[96296]: rust-lang/rust#96296
[96455]: rust-lang/rust#96455
[96737]: rust-lang/rust#96737
[96868]: rust-lang/rust#96868
[96881]: rust-lang/rust#96881
[96885]: rust-lang/rust#96885
[96959]: rust-lang/rust#96959
[97034]: rust-lang/rust#97034
[97202]: rust-lang/rust#97202
[97316]: rust-lang/rust#97316
[97652]: rust-lang/rust#97652
[97675]: rust-lang/rust#97675
[97803]: rust-lang/rust#97803
[97837]: rust-lang/rust#97837
[97867]: rust-lang/rust#97867
[cargo/10713]: rust-lang/cargo#10713
[cargo/10755]: rust-lang/cargo#10755

[`array::from_fn`]: https://doc.rust-lang.org/stable/std/array/fn.from_fn.html
[`Box::into_pin`]: https://doc.rust-lang.org/stable/std/boxed/struct.Box.html#method.into_pin
[`BinaryHeap::try_reserve_exact`]: https://doc.rust-lang.org/stable/alloc/collections/binary_heap/struct.BinaryHeap.html#method.try_reserve_exact
[`BinaryHeap::try_reserve`]: https://doc.rust-lang.org/stable/std/collections/struct.BinaryHeap.html#method.try_reserve
[`OsString::try_reserve`]: https://doc.rust-lang.org/stable/std/ffi/struct.OsString.html#method.try_reserve
[`OsString::try_reserve_exact`]: https://doc.rust-lang.org/stable/std/ffi/struct.OsString.html#method.try_reserve_exact
[`PathBuf::try_reserve`]: https://doc.rust-lang.org/stable/std/path/struct.PathBuf.html#method.try_reserve
[`PathBuf::try_reserve_exact`]: https://doc.rust-lang.org/stable/std/path/struct.PathBuf.html#method.try_reserve_exact
[`Path::try_exists`]: https://doc.rust-lang.org/stable/std/path/struct.Path.html#method.try_exists
[`Ref::filter_map`]: https://doc.rust-lang.org/stable/std/cell/struct.Ref.html#method.filter_map
[`RefMut::filter_map`]: https://doc.rust-lang.org/stable/std/cell/struct.RefMut.html#method.filter_map
[`NonNull::<slice>::len`]: https://doc.rust-lang.org/stable/std/ptr/struct.NonNull.html#method.len
[`ToOwned::clone_into`]: https://doc.rust-lang.org/stable/std/borrow/trait.ToOwned.html#method.clone_into
[`Ipv6Addr::to_ipv4_mapped`]: https://doc.rust-lang.org/stable/std/net/struct.Ipv6Addr.html#method.to_ipv4_mapped
[`unix::io::AsFd`]: https://doc.rust-lang.org/stable/std/os/unix/io/trait.AsFd.html
[`unix::io::BorrowedFd<'fd>`]: https://doc.rust-lang.org/stable/std/os/unix/io/struct.BorrowedFd.html
[`unix::io::OwnedFd`]: https://doc.rust-lang.org/stable/std/os/unix/io/struct.OwnedFd.html
[`windows::io::AsHandle`]: https://doc.rust-lang.org/stable/std/os/windows/io/trait.AsHandle.html
[`windows::io::BorrowedHandle<'handle>`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.BorrowedHandle.html
[`windows::io::OwnedHandle`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.OwnedHandle.html
[`windows::io::HandleOrInvalid`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.HandleOrInvalid.html
[`windows::io::HandleOrNull`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.HandleOrNull.html
[`windows::io::InvalidHandleError`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.InvalidHandleError.html
[`windows::io::NullHandleError`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.NullHandleError.html
[`windows::io::AsSocket`]: https://doc.rust-lang.org/stable/std/os/windows/io/trait.AsSocket.html
[`windows::io::BorrowedSocket<'handle>`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.BorrowedSocket.html
[`windows::io::OwnedSocket`]: https://doc.rust-lang.org/stable/std/os/windows/io/struct.OwnedSocket.html
[`thread::scope`]: https://doc.rust-lang.org/stable/std/thread/fn.scope.html
[`thread::Scope`]: https://doc.rust-lang.org/stable/std/thread/struct.Scope.html
[`thread::ScopedJoinHandle`]: https://doc.rust-lang.org/stable/std/thread/struct.ScopedJoinHandle.html

[`array::from_ref`]: https://doc.rust-lang.org/stable/std/array/fn.from_ref.html
[`slice::from_ref`]: https://doc.rust-lang.org/stable/std/slice/fn.from_ref.html
[`intrinsics::copy`]: https://doc.rust-lang.org/stable/std/intrinsics/fn.copy.html
[`intrinsics::copy_nonoverlapping`]: https://doc.rust-lang.org/stable/std/intrinsics/fn.copy_nonoverlapping.html
[`<*const T>::copy_to`]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.copy_to
[`<*const T>::copy_to_nonoverlapping`]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.copy_to_nonoverlapping
[`<*mut T>::copy_to`]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.copy_to-1
[`<*mut T>::copy_to_nonoverlapping`]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.copy_to_nonoverlapping-1
[`<*mut T>::copy_from`]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.copy_from
[`<*mut T>::copy_from_nonoverlapping`]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.copy_from_nonoverlapping
[`str::from_utf8`]: https://doc.rust-lang.org/stable/std/str/fn.from_utf8.html
[`Utf8Error::error_len`]: https://doc.rust-lang.org/stable/std/str/struct.Utf8Error.html#method.error_len
[`Utf8Error::valid_up_to`]: https://doc.rust-lang.org/stable/std/str/struct.Utf8Error.html#method.valid_up_to
[`Condvar::new`]: https://doc.rust-lang.org/stable/std/sync/struct.Condvar.html#method.new
[`Mutex::new`]: https://doc.rust-lang.org/stable/std/sync/struct.Mutex.html#method.new
[`RwLock::new`]: https://doc.rust-lang.org/stable/std/sync/struct.RwLock.html#method.new
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. finished-final-comment-period The final comment period is finished for this PR / Issue. needs-fcp This change is insta-stable, so needs a completed FCP to proceed. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.