-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
yamux: Switch to upstream implementation while keeping the controller API #320
Conversation
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome! This simplifies things a lot! 🎉
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Did enabling auto-tuning improve performance in any significant way? |
Hi, I've only noticed an improvement in our synthetic benchmarks for the notification protocol on
Before this PR we saw the following:
This reflects a ~20% improvement on this benchmark (before and after). However, I'm not entirely sure about the stability of the data we get out of this. I've not seen anything else in versi-network. CPU-wise, we look to be in the same ballpark. I would still like to run some tests. The update's main goal was to filter out compatibility issues between libp2p <-> litep2p. |
Thanks for the info! I'm surprised that libp2p is faster, considering I spent a quite a bit of time optimizing the notification paths in litep2p and have a recollection of it being faster, albeit in microbenchmarks. Granted the architectures are very different so could be that this is the price litep2p pays for all the message passing it does or my benchmarks were incorrect. AFAIR the receiver side is quite copy-heavy so if these numbers are unacceptable, I think there are still ways to optimize further. |
## [0.9.1] - 2025-01-19 This release enhances compatibility between litep2p and libp2p by backporting the latest Yamux updates. Additionally, it includes various improvements and fixes to boost the stability and performance of the WebSocket stream and the multistream-select protocol. ### Changed - yamux: Switch to upstream implementation while keeping the controller API ([#320](#320)) - req-resp: Replace SubstreamSet with FuturesStream ([#321](#321)) - cargo: Bring up to date multiple dependencies ([#324](#324)) - build(deps): bump hickory-proto from 0.24.1 to 0.24.3 ([#323](#323)) - build(deps): bump openssl from 0.10.66 to 0.10.70 ([#322](#322)) ### Fixed - websocket/stream: Fix unexpected EOF on `Poll::Pending` state poisoning ([#327](#327)) - websocket/stream: Avoid memory allocations on flushing ([#325](#325)) - multistream-select: Enforce `io::error` instead of empty protocols ([#318](#318)) - multistream: Do not wait for negotiation in poll_close ([#319](#319)) cc @paritytech/networking --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
This PR updates litep2p to version 0.9.1. The yamux config is entirely removed to mirror the libp2p yamux upstream version. While at it, I had to bump indexmap and URL as well. ## [0.9.1] - 2025-01-19 This release enhances compatibility between litep2p and libp2p by using the latest Yamux upstream version. Additionally, it includes various improvements and fixes to boost the stability and performance of the WebSocket stream and the multistream-select protocol. ### Changed - yamux: Switch to upstream implementation while keeping the controller API ([#320](paritytech/litep2p#320)) - req-resp: Replace SubstreamSet with FuturesStream ([#321](paritytech/litep2p#321)) - cargo: Bring up to date multiple dependencies ([#324](paritytech/litep2p#324)) - build(deps): bump hickory-proto from 0.24.1 to 0.24.3 ([#323](paritytech/litep2p#323)) - build(deps): bump openssl from 0.10.66 to 0.10.70 ([#322](paritytech/litep2p#322)) ### Fixed - websocket/stream: Fix unexpected EOF on `Poll::Pending` state poisoning ([#327](paritytech/litep2p#327)) - websocket/stream: Avoid memory allocations on flushing ([#325](paritytech/litep2p#325)) - multistream-select: Enforce `io::error` instead of empty protocols ([#318](paritytech/litep2p#318)) - multistream: Do not wait for negotiation in poll_close ([#319](paritytech/litep2p#319)) cc @paritytech/networking --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
This PR updates litep2p to version 0.9.1. The yamux config is entirely removed to mirror the libp2p yamux upstream version. While at it, I had to bump indexmap and URL as well. ## [0.9.1] - 2025-01-19 This release enhances compatibility between litep2p and libp2p by using the latest Yamux upstream version. Additionally, it includes various improvements and fixes to boost the stability and performance of the WebSocket stream and the multistream-select protocol. ### Changed - yamux: Switch to upstream implementation while keeping the controller API ([#320](paritytech/litep2p#320)) - req-resp: Replace SubstreamSet with FuturesStream ([#321](paritytech/litep2p#321)) - cargo: Bring up to date multiple dependencies ([#324](paritytech/litep2p#324)) - build(deps): bump hickory-proto from 0.24.1 to 0.24.3 ([#323](paritytech/litep2p#323)) - build(deps): bump openssl from 0.10.66 to 0.10.70 ([#322](paritytech/litep2p#322)) ### Fixed - websocket/stream: Fix unexpected EOF on `Poll::Pending` state poisoning ([#327](paritytech/litep2p#327)) - websocket/stream: Avoid memory allocations on flushing ([#325](paritytech/litep2p#325)) - multistream-select: Enforce `io::error` instead of empty protocols ([#318](paritytech/litep2p#318)) - multistream: Do not wait for negotiation in poll_close ([#319](paritytech/litep2p#319)) cc @paritytech/networking --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> (cherry picked from commit 42e9de7)
This PR relies on the libp2p-yamux crate for the core functionality of our multiplexer.
The main goal is to bring complete compatibility between libp2p and litep2p on the yamux layer, and remove 90% of the yamux code in favor of the upstream implementation while keeping the controller API in place.
The upstream crate brings in multiple fixes for substreams while minimally impacting our dependency tree.
The downside of the upstream implementation is that the controller API has been removed. Adjusting to the new API would be a massive breaking change for all transport layers. Therefore, we keep the controller API which integrates seamlessly with the upstream yamux. No other changes were present to the controller API in the upstream implementation.
Yamux changelog
The changelog includes the fixes from the upstream since the moment we have inlined the crate in litep2p:
Next Steps
cc @paritytech/networking