Skip to content
This repository has been archived by the owner on Sep 6, 2022. It is now read-only.

Network Resource Manager interface #229

Merged
merged 57 commits into from
Jan 17, 2022
Merged

Network Resource Manager interface #229

merged 57 commits into from
Jan 17, 2022

Conversation

vyzo
Copy link
Contributor

@vyzo vyzo commented Dec 16, 2021

See libp2p/go-libp2p#1260 for the proposal/design discussion.

@vyzo vyzo marked this pull request as draft December 16, 2021 12:48
network/rcmgr.go Outdated Show resolved Hide resolved
// TransactionalScope is a mixin interface for transactional scopes.
type TransactionalScope interface {
// Done ends the transaction scope and releases associated resources.
Done()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have any examples of how we think we might want to use these interfaces? We should start with some toy applications before we spend too much time on these APIs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The most basic hooks will be inside the transport and muxer implementations.
We need to cater to both them, at the bowels of the stack, and higher level users.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add testable examples to the godocs? https://go.dev/blog/examples

Copy link
Contributor

@marten-seemann marten-seemann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial pass.

network/rcmgr.go Outdated Show resolved Hide resolved
network/rcmgr.go Outdated Show resolved Hide resolved
network/rcmgr.go Outdated Show resolved Hide resolved
network/rcmgr.go Outdated Show resolved Hide resolved
network/rcmgr.go Outdated Show resolved Hide resolved
Copy link
Contributor

@marten-seemann marten-seemann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick review today, more to follow tomorrow.

network/rcmgr.go Outdated Show resolved Hide resolved
network/rcmgr.go Outdated Show resolved Hide resolved
network/rcmgr.go Outdated Show resolved Hide resolved
network/rcmgr.go Outdated Show resolved Hide resolved
network/rcmgr.go Show resolved Hide resolved
vyzo and others added 3 commits December 21, 2021 17:02
Co-authored-by: Marten Seemann <martenseemann@gmail.com>
Co-authored-by: Marten Seemann <martenseemann@gmail.com>
network/rcmgr.go Outdated Show resolved Hide resolved
1. Introduce transactions in all scopes
2. Limit the view of stream/connection scope for users, to avoid the Done footgun
3. Move OpenStream to the resource manager
network/rcmgr.go Outdated Show resolved Hide resolved
vyzo and others added 5 commits January 5, 2022 17:18
Co-authored-by: Marten Seemann <martenseemann@gmail.com>
…mory status

so that we don't have to undo reservations if there is too much pressure.
@vyzo vyzo marked this pull request as ready for review January 8, 2022 16:10
network/rcmgr.go Outdated Show resolved Hide resolved
network/rcmgr.go Outdated Show resolved Hide resolved
network/rcmgr.go Show resolved Hide resolved
network/rcmgr.go Outdated Show resolved Hide resolved
network/rcmgr.go Outdated Show resolved Hide resolved
vyzo and others added 3 commits January 13, 2022 14:58
Co-authored-by: raulk <raul@protocol.ai>
Co-authored-by: raulk <raul@protocol.ai>
Co-authored-by: raulk <raul@protocol.ai>
network/rcmgr.go Outdated Show resolved Hide resolved
ViewTransient(func(ResourceScope) error) error

// ViewService retrieves a service-specific scope.
ViewService(string, func(ServiceScope) error) error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does service deal with transient connections too? How does hole punching figure into this (e.g. when secondary connections associated with the service are made)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the service does not deal with conns at all, it only deals with streams.
Conns are really shared resources, so there isn't much of a point to have them owned by a service.

network/rcmgr.go Show resolved Hide resolved
transport/transport.go Show resolved Hide resolved
network/rcmgr.go Outdated
Stat() ScopeStat

// BeginTransaction creates a new transactional scope rooted at this scope
BeginTransaction() (TransactionalScope, error)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Under what kinds of situations are we expecting an error here? Is this just room for growth in case anything comes up, or do we have some in mind?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We currently error if the scope has been closed. But it's good to have room for growth too.

network/rcmgr.go Show resolved Hide resolved
Comment on lines +251 to +254
NumStreamsInbound int
NumStreamsOutbound int
NumConnsInbound int
NumConnsOutbound int
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How (if at all) are we planning to deal with these in the context of relayed or DCUtR connections?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you mean here? Relayed connections are just a virtual connection on top of some other connection and don't use fds.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do count them however in stat.

@github-actions
Copy link

gocompat says:

Branch 'master' set up to track remote branch 'master' from 'origin'.
"github.com/libp2p/go-libp2p-core/mux".Multiplexer DeclarationTypeChanged
"github.com/libp2p/go-libp2p-core/mux".MuxedConn DeclarationTypeChanged
"github.com/libp2p/go-libp2p-core/mux".MuxedStream DeclarationTypeChanged
"github.com/libp2p/go-libp2p-core/mux".NoopHandler TopLevelDeclarationDeleted
"github.com/libp2p/go-libp2p-core/network".Network InterfaceChanged
"github.com/libp2p/go-libp2p-core/network".Stream InterfaceChanged
"github.com/libp2p/go-libp2p-core/network".Conn InterfaceChanged
"github.com/libp2p/go-libp2p-core/transport".CapableConn InterfaceChanged
"github.com/libp2p/go-libp2p-core/transport".TransportNetwork InterfaceChanged
"github.com/libp2p/go-libp2p-core/transport".Upgrader InterfaceChanged

@marten-seemann marten-seemann merged commit e4c76cf into master Jan 17, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants