Skip to content
This repository has been archived by the owner on Sep 6, 2022. It is now read-only.

Network Resource Manager interface #229

Merged
merged 57 commits into from
Jan 17, 2022
Merged
Show file tree
Hide file tree
Changes from 48 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
cfce3f5
add resource manager interfaces
vyzo Dec 16, 2021
044fbea
add scope accessors to streams and conns
vyzo Dec 16, 2021
193ee25
add ResourceManager accessor to the Network interface
vyzo Dec 16, 2021
2afdeee
allow initially unattached streams.
vyzo Dec 16, 2021
46cbcfe
introduce service scopes, canonicalize ownership interface through se…
vyzo Dec 21, 2021
d4cfc93
make system scope explicit
vyzo Dec 21, 2021
586f1d0
make memory stat an int64
vyzo Dec 21, 2021
b3f9e1d
make the system scope a generic resource scope, introduce the DMZ
vyzo Dec 21, 2021
35ec4fd
fix typo
vyzo Dec 21, 2021
081233f
fix typo
vyzo Dec 21, 2021
5767cbb
Merge branch 'master' into feat/rcmgr
vyzo Dec 21, 2021
aeb9233
rename DMZ to transient scope, remove OpenConnection from PeerScope
vyzo Dec 22, 2021
d997bbb
remove ncopy param from GrowBuffer
vyzo Dec 22, 2021
8f759d3
remove protocols from OpenStream
vyzo Dec 22, 2021
dcef937
document nil receiver contract, fix protocol scope protocol accessor …
vyzo Dec 22, 2021
c52807c
remove nil receiver contract requirement
vyzo Dec 23, 2021
76a25c9
flesh out stat struct
vyzo Dec 23, 2021
25d4931
turn resource manager scope accessors into viewers
vyzo Dec 23, 2021
deb1c16
interface refiniments
vyzo Dec 24, 2021
1f5a414
add Buffer interface
vyzo Dec 27, 2021
6400bea
fix typo
vyzo Dec 28, 2021
3357752
fix typo
vyzo Dec 28, 2021
d99bef0
fix typo
vyzo Dec 28, 2021
c452665
rename user scopes to plain names, management scopes as such
vyzo Dec 28, 2021
219e9bc
rename BeginTxn to BeginTransaction
vyzo Dec 28, 2021
3ecf62c
RIP Buffers
vyzo Dec 28, 2021
535c095
make ErrResourceLimitExceeded a temporary error; move rcmgr errors wi…
vyzo Dec 29, 2021
caacd96
unexport TemporaryError
vyzo Dec 30, 2021
a5d2e41
null resource manager stub
vyzo Dec 31, 2021
b948cd7
unexport the null stubs, make entry point a variable
vyzo Dec 31, 2021
48c94b6
don't rely on typed nils but instead use actual null object instances
vyzo Dec 31, 2021
5fcc7b6
add Scope to the CapableConn interface
vyzo Dec 31, 2021
625a159
rename ConnectionScope to ConnScope for consistency
vyzo Dec 31, 2021
f205ca6
fix typo
vyzo Dec 31, 2021
31b6a33
rename ConnectionManagementScope to ConnManagementScope
vyzo Dec 31, 2021
59c3e97
Merge branch 'master' of github.com:libp2p/go-libp2p-core into feat/r…
vyzo Jan 4, 2022
f4f1520
add the ConnManagementScope to Upgrader.Upgrade
marten-seemann Jan 4, 2022
d2ff788
fix argument name
marten-seemann Jan 4, 2022
d1a82f1
godocs for ResourceManager
vyzo Jan 4, 2022
541324d
introduce MemoryStatus indicator in ReserveMemory
vyzo Jan 5, 2022
cafe95a
use uint8 for MemoryStatus
vyzo Jan 5, 2022
84d55d3
rework reservation interface to pass priority instead of returning me…
vyzo Jan 6, 2022
d9136f0
improve comment
vyzo Jan 6, 2022
2192774
fix typo
vyzo Jan 6, 2022
005b84b
export the NullScope
marten-seemann Jan 7, 2022
9d473a8
Stream.SetProtocol can return an error
vyzo Jan 9, 2022
0c1046c
merge the mux package into network
marten-seemann Jan 13, 2022
0391e67
pass the PeerScope to Multiplexer.NetConn
marten-seemann Jan 13, 2022
d3d9ac7
Update network/rcmgr.go
vyzo Jan 13, 2022
140d0ed
Update network/rcmgr.go
vyzo Jan 13, 2022
7a1c47e
Update network/rcmgr.go
vyzo Jan 13, 2022
e2bac5c
Update network/rcmgr.go
vyzo Jan 14, 2022
4bfde8e
remove reference to deprecated mux.MuxedConn
vyzo Jan 14, 2022
68e492f
rename transaction to span
vyzo Jan 14, 2022
953320f
indicate bytes in ReserveMemory
vyzo Jan 14, 2022
06aa6f9
break ResourceManager View methods into Viewer interface.
vyzo Jan 14, 2022
6b8d8bf
add experimental interface warning
vyzo Jan 14, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 9 additions & 91 deletions mux/mux.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,99 +5,17 @@
package mux

import (
"context"
"errors"
"io"
"net"
"time"
"github.com/libp2p/go-libp2p-core/network"
)

// ErrReset is returned when reading or writing on a reset stream.
var ErrReset = errors.New("stream reset")
// Deprecated: use network.ErrReset instead.
var ErrReset = network.ErrReset

// Stream is a bidirectional io pipe within a connection.
type MuxedStream interface {
io.Reader
io.Writer
// Deprecated: use network.MuxedStream instead.
type MuxedStream = network.MuxedStream

// Close closes the stream.
//
// * Any buffered data for writing will be flushed.
// * Future reads will fail.
// * Any in-progress reads/writes will be interrupted.
//
// Close may be asynchronous and _does not_ guarantee receipt of the
// data.
//
// Close closes the stream for both reading and writing.
// Close is equivalent to calling `CloseRead` and `CloseWrite`. Importantly, Close will not wait for any form of acknowledgment.
// If acknowledgment is required, the caller must call `CloseWrite`, then wait on the stream for a response (or an EOF),
// then call Close() to free the stream object.
//
// When done with a stream, the user must call either Close() or `Reset()` to discard the stream, even after calling `CloseRead` and/or `CloseWrite`.
io.Closer
// Deprecated: use network.MuxedConn instead.
type MuxedConn = network.MuxedConn

// CloseWrite closes the stream for writing but leaves it open for
// reading.
//
// CloseWrite does not free the stream, users must still call Close or
// Reset.
CloseWrite() error

// CloseRead closes the stream for reading but leaves it open for
// writing.
//
// When CloseRead is called, all in-progress Read calls are interrupted with a non-EOF error and
// no further calls to Read will succeed.
//
// The handling of new incoming data on the stream after calling this function is implementation defined.
//
// CloseRead does not free the stream, users must still call Close or
// Reset.
CloseRead() error

// Reset closes both ends of the stream. Use this to tell the remote
// side to hang up and go away.
Reset() error

SetDeadline(time.Time) error
SetReadDeadline(time.Time) error
SetWriteDeadline(time.Time) error
}

// NoopHandler do nothing. Resets streams as soon as they are opened.
var NoopHandler = func(s MuxedStream) { s.Reset() }

// MuxedConn represents a connection to a remote peer that has been
// extended to support stream multiplexing.
//
// A MuxedConn allows a single net.Conn connection to carry many logically
// independent bidirectional streams of binary data.
//
// Together with network.ConnSecurity, MuxedConn is a component of the
// transport.CapableConn interface, which represents a "raw" network
// connection that has been "upgraded" to support the libp2p capabilities
// of secure communication and stream multiplexing.
type MuxedConn interface {
// Close closes the stream muxer and the the underlying net.Conn.
io.Closer

// IsClosed returns whether a connection is fully closed, so it can
// be garbage collected.
IsClosed() bool

// OpenStream creates a new stream.
OpenStream(context.Context) (MuxedStream, error)

// AcceptStream accepts a stream opened by the other side.
AcceptStream() (MuxedStream, error)
}

// Multiplexer wraps a net.Conn with a stream multiplexing
// implementation and returns a MuxedConn that supports opening
// multiple streams over the underlying net.Conn
type Multiplexer interface {

// NewConn constructs a new connection
NewConn(c net.Conn, isServer bool) (MuxedConn, error)
}
// Deprecated: use network.Multiplexer instead.
type Multiplexer = network.Multiplexer
8 changes: 8 additions & 0 deletions network/conn.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ type Conn interface {
ConnSecurity
ConnMultiaddrs
ConnStat
ConnScoper

// ID returns an identifier that uniquely identifies this Conn within this
// host, during this run. Connection IDs may repeat across restarts.
Expand Down Expand Up @@ -65,3 +66,10 @@ type ConnStat interface {
// Stat stores metadata pertaining to this conn.
Stat() ConnStats
}

// ConnScoper is the interface that one can mix into a connection interface to give it a resource
// management scope
type ConnScoper interface {
// Scope returns the user view of this connection's resource scope
Scope() ConnScope
}
21 changes: 20 additions & 1 deletion network/errors.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,17 @@
package network

import "errors"
import (
"errors"
"net"
)

type temporaryError string

func (e temporaryError) Error() string { return string(e) }
func (e temporaryError) Temporary() bool { return true }
func (e temporaryError) Timeout() bool { return false }

var _ net.Error = temporaryError("")

// ErrNoRemoteAddrs is returned when there are no addresses associated with a peer during a dial.
var ErrNoRemoteAddrs = errors.New("no remote addresses")
Expand All @@ -12,3 +23,11 @@ var ErrNoConn = errors.New("no usable connection to peer")
// ErrTransientConn is returned when attempting to open a stream to a peer with only a transient
// connection, without specifying the UseTransient option.
var ErrTransientConn = errors.New("transient connection to peer")

// ErrResourceLimitExceeded is returned when attempting to perform an operation that would
// exceed system resource limits.
var ErrResourceLimitExceeded = temporaryError("resource limit exceeded")

// ErrResourceScopeClosed is returned when attemptig to reserve resources in a closed resource
// scope.
var ErrResourceScopeClosed = errors.New("resource scope closed")
95 changes: 95 additions & 0 deletions network/mux.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
package network

import (
"context"
"errors"
"io"
"net"
"time"
)

// ErrReset is returned when reading or writing on a reset stream.
var ErrReset = errors.New("stream reset")

// MuxedStream is a bidirectional io pipe within a connection.
type MuxedStream interface {
io.Reader
io.Writer

// Close closes the stream.
//
// * Any buffered data for writing will be flushed.
// * Future reads will fail.
// * Any in-progress reads/writes will be interrupted.
//
// Close may be asynchronous and _does not_ guarantee receipt of the
// data.
//
// Close closes the stream for both reading and writing.
// Close is equivalent to calling `CloseRead` and `CloseWrite`. Importantly, Close will not wait for any form of acknowledgment.
// If acknowledgment is required, the caller must call `CloseWrite`, then wait on the stream for a response (or an EOF),
// then call Close() to free the stream object.
//
// When done with a stream, the user must call either Close() or `Reset()` to discard the stream, even after calling `CloseRead` and/or `CloseWrite`.
io.Closer

// CloseWrite closes the stream for writing but leaves it open for
// reading.
//
// CloseWrite does not free the stream, users must still call Close or
// Reset.
CloseWrite() error

// CloseRead closes the stream for reading but leaves it open for
// writing.
//
// When CloseRead is called, all in-progress Read calls are interrupted with a non-EOF error and
// no further calls to Read will succeed.
//
// The handling of new incoming data on the stream after calling this function is implementation defined.
//
// CloseRead does not free the stream, users must still call Close or
// Reset.
CloseRead() error

// Reset closes both ends of the stream. Use this to tell the remote
// side to hang up and go away.
Reset() error

SetDeadline(time.Time) error
SetReadDeadline(time.Time) error
SetWriteDeadline(time.Time) error
}

// MuxedConn represents a connection to a remote peer that has been
// extended to support stream multiplexing.
//
// A MuxedConn allows a single net.Conn connection to carry many logically
// independent bidirectional streams of binary data.
//
// Together with network.ConnSecurity, MuxedConn is a component of the
// transport.CapableConn interface, which represents a "raw" network
// connection that has been "upgraded" to support the libp2p capabilities
// of secure communication and stream multiplexing.
type MuxedConn interface {
// Close closes the stream muxer and the the underlying net.Conn.
io.Closer

// IsClosed returns whether a connection is fully closed, so it can
// be garbage collected.
IsClosed() bool

// OpenStream creates a new stream.
OpenStream(context.Context) (MuxedStream, error)

// AcceptStream accepts a stream opened by the other side.
AcceptStream() (MuxedStream, error)
}

// Multiplexer wraps a net.Conn with a stream multiplexing
// implementation and returns a MuxedConn that supports opening
// multiple streams over the underlying net.Conn
type Multiplexer interface {
// NewConn constructs a new connection
NewConn(c net.Conn, isServer bool, scope PeerScope) (MuxedConn, error)
}
3 changes: 3 additions & 0 deletions network/network.go
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,9 @@ type Network interface {
// listens. It expands "any interface" addresses (/ip4/0.0.0.0, /ip6/::) to
// use the known local interfaces.
InterfaceListenAddresses() ([]ma.Multiaddr, error)

// ResourceManager returns the ResourceManager associated with this network
ResourceManager() ResourceManager
}

// Dialer represents a service that can dial out to peers
Expand Down
Loading