Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: introduce tracing #4282

Merged
merged 125 commits into from
Nov 2, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
125 commits
Select commit Hold shift + click to select a range
393ae2f
feaat:tracing
eserilev Aug 1, 2023
b3f055a
move error span
eserilev Aug 3, 2023
9780a07
remove tracing-subscriber from library crate
eserilev Aug 4, 2023
c9e4a3b
update logging
eserilev Aug 10, 2023
bf2fa39
remove reduncdant logs
eserilev Aug 10, 2023
8b30438
resolve conflict
eserilev Aug 10, 2023
6c12589
autonat tracing
eserilev Aug 30, 2023
ae55e8b
dcutr changes
eserilev Aug 30, 2023
3ce8b02
relay server
eserilev Aug 30, 2023
ccdf589
floodsub
eserilev Aug 30, 2023
d23d9a1
gossipsub
eserilev Aug 30, 2023
dce4a61
identity
eserilev Aug 30, 2023
8466290
merge conflicts
eserilev Aug 30, 2023
60c4c23
kad
eserilev Aug 30, 2023
1344b7f
mdns
eserilev Aug 30, 2023
ec8f0f1
perf
eserilev Aug 30, 2023
2000e42
rendezvous
eserilev Aug 30, 2023
3f40960
request-response
eserilev Aug 30, 2023
d990488
chat example changes
eserilev Aug 30, 2023
3e7af9b
distributed key value store example
eserilev Aug 30, 2023
f4d9944
inline tracing spans
eserilev Aug 31, 2023
ea8683f
add tracing to the rest of the examples
eserilev Aug 31, 2023
9b5b439
merge conflicts
eserilev Sep 6, 2023
8eb986f
fully qualified tracing usage
eserilev Sep 6, 2023
79a533b
fix
eserilev Sep 6, 2023
1a5c1ee
fix cargo newline issue
eserilev Sep 6, 2023
f6b2f6e
fully qualify trace usage
eserilev Sep 6, 2023
4796bd9
Merge branch 'master' into feat-tracing
thomaseizinger Sep 21, 2023
a2e9834
Fix clippy lints
thomaseizinger Sep 21, 2023
e8d7745
Fix formatting
thomaseizinger Sep 21, 2023
b8473fd
Fix wasm compile error
thomaseizinger Sep 21, 2023
b9a7949
added improved tracing for autonat, dcutr, floodsub, gossipsub
eserilev Sep 21, 2023
97458ad
tracing improvements to kad and identify
eserilev Sep 21, 2023
2a42e73
mdns tracing improvements
eserilev Sep 21, 2023
6a7fbfe
perf tracing improvements
eserilev Sep 21, 2023
8ec399a
the rest of the protocols
eserilev Sep 21, 2023
a723629
remove tracing from core crate
eserilev Sep 21, 2023
8229030
removed log dependency from identity crate
eserilev Sep 21, 2023
0c0e13f
swap log w/ trace for interop-tests
eserilev Sep 21, 2023
eba436a
browser-webrtc example tracing
eserilev Sep 21, 2023
863ad7a
dcutr example tracing
eserilev Sep 21, 2023
06bfcd2
memory-connection-limits tracing
eserilev Sep 21, 2023
9885250
multistream-select tracing
eserilev Sep 21, 2023
365e21d
server tracing
eserilev Sep 21, 2023
f523e07
muxer crate tracing
eserilev Sep 22, 2023
2df883f
yamux, test harness tracing
eserilev Sep 22, 2023
b45c345
autonat tracing, plus remove unwrap
eserilev Sep 22, 2023
186cd05
keypair tracing fix
eserilev Sep 22, 2023
7b7e4af
upnp and a few other crates tracing
eserilev Sep 22, 2023
d50fed5
swarm tracing
eserilev Sep 22, 2023
9e60720
swarm test tracing
eserilev Sep 22, 2023
317e0dc
cargo fmt
eserilev Sep 22, 2023
76c8f06
tracing cleanup
eserilev Sep 22, 2023
886297b
remove '.' from tracing messages
eserilev Sep 22, 2023
913ec28
tracing dns crate, plus alphabatize dependancies
eserilev Sep 22, 2023
a227415
noise crate tracing
eserilev Sep 22, 2023
8f75357
plaintext crate tracing
eserilev Sep 22, 2023
27c49ee
pnet crate tracing
eserilev Sep 22, 2023
73b9c58
added quic tracing
eserilev Sep 22, 2023
fc4bc36
tcp crate tracing
eserilev Sep 22, 2023
49db005
webrtc crate tracing
eserilev Sep 22, 2023
0935901
weebrtc-websys crate tracing
eserilev Sep 22, 2023
e9c495a
Merge branch 'master' of https://github.com/libp2p/rust-libp2p into f…
eserilev Sep 22, 2023
8e8980f
websocket crate tracing
eserilev Sep 23, 2023
62628f4
fix dependencies
eserilev Sep 23, 2023
8497c2d
changes based on feedback
eserilev Sep 23, 2023
b6f8e50
cargo fmt
eserilev Sep 23, 2023
dd405ba
resovle conflicts
eserilev Sep 23, 2023
09ea23a
personal review
eserilev Sep 23, 2023
5263e7c
merge
eserilev Sep 24, 2023
cbbf21c
remove env logger dependency
eserilev Oct 1, 2023
540f129
fmt
eserilev Oct 1, 2023
47767b1
fixes
eserilev Oct 1, 2023
92eb669
fmt
eserilev Oct 1, 2023
fc40b56
fmt
eserilev Oct 1, 2023
294f1e0
fixes based on feedback
eserilev Oct 1, 2023
897dea0
remove unnecessary peer id ref
eserilev Oct 1, 2023
587b1ac
fixing tracing subscriber issue
eserilev Oct 5, 2023
a53da1f
fmt
eserilev Oct 5, 2023
612d933
add mdns + ping example
eserilev Oct 5, 2023
edaea4f
otel example
eserilev Oct 5, 2023
9e8774e
Merge branch 'master' into feat-tracing
thomaseizinger Oct 16, 2023
6d070c4
Merge branch 'master' into feat-tracing
thomaseizinger Oct 25, 2023
09f9e3f
Merge branch 'master' into feat-tracing
thomaseizinger Oct 25, 2023
4ba5c90
Fix compile errors
thomaseizinger Oct 25, 2023
9f856a3
Revert bad updates
thomaseizinger Oct 25, 2023
651da5a
Add more spans and extend metrics example
thomaseizinger Oct 25, 2023
70a8e26
Add more docs
thomaseizinger Oct 25, 2023
8839f36
Remove span from select
thomaseizinger Oct 25, 2023
4c574f3
Move muxer spans into `Connection`
thomaseizinger Oct 25, 2023
d5ebef0
Add spans for `Swarm`
thomaseizinger Oct 25, 2023
52048e6
Add local peer id to `Swarm::poll` span
thomaseizinger Oct 25, 2023
ffe04d4
Move spans to concrete `poll` functions
thomaseizinger Oct 25, 2023
5a3514b
Move stream muxer spans to concrete impl
thomaseizinger Oct 25, 2023
ccf60a4
Remove unused imports
thomaseizinger Oct 25, 2023
20beaad
Instrument dialing of new connections
thomaseizinger Oct 25, 2023
fee6485
Remove parents
thomaseizinger Oct 25, 2023
eac2630
Set `follows_from` instead
thomaseizinger Oct 25, 2023
d6e8eb7
Fix doc link
thomaseizinger Oct 25, 2023
40c7517
Fix libp2p doc tests
thomaseizinger Oct 25, 2023
c6e2222
Fix bad char
thomaseizinger Oct 25, 2023
77f240e
Merge branch 'master' into feat-tracing
thomaseizinger Oct 27, 2023
4a0b030
merge
eserilev Oct 30, 2023
9659490
resolve merge conflicts
eserilev Oct 30, 2023
702c664
Merge branch 'master' into feat-tracing
thomaseizinger Oct 31, 2023
c75d265
Fix formatting
thomaseizinger Oct 31, 2023
4874464
Fix compile errors and delete otel-example
thomaseizinger Oct 31, 2023
94f9caa
Add spans for durations of connections
thomaseizinger Oct 31, 2023
8d84da4
Remove fields already present in parent span
thomaseizinger Oct 31, 2023
93350b1
Fix link
thomaseizinger Oct 31, 2023
e827911
Fix doc tests and warnings
thomaseizinger Oct 31, 2023
b136d49
Remove `local_peer_id` from `Swarm::poll`
thomaseizinger Oct 31, 2023
eccc6c1
dont upgrade debug to error
eserilev Nov 1, 2023
0f34558
add tracing decorator to handlers
eserilev Nov 1, 2023
88f256e
add tracing decorator to network behaviours
eserilev Nov 1, 2023
3cdf295
merge
eserilev Nov 1, 2023
c18fbd3
Merge branch 'master' into feat-tracing
thomaseizinger Nov 2, 2023
7b55185
Remove unused imports
thomaseizinger Nov 2, 2023
949ae15
Fix logger init in test
thomaseizinger Nov 2, 2023
fd2ba76
Merge remote-tracking branch 'origin/feat-tracing' into feat-tracing
thomaseizinger Nov 2, 2023
c3595da
Unify init of subscriber across examples
thomaseizinger Nov 2, 2023
6df4a94
Serve metrics on random port to avoid panic
thomaseizinger Nov 2, 2023
b5c2233
Fix bad merge
thomaseizinger Nov 2, 2023
cb72fbf
Merge branch 'master' into feat-tracing
thomaseizinger Nov 2, 2023
1cdaaa7
Merge branch 'master' into feat-tracing
thomaseizinger Nov 2, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions examples/dcutr/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -172,10 +172,10 @@ fn main() -> Result<(), Box<dyn Error>> {
.build();

swarm
.listen_on("/ip4/0.0.0.0/udp/0/quic-v1".parse().unwrap())
.listen_on("/ip4/0.0.0.0/udp/4001/quic-v1".parse().unwrap())
.unwrap();
swarm
.listen_on("/ip4/0.0.0.0/tcp/0".parse().unwrap())
.listen_on("/ip6/::/udp/4001/quic-v1".parse().unwrap())
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
.unwrap();

// Wait to listen on all interfaces.
Expand Down
2 changes: 2 additions & 0 deletions examples/ping-example/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,5 @@ async-std = { version = "1.12", features = ["attributes"] }
async-trait = "0.1"
futures = "0.3.28"
libp2p = { path = "../../libp2p", features = ["async-std", "dns", "macros", "noise", "ping", "tcp", "websocket", "yamux"] }
tracing = { version = "0.1.37", features = ["log"] }
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
9 changes: 9 additions & 0 deletions examples/ping-example/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,19 @@ use libp2p::{
swarm::{keep_alive, NetworkBehaviour, SwarmBuilder, SwarmEvent},
tcp, yamux, Multiaddr, PeerId, Transport,
};
use tracing::level_filters::LevelFilter;
use tracing_subscriber::EnvFilter;
use std::error::Error;

#[async_std::main]
async fn main() -> Result<(), Box<dyn Error>> {

let env_filter = EnvFilter::builder()
.with_default_directive(LevelFilter::DEBUG.into())
.from_env_lossy();

tracing_subscriber::fmt().with_env_filter(env_filter).init();

let local_key = identity::Keypair::generate_ed25519();
let local_peer_id = PeerId::from(local_key.public());
println!("Local peer id: {local_peer_id:?}");
Expand Down
1 change: 1 addition & 0 deletions protocols/ping/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ libp2p-identity = { workspace = true }
log = "0.4.19"
rand = "0.8"
void = "1.0"
tracing = "0.1.37"
eserilev marked this conversation as resolved.
Show resolved Hide resolved

[dev-dependencies]
async-std = "1.6.2"
Expand Down
8 changes: 4 additions & 4 deletions protocols/ping/src/handler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -258,11 +258,11 @@ impl ConnectionHandler for Handler {
match fut.poll_unpin(cx) {
Poll::Pending => {}
Poll::Ready(Err(e)) => {
log::debug!("Inbound ping error: {:?}", e);
tracing::debug!("Inbound ping error: {:?}", e);
self.inbound = None;
}
Poll::Ready(Ok(stream)) => {
log::trace!("answered inbound ping from {}", self.peer);
tracing::trace!("answered inbound ping from {}", self.peer);
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved

// A ping from a remote peer has been answered, wait for the next.
self.inbound = Some(protocol::recv_ping(stream).boxed());
Expand All @@ -273,7 +273,7 @@ impl ConnectionHandler for Handler {
loop {
// Check for outbound ping failures.
if let Some(error) = self.pending_errors.pop_back() {
log::debug!("Ping failure: {:?}", error);
tracing::debug!("Ping failure: {:?}", error);
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved

self.failures += 1;

Expand All @@ -295,7 +295,7 @@ impl ConnectionHandler for Handler {
break;
}
Poll::Ready(Ok((stream, rtt))) => {
log::debug!("latency to {} is {}ms", self.peer, rtt.as_millis());
tracing::debug!("latency to {} is {}ms", self.peer, rtt.as_millis());
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved

self.failures = 0;
self.interval.reset(self.config.interval);
Expand Down
2 changes: 2 additions & 0 deletions swarm/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ multistream-select = { workspace = true }
[target.'cfg(not(any(target_os = "emscripten", target_os = "wasi", target_os = "unknown")))'.dependencies]
async-std = { version = "1.6.2", optional = true }
tokio = { version = "1.29", features = ["rt"], optional = true }
tracing = "0.1.37"
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved

[features]
macros = ["dep:libp2p-swarm-derive"]
Expand Down
6 changes: 6 additions & 0 deletions swarm/src/connection.rs
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,12 @@ impl ConnectionId {
}
}

impl fmt::Display for ConnectionId {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.0)
}
}

/// Information about a successfully established connection.
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct Connected {
Expand Down
6 changes: 6 additions & 0 deletions swarm/src/connection/pool.rs
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,8 @@ where
{
/// Creates a new empty `Pool`.
pub(crate) fn new(local_id: PeerId, config: PoolConfig) -> Self {
let span = tracing::error_span!("peer id:", %local_id, connection_id = tracing::field::Empty);
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
let _guard = span.enter();
let (pending_connection_events_tx, pending_connection_events_rx) = mpsc::channel(0);
let executor = match config.executor {
Some(exec) => ExecSwitch::Executor(exec),
Expand Down Expand Up @@ -495,6 +497,10 @@ where
) {
let connection = connection.extract();

let span = tracing::error_span!("peer id:", %obtained_peer_id, connection_id = tracing::field::Empty);
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
let _guard = span.enter();
span.record("connection id", tracing::field::display(id));
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved

let conns = self.established.entry(obtained_peer_id).or_default();
self.counters.inc_established(endpoint);

Expand Down
Loading