Refactor: Improve reliability of peerd connections #792

TheCharlatan · 2022-11-28T17:49:11Z

This includes:

Tracking the swapd connection status
Unique peerd Id's
Taker reconnects on checkpoint restore
Reporting back on initial connect failure

This allows us to consistently inform the user on the connection status of the swap. Peerd now emits Disconnected and Reconnected requests to farcasterd, which in turn can route them on to swapd.

The connect command now allows a user to manually trigger a connect for a restored swap that was previously unable to connect.

…r swap

was calling colorize func, not swap_id itself

Handle Connect msg properly for running and restoring swapd Peerd: Add a Pong to our own hacky handshake Peerd: Only insert protocol messages to unchecked_msg_cache

Farcaster: Correct handling of restoring tsm connection

Rustfmt

Ensure ConnectSuccess message is only expected for Taker swaps Handle new and failed connections in a coherent manner Trigger peerd termination externally, not internally

log swapids if awaiting connection

Peerd: Emit Reconnected only after checking cached messages Do not reconnect as a listener-spawned peerd

h4sh3d

Currently testing swaps against fc.dev. This is my first round of review.

I can see this in my logs, not sure what triggers it and if it is related to this work or not:

[2022-11-29T10:18:23Z WARN  farcaster_node::swapd::swap_state] Not on RefundSig state

This is happening as Bob.

src/event.rs

src/farcasterd/runtime.rs

src/farcasterd/trade_state_machine.rs

src/grpcd/runtime.rs

TheCharlatan · 2022-11-29T10:38:00Z

I can see this in my logs, not sure what triggers it and if it is related to this work or not:
[2022-11-29T10:18:23Z WARN  farcaster_node::swapd::swap_state] Not on RefundSig state
This is happening as Bob.

This is unrelated, it's also logged on main.

…peerd

…ntRebase

h4sh3d

Last round, mostly minor comments, feel free to discard the ones that you consider not important.

src/farcasterd/runtime.rs

src/farcasterd/trade_state_machine.rs

src/swapd/runtime.rs

src/farcasterd/trade_state_machine.rs

src/peerd/runtime.rs

TheCharlatan and others added 30 commits November 14, 2022 23:15

Restore: Re-establish connection when restoring a taker swap

2e84852

Swapd: Improve connection information in swap info request

84a5168

This allows us to consistently inform the user on the connection status of the swap. Peerd now emits Disconnected and Reconnected requests to farcasterd, which in turn can route them on to swapd.

Cli: Implement Connect command

f69f82b

The connect command now allows a user to manually trigger a connect for a restored swap that was previously unable to connect.

Farcasterd: Handle cases where tsm is not found after cli call

56c1112

Farcasterd: Let swapd know of connected peerd on taker restore

ce9ea76

Tests: Add functional swap test for restoring and reconnecting a take…

10d7f1f

…r swap

Trade State Machine: Log listen failed on make offer

de7980a

Grpc: Add connect_swap endpoint

3152119

Tests: Rename setup_clients to launch_farcasterd_pair

84d91f6

Merge remote-tracking branch 'upstream/main' into restoreSwapConnect

faddc06

Merge remote-tracking branch 'upstream/main' into restoreSwapConnect

bf7341b

Merge remote-tracking branch 'upstream/main' into restoreSwapConnect

dbf904a

log swap_id from request in case sm_name not set

32b7c2a

increase logging

0cff81f

fix swap_id logging

e720499

was calling colorize func, not swap_id itself

fixup! increase logging

b80916f

Update sequencediagram

ad51a61

log swapid with check

fc7f534

Peerd: Connect to remote peer from within the runtime

1277b76

Handle Connect msg properly for running and restoring swapd Peerd: Add a Pong to our own hacky handshake Peerd: Only insert protocol messages to unchecked_msg_cache

log service availability when not transitioning

0919454

Farcaster: Correct handling of tsm's awaiting a Connect

58ccee8

Farcaster: Correct handling of restoring tsm connection

log swap_id with all attempt_...swapd_running msgs

38c03b8

Rustfmt

Farcaster: Normalize connection handling

0212ea4

Ensure ConnectSuccess message is only expected for Taker swaps Handle new and failed connections in a coherent manner Trigger peerd termination externally, not internally

log peerd termination upon cleanup

3ebfd3f

log swapids if awaiting connection

Peerd: Use the same function for connecting and re-connecting

a0dd2e5

Peerd: Emit Reconnected only after checking cached messages Do not reconnect as a listener-spawned peerd

log PID with peerd when launching & terminating

acfe826

demote noisy next() logging

458222c

warn if non-farcasterd service is broker

1b89853

log PID when receiving protocol messages

5196e6e

peerd: send receipt to peer only after forwarding

e272427

TheCharlatan added 4 commits November 28, 2022 19:07

Tests: Correct compilation error

4769991

Logging: Cleanup info level connection logging

0a1e1b6

Farcaster runtime: Fix double reference

bfeaf2c

Tests: Test reconnect during making/taking an offer

45612b2

This was referenced Nov 28, 2022

Restore: Re-establish connection when restoring a taker swap #752

Closed

Peerd connect treatmeant #787

Closed

reattempt connect if failed during init connect #729

Closed

TheCharlatan marked this pull request as ready for review November 28, 2022 20:14

Diagrams: Update with new connection procedure

f593513

TheCharlatan added refactor farcaster.dev testing Currently apply (or to be applied) on farcaster.dev backend reliability Feature increasing the reliability of the service labels Nov 28, 2022

TheCharlatan changed the title ~~Peerd connect treatment rebase~~ Refactor: Improve reliability of peerd connections Nov 28, 2022

h4sh3d reviewed Nov 29, 2022

View reviewed changes

src/event.rs Outdated Show resolved Hide resolved

src/farcasterd/runtime.rs Outdated Show resolved Hide resolved

src/farcasterd/trade_state_machine.rs Outdated Show resolved Hide resolved

src/grpcd/runtime.rs Outdated Show resolved Hide resolved

TheCharlatan added 4 commits November 29, 2022 11:41

Event: Remove debug log line with conflicting message

65329a3

TSM: Add helper function for getting swaps connected with a specific …

b19a007

…peerd

TSM: Log error is maker peerd is none

f1dd5fa

Merge remote-tracking branch 'upstream/main' into peerdConnectTreatme…

2f83e4c

…ntRebase

TheCharlatan mentioned this pull request Nov 29, 2022

Config: Define global static bind address #766

Merged

Merge remote-tracking branch 'upstream/main' into peerdConnectTreatme…

a93a295

…ntRebase

h4sh3d reviewed Nov 30, 2022

View reviewed changes

TheCharlatan added 5 commits November 30, 2022 16:56

Farcaster: Unify handling of Disconnected and Reconnected message

21f38e7

Farcaster: Log swap id on clean up service terminating

11a92b7

Farcaster: Improve various logging statements

1d1ec6e

Remove some needless whitespace

6e23cf1

TSM: Update diagram

ed4247f

h4sh3d approved these changes Dec 1, 2022

View reviewed changes

h4sh3d merged commit 5d8a61f into farcaster-project:main Dec 1, 2022

Lederstrumpf mentioned this pull request Dec 9, 2022

performance tracking #810

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor: Improve reliability of peerd connections #792

Refactor: Improve reliability of peerd connections #792

TheCharlatan commented Nov 28, 2022 •

edited

Loading

h4sh3d left a comment

TheCharlatan commented Nov 29, 2022

h4sh3d left a comment

Refactor: Improve reliability of peerd connections #792

Refactor: Improve reliability of peerd connections #792

Conversation

TheCharlatan commented Nov 28, 2022 • edited Loading

h4sh3d left a comment

Choose a reason for hiding this comment

TheCharlatan commented Nov 29, 2022

h4sh3d left a comment

Choose a reason for hiding this comment

TheCharlatan commented Nov 28, 2022 •

edited

Loading