memory leak during block syncing

[stefantalpalaru]

To replicate, run a local sim in one terminal tab:

make NIMFLAGS="-d:insecure" eth2_network_simulation

In another tab, start the Prometheus daemon (after the sim starts running, so the conf file has been generated):

cd tests/simulation/prometheus
prometheus

Wait a couple of minutes, then connect an external node that will start syncing blocks (from a third tab):

./tests/simulation/run_node.sh 4

You can do a quick-and-dirty metric visualisation from Prometheus' web interface. Relevant graphs (you also want the "metrics" branch of "vendor/nim-chronos"):

GC's heap:

Pending futures:

[stefantalpalaru]

Fixed by #874

[stefantalpalaru]

It's back ("Blob" is "seq[byte]"):

[dryajov]

Potentially addressed by vacp2p/nim-libp2p#187

[stefantalpalaru]

Now we're leaking strings:

[dryajov]

yes, notices that, but wasn't 100% sure, seq[byte] is gone completely?

[stefantalpalaru]

seq[byte] is gone completely?

No, but looks like it slowed down considerably. Here it is in the second node:

[dryajov]

Latest master for libp2p has some additional fixes that might address some of this further, but I've definitely seen string leaks even with those, so more digging is required.

[stefantalpalaru]

Metrics from a witti client runing on an empty db show memory leaks, open file descriptor leaks and libp2p connection leaks.

make clean-witti
make NIMFLAGS="-d:insecure" SCRIPT_PARAMS="--skipGoerliKey" witti

[dryajov]

Metrics from a witti client runing on an empty db show memory leaks, open file descriptor leaks and libp2p connection leaks.

make clean-witti
make NIMFLAGS="-d:insecure" SCRIPT_PARAMS="--skipGoerliKey" witti

this is potentially fixed in latest master in libp2p, nbc needs to be updated with it.

[arnetheduck]

resolved by status-im/nim-chronos#130