Memory leak

[stakeworks]

Ubuntu: 20.04 LTS
8 cores
30 GB RAM
bifrost 0.9.44-7af39eb

After upgrade to bifrost 0.9.44-7af39eb memory usage increases a lot! (the memory issue also happened after upgrade to robonomics 2.0.0-17b47a4bd7c)

Don't know if it's related but also getting these messages:

Labels
alertname = UnboundedChannelPersistentlyLarge
chain = bifrost
entity = mpsc_finality_notification_stream
instance = localhost:19635
job = bifrost-kusama-cmn12
monitor = CMN12
severity = warning
Annotations
message = Channel mpsc_finality_notification_stream on node localhost:19635 contains more than 200 items for more than 5 minutes. Node might be frozen.

[Akagi201]

It seems that it is releated to this issue paritytech/substrate#11604

[koute]

If you have the following at the start of your logs:

[Relaychain] BEEFY is still experimental, usage on a production network is discouraged.

...and you're not actually using BEEFY then it's most likely the same issue.

If so here's a fix you can test out: paritytech/substrate#11694

[stakeworks]

Looks like it's been solved in version 0.9.52, memory usage is still normal after an hour. Will keep an eye on it for the next days.

[ark930]

Fixed by the version bifrost-v0.9.52: https://github.com/bifrost-finance/bifrost/releases/tag/bifrost-v0.9.52-1