bench-bot is not responding to commands sometimes

[joao-paulo-parity]

e.g. paritytech/polkadot#4777 (comment)

might be related to #68

or might be related to some other problem which is not visible from the logs at the moment #82

[joao-paulo-parity]

My current guess is related to the proxy we're using for receiving events (https://github.com/paritytech/devops/issues/1070). Specifically after we reconnect:

ERROR (server):                                                                                        
    type: "error"                                                                                      
INFO (server): Connected    

It seems that although the server says it's "Connected" again, we might no longer receive events.

Here's the trail of code sources leading up to this assumption:

• https://github.com/probot/probot/blob/ce9688493029f32a5abdeb3a4fb91d56d00c18e5/src/helpers/webhook-proxy.ts
• https://github.com/EventSource/eventsource/blob/9dd06876ab43af37c3313c679fbdc7f722293a0d/lib/eventsource.js#L65
• https://github.com/probot/smee-client/blob/ac4fe48924217f426889f4860d5e2a22c453be95/index.ts#L64
• https://github.com/EventSource/eventsource/blob/9dd06876ab43af37c3313c679fbdc7f722293a0d/lib/eventsource.js#L178

If the guess is correct then we can try to restart the whole server when an error happens instead of reconnecting, but that implies a refactor in the server's setup.

This problem might originate from the library we're using, https://github.com/EventSource/eventsource, which does not appear to be well-maintained. If this bot were to be rewritten in another language as suggested by #54 then the error could end up being solved by using a (better and) different implementation of EventSource, e.g. in processbot we're using a different library (https://github.com/paritytech/parity-processbot/blob/2f8ffedb62a2abc1a3b50c983087e80c1d8677be/src/main.rs#L72).