Investigate web3 event listener cancellation and termination errors

[bgins]

General Description

We have been seeing some errors reported on cancellation or program termination as of #432. Let's see if we are handling cleanup correctly and what we can do about the errors.

Which system(s) or functionality does this affect

This change affects the job creators, resource providers, mediator, and solver. It also interacts with the web3 protocol, but nothing needs to change in the protocol.

Describe the changes, and how this affects/ interacts with each system.

We have been seeing errors on shutting down services like:

^C2024-11-27T09:32:45-08:00 ERR ../../../../../usr/local/go/src/runtime/asm_arm64.s:1223 > error starting listeners: cancel by context reconnect in 2 seconds
2024-11-27T09:32:45-08:00 ERR ../../../../../usr/local/go/src/runtime/asm_arm64.s:1223 > error starting listeners: cancel by context reconnect in 2 seconds
2024-11-27T09:32:45-08:00 ERR ../../../../../usr/local/go/src/runtime/asm_arm64.s:1223 > error starting listeners: cancel by context reconnect in 2 seconds
2024-11-27T09:32:45-08:00 ERR ../../../../../usr/local/go/src/runtime/asm_arm64.s:1223 > error starting listeners: cancel by context reconnect in 2 seconds
2024-11-27T09:32:45-08:00 ERR ../../../../../usr/local/go/src/runtime/asm_arm64.s:1223 > error starting listeners: cancel by context reconnect in 2 seconds
2024-11-27T09:32:45-08:00 ERR ../../../../../usr/local/go/src/runtime/asm_arm64.s:1223 > error starting listeners: cancel by context reconnect in 2 seconds

We have also been seeing errors at program termination when running the CLI run command:

2024-11-27T09:33:37-08:00 ERR ../../../../../usr/local/go/src/runtime/asm_arm64.s:1223 > error starting listeners: cancel by mediation MediationRequested event subscribe error %!w(<nil>) reconnect in 2 seconds
2024-11-27T09:33:37-08:00 ERR ../../../../../usr/local/go/src/runtime/asm_arm64.s:1223 > error starting listeners: cancel by mediation MediationRequested event subscribe error %!w(<nil>) reconnect in 2 seconds
2024-11-27T09:33:37-08:00 ERR ../../../../../usr/local/go/src/runtime/asm_arm64.s:1223 > error starting listeners: cancel by storage DealStateChange event subscribe error %!w(<nil>) reconnect in 2 seconds
2024-11-27T09:33:37-08:00 ERR ../../../../../usr/local/go/src/runtime/asm_arm64.s:1223 > error starting listeners: cancel by token Transfer event subscribe error %!w(<nil>) reconnect in 2 seconds
2024-11-27T09:33:37-08:00 ERR ../../../../../usr/local/go/src/runtime/asm_arm64.s:1223 > error starting listeners: cancel by job JobAdded event subscribe error %!w(<nil>) reconnect in 2 seconds
2024-11-27T09:33:37-08:00 ERR ../../../../../usr/local/go/src/runtime/asm_arm64.s:1223 > error starting listeners: cancel by pow newPowRound event subscribe error %!w(<nil>) reconnect in 2 seconds

Let's investigate these errors and decide what to do about them.