decaying connmgr tags for message delivery

[yusefnapora]

This adds decaying connection manager tags that give a boost to peers that deliver messages. closes #306

I was hoping to finish and fix the test over the weekend, but didn't get a chance. It seems to basically work, but some of the honest peers aren't connected at the end of the test.

I'll sort it out in the morning, just wanted to get the branch up in case @vyzo wants to look through before then.

[yusefnapora]

@vyzo the tag tracer is still missing "near first" message deliveries. Can you think of a good way to do this without duplicating the delivery record tracking from the score tracer?

[vyzo]

I think we can get away very simply by starting tracking in Validate and stopping all tracking when the message is Delivered or Rejected. The Duplicate should only fire while still validating.
So basically we can have a lightweight version of what we have in the score tracer.

[yusefnapora]

I squashed a few things and marked this as "ready for review", but we still need to wait until the dependency PRs are merged.

[vyzo]

this looks good, just a comment.

Note: we'll want to go mod tidy this when it's time to merge.

[yusefnapora]

hmm, I added another 5 minutes to the travis timeout, but we're still hitting it. I can't imagine the tests in this branch take an extra 5m to run... I wonder if I could be deadlocking somewhere?

[yusefnapora]

okay, I finally wised up and figured out how to enable the race detector by default when I run tests in IntelliJ. With it enable, we hit the max goroutine limit in TestGossipsubConnTagMessageDeliveries. But reducing the number of peers in the test seems to make it happy.

[vyzo]

we should be getting output in that case.

[yusefnapora]

hmm, that's true. It did also take a few minutes on my laptop before it failed, so it might actually be taking more than 5m on travis

[yusefnapora]

I got the end-to-end test passing by lowering the number of test peers again, but now the unit test is failing, and it's a little confusing.

=== RUN   TestTagTracerDeliveryTags
    TestTagTracerDeliveryTags: tag_tracer_test.go:138: expected delivery tag value = 10, got 11
    TestTagTracerDeliveryTags: tag_tracer_test.go:145: expected delivery tag value = 0, got 1

This is just after adding 50 minutes to the fake clock, and checking that the tag value has decreased.

On my laptop, it gets the expected value, but it always seems to be off by one on Travis. The reason I'm confused is that the result should only depend on the fake clock we pass into the decayer, not the real clock. I tried sleeping the real clock for a second after bumping the fake one, to give the Decayer's processing routine time to run, but it still fails.

I'm going to try increasing the sleep time to five seconds, but I'll probably be asleep myself by the time it's done running. Anyway, Vyzo, perhaps you could take a look if it's still failing.

[vyzo]

Hard to tell what's happening with the tests on travis, code looks good to me.

[vyzo]

at last, green checkmaks!