veneur-proxy: Add option for GRPC streaming to veneur-global

[marvelm]

Context

I've switched the veneur pipeline in our production clusters to using gRPC in order to address a bug. Unfortunately, in large clusters the batches produced by veneur are significantly larger that the default MaxRecvSize (4MiB) in the gRPC library.

Here's the error message:

veneur-pfj5s veneur time=“2022-11-17T18:46:21Z” level=error msg=“Failed to forward to an upstream Veneur” destination=“veneur-proxy:8128” error=“rpc error: code = ResourceExhausted desc = grpc: received message larger than max (4798168 vs. 4194304)” grpcstate=READY metrics=19408 protocol=grpc

This error was easily resolved by enabling gRPC streaming in the veneur DaemonSet (configuration).

After enabling this streaming feature, I discovered that problem had now moved upstream to veneur-proxy. Veneur-proxy was now logging the same error. Unfortunately, gRPC streaming is not configurable within veneur-proxy. So, I've had to add the feature.

We've been running this patch in all of our clusters for the past week.

[Oscil8]

I'd make sure to run tests; which you can do by running docker build .

[marvelm]

Hi @Oscil8. Thanks for taking the time to review this code. I wanted to add a unit test, but it's difficult to test this toggle in isolation. So, I've adapted an existing test and I've looked at the coverage to ensure that the happy path is being hit.

[Oscil8]

LGTM; I'll get a release cut after you merge

[marvelm]

@Oscil8 I can't merge this PR as I don't have write access to this repo 😅