increase initialWindowSize and initialConnWindowSize by 32x

[glycerine]

Improves throughput by 5x on a 2.7msec latency link.

Partially addresses #1043.

[petermattis]

A larger window size means more memory that can be buffered by gRPC without being acknowledged by the application. In addition to the extra memory usage, you've likely hurt latency for small requests sharing the same connection.

This change increases the stream and connection window sizes by 2048x for only a 5.5x throughput improvement. I think there is a less radical change that can provide most of the benefit. In my testing with CockroachDB, bumping the stream and connection window sizes to 2MB (increase of 32x) was the minimum size that provided close to max throughput on a link with 60ms RTTs. As a stop-gap until the window sizes dynamically adjust, I'd be much more comfortable with a change which left the defaults unchanged but made the stream and window sizes configurable.

[glycerine]

a change which left the defaults unchanged but made the stream and window sizes configurable

I backed the multiplier back to 64x (4MB). I tried 32x but that was still loosing 8% compared to 64x on my 2.7msec latency link.

Options are great, but the defaults should be sane too. I would think. Because most users won't ever dig deep enough to discover the option.

you've likely hurt latency for small requests sharing the same connection

This comment I don't follow the reasoning, but I would like to. How would latency for small requests sharing the same connection be hurt by a bigger window; I don't see how their frames would be delayed since they are likely to be sent and reach the receiver sooner.

[petermattis]

How would latency for small requests sharing the same connection be hurt by a bigger window; I don't see how their frames would be delayed since they are likely to be sent and reach the receiver sooner.

Ah, ignore me. I misremembered how framing when writing data to a connection works. See http2Client.Write. Data is interleaved in 16 KB chunks regardless of the stream/connection window sizes.

[glycerine]

a change which left the defaults unchanged but made the stream and window sizes configurable

I backed the multiplier back to 64x (4MB). I tried 32x but that was still loosing 8% compared to 64x on my 2.7msec latency link.

Options are great, but the defaults should be sane too. I would think. Because most users won't ever dig deep enough to discover the option.

you've likely hurt latency for small requests sharing the same connection

This comment I don't follow the reasoning, but I would like to. How would latency for small requests sharing the same connection be hurt by a bigger window; I don't see how their frames would be delayed since they are likely to be sent and reach the receiver sooner.

[glycerine]

Travis tests in transport timeout at 64x, even though they pass locally on my laptop. Trying at 32x.

[glycerine]

Yes, this (32x) looks good. It gives a 2MB initialWindowSize, and 32MB initialConnWindowSize. These seem to be pretty sane defaults, at least for my 2.7msec end-to-end wan setup.

[MakMukhi]

Implemented by #1210

[glycerine]

This was not implemented by #1210, and so should not have been closed.

#1210 allows the defaults to be changed, but this PR actually improves the defaults. Hence they will less often need to be changed.

I just re-tested this against tip, and on a high latency 109msec link, between New Jersey and Australia, these defaults double the throughput on tip (e.g. for #1043).