store-gateway: reduce garbage from index header

[bboreham]

What this PR does

This one call to io.ReadAll was generating 6% of all garbage in some production store-gateways recently.

Since gzip puts the size at the end of the compressed data, we can use that size to pre-allocate the buffer and avoid resizing.
Code is adapted from some that used to be inside gRPC, from grpc/grpc-go#3048

It is annoying to add 16 lines of code for this benefit, so I looked at whether we could have a utility function doing it.

• pkg/util/http.go has similar code but also handles LZ4.
• pkg/distributor/otel.go also handles gzip and LZ4, and pools the buffers.

Both of the above, and some other similar places, have a maximum size they will read.

Which issue(s) this PR fixes or relates to

Fixes #

Checklist

• Tests updated.
• Documentation added.
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
• about-versioning.md updated with experimental features.

[dimitarvdimitrov]

this looks good in general. Can't we change util.ParseProtoReader to do this estimation? It currently takes expectedSize I guess we can take -1 and let it try to estimate the size based on the method here. Do you want to try that or should I?

[jhalterman]

I don't have an opinion on having a separate function vs updating the existing one, but if we do have a new function, can it also live in the util package for better reuse?