Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release-v2.7] [DOC] Update gRPC compression documentation #4634

Merged
merged 2 commits into from
Jan 28, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/sources/tempo/api_docs/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -900,4 +900,9 @@ service StreamingQuerier {
rpc MetricsQueryRange(QueryRangeRequest) returns (stream QueryRangeResponse) {}
}
```

{{< admonition type="note" >}}
gRPC compression is disabled by default.
Refer to [gRPC compression configuration](https://grafana.com/docs/tempo/<TEMPO_VERSION>/configuration/#grpc-compression) for more information.
{{< /admonition >}}
<!-- vale Grafana.GooglePassive = YES -->
26 changes: 26 additions & 0 deletions docs/sources/tempo/configuration/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ The Tempo configuration options include:
- [Server](#server)
- [Distributor](#distributor)
- [Set max attribute size to help control out of memory errors](#set-max-attribute-size-to-help-control-out-of-memory-errors)
- [gRPC compression](#grpc-compression)
- [Ingester](#ingester)
- [Metrics-generator](#metrics-generator)
- [Query-frontend](#query-frontend)
Expand Down Expand Up @@ -267,6 +268,31 @@ Use the `tempo_distributor_attributes_truncated_total` metric to track how many

For additional information, refer to [Troubleshoot out-of-memory errors](https://grafana.com/docs/tempo/<TEMPO_VERSION>/troubleshooting/out-of-memory-errors/).

### gRPC compression

If you notice increased network traffic or issues, check the gRPC compression settings.

Tempo 2.7 disabled gRPC compression in the querier and distributor for performance reasons. ([#4429](https://github.com/grafana/tempo/pull/4429))
Benchmark testing suggested that without compression, queriers and distributors used less CPU and memory.

However, you may notice an increase in ingester data and network traffic especially for larger clusters.

You can configure the gRPC compression in the `querier`, `ingester`, and `metrics_generator` clients of the distributor.
To re-enable the compression, use `snappy` with the following settings:

```yaml
ingester_client:
grpc_client_config:
grpc_compression: "snappy"
metrics_generator_client:
grpc_client_config:
grpc_compression: "snappy"
querier:
frontend_worker:
grpc_client_config:
grpc_compression: "snappy"
```

## Ingester

For more information on configuration options, refer to [this file](https://github.com/grafana/tempo/blob/main/modules/ingester/config.go).
Expand Down
22 changes: 6 additions & 16 deletions docs/sources/tempo/release-notes/v2-7.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,22 +206,12 @@ max_spans_per_span_set

### gRPC compression disabled

Disable gRPC compression in the querier and distributor for performance reasons. ([#4429](https://github.com/grafana/tempo/pull/4429)) Check the gRPC compression settings if you see network issues.
If you would like to re-enable it, we recommend 'snappy'.
Use the following settings:

```yaml
ingester_client:
grpc_client_config:
grpc_compression: "snappy"
metrics_generator_client:
grpc_client_config:
grpc_compression: "snappy"
querier:
frontend_worker:
grpc_client_config:
grpc_compression: "snappy"
```
This release disables gRPC compression in the querier and distributor for performance reasons. ([#4429](https://github.com/grafana/tempo/pull/4429))
Our benchmark suggests that without compression, queriers and distributors use less CPU and memory.

If you notice increased network traffic or issues, check the gRPC compression settings.

Refer to [gRPC compression configuration](https://grafana.com/docs/tempo/<TEMPO_VERSION>/configuration/#grpc-compression) for more information.

### Other upgrade considerations

Expand Down
22 changes: 7 additions & 15 deletions docs/sources/tempo/setup/upgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,22 +98,14 @@ For Jaeger exporting, set `OTEL_TRACES_EXPORTER=jaeger`.For more information, re

### gRPC compression disabled

Disable gRPC compression in the querier and distributor for performance reasons. ([#4429](https://github.com/grafana/tempo/pull/4429)) Check the gRPC compression settings if you see network issues.
If you would like to re-enable it, we recommend 'snappy'.
Use the following settings:
This release disables gRPC compression in the querier and distributor for performance reasons. ([#4429](https://github.com/grafana/tempo/pull/4429))
Our benchmark suggests that without compression, queriers and distributors use less CPU and memory.

```
ingester_client:
grpc_client_config:
grpc_compression: "snappy"
metrics_generator_client:
grpc_client_config:
grpc_compression: "snappy"
querier:
frontend_worker:
grpc_client_config:
grpc_compression: "snappy"
```
However, you may notice an increase in ingester data and network traffic.

If you notice increased network traffic or issues, check the gRPC compression settings.

Refer to [gRPC compression configuration](https://grafana.com/docs/tempo/<TEMPO_VERSION>/configuration/#grpc-compression) for more information.

### Added, updated, removed, or renamed configuration parameters

Expand Down