This release contains 531 PRs from 60 authors, including new contributors Benoit Schipper, Derek Cadzow, Edwin, Itay Kalfon, Ivan Farré Vicente, Jan O. Rundshagen, Jorge Turrado Ferrero, Lukas Monkevicius, Mickaël Canévet, Rafael Sathler, Rajakavitha Kodhandapani, Tim Kotowski, Vladimir Varankin, Zach, Zach Day, Zirko, blut, github-actions[bot], ncharaf, zhehao-grafana. Thank you!

Grafana Mimir version 2.12.0 release notes

Grafana Labs is excited to announce version 2.12 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bug fixes in this release.
For the complete list of changes, refer to the CHANGELOG.

Features and enhancements

Added support to only count series that are considered active through the Cardinality API endpoint /api/v1/cardinality/label_names by passing the count_method parameter.
If set to active it counts only series that are considered active according to the -ingester.active-series-metrics-idle-timeout flag setting rather than counting all in-memory series.
The "Store-gateway: bucket tenant blocks" admin page contains a new column "No Compact".
If block no compaction marker is set, it specifies the reason and the date the marker is added.
The estimated number of compaction jobs based on the current bucket-index is now computed by the compactor.
The result is tracked by the new cortex_bucket_index_compaction_jobs metric.
If this computation fails, the cortex_bucket_index_compaction_jobs_errors_total metric is updated instead.
The estimated number of compaction jobs is also shown in Top tenants, Tenants, and Compactor dashboards.
Added mimir-distroless container image built upon a distroless image (gcr.io/distroless/static-debian12).
This improvement minimizes attack surfaces and potential CVEs by trimming down the dependencies within the image.
After comprehensive testing, the Mimir maintainers plan to shift from the current image to the distroless version.

Additionally, the following previously experimental features are now considered stable:

The number of pre-allocated workers used to forward push requests to the ingesters, configurable via the -distributor.reusable-ingester-push-workers CLI flag on distributors.
It now defaults to 2000.
Note that this is a performance optimization, and not a limiting feature.
If not enough workers available, new goroutines will be spawned.
The number of gRPC server workers used to serve the requests, configurable via the -server.grpc.num-workers CLI flag.
It now defaults to 100.
Note that this is the number of pre-allocated long-lived workers, and not a limiting feature.
If not enough workers are available, new goroutines will be spawned.
The maximum number of concurrent index header loads across all tenants, configurable via the -blocks-storage.bucket-store.index-header.lazy-loading-concurrency CLI flag on store-gateways.
It defaults to 4.
The maximum time to wait for the query-frontend to become ready before rejecting requests, configurable via the -query-frontend.not-running-timeout CLI flag on query-frontends.
It now defaults to 2s.
The CLI flag that allows queriers to reduce pressure on ingesters by initially querying only the minimum set of ingesters required to reach quorum, -querier.minimize-ingester-requests.
It is now enabled by default.
Spread-minimizing token-related CLI flags: -ingester.ring.token-generation-strategy, -ingester.ring.spread-minimizing-zones and -ingester.ring.spread-minimizing-join-ring-in-order.
You can read more about this feature in our blog post.

Important changes

In Grafana Mimir 2.12 the following behavior has changed:

Store-gateway now persists a sparse version of the index-header to disk on construction and loads sparse index-headers from disk instead of the whole index-header.
This improves the speed at which index headers are lazy-loaded from disk by up to 90%. The added disk usage is in the order of 1-2%.
Alertmanager deprecated the v1 API. All v1 API endpoints now respond with a JSON deprecation notice and a status code of 410.
All endpoints have a v2 equivalent.
The list of endpoints is:
- <alertmanager-web.external-url>/api/v1/alerts
- <alertmanager-web.external-url>/api/v1/receivers
- <alertmanager-web.external-url>/api/v1/silence/{id}
- <alertmanager-web.external-url>/api/v1/silences
- <alertmanager-web.external-url>/api/v1/status
Exemplar's label traceID has been changed to trace_id to be consistent with the OpenTelemetry standard.
Errors returned by ingesters now contain only gRPC status codes.
Previously they contained both gRPC and HTTP status codes.

{{< admonition type="warning" >}}
To guarantee backwards compatibility when migrating from a version prior to 2.11, it's necessary to first migrate to version 2.11, and then to version 2.12.
Otherwise, it might happen that during the migration, some ingester errors with HTTP status code 4xx won't be recognized, and the corresponding request will be repeated.
{{< /admonition >}}
Responses with gRPC status codes are now reported as status_code labels in the cortex_request_duration_seconds and cortex_ingester_client_request_duration_seconds metrics.
Responses with HTTP 4xx status codes are now treated as errors and used in status_code label of request duration metric.

The default value of the following CLI flags have been changed:

-blocks-storage.tsdb.head-postings-for-matchers-cache-max-bytes from 10MB to 100MB.
-blocks-storage.tsdb.block-postings-for-matchers-cache-max-bytes from 10MB to 100MB.
-blocks-storage.bucket-store.tenant-sync-concurrency from 10 to 1.
-query-frontend.max-cache-freshness from 1m to 10m.
-distributor.write-requests-buffer-pooling-enabled from false to true.
-locks-storage.bucket-store.block-sync-concurrency from 20 to 4.
-memberlist.stream-timeout from 10s to 2s.
-server.report-grpc-codes-in-instrumentation-label-enabled from false to true.

The following deprecated configuration options are removed in Grafana Mimir 2.12:

The YAML setting frontend.cache_unaligned_requests.
Experimental CLI flag -querier.prefer-streaming-chunks-from-ingesters.

The following configuration options are deprecated and will be removed in Grafana Mimir 2.14:

The CLI flag -ingester.limit-inflight-requests-using-grpc-method-limiter.
It now defaults to true.
The CLI flag -ingester.return-only-grpc-errors.
It now defaults to true.

{{< admonition type="warning" >}}
To guarantee backwards compatibility when migrating from a version prior to 2.11, it's necessary to first migrate to version 2.11, and then to version 2.12.
Otherwise, it might happen that during the migration, some ingester errors with HTTP status code 4xx won't be recognized, and the corresponding request will be repeated.
{{< /admonition >}}
The CLI flag -ingester.client.report-grpc-codes-in-instrumentation-label-enabled.
It now defaults to true.
The CLI flag -distributor.limit-inflight-requests-using-grpc-method-limiter.
It now defaults to true.
The CLI flag -distributor.enable-otlp-metadata-storage.
It now defaults to true.
The CLI flag -querier.max-query-into-future.

The following metrics are removed or deprecated:

cortex_bucket_store_blocks_loaded_by_duration has been removed.
cortex_distributor_sample_delay_seconds has been deprecated and will be removed in Mimir 2.14.

Experimental features

Grafana Mimir 2.12 includes new features that are considered experimental and disabled by default.
Use them with caution and report any issues you encounter:

The maximum number of tenant IDs that may be for a federated query can be configured via the -tenant-federation.max-tenants CLI flag on query-frontends.
By default, it's 0, meaning that the limit is disabled.
Sharding of active series queries can be enabled via the -query-frontend.shard-active-series-queries CLI flag on query-frontends.
Timely head compaction can be enabled via the -blocks-storage.tsdb.timely-head-compaction-enabled on ingesters.
If enabled, the head compaction happens when the min block range can no longer be appended, without requiring 1.5x the chunk range worth of data in the head.
Streaming of responses from querier to query-frontend can be enabled via the -querier.response-streaming-enabled CLI flag on queriers.
This is currently supported only for responses from the /api/v1/cardinality/active_series endpoint.
The maximum response size for active series queries, in bytes, can be set via the -querier.active-series-results-max-size-bytes CLI flag on queriers.
Metric relabeling on a per-tenant basis can be forcefully disabled via the -distributor.metric-relabeling-enabled CLI flag on rulers.
Metrics relabeling is enabled by default.
Query Queue Load Balancing by Query Component. Tenant query queues in the query-scheduler can now be split into subqueues by which query component is expected to be utilized to complete the query: ingesters, store-gateways, both, or uncategorized.
Dequeuing queries for a given tenant will rotate through the query component subqueues via simple round-robin.
In the event that the one of the query components (ingesters or store-gateways) experience a slowdown, queries only utilizing the other query component can continue to be serviced.
This feature is recommended to be enabled.
The following CLI flags must be set to true in order to be in effect:
- -query-frontend.additional-query-queue-dimensions-enabled on the query-frontend.
- -query-scheduler.additional-query-queue-dimensions-enabled on the query-scheduler.
Owned series tracking in ingesters can be enabled via the -ingester.track-ingester-owned-series CLI flag.
When enabled, ingesters will track the number of in-memory series that still map to the ingester based on the ring state.
These counts are more reactive to ring and shard changes than in-memory series, and can be used when enforcing tenant series limits by enabling the -ingester.use-ingester-owned-series-for-limits CLI flag.
This feature requires zone-aware replication to be enabled, and the replication factor to be equal to the number of zones.

Bug fixes

Distributor: fixed an issue where -distributor.metric-relabeling-enabled could cause distributors to panic.
Distributor: fix an issue where -distributor.metric-relabeling-enabled could cause distributors to write unsorted labels and corrupt blocks.
Ingester: errors encountered while iterating through chunks or samples in response to a query request aren't ignored anymore.
Compactor: out-of-order blocks aren't allowed to prevent timely compaction anymore.
Querier: requests to store-gateway when a query gets canceled aren't retried anymore.
Querier: status code 499 is now returned instead of 500 when a request to remote read endpoint gets canceled.
Querier: fixed an issue where -querier.max-fetched-series-per-query wasn't applied to /series endpoint in case series loaded from ingesters.
Querier: fixed an issue with the remote-read requests HTTP status code translations.
Previously, remote-read had conflicting behaviours: when returning samples all internal errors were translated to HTTP 400, while when returning chunks all internal errors were translated to HTTP 500.
With this fix, all validation errors will be translated into HTTP 400 errors, while all other errors will be translated into HTTP 500 errors.
Query-frontend: the cortex_query_frontend_queries_total metric incorrectly reported op="query" for any request which wasn't a range query.
Now the op label value can be one of the following:
- query: instant query
- query_range: range query
- cardinality: cardinality query
- label_names_and_values: label names / values query
- active_series: active series query
- other: any other request
Ruler: fixed an issue where "failed to remotely evaluate query expression, will retry" messages were logged without context such as the trace ID and didn't appear in trace events.
Ruler: requests to remote querier when server's response exceeds its configured max payload size aren't retried anymore.
Ruler: fixed a regression that caused client errors to be tracked in cortex_ruler_write_requests_failed_total metric.
Ruler: fixed an issue with recording rule result being corruption due to an usage of a bad native histogram pointer.

Helm chart improvements

The Grafana Mimir and Grafana Enterprise Metrics Helm charts are released independently.
Refer to the Grafana Mimir Helm chart documentation.

Changelog

2.12.0

Grafana Mimir

[CHANGE] Alertmanager: Deprecates the v1 API. All v1 API endpoints now respond with a JSON deprecation notice and a status code of 410. All endpoints have a v2 equivalent. The list of endpoints is: #7103
- <alertmanager-web.external-url>/api/v1/alerts
- <alertmanager-web.external-url>/api/v1/receivers
- <alertmanager-web.external-url>/api/v1/silence/{id}
- <alertmanager-web.external-url>/api/v1/silences
- <alertmanager-web.external-url>/api/v1/status
[CHANGE] Ingester: Increase default value of -blocks-storage.tsdb.head-postings-for-matchers-cache-max-bytes and -blocks-storage.tsdb.block-postings-for-matchers-cache-max-bytes to 100 MiB (previous default value was 10 MiB). #6764
[CHANGE] Validate tenant IDs according to documented behavior even when tenant federation is not enabled. Note that this will cause some previously accepted tenant IDs to be rejected such as those longer than 150 bytes or containing | characters. #6959
[CHANGE] Ruler: don't use backoff retry on remote evaluation in case of 4xx errors. #7004
[CHANGE] Server: responses with HTTP 4xx status codes are now treated as errors and used in status_code label of request duration metric. #7045
[CHANGE] Memberlist: change default for -memberlist.stream-timeout from 10s to 2s. #7076
[CHANGE] Memcached: remove legacy thanos_cache_memcached_* and thanos_memcached_* prefixed metrics. Instead, Memcached and Redis cache clients now emit thanos_cache_* prefixed metrics with a backend label. #7076
[CHANGE] Ruler: the following metrics, exposed when the ruler is configured to discover Alertmanager instances via service discovery, have been renamed: #7057
- prometheus_sd_failed_configs renamed to cortex_prometheus_sd_failed_configs
- prometheus_sd_discovered_targets renamed to cortex_prometheus_sd_discovered_targets
- prometheus_sd_received_updates_total renamed to cortex_prometheus_sd_received_updates_total
- prometheus_sd_updates_delayed_total renamed to cortex_prometheus_sd_updates_delayed_total
- prometheus_sd_updates_total renamed to cortex_prometheus_sd_updates_total
- prometheus_sd_refresh_failures_total renamed to cortex_prometheus_sd_refresh_failures_total
- prometheus_sd_refresh_duration_seconds renamed to cortex_prometheus_sd_refresh_duration_seconds
[CHANGE] Query-frontend: the default value for -query-frontend.not-running-timeout has been changed from 0 (disabled) to 2s. The configuration option has also been moved from "experimental" to "advanced". #7127
[CHANGE] Store-gateway: to reduce disk contention on HDDs the default value for blocks-storage.bucket-store.tenant-sync-concurrency has been changed from 10 to 1 and the default value for blocks-storage.bucket-store.block-sync-concurrency has been changed from 20 to 4. #7136
[CHANGE] Store-gateway: Remove deprecated CLI flags -blocks-storage.bucket-store.index-header-lazy-loading-enabled and -blocks-storage.bucket-store.index-header-lazy-loading-idle-timeout and their corresponding YAML settings. Instead, use -blocks-storage.bucket-store.index-header.lazy-loading-enabled and -blocks-storage.bucket-store.index-header.lazy-loading-idle-timeout. #7521
[CHANGE] Store-gateway: Mark experimental CLI flag -blocks-storage.bucket-store.index-header.lazy-loading-concurrency and its corresponding YAML settings as advanced. #7521
[CHANGE] Store-gateway: Remove experimental CLI flag -blocks-storage.bucket-store.index-header.sparse-persistence-enabled since this is now the default behavior. #7535
[CHANGE] All: set -server.report-grpc-codes-in-instrumentation-label-enabled to true by default, which enables reporting gRPC status codes as status_code labels in the cortex_request_duration_seconds metric. #7144
[CHANGE] Distributor: report gRPC status codes as status_code labels in the cortex_ingester_client_request_duration_seconds metric by default. #7144
[CHANGE] Distributor: CLI flag -ingester.client.report-grpc-codes-in-instrumentation-label-enabled has been deprecated, and its default value is set to true. #7144
[CHANGE] Ingester: CLI flag -ingester.return-only-grpc-errors has been deprecated, and its default value is set to true. To ensure backwards compatibility, during a migration from a version prior to 2.11.0 to 2.12 or later, -ingester.return-only-grpc-errors should be set to false. Once all the components are migrated, the flag can be removed. #7151
[CHANGE] Ingester: the following CLI flags have been moved from "experimental" to "advanced": #7169
- -ingester.ring.token-generation-strategy
- -ingester.ring.spread-minimizing-zones
- -ingester.ring.spread-minimizing-join-ring-in-order
[CHANGE] Query-frontend: the default value of the CLI flag -query-frontend.max-cache-freshness (and its respective YAML configuration parameter) has been changed from 1m to 10m. #7161
[CHANGE] Distributor: default the optimization -distributor.write-requests-buffer-pooling-enabled to true. #7165
[CHANGE] Tracing: Move query information to span attributes instead of span logs. #7046
[CHANGE] Distributor: the default value of circuit breaker's CLI flag -ingester.client.circuit-breaker.cooldown-period has been changed from 1m to 10s. #7310
[CHANGE] Store-gateway: remove cortex_bucket_store_blocks_loaded_by_duration. cortex_bucket_store_series_blocks_queried is better suited for detecting when compactors are not able to keep up with the number of blocks to compact. #7309
[CHANGE] Ingester, Distributor: the support for rejecting push requests received via gRPC before reading them into memory, enabled via -ingester.limit-inflight-requests-using-grpc-method-limiter and -distributor.limit-inflight-requests-using-grpc-method-limiter, is now stable and enabled by default. The configuration options have been deprecated and will be removed in Mimir 2.14. #7360
[CHANGE] Distributor: Change-distributor.enable-otlp-metadata-storage flag's default to true, and deprecate it. The flag will be removed in Mimir 2.14. #7366
[CHANGE] Store-gateway: Use a shorter TTL for cached items related to temporary blocks. #7407 #7534
[CHANGE] Standardise exemplar label as "trace_id". #7475
[CHANGE] The configuration option -querier.max-query-into-future has been deprecated and will be removed in Mimir 2.14. #7496
[CHANGE] Distributor: the metric cortex_distributor_sample_delay_seconds has been deprecated and will be removed in Mimir 2.14. #7516
[CHANGE] Query-frontend: The deprecated YAML setting frontend.cache_unaligned_requests has been moved to limits.cache_unaligned_requests. #7519
[CHANGE] Querier: the CLI flag -querier.minimize-ingester-requests has been moved from "experimental" to "advanced". #7638
[FEATURE] Introduce -server.log-source-ips-full option to log all IPs from Forwarded, X-Real-IP, X-Forwarded-For headers. #7250
[FEATURE] Introduce -tenant-federation.max-tenants option to limit the max number of tenants allowed for requests when federation is enabled. #6959
[FEATURE] Cardinality API: added a new count_method parameter which enables counting active label names. #7085
[FEATURE] Querier / query-frontend: added -querier.promql-experimental-functions-enabled CLI flag (and respective YAML config option) to enable experimental PromQL functions. The experimental functions introduced are: mad_over_time(), sort_by_label() and sort_by_label_desc(). #7057
[FEATURE] Alertmanager API: added -alertmanager.grafana-alertmanager-compatibility-enabled CLI flag (and respective YAML config option) to enable an experimental API endpoints that support the migration of the Grafana Alertmanager. #7057
[FEATURE] Alertmanager: Added -alertmanager.utf8-strict-mode-enabled to control support for any UTF-8 character as part of Alertmanager configuration/API matchers and labels. It's default value is set to false. #6898
[FEATURE] Querier: added histogram_avg() function support to PromQL. #7293
[FEATURE] Ingester: added -blocks-storage.tsdb.timely-head-compaction flag, which enables more timely head compaction, and defaults to false. #7372
[FEATURE] Compactor: Added /compactor/tenants and /compactor/tenant/{tenant}/planned_jobs endpoints that provide functionality that was provided by tools/compaction-planner -- listing of planned compaction jobs based on tenants' bucket index. #7381
[FEATURE] Add experimental support for streaming response bodies from queriers to frontends via -querier.response-streaming-enabled. This is currently only supported for the /api/v1/cardinality/active_series endpoint. #7173
[FEATURE] Release: Added mimir distroless docker image. #7371
[FEATURE] Add support for the new grammar of {"metric_name", "l1"="val"} to promql and some of the exposition formats. #7475 #7541
[ENHANCEMENT] Distributor: Add a new metric cortex_distributor_otlp_requests_total to track the total number of OTLP requests. #7385
[ENHANCEMENT] Vault: add lifecycle manager for token used to authenticate to Vault. This ensures the client token is always valid. Includes a gauge (cortex_vault_token_lease_renewal_active) to check whether token renewal is active, and the counters cortex_vault_token_lease_renewal_success_total and cortex_vault_auth_success_total to see the total number of successful lease renewals / authentications. #7337
[ENHANCEMENT] Store-gateway: add no-compact details column on store-gateway tenants admin UI. #6848
[ENHANCEMENT] PromQL: ignore small errors for bucketQuantile #6766
[ENHANCEMENT] Distributor: improve efficiency of some errors #6785
[ENHANCEMENT] Ruler: exclude vector queries from being tracked in cortex_ruler_queries_zero_fetched_series_total. #6544
[ENHANCEMENT] Ruler: local storage backend now supports reading a rule group via /config/api/v1/rules/{namespace}/{groupName} configuration API endpoint. #6632
[ENHANCEMENT] Query-Frontend and Query-Scheduler: split tenant query request queues by query component with query-frontend.additional-query-queue-dimensions-enabled and query-scheduler.additional-query-queue-dimensions-enabled. #6772
[ENHANCEMENT] Distributor: support disabling metric relabel rules per-tenant via the flag -distributor.metric-relabeling-enabled or associated YAML. #6970
[ENHANCEMENT] Distributor: -distributor.remote-timeout is now accounted from the first ingester push request being sent. #6972
[ENHANCEMENT] Storage Provider: -<prefix>.s3.sts-endpoint sets a custom endpoint for AWS Security Token Service (AWS STS) in s3 storage provider. #6172
[ENHANCEMENT] Querier: add cortex_querier_queries_storage_type_total metric that indicates how many queries have executed for a source, ingesters or store-gateways. Add cortex_querier_query_storegateway_chunks_total metric to count the number of chunks fetched from a store gateway. #7099,#7145
[ENHANCEMENT] Query-frontend: add experimental support for sharding active series queries via -query-frontend.shard-active-series-queries. #6784
[ENHANCEMENT] Distributor: set -distributor.reusable-ingester-push-workers=2000 by default and mark feature as advanced. #7128
[ENHANCEMENT] All: set -server.grpc.num-workers=100 by default and mark feature as advanced. #7131
[ENHANCEMENT] Distributor: invalid metric name error message gets cleaned up to not include non-ascii strings. #7146
[ENHANCEMENT] Store-gateway: add source, level, and out_or_order to cortex_bucket_store_series_blocks_queried metric that indicates the number of blocks that were queried from store gateways by block metadata. #7112 #7262 #7267
[ENHANCEMENT] Compactor: After updating bucket-index, compactor now also computes estimated number of compaction jobs based on current bucket-index, and reports the result in cortex_bucket_index_estimated_compaction_jobs metric. If computation of jobs fails, cortex_bucket_index_estimated_compaction_jobs_errors_total is updated instead. #7299
[ENHANCEMENT] Mimir: Integrate profiling into tracing instrumentation. #7363
[ENHANCEMENT] Alertmanager: Adds metric cortex_alertmanager_notifications_suppressed_total that counts the total number of notifications suppressed for being silenced, inhibited, outside of active time intervals or within muted time intervals. #7384
[ENHANCEMENT] Query-scheduler: added more buckets to cortex_query_scheduler_queue_duration_seconds histogram metric, in order to better track queries staying in the queue for longer than 10s. #7470
[ENHANCEMENT] A type label is added to prometheus_tsdb_head_out_of_order_samples_appended_total metric. #7475
[ENHANCEMENT] Distributor: Optimize OTLP endpoint. #7475
[ENHANCEMENT] API: Use github.com/klauspost/compress for faster gzip and deflate compression of API responses. #7475
[ENHANCEMENT] Ingester: Limiting on owned series (-ingester.use-ingester-owned-series-for-limits) now prevents discards in cases where a tenant is sharded across all ingesters (or shuffle sharding is disabled) and the ingester count increases. #7411
[ENHANCEMENT] Block upload: include converted timestamps in the error message if block is from the future. #7538
[ENHANCEMENT] Query-frontend: Introduce -query-frontend.active-series-write-timeout to allow configuring the server-side write timeout for active series requests. #7553 #7569
[BUGFIX] Ingester: don't ignore errors encountered while iterating through chunks or samples in response to a query request. #6451
[BUGFIX] Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks #6766
[BUGFIX] Fix issue where concatenatingChunkIterator can obscure errors #6766
[BUGFIX] Fix panic during tsdb Commit #6766
[BUGFIX] tsdb/head: wlog exemplars after samples #6766
[BUGFIX] Ruler: fix issue where "failed to remotely evaluate query expression, will retry" messages are logged without context such as the trace ID and do not appear in trace events. #6789
[BUGFIX] Ruler: do not retry requests to remote querier when server's response exceeds its configured max payload size. #7216
[BUGFIX] Querier: fix issue where spans in query request traces were not nested correctly. #6893
[BUGFIX] Fix issue where all incoming HTTP requests have duplicate trace spans. #6920
[BUGFIX] Querier: do not retry requests to store-gateway when a query gets canceled. #6934
[BUGFIX] Querier: return 499 status code instead of 500 when a request to remote read endpoint gets canceled. #6934
[BUGFIX] Querier: fix issue where -querier.max-fetched-series-per-query is not applied to /series endpoint if the series are loaded from ingesters. #7055
[BUGFIX] Distributor: fix issue where -distributor.metric-relabeling-enabled may cause distributors to panic #7176
[BUGFIX] Distributor: fix issue where -distributor.metric-relabeling-enabled may cause distributors to write unsorted labels and corrupt blocks #7326
[BUGFIX] Query-frontend: the cortex_query_frontend_queries_total report incorrectly reported op="query" for any request which wasn't a range query. Now the op label value can be one of the following: #7207
- query: instant query
- query_range: range query
- cardinality: cardinality query
- label_names_and_values: label names / values query
- active_series: active series query
- other: any other request
[BUGFIX] Fix performance regression introduced in Mimir 2.11.0 when uploading blocks to AWS S3. #7240
[BUGFIX] Query-frontend: fix race condition when sharding active series is enabled (see above) and response is compressed with snappy. #7290
[BUGFIX] Query-frontend: "query stats" log unsuccessful replies from downstream as "failed". #7296
[BUGFIX] Packaging: remove reload from systemd file as mimir does not take into account SIGHUP. #7345
[BUGFIX] Compactor: do not allow out-of-order blocks to prevent timely compaction. #7342
[BUGFIX] Update google.golang.org/grpc to resolve occasional issues with gRPC server closing its side of connection before it was drained by the client. #7380
[BUGFIX] Query-frontend: abort response streaming for active_series requests when the request context is canceled. #7378
[BUGFIX] Compactor: improve compaction of sporadic blocks. #7329
[BUGFIX] Ruler: fix regression that caused client errors to be tracked in cortex_ruler_write_requests_failed_total metric. #7472
[BUGFIX] promql: Fix Range selectors with an @ modifier are wrongly scoped in range queries. #7475
[BUGFIX] Fix metadata API using wrong JSON field names. #7475
[BUGFIX] Ruler: fix native histogram recording rule result corruption. #7552
[BUGFIX] Querier: fix HTTP status code translations for remote read requests. Previously, remote-read had conflicting behaviours: when returning samples all internal errors were translated to HTTP 400; when returning chunks all internal errors were translated to HTTP 500. #7487
[BUGFIX] Query-frontend: Fix memory leak on every request. #7654

Mixin

[CHANGE] The job label matcher for distributor and gateway have been extended to include any deployment matching distributor.* and cortex-gw.* respectively. This change allows to match custom and multi-zone distributor and gateway deployments too. #6817
[ENHANCEMENT] Dashboards: Add panels for alertmanager activity of a tenant #6826
[ENHANCEMENT] Dashboards: Add graphs to "Slow Queries" dashboard. #6880
[ENHANCEMENT] Dashboards: Update all deprecated "graph" panels to "timeseries" panels. #6864 #7413 #7457
[ENHANCEMENT] Dashboards: Make most columns in "Slow Queries" sortable. #7000
[ENHANCEMENT] Dashboards: Render graph panels at full resolution as opposed to at half resolution. #7027
[ENHANCEMENT] Dashboards: show query-scheduler queue length on "Reads" and "Remote Ruler Reads" dashboards. #7088
[ENHANCEMENT] Dashboards: Add estimated number of compaction jobs to "Compactor", "Tenants" and "Top tenants" dashboards. #7449 #7481
[ENHANCEMENT] Recording rules: add native histogram recording rules to cortex_request_duration_seconds. #7528
[ENHANCEMENT] Dashboards: Add total owned series, and per-ingester in-memory and owned series to "Tenants" dashboard. #7511
[BUGFIX] Dashboards: drop step parameter from targets as it is not supported. #7157
[BUGFIX] Recording rules: drop rules for metrics removed in 2.0: cortex_memcache_request_duration_seconds and cortex_cache_request_duration_seconds. #7514

Jsonnet

[CHANGE] Distributor: Increase JAEGER_REPORTER_MAX_QUEUE_SIZE from the default (100) to 1000, to avoid dropping tracing spans. #7259
[CHANGE] Querier: Increase JAEGER_REPORTER_MAX_QUEUE_SIZE from 1000 to 5000, to avoid dropping tracing spans. #6764
[CHANGE] rollout-operator: remove default CPU limit. #7066
[CHANGE] Store-gateway: Increase JAEGER_REPORTER_MAX_QUEUE_SIZE from the default (100) to 1000, to avoid dropping tracing spans. #7068
[CHANGE] Query-frontend, ingester, ruler, backend and write instances: Increase JAEGER_REPORTER_MAX_QUEUE_SIZE from the default (100), to avoid dropping tracing spans. #7086
[CHANGE] Ring: relaxed the hash ring heartbeat period and timeout for distributor, ingester, store-gateway and compactor: #6860
- -distributor.ring.heartbeat-period set to 1m
- -distributor.ring.heartbeat-timeout set to 4m
- -ingester.ring.heartbeat-period set to 2m
- -store-gateway.sharding-ring.heartbeat-period set to 1m
- -store-gateway.sharding-ring.heartbeat-timeout set to 4m
- -compactor.ring.heartbeat-period set to 1m
- -compactor.ring.heartbeat-timeout set to 4m
[CHANGE] Ruler-querier: the topology spread constrain max skew is now configured through the configuration option ruler_querier_topology_spread_max_skew instead of querier_topology_spread_max_skew. #7204
[CHANGE] Distributor: -server.grpc.keepalive.max-connection-age lowered from 2m to 60s and configured -shutdown-delay=90s and termination grace period to 100 seconds in order to reduce the chances of failed gRPC write requests when distributors gracefully shutdown. #7361
[FEATURE] Added support for the following root-level settings to configure the list of matchers to apply to node affinity: #6782 #6829
- alertmanager_node_affinity_matchers
- compactor_node_affinity_matchers
- continuous_test_node_affinity_matchers
- distributor_node_affinity_matchers
- ingester_node_affinity_matchers
- ingester_zone_a_node_affinity_matchers
- ingester_zone_b_node_affinity_matchers
- ingester_zone_c_node_affinity_matchers
- mimir_backend_node_affinity_matchers
- mimir_backend_zone_a_node_affinity_matchers
- mimir_backend_zone_b_node_affinity_matchers
- mimir_backend_zone_c_node_affinity_matchers
- mimir_read_node_affinity_matchers
- mimir_write_node_affinity_matchers
- mimir_write_zone_a_node_affinity_matchers
- mimir_write_zone_b_node_affinity_matchers
- mimir_write_zone_c_node_affinity_matchers
- overrides_exporter_node_affinity_matchers
- querier_node_affinity_matchers
- query_frontend_node_affinity_matchers
- query_scheduler_node_affinity_matchers
- rollout_operator_node_affinity_matchers
- ruler_node_affinity_matchers
- ruler_node_affinity_matchers
- ruler_querier_node_affinity_matchers
- ruler_query_frontend_node_affinity_matchers
- ruler_query_scheduler_node_affinity_matchers
- store_gateway_node_affinity_matchers
- store_gateway_node_affinity_matchers
- store_gateway_zone_a_node_affinity_matchers
- store_gateway_zone_b_node_affinity_matchers
- store_gateway_zone_c_node_affinity_matchers
[FEATURE] Ingester: Allow automated zone-by-zone downscaling, that can be enabled via the ingester_automated_downscale_enabled flag. It is disabled by default. #6850
[ENHANCEMENT] Alerts: Add MimirStoreGatewayTooManyFailedOperations warning alert that triggers when Mimir store-gateway report error when interacting with the object storage. #6831
[ENHANCEMENT] Querier HPA: improved scaling metric and scaling policies, in order to scale up and down more gradually. #6971
[ENHANCEMENT] Rollout-operator: upgraded to v0.13.0. #7469
[ENHANCEMENT] Rollout-operator: add tracing configuration to rollout-operator container (when tracing is enabled and configured). #7469
[ENHANCEMENT] Query-frontend: configured -shutdown-delay, -server.grpc.keepalive.max-connection-age and termination grace period to reduce the likelihood of queries hitting terminated query-frontends. #7129
[ENHANCEMENT] Autoscaling: add support for KEDA's ignoreNullValues option for Prometheus scaler. #7471
[BUGFIX] Update memcached-exporter to 0.14.1 due to CVE-2023-39325. #6861

Mimirtool

[FEATURE] Add command migrate-utf8 to migrate Alertmanager configurations for Alertmanager versions 0.27.0 and later. #7383
[ENHANCEMENT] Add template render command to render locally a template. #7325
[ENHANCEMENT] Add --extra-headers option to mimirtool rules command to add extra headers to requests for auth. #7141
[ENHANCEMENT] Analyze Prometheus: set tenant header. #6737
[ENHANCEMENT] Add argument --output-dir to mimirtool alertmanager get where the config and templates will be written to and can be loaded via mimirtool alertmanager load #6760
[BUGFIX] Analyze rule-file: .metricsUsed field wasn't populated. #6953

Mimir Continuous Test

[ENHANCEMENT] Include comparison of all expected and actual values when any float sample does not match. #6756

Query-tee

[BUGFIX] Fix issue where Host HTTP header was not being correctly changed for the proxy targets. #7386
[ENHANCEMENT] Allow using the value of X-Scope-OrgID for basic auth username in the forwarded request if URL username is set as __REQUEST_HEADER_X_SCOPE_ORGID__. #7452

Documentation

[CHANGE] No longer mark OTLP distributor endpoint as experimental. #7348
[ENHANCEMENT] Added runbook for KubePersistentVolumeFillingUp alert. #7297
[ENHANCEMENT] Add Grafana Cloud recommendations to OTLP documentation. #7375
[BUGFIX] Fixed typo on single zone->zone aware replication Helm page. #7327

Tools

[CHANGE] copyblocks: The flags for copyblocks have been changed to align more closely with other tools. #6607
[CHANGE] undelete-blocks: undelete-blocks-gcs has been removed and replaced with undelete-blocks, which supports recovering deleted blocks in versioned buckets from ABS, GCS, and S3-compatible object storage. #6607
[FEATURE] copyprefix: Add tool to copy objects between prefixes. Supports ABS, GCS, and S3-compatible object storage. #6607

All changes in this release: mimir-2.11.0...mimir-2.12.0

What's Changed

Update grafana/grafana Docker tag to v10.2.2 (main) by @renovate in #6744
Update Helm release grafana-agent-operator to v0.3.11 (main) by @renovate in #6743
fix(deps): update module github.com/aws/aws-sdk-go to v1.48.4 (main) by @renovate in #6745
fix(deps): update module github.com/failsafe-go/failsafe-go to v0.4.1 (main) by @renovate in #6746
compactor: mark corrupted blocks for no compaction to avoid blocking by @ortuman in #6588
Docs: Fix flag syntax by @osg-grafana in #6750
docs: bump required agent version by @krajorama in #6752
Add option distributor.ingestion-burst-factor by @treid314 in #6662
Implement series limit using ingester own series by @pstibrany in #6718
[2.10] Backport native histograms documentation by @krajorama in #6757
docs: missed newline by @krajorama in #6759
Merge release-2.10 to main by @krajorama in #6758
Improve series deduplication for /active_series endpoint by @flxbk in #6717
Make align-queries-with-step a per-tenant setting by @56quarters in #6714
Rename "ring" to "ingester ring" by @pstibrany in #6762
jsonnet: Remove explicit config for align-queries-with-step by @56quarters in #6765
Add debug message to track tenants sending queries that are not able to benefit from caches. by @wilfriedroset in #6732
store-gateway: fix duration metrics by @dimitarvdimitrov in #6616
Enable StatefulSetAutoDeletePVC feature for alertmanager, compactor, ingester & store-gateway StatefulSets by @ivanfavi in #6106
Draft release notes for 2.11 by @leizor in #6702
Prepare release 2.11.0-rc.0 CHANGELOG.md by @leizor in #6770
continuous-test: Include comparison of all expected and actual values when any float sample does not match by @charleskorn in #6756
Docs: Move configuration-parameters subdirectory from references/ to configure/ by @osg-grafana in #6761
Upgrade to latest mimir-prometheus@main by @aknuds1 in #6766
Jsonnet: add newDistributorDeployment() utility by @pracucci in #6774
Jsonnet: add newDistributorContainer() utility by @pracucci in #6775
Upstream internal jsonnet changes by @pstibrany in #6764
Distributor: Refactor OTLP handler by @aknuds1 in #6719
feat/add discovered metrics to metricsUsed by @56quarters in #6773
Jsonnet/Helm: make memcached connection limit configurable by @lukas-unity in #6715
Un-revert #6451 by @charleskorn in #6780
Jsonnet: add distributor_node_affinity_matchers setting support by @pracucci in #6782
Query-frontend worker: put back 'continue' removed in error by @bboreham in #6768
Exclude vector queries from being counted in metric for rules with zero fetched series by @zenador in #6544
Increase default timeout for index-queries cache requests by @56quarters in #6786
Use BucketIndexBlocksFinder instead of BucketScanBlocksFinder by @leizor in #6779
Fix issue where failed remote rule evaluation errors are logged without context such as trace IDs by @charleskorn in #6789
distributor: more efficient labels error messages by @bboreham in #6785
Update github.com/thanos-io/objstore digest to bffedaa (main) by @renovate in #6741
Run make mixin-screenshots by @leizor in #6771
Update github.com/alecthomas/units digest to 9a357b5 (main) by @renovate in #6800
Update golang.org/x/exp digest to 6522937 (main) by @renovate in #6803
Fix syntax error in renovate.json by @pstibrany in #6807
Update module github.com/aws/aws-sdk-go to v1.48.11 (main) by @renovate in #6805
Move heap ballast util function to dskit by @bboreham in #6810
Move change log entry of 6106 into helm changelog by @dimitarvdimitrov in #6769
store-gateway: use bucket index instead of scanning the bucket by @dimitarvdimitrov in #6808
Make distributor log ingester details when a push request fails by @duricanikolic in #6801
querier: default to using batch.NewChunkMergeIterator by @dimitarvdimitrov in #6814
TestQuerierWithBlocksStorageOnMissingBlocksFromStorage: start compactor earlier by @dimitarvdimitrov in #6815
Update alpine Docker tag to v3.18.5 (main) by @renovate in #6804
Fix helm build script when running out of build-image in a non-bash shell by @pracucci in #6820
Mixin: extend distributor and gateway job label matchers to include multi-zone deployments by @pracucci in #6817
Fix flaky ring tests by @pstibrany in #6824
Move util to format LabelAdaptor string to mimirpb package by @bboreham in #6822
ingester: simplify metadata limit error by @bboreham in #6821
fix up bad merge giving compile errors by @bboreham in #6827
mimirtool: clarify what auth flags do by @dimitarvdimitrov in #6798
Use spanlogger for Ruler API calls by @56quarters in #6823
[dashboards] Add panels for alertmanager activity of a tenant by @vaxvms in #6826
Jsonnet: add node_affinity_matchers setting support for every Mimir component by @pracucci in #6829
Jsonnet: rename with_anti_affinity to withAntiAffinity by @pracucci in #6832
Add MimirStoreGatewayTooManyFailedOperations alert by @wilfriedroset in #6831
ruler: fix TestRulerMetricsForInvalidQueriesAndNoFetchedSeries by @dimitarvdimitrov in #6843
Update mimir-prometheus by @pracucci in #6846
Clarify grafana agent native histogram support by @callmehyde in #6793
Release mimir-distributed Helm chart 5.2.0-weekly.267 by @grafanabot in #6825
query-frontend: inject query cache keys for LabelValues/Cardinality requests by @dimitarvdimitrov in #6849
storegateway: small refactor to simplify by @bboreham in #5444
Docs: Add an introduction; add range query caching info by @osg-grafana in #6857
Add unit test to compare histogram_quantile results with and without query sharding by @zenador in #6525
Doc: Refer to native history bucket limit by @osg-grafana in #6859
ingester client: context canceled is not an error by @bboreham in #6809
Store-gateway: Add no-compact column on store-gateway/tenants admin UI by @ying-jeanne in #6848
Helm/Jsonnet: update memcached-exporter to 0.14.1 by @krajorama in #6861
[mimir-distributed-release-5.1] Helm/Jsonnet: update memcached-exporter to 0.14.1 by @krajorama in #6863
fix(deps): update module github.com/aws/aws-sdk-go to v1.48.16 (main) by @renovate in #6869
extend histogram cortex_distributor_sample_delay_seconds_bucket to track negative delays by @replay in #6838
Mixin: remove last graph panel by @dimitarvdimitrov in #6864
e2e mimir client: add ability to inject headers and roundtrippers by @dimitarvdimitrov in #6851
Release mimir-distributed Helm chart 5.2.0-weekly.268 by @grafanabot in #6882
chore: mimirtool prometheus analyze set tenant header by @jmichalek132 in #6737
query-frontend: add special error to prevent caching of LabelValues and LabelValuesCardinality requests by @dimitarvdimitrov in #6885
Distributor: Use pooled buffers for reading/decompressing request body by @aknuds1 in #6836
Update docs on CacheKeyGenerator by @dimitarvdimitrov in #6890
Fix formatting of comment posted during build image CI builds by @charleskorn in #6872
Add experimental support to write incoming data to a Kafka-compatible backend by @pracucci in #6888
Fix issue where querier trace spans are not nested correctly by @charleskorn in #6893
Upgrade dskit by @charleskorn in #6875
util.RequestBuffers: Add tests by @aknuds1 in #6891
Update alpine Docker tag to v3.19.0 (main) by @renovate in #6876
fix(deps): update module github.com/minio/minio-go/v7 to v7.0.65 (main) by @renovate in #6873
chore(deps): update grafana/agent docker tag to v0.38.1 (main) by @renovate in #6878
Update 2.9 changelog and release notes by @dimitarvdimitrov in #6899
Update 2.10 changelog and release notes by @dimitarvdimitrov in #6900
util.RequestBuffers: Fix test flakiness by @aknuds1 in #6905
Introduce type for Unix timestamps in JSON by @pstibrany in #6883
Kafka backend prototype: set metadata min age == max age by @pracucci in #6894
Update all release-2.10 release notes by @colega in #6908
jsonnet: Add ability to have rollout-operator scale down ingesters by @56quarters in #6850
k6: Pass auth to write path via headers not URL by @56quarters in #6915
Distributor: simplify context propagation by @dimitarvdimitrov in #6889
Add project ID option for k6 cloud tests by @chencs in #6916
Add flag to enable remote rule evaluation in microservices Docker Compose setup by @charleskorn in #6919
Use correct port in ruler helper script for microservices Docker Compose setup by @charleskorn in #6918
fix(deps): update golang.org/x/exp digest to f3f8817 (main) by @renovate in #6867
chore(deps): update anchore/sbom-action action to v0.15.1 (main) by @renovate in #6877
fix(deps): update module go.opentelemetry.io/collector/pdata to v1.0.0 (main) by @renovate in #6874
fix(deps): update module github.com/failsafe-go/failsafe-go to v0.4.2 (main) by @renovate in #6870
fix(deps): update module github.com/go-openapi/strfmt to v0.21.9 (main) by @renovate in #6871
Update golang Docker tag to v1.21.5 (main) by @renovate in #6868
Compactor: add function to remove no compact marker by @ying-jeanne in #6917
Ingester remains in the LEAVING state if starting() terminates by @duricanikolic in #6923
apply multidimensional query request queuing: supply queue dimensions from frontend & utilize in scheduler by @francoposa in #6772
Update dskit and fix issue where all incoming HTTP requests have duplicate trace spans by @charleskorn in #6920
Add trace span for time spent lazily loading index-header in store-gateway by @charleskorn in #6922
Fix typo in log message by @pracucci in #6933
e2e client support for cardinality by @dimitarvdimitrov in #6852
Dashboards: add graphs to "Slow queries" by @dimitarvdimitrov in #6880
Helm: release 5.1.4 by @krajorama in #6865
k6: Split usernames and tenant IDs by read/write by @56quarters in #6930
ingester: add experimental support for consuming records from kafka by @dimitarvdimitrov in #6929
Merge mimir-distributed-release-5.1 to main by @dimitarvdimitrov in #6936
chore(deps): update grafana/mimirtool docker tag to v2.10.5 (main) by @renovate in #6945
Update Mimir images to use 2.10.5 by @dimitarvdimitrov in #6937
feat(helm): add pure Ingress option instead of the gateway service by @mcanevet in #6932
Release mimir-distributed Helm chart 5.2.0-weekly.269 by @grafanabot in #6949
ingest consumer: handle Push errors by @dimitarvdimitrov in #6940
ingest consumer: more granular error handling, committer sanity check by @dimitarvdimitrov in #6951
Fix context cancellation handling when request is canceled while querying store-gateway by @pracucci in #6934
Fix .metricsUsed in mimirtool analyze rule-file by @colega in #6953
Get rid of validation/errors.go by @duricanikolic in #6955
Adjust renovate.json for 2.11 release by @leizor in #6796
add additional queue dimensions to query scheduler queue duration histogram by @francoposa in #6960
Chore: remove unused storepb.StoreServer by @pracucci in #6958
store-gateway: remove obsolete comments for fine-grained chunks caching by @dimitarvdimitrov in #6892
Slow queries dashboard: unhide parameters, remove length calculation by @dimitarvdimitrov in #6913
Delete docs on removed configuration parameters by @leizor in #6967
Merge release-2.11 to main by @leizor in #6963
fix and test label query optional time param parsing for additional queue dimensions by @francoposa in #6969
Support disabling metric relabeling per tenant by @Logiraptor in #6970
Improve validation.LimitError usages by @duricanikolic in #6954
ingest consumer: commit offset on shutdown by @dimitarvdimitrov in #6974
Upgrade mimir-prometheus by @pracucci in #6975
Mimirtool: add --output-dir to alertmanager get by @edwintye in #6760
Update module golang.org/x/crypto to v0.17.0 [SECURITY] (main) by @renovate in #6962
Update go-openapi packages (main) by @renovate in #6946
fix(deps): update golang.org/x/exp digest to dc181d7 (main) by @renovate in #6943
Remove PrioritizeRecoverableErr method no longer used by @56quarters in #6976
Distributor: start remote timeout on first callback by @colega in #6972
Jsonnet: improve querier HPA to scale up and down more gradually by @pracucci in #6971
Refactor Distributor.push() by @colega in #6978
Release mimir-distributed Helm chart 5.2.0-weekly.270 by @grafanabot in #6998
Update module github.com/twmb/franz-go to v1.15.4 (main) by @renovate in #6997
Update github.com/thanos-io/objstore digest to 9f421f2 (main) by @renovate in #6993
Merge release-2.11 to main by @leizor in #7002
Add 2.11 to backwards compatibility test by @leizor in #7003
fix(deps): update module github.com/minio/minio-go/v7 to v7.0.66 (main) by @renovate in #6996
Update module github.com/Azure/azure-sdk-for-go/sdk/storage/azblob to v1.2.1 (main) by @renovate in #6947
chore(deps): update grafana/grafana docker tag to v10.2.3 (main) by @renovate in #6994
fix(deps): update module github.com/prometheus/client_golang to v1.18.0 (main) by @renovate in #7005
fix(deps): update golang.org/x/exp digest to 02704c9 (main) by @renovate in #7006
"Slow Queries" dashboard: make more fields sortable by @dimitarvdimitrov in #7000
fix(deps): update module github.com/aws/aws-sdk-go to v1.49.11 (main) by @renovate in #7009
fix(deps): update module go.uber.org/goleak to v1.3.0 (main) by @renovate in #7010
fix(deps): update module golang.org/x/time to v0.5.0 (main) by @renovate in #7011
fix(deps): update module sigs.k8s.io/kustomize/kyaml to v0.16.0 (main) by @renovate in #7012
fix(deps): update module google.golang.org/protobuf to v1.32.0 (main) by @renovate in #7017
chore(deps): update prom/prometheus docker tag to v2.48.1 (main) by @renovate in #7007
chore(deps): update registry.k8s.io/kustomize/kustomize docker tag to v5.3.0 (main) by @renovate in #7008
chore(deps): update actions/setup-go action to v5 (main) by @renovate in #7020
fix(deps): update module github.com/alecthomas/chroma/v2 to v2.12.0 (main) by @renovate in #7014
fix(deps): update module github.com/spf13/afero to v1.11.0 (main) by @renovate in #7018
prometheus codec time param parsing docstrings and emulating upstream prometheus behavior by @francoposa in #6985
fix(deps): update github.com/thanos-io/objstore digest to 61cfed8 (main) by @renovate in #7024
mimir-build-image: update to Debian Bookworm by @colega in #6980
helm: omit backend: s3 when minio is disabled by @dimitarvdimitrov in #6999
Kafka ingestion experiment: add PartitionReader.WaitReadConsistency() by @pracucci in #6982
mimir-mixin: update github.com/grafana/jsonnet-libs by @dimitarvdimitrov in #7027
slow queries dashboard: rename loki_datasource by @dimitarvdimitrov in #7028
Always validate tenant IDs and introduce max tenants setting by @56quarters in #6959
Fix panic in TestPartitionOffsetWatcher_Concurrency by @pracucci in #7029
chore(deps): update prom/memcached-exporter docker tag to v0.14.2 (main) by @renovate in #7025
mixin: make autoscaling dashboards ignore multiple series from keda metrics by @dimitarvdimitrov in #7001
fix(deps): update module cloud.google.com/go/storage to v1.36.0 (main) by @renovate in #7013
fix(deps): update module github.com/aws/aws-sdk-go to v1.49.13 (main) by @renovate in #7035
Slow queries dashboard: Fix loki datasource variable name by @dimitarvdimitrov in #7039
Fix flaky TestLoader_ShouldCacheIndexNotFoundOnBackgroundUpdates by @pracucci in #7040
Remote querier: use backoff retry only with 5xx errors by @duricanikolic in #7004
fix(deps): update go-openapi packages (main) by @renovate in #6995
chore(deps): update alpine/helm docker tag to v3.13.3 (main) by @renovate in #6944
fix(deps): update module github.com/google/go-github/v32 to v57 (main) by @renovate in #7038
[query-frontend] Set http request headers from querymiddleware request options by @flxbk in #7033
Add strong read consistency support in ingester for experimental ingest storage by @pracucci in #7030
Add 'msteams' to TestMultitenantAlertmanager_FirewallShouldBlockHTTPBasedReceiversWhenEnabled by @pracucci in #7050
Enable sharding for active_series requests by @flxbk in #6784
query-frontend: remove duplicate defaulting of instant query time param by @dimitarvdimitrov in #7026
Move owned series check logging/metric inside of updateAllTenants by @pr00se in #7044
Allow server to return HTTP 4xx errors by @duricanikolic in #7045
Release mimir-distributed Helm chart 5.2.0-weekly.271 by @grafanabot in #7049
Fix max_fetched_series_per_query not applied to /series from ingesters by @krajorama in #7055
Enforce use of context.WithCancelCause in non-test code by @charleskorn in #6921
fix(deps): update dependency puppeteer to v21.7.0 (main) by @renovate in #7069
chore(deps): update otel/opentelemetry-collector-contrib docker tag to v0.91.0 (main) by @renovate in #7067
fix(deps): update module github.com/aws/aws-sdk-go to v1.49.16 (main) by @renovate in #7064
chore(deps): update anchore/sbom-action action to v0.15.2 (main) by @renovate in #7063
fix(deps): update golang.org/x/exp digest to be819d1 (main) by @renovate in #7062
fix(deps): update module golang.org/x/sync to v0.6.0 (main) by @renovate in #7070
Release mimir-distributed Helm chart 5.2.0-weekly.272 by @grafanabot in #7072
chore(deps): update actions/download-artifact action to v4 (main) by @renovate in #7019
new Mimir / Reads dashboard panel for scheduler queue duration, breakout by additional queue dimensions by @francoposa in #7059
Remove default CPU limit for rollout-operator pod by @charleskorn in #7066
Revert #6922 by @charleskorn in #7065
fix(deps): update module google.golang.org/grpc to v1.60.1 (main) by @renovate in #7016
Prepare CHANGELOG.md for next helm release by @flxbk in #7079
Update dskit to latest main by @56quarters in #7076
Allow S3 Storage Provider to support sts_endpoint by @TimKotowski in #6990
Increase default Jaeger queue size for store-gateways and queriers by @charleskorn in #7068
Fix formatting in querier component doc by @jhalterman in #7078
docs: update link to Grafana Agent Operator CRDs by @rfratto in #7084
Helm chart release 5.2.0-rc.0 by @flxbk in #7093
ingest: include gRPC status code validation only when running ingester by @dimitarvdimitrov in #7095
ingest: Add dev environment by @dimitarvdimitrov in #7096
Increase default Jaeger queue size for some components and make queue size for all components configurable in Helm chart by @charleskorn in #7086
Mimir / Reads dashboard query scheduler queue duration breakout improvements by @francoposa in #7098
Update dskit by @pstibrany in #7092
Handle context.Canceled in active series requests by @flxbk in #7102
Small fixes for /active_series by @flxbk in #7106
Repurpose client within tools/copyblocks to add undelete-blocks and copyprefix by @andyasp in #6607
ingest storage: per-query X-Read-Consistency HTTP header by @dimitarvdimitrov in #7091
Update dskit by @pr00se in #7109
scheduler queue benchmark: factor out & clarify functions by @francoposa in #7101
Update dskit to latest main by @56quarters in #7111
Update make docs procedure and add workflow to keep it up to date by @jdbaldry in #5794
querier: remove noop if by @dimitarvdimitrov in #7108
Add additional querier metrics by @jhalterman in #7099
Support active count_method in label names cardinality by @Logiraptor in #7085
Add query-scheduler queue length to Reads and Remote Ruler Reads dashboards by @charleskorn in #7088
fix(deps): update github.com/grafana/dskit digest to f30e65d (main) by @renovate in #7114
chore(deps): update anchore/sbom-action action to v0.15.3 (main) by @renovate in #7116
fix(deps): update golang.org/x/exp digest to db7319d (main) by @renovate in #7115
chore(deps): update helm release grafana-agent-operator to v0.3.14 (main) by @renovate in #7118
chore(deps): update golang docker tag to v1.21.6 (main) by @renovate in #7117
fix(deps): update module github.com/aws/aws-sdk-go to v1.49.21 (main) by @renovate in #7121
chore(deps): update memcached docker tag to v1.6.23 (main) by @renovate in #7120
Update vendored mimir-prometheus by @pracucci in #7057
chore(deps): update grafana/agent docker tag to v0.39.0 (main) by @renovate in #7123
fix(deps): update module go.opentelemetry.io/collector/pdata to v1.0.1 (main) by @renovate in #7122
fix(deps): update module google.golang.org/api to v0.156.0 (main) by @renovate in #7015
Helm chart release 5.2.0 by @flxbk in #7089
Release mimir-distributed Helm chart 5.2.0-weekly.273 by @grafanabot in #7124
Fix missing "annotation" key in create-bucket-job template of helm chart by @narqo in #7113
Upgrade rollout-operator to v0.10.1 by @pracucci in #7125
Change -query-frontend.not-running-timeout default value to 2s and mark it as advanced by @pracucci in #7127
Use 2K reusable ingester push workers by default by @colega in #7128
Set -server.grpc.num-workers=100 by default by @colega in #7131
mimirpb: add DeepCopyTimeseries extra option to make a deep copy hi… by @ortuman in #7130
Jsonnet / Helm: reduce likelihood of queries hitting terminated query-frontends by @pracucci in #7129
Merge mimir-distributed-release-5.2 to main by @flxbk in #7097
Make TestBlocksStoreQuerier_ShouldReturnContextCanceledIfContextWasCanceledWhileRunningRequestOnStoreGateway easier to debug when flaky by @pracucci in #7134
Documentation: Document ingester read path limiting by @aknuds1 in #7110
Reduce sync concurrency in store-gateway by default to reduce disk contention by @andyasp in #7136
Perform initial owned series calculation before starting lifecycler by @pr00se in #7087
Update vendored mimir-prometheus by @pracucci in #7138
Chore: simplify runAsync() and runAsyncAfter() usage in ingest storage tests by @pracucci in #7147
Ingest storage: configure BrokerMaxReadBytes on Kafka reader by @pracucci in #7148
Report gRPC status codes as labels in request duration metrics by @duricanikolic in #7144
Ingest storage local dev env: configure Mimir components to start after Kafka is healthy by @pracucci in #7153
remove non-ascii characters from invalid metric names in error message by @replay in #7146
/active_series: generate correct request shards for incoming GET requests, handle gRPC errors by @flxbk in #7133
Update to latest dskit for cache changes by @56quarters in #7155
[BUGFIX] Update nginx deployment to mount nginx.conf as file by @blut in #7150
Update opentelemetry-go monorepo to v1.22.0 (main) by @renovate in #7071
Jsonnet / Helm: relax the hash ring heartbeat period and timeout for distributor, ingester, store-gateway and compactor by @dimitarvdimitrov in #6860
mimir-mixin: drop unsupported step target parameter by @krajorama in #7157
Update values.yaml documentation and example by @benoitschipper in #7135
Make ingesters return gRPC errors only by @duricanikolic in #7151
Update RELEASE.md with tentative 2024 release schedule by @duricanikolic in #7159
Rename metric to cortex_querier_query_storegateway_chunks_total by @jhalterman in #7145
CI: use a single cache for tests by @bboreham in #5266
Set distributor.remote-timeout=10s for rulers in jsonnet/helm by @leizor in #7143
Add integration test using ingest storage by @pracucci in #7160
fix additional queue dimensions unsupported request logging by @francoposa in #7164
mimir ruler reads dashboard query scheduler queue duration breakout improvements by @francoposa in #7163
mimir ruler reads dashboard query scheduler row - fix job matchers by @francoposa in #7166
Update MAINTAINERS.md by @derek-cadzow in #7077
Update make docs procedure by @github-actions in #7167
Ingester: mark spread-minimizing tokens-related CLI flags as advanced by @duricanikolic in #7169
Replace servicediscovery package with dskit's by @npazosmendez in #7171
Changelog: rephrase the changes related to STS endpoint configuration by @narqo in #7170
Alertmanager: Support uploading Grafana Alertmanager Configuration an… by @gotjosh in #6682
Changed -query-frontend.max-cache-freshness default value from 1m to 10m by @pracucci in #7161
Enable distributor write requests buffer pooling by default by @andyasp in #7165
Mimirtool: add the ability to include extra headers with requests made to the server by @snowzach in #7141
Fix panic as a result of duplicate write request cleanup by @Logiraptor in #7176
Write query parameters to span attributes instead of events by @chencs in #7046
Update golang.org/x/exp digest to 1b97071 (main) by @renovate in #7179
Update grafana/agent Docker tag to v0.39.1 (main) by @renovate in #7183
Update Helm release grafana-agent-operator to v0.3.15 (main) by @renovate in #7180
Update anchore/sbom-action action to v0.15.4 (main) by @renovate in #7181
Update github.com/thanos-io/objstore digest to 6ecabdd (main) by @renovate in #7178
Update Helm release rollout-operator to v0.13.0 (main) by @renovate in #7184
Update alpine/helm Docker tag to v3.14.0 (main) by @renovate in #7185
Update module github.com/aws/aws-sdk-go to v1.50.0 (main) by @renovate in #7186
Update module github.com/prometheus/common to v0.46.0 (main) by @renovate in #7187
Increase log level for problems creating index header readers by @56quarters in #7177
Add conditional read-after–write support to rules evaluation by @pracucci in #7142
querier: don't log errors on stream connection reset by @narqo in #7168
Documented ingeter migration to spread-minimizing tokens by @duricanikolic in #7174
Release mimir-distributed Helm chart 5.2.0-weekly.274 by @grafanabot in #7191
Update make docs procedure by @github-actions in #7197
Alertmanager: Update to latest main by @grobinson-grafana in #7103
Improve read consistency observability by @pracucci in #7193
Revert "helm: omit backend: s3 when minio is disabled" by @dimitarvdimitrov in #7199
[mimir-distributed-release-5.2] Revert "helm: omit backend: s3 when minio is disabled" by @grafanabot in #7201
Slow queries dashboard: bug fixes and a new panel by @dimitarvdimitrov in #7194
Helm: release 5.2.1 by @dimitarvdimitrov in #7202
Merge mimir-distributed-release-5.2 to main by @dimitarvdimitrov in #7203
Jsonnet: allow to configure ruler-querier max skew separately from querier by @pracucci in #7204
query scheduler test for multidimensional queueing effectiveness scenarios by @francoposa in #7162
Fix missing PR number from CHANGELOG.md by @grobinson-grafana in #7208
downgrade integration test certificate elliptic curve algo for boringcrypto FIPS-only compatibility by @francoposa in #7206
Fix 'op' label values for cortex_query_frontend_queries_total metric by @pracucci in #7207
distributor: make unit tests more flexible by @dimitarvdimitrov in #7205
Alertmanager: Support UTF-8 by @grobinson-grafana in #6898
Chore: remove superfluous errors.Cause() usage in pkg/storage by @pracucci in #7212
Remove errors.Cause() usage from store-gateway and its clients by @pracucci in #7213
Remove errors.Cause() usage from cardinality analysis handler by @pracucci in #7214
distributor: use pointers to mockIngester in tests by @dimitarvdimitrov in #7223
Do not use grpc/status.FromError() and gogo/status.FromError() by @pracucci in #7224
Alertmanager: Support UTF-8 characters in matchers and labels small followup by @gotjosh in #7221
Add querier metric for block source and compaction level by @jhalterman in #7112
Ensure we never use errors.Cause() by @pracucci in #7215
Add Dockerfile for tools/copyblocks by @pstibrany in #7211
Extensible "Rollout progress" workload grouping by @colega in #7228
Enable Nginx Proxy HTTP/1.1 in Mimir for Istio Envoy Sidecar Compatibility by @rafilkmp3 in #5040
ruler: add more details to failures for TestRulerEvaluationDelay by @dimitarvdimitrov in #7217
Fix s3 upload performance regression by @pracucci in #7241
Update alpine Docker tag to v3.19.1 (main) by @renovate in #7238
chore(deps): update anchore/sbom-action action to v0.15.5 (main) by @renovate in #7239
fix(deps): update module github.com/aws/aws-sdk-go to v1.50.5 (main) by @renovate in #7242
fix(deps): update module github.com/go-openapi/swag to v0.22.9 (main) by @renovate in #7243
Update mimir-prometheus to c2a5e159 by @krajorama in #7219
Update dskit to latest main and implements LogSourceIPsFull flag by @bubu11e in #7250
Update mimir-prometheus by @pracucci in #7251
Native histogram: validate schema number before reducing resolution by @duricanikolic in #7252
Update Alertmanager to latest main by @grobinson-grafana in #7254
Update dskit to latest main by @56quarters in #7258
Increase distributor jaeger max queue size by @jhesketh in #7259
Reuse Histogram objects in tools/tsdb-chunks and tools/tsdb-print-chunk by @duricanikolic in #7260
Update about-grafana-mimir-architecture by @narqo in #7234
Include out-or-order label in blocks queried metric by @56quarters in #7262
Add utf8-strict-mode flag for mimirtool by @grobinson-grafana in #7227
rulestore/local: add support for GetRuleGroup endpoint by @narqo in #7248
Add out of order field to block index by @jhalterman in #7267
Add gitignore files for mimir tools by @jhalterman in #7271
frontend: improve shardActiveSeriesMiddleware performance when merging responses by @ortuman in #7261
Native histograms: optimize chunk iterator usage by @duricanikolic in #7274
Release mimir-distributed Helm chart 5.2.0-weekly.275 by @grafanabot in #7245
helm: align grpc server connection lifetime settings with jsonnet by @narqo in #7269
querymiddleware: Fix race condition in shardActiveSeriesMiddleware by @narqo in #7290
ruler: build user-agent header at runtime by @narqo in #7264
helm: sync server.grpc.keepalive.max-connection-idle with jsonnet by @narqo in #7298
Compactor: export estimated number of compaction jobs based on bucket-index by @pstibrany in #7299
Add KubePersistentVolumeFillingUp runbook by @pracucci in #7297
Internal: remove unnecessary parameter to NoCompactionMarkFilter by @pstibrany in #7301
Name query metrics for easier discovery by @56quarters in #7302
fix(deps): update module github.com/aws/aws-sdk-go to v1.50.11 (main) by @renovate in #7288
fix(deps): update module github.com/klauspost/compress to v1.17.6 (main) by @renovate in #7291
chore(deps): update anchore/sbom-action action to v0.15.8 (main) by @renovate in #7286
chore(deps): update grafana/agent docker tag to v0.39.2 (main) by @renovate in #7287
chore(deps): update grafana/grafana docker tag to v10.3.1 (main) by @renovate in #7292
fix(deps): update module github.com/failsafe-go/failsafe-go to v0.4.4 (main) by @renovate in #7289
Chore: removed unused parameter from GenerateBlockFromSpec() by @pracucci in #7303
Update mimir-prometheus by @pracucci in #7293
Release mimir-distributed Helm chart 5.3.0-weekly.276 by @grafanabot in #7294
Open circuit breakers on timeouts and per-instance limit errors only by @duricanikolic in #7310
Get rid of iterators.chunkIterator and iterators.chunkMergeIterator by @duricanikolic in #7313
Compactor: Language fixes by @aknuds1 in #7315
Do not register compat metrics in mimirtool by @grobinson-grafana in #7314
Compactor: Un-export symbols that don't need to be exported by @aknuds1 in #7317
Circuit breakers: add client.ErrCircuitBreakerOpen type by @duricanikolic in #7324
Add mimirpb.CIRCUIT_BREAKER_OPEN error cause by @duricanikolic in #7330
store-gateway: remove cortex_bucket_store_blocks_loaded_by_duration by @dimitarvdimitrov in #7309
ruler: don't retry on non-retriable error by @narqo in #7216
Update Alertmanager to f69a508 by @grobinson-grafana in #7332
Helm: add ruler specific service account by @QuantumEnigmaa in #7132
frontend/transport: Log non-2xx replies from downstream as unsuccesful by @narqo in #7296
querymiddleware: Pool snappy writer in shard activity series by @narqo in #7308
Helm: make PSP configurable by @QuantumEnigmaa in #7190
Helm - Templatable host for gateway ingress/route by @Itaykal in #7218
[Docs] Update migrate-from-single-zone-with-helm.md by @eamonryan in #7327
Always sort labels in distributors by @Logiraptor in #7326
Do not check for ingester ring state before creating TSDB, or compacting / shipping blocks by @pracucci in #7322
Compactor: String format compaction plan as comma separated blocks by @aknuds1 in #7321
Add a lifetime manager for Vault authentication tokens by @fayzal-g in #7337
Update github.com/grafana/dskit digest to f245b48 (main) by @renovate in #7283
Packaging: remove reload from systemd file as mimir does not take into account SIGHUP by @wilfriedroset in #7345
Docs: No longer mark OTLP endpoint as experimental by @aknuds1 in #7348
Update golang.org/x/exp digest to 2c58cdc (main) by @renovate in #7352
Update module github.com/aws/aws-sdk-go to v1.50.15 (main) by @renovate in #7353
Update module github.com/minio/minio-go/v7 to v7.0.67 (main) by @renovate in #7354
Update dependency puppeteer to v21.11.0 (main) by @renovate in #7355
Update helm/kind-action action to v1.9.0 (main) by @renovate in #7357
Update module cloud.google.com/go/storage to v1.37.0 (main) by @renovate in #7358
Jsonnet / Helm: improve distributors graceful shutdown by @pracucci in #7361
Release mimir-distributed Helm chart 5.3.0-weekly.277 by @grafanabot in #7362
Distributor: Make -distributor.enable-otlp-metadata-storage flag default to true, and deprecate by @aknuds1 in #7366
Mark -ingester.limit-inflight-requests-using-grpc-method-limiter and -distributor.limit-inflight-requests-using-grpc-method-limiter as stable and enable it by default by @pracucci in #7360
Do not consider out-of-order blocks when filtering compactable jobs by @jhalterman in #7342
mimir: Inject span profiler into tracer by @narqo in #7363
Add experimental partitions ring lifecycler support by @pracucci in #7349
feat(helm): Adding KEDA autoscaling support by @beatkind in #7282
Add timely head compaction support by @jhalterman in #7372
Fix type error in autoscaling jsonnet by @56quarters in #7374
Become temporary maintainer of docs until another technical writer is found by @jdbaldry in #7378
vendor: Bump grpc-go to 1.61 latest by @narqo in #7380
Add page for displaying compaction jobs computed from bucket-index. by @pstibrany in #7381
query-tee: override correct Host attribute by @dimitarvdimitrov in #7386
Use partitions ring in write path and ingesters consumption by @pracucci in #7376
Fix TestPartitionReader_WaitReadConsistency flakyness by @pracucci in #7391
mimirtool: Add new migrate-utf8 command by @grobinson-grafana in #7383
active series: abort streaming on context cancelled by @flxbk in #7387
Introduce new metric to track OTLP requests by @fayzal-g in #7385
Alertmanager: Update Alertmanager to commit 80b3cb0 by @grobinson-grafana in #7384
prepare-partition-downscale: Switch to Active state only if partition is currently Inactive. by @pstibrany in #7394
Add partitions ring support to Distributor.QueryStream() by @pracucci in #7388
Update Vault token renewal test to use e2e.GreaterOrEqual by @fayzal-g in #7398
ingester: Make usagestats vars package-global by @pstibrany in #7395
Make active series benchmark more realistic, improve performance of context cancellation check. by @flxbk in #7396
Query-frontend activity log enhancements by @pstibrany in #7400
Add partitions ring support to more Distributor querying functions by @pracucci in #7393
Add partitions ring support to Distributor.UserStats() by @pracucci in #7402
Improve compaction of sporadic blocks by @jhalterman in #7329
Add new command on mimirtool used to render template by @ncharaf in #7325
Clean up a few changelog entries by @56quarters in #7408
[Pull Request Template] change file path by @ncharaf in #7410
Update module github.com/aws/aws-sdk-go to v1.50.20 (main) by @renovate in #7419
Update grafana/grafana Docker tag to v10.3.3 (main) by @renovate in #7418
feat: Adding global kedaAutoscaling section by @beatkind in #7392
added the right pointer for OTLP endpoint by @zhehao-grafana in #7375
fix(deps): update github.com/grafana/dskit digest to ce15a83 (main) by @renovate in #7415
Replace all blockquote admonitions with the shortcode by @jdbaldry in #7420
Add partitions support to Distributor.ActiveSeries() by @pracucci in #7404
Note how to run Mimir Alertmanager as part of a monolithic deployment by @Rajakavitha1 in #7414
Added partitions support to Distributor.LabelValuesCardinality() by @pracucci in #7423
Release mimir-distributed Helm chart 5.3.0-weekly.278 by @grafanabot in #7429
Update dskit by @pracucci in #7430
Convert mimir-mixin from deprecated graph panel type to timeseries by @narqo in #7413
Remove unused logger argument by @56quarters in #7438
Update dskit by @pr00se in #7439
Require ingester zone to be configured when running ingest storage by @pracucci in #7432
Revert "Require ingester zone to be configured when running ingest storage" by @pracucci in #7443
Fix premature context cancellation in Distributor.QueryStream() when experimental ingest storage is enabled by @pracucci in #7437
Switch ingester limiter to use ingesterRing rather than lifecycler by @pr00se in #7440
Set series cache TTL different based on block metadata by @56quarters in #7407
storegateway: Reduce number of overlapping iterator interfaces into single iterator[S] by @narqo in #7451
fix(deps): update github.com/thanos-io/objstore digest to a8d75c5 (main) by @renovate in #7351
Modify ingester's limiter to work with ingest store. by @pstibrany in #7424
Add dashboard UI check to all dashboards using uid based on filename hash by @pstibrany in #7446
Add optional querier response streaming by @flxbk in #7173
query-tee: use X-Scope-OrgID for basic auth by @dimitarvdimitrov in #7452
mimir-mixin: Fix "unexpected type object, expected array" by @narqo in #7457
Update dskit to latest commit by @56quarters in #7458
chore(deps): update grafana/doc-validator docker tag to v4.1.0 (main) by @renovate in #7463
fix(deps): update module github.com/klauspost/compress to v1.17.7 (main) by @renovate in #7462
fix(deps): update module github.com/aws/aws-sdk-go to v1.50.25 (main) by @renovate in #7460
chore(deps): update prom/prometheus docker tag to v2.50.0 (main) by @renovate in #7465
chore(deps): update otel/opentelemetry-collector-contrib docker tag to v0.95.0 (main) by @renovate in #7464
fix(deps): update golang.org/x/exp digest to 814bf88 (main) by @renovate in #7416
Revert "fix(deps): update github.com/thanos-io/objstore digest to a8d75c5 (main)" by @flxbk in #7459
fix(deps): update module cloud.google.com/go/storage to v1.38.0 (main) by @renovate in #7467
Add estimated compaction jobs based on bucket-index to Mimir dashboards by @pstibrany in #7449
Jsonnet / Helm: upgrade rollout-operator to v0.13.0 by @pracucci in #7469
Add more buckets to cortex_query_scheduler_queue_duration_seconds by @pracucci in #7470
Support KEDA's ignoreNullValues field. by @pstibrany in #7471
Chunk iterators: Optimize histogram memory allocations by @duricanikolic in #7427
Update alpine/helm Docker tag to v3.14.1 (main) by @renovate in #7417
[Build] Add and Use distroless image in e2e tests by @ying-jeanne in #7371
Fix regression that caused client errors to be tracked in cortex_ruler_write_requests_failed_total by @pracucci in #7472
Add cardinality API qps to Overview dashboard by @flxbk in #6720
Release mimir-distributed Helm chart 5.3.0-weekly.279 by @grafanabot in #7476
Upgrade to latest mimir-prometheus@main by @aknuds1 in #7475
Helm / Gateway: Allow to configure whether or not NGINX binds IPv6 by @wilfriedroset in #7421
Update CODEOWNERS by @grobinson-grafana in #7490
Update dskit by @pracucci in #7492
fix: Update KEDA related information by @JorTurFer in #7480
Better explain Estimated Compaction Jobs in compactor dashboard. by @pstibrany in #7481
Remove docs writer from maintaining proposals by @jdbaldry in #7493
fix(deps): update github.com/thanos-io/objstore digest to c3ccc5d (main) by @renovate in #7466
mixin: add extra alert options by @jmichalek132 in #7442
changelog: Add a note on updating all panels to "timeseries" in mixin by @narqo in #7502
Revert "fix(deps): update github.com/thanos-io/objstore digest to c3ccc5d (main)" by @dimitarvdimitrov in #7505
Querier: deprecates max-query-into-future by @wilfriedroset in #7496
Wait for owned series recomputation before lowering local series limit by @pr00se in #7411
Docs: Explain why types of caches should be separate by @56quarters in #7498
Update dskit to latest commit by @56quarters in #7510
Drop useless recording rules for non existent cache metrics by @krajorama in #7514
querier: fix remote read error translation by @dimitarvdimitrov in #7487
Deprecate metric cortex_distributor_sample_delay_seconds by @krajorama in #7516
Remove the deprecated frontend.cache_unaligned_requests YAML config by @56quarters in #7519
helm: Fix siToBytes to work with power-of-ten suffixes by @narqo in #7506
mixin: native histogram recording rule: cortex_request_duration_seconds by @krajorama in #7528
Remove deprecated store-gateway index-header settings. by @56quarters in #7521
[mimir-mixin] Add owned series to tenant dashboard by @pr00se in #7511
Release mimir-distributed Helm chart 5.3.0-weekly.280 by @grafanabot in #7527
Jsonnet: Don't explicitly set default values for index-header by @56quarters in #7532
Make persistence of sparse headers default behavior by @56quarters in #7535
Block upload: include converted timestamp in the error message by @pstibrany in #7538
Upgrade mimir-prometheus to latest main by @aknuds1 in #7541
Initialize Tanka with 1.29. by @pstibrany in #7544
Set TTL for cached postings based on block metadata by @56quarters in #7534
Querier: improve merging of batchStream objects by @duricanikolic in #7478
Update module google.golang.org/protobuf to v1.33.0 [SECURITY] (main) by @renovate in #7549
Update grafana/doc-validator Docker tag to v4.1.1 (main) by @renovate in #7524
chore(deps): update helm release grafana-agent-operator to v0.3.17 (main) by @renovate in #7525
chore(deps): update alpine/helm docker tag to v3.14.2 (main) by @renovate in #7523
chore(deps): update memcached docker tag to v1.6.24 (main) by @renovate in #7526
Remove index_header_lazy_loading_enabled docker-compose config by @flxbk in #7551
ruler: fix native histogram recording rule result corruption by @krajorama in #7552
[query-frontend] Set write deadline for /active_series requests by @flxbk in #7553
chore(deps): update golang docker tag to v1.21.8 (main) by @pr00se in #7557
Make owned series service in ingester work with partitions ring. by @pstibrany in #7508
Add Casie as maintainer by @pracucci in #7560
Update requirements doc with correct frontend parameter by @aldernero in #7566
[Helm] Release 5.2.2 by @Logiraptor in #7555
[query-frontend] /active_series: cancel request context when write deadline is reached by @flxbk in #7569
chore(deps): update module github.com/go-jose/go-jose/v3 to v3.0.3 [security] (main) by @renovate in #7571
Merge mimir-distributed-release-5.2 to main by @Logiraptor in #7568
[release-2.12] Preparing release notes for 2.12 by @grafanabot in #7600
Update the release version in CHANGELOG.md and VERSION by @duricanikolic in #7599
Run make mixin-screenshots for release 2.12 by @duricanikolic in #7602
Move -querier.minimize-ingester-requests from experimental to advanced by @duricanikolic in #7649
[release-2.12] Get rid of -querier.prefer-streaming-chunks-from-ingesters by @grafanabot in #7661
[query-frontend] Close response body in request handler (#7654) by @flxbk in #7663
Update VERSION for release 2.12.0-rc.1 by @duricanikolic in #7671
Prepare 2.12 final release by @pr00se in #7781
Update screenshots for 2.12 by @pr00se in #7796

New Contributors

@ivanfavi made their first contribution in #6106
@lukas-unity made their first contribution in #6715
@callmehyde made their first contribution in #6793
@mcanevet made their first contribution in #6932
@edwintye made their first contribution in #6760
@TimKotowski made their first contribution in #6990
@rfratto made their first contribution in #7084
@blut made their first contribution in #7150
@benoitschipper made their first contribution in #7135
@derek-cadzow made their first contribution in #7077
@github-actions made their first contribution in #7167
@snowzach made their first contribution in #7141
@rafilkmp3 made their first contribution in #5040
@QuantumEnigmaa made their first contribution in #7132
@Itaykal made their first contribution in #7218
@ncharaf made their first contribution in #7325
@zhehao-grafana made their first contribution in #7375
@JorTurFer made their first contribution in #7480

Full Changelog: mimir-2.11.0...mimir-2.12.0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

2.12.0

Grafana Mimir version 2.12.0 release notes

Features and enhancements

Important changes

Experimental features

Bug fixes

Helm chart improvements

Changelog

2.12.0

Grafana Mimir

Mixin

Jsonnet

Mimirtool

Mimir Continuous Test

Query-tee

Documentation

Tools

What's Changed

New Contributors

Contributors