diff --git a/content/en/blog/2024/collector-roadmap/can-haz-collector.png b/content/en/blog/2024/collector-roadmap/can-haz-collector.png deleted file mode 100644 index 941e0bc2a925..000000000000 Binary files a/content/en/blog/2024/collector-roadmap/can-haz-collector.png and /dev/null differ diff --git a/content/en/blog/2024/collector-roadmap/index.md b/content/en/blog/2024/collector-roadmap/index.md deleted file mode 100644 index 3a1d5bafa3ac..000000000000 --- a/content/en/blog/2024/collector-roadmap/index.md +++ /dev/null @@ -1,97 +0,0 @@ ---- -title: The roadmap to v1 for the OpenTelemetry Collector -linkTitle: Collector Roadmap -date: 2024-05-06 -# prettier-ignore -cSpell:ignore: Antipatterns Boten Broadbridge Helmuth Hrabovcak Ishan Jaglowski OTTL Pantuza pushback Shishi Vijay -author: '[Alex Boten](https://github.com/codeboten) (Honeycomb)' ---- - -The [OpenTelemetry Collector](/docs/collector/) is a very popular component in -OpenTelemetry that has been under heavy development for quite some time. It is a -binary that allows many formats of telemetry to be sent to it, transformed, and -emitted to a destination. Much has been said about the Collector over the past -few years in various blog posts and talks. Here's a small list of talks about -the Collector if you haven't had the chance to learn about it: - -- [Connected Observability Pipelines in the OpenTelemetry Collector - Dan Jaglowski](https://www.youtube.com/watch?v=uPpZ23iu6kI) -- [OpenTelemetry Collector Antipatterns - Adriana Villela](/blog/2024/otel-collector-anti-patterns/) -- [Telemetry Showdown: Fluent Bit Vs. OpenTelemetry Collector - Henrik Rexed](https://www.youtube.com/watch?v=ykq1F_3PmJw) -- [OpenTelemetry Collector Deployment Patterns - Juraci Kröhling](https://www.youtube.com/watch?v=WhRrwSHDBFs) -- [OTTL Me Why Transforming Telemetry in the OpenTelemetry Collector Just Got Better - Tyler Helmuth & Evan Bradley](https://www.youtube.com/watch?v=uVs0oUV72CE) -- [Manage OpenTelemetry Collectors at scale with Ansible - Ishan Jain](/blog/2024/scaling-collectors/) - -The Collector has been a core component for organizations looking to adopt -OpenTelemetry as part of their strategy to improve the telemetry emitted by -their systems. Organizations around the world have already adopted it and -successfully process large amounts of data through pipelines as documented by -these various talks: - -- [Adopting OpenTelemetry Collector @ eBay - Swapping Engines Mid Flight - Vijay Samuel](https://www.youtube.com/watch?v=tZJd6W-CIcU) -- [Ingesting 6.5 Tb of Telemetry Data Daily Through OpenTelemetry Protocol and Collectors - Gustavo Pantuza](https://www.youtube.com/watch?v=aDysORX1zIs) -- [Today, Not Tomorrow: Scalable Strategies for Migrating to OpenTelemetry - Jason Anderson & Kevin Broadbridge](https://www.youtube.com/watch?v=iPGd9_aYu-A) -- [How and Why You Should Adopt and Expose OSS Interfaces Like OTel & Prometheus - Daniel Hrabovcak & Shishi Chen](https://www.youtube.com/watch?v=D71fK2MFreI) -- [Why, How to, and Issues: Tail-Based Sampling in the OpenTelemetry Collector - Reese Lee](https://www.youtube.com/watch?v=l4PeclHKl7I) - -A few months ago, there was an -[ask from the community](https://github.com/open-telemetry/community/issues/1971) -to declare the OpenTelemetry Collector stable. - -![Can haz Collector v1?](can-haz-collector.png) - -Now you might be asking yourself "Why would anyone want the Collector to be -declared stable? You just told me it's already used in production!" It's true, -the Collector and its configuration have been fairly stable for core components -for some time. However, being "unofficially stable" is not good enough for a -wide variety of organizations who wish to adopt the Collector: - -- An official v1 will signal that the OpenTelemetry community is ready to - provide long term support and not introduce backwards incompatible changes - without bumping the major version. -- Organizations with policies not to use pre-release software will be able to - start adopting the Collector. -- Stability in the Collector helps the community move OpenTelemetry to become a - CNCF Graduated project. - -The request to stabilize was met with pushback from maintainers since calling -anything 1.0 has a way of setting expectations indefinitely. This led to a -series of discussions and meetings that brought together the maintainers of the -Collector to decide on what a 1.0 really means for the Collector. - -And after a lot of back and forth, we decided on a limited scope of what we -wanted to focus on: - -1. A distribution of the Collector that only includes an OTLP receiver and an - OTLP exporter. -2. Individual Go modules that the Collector components rely upon must also be - marked as stable as per the project's - [versioning guidelines](https://github.com/open-telemetry/opentelemetry-collector/blob/main/VERSIONING.md#public-api-expectations). - -Aside from this, there were a few areas the contributors wanted to improve based -on user feedback: - -- The telemetry generated by the Collector about itself: - - Traces, metrics, and logs must be available via OTLP. - - The configuration for the telemetry must follow the configuration schema. -- The scalability of the Collector: - - Handling for queueing, back pressure, and errors must be improved. - - Clear benchmarks and performance expectations for end users. -- Overall documentation befitting of a stable piece of critical infrastructure. - -The -[roadmap](https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/ga-roadmap.md) -was published in the Collector's repository and milestones were created to track -the work underway. To ensure the effort can be successful, the scope of the -deliverable was limited to provide: - -- a clear and achievable goal -- the focus needed to not get distracted -- a signal to new contributors of where the project is focusing - -There is much to do as you can see on the -[project board](https://github.com/orgs/open-telemetry/projects/83), but there -is a lot of excitement around this effort. If you're keen on helping, reach out -either by commenting on any of the open issues in GitHub, or attending the -Collector SIG call that happens weekly on Wednesdays. For a quick overview of -the 1.0 progress you can checkout the tracking -[issue](https://github.com/open-telemetry/opentelemetry-collector/issues/9375). diff --git a/content/en/docs/collector/deployment/gateway.md b/content/en/docs/collector/deployment/gateway.md index 5f0547c8b1f0..fcddb5706697 100644 --- a/content/en/docs/collector/deployment/gateway.md +++ b/content/en/docs/collector/deployment/gateway.md @@ -212,6 +212,47 @@ Cons: - Added latency in case of cascaded collectors - Higher overall resource usage (costs) +## Additional Considerations Deploying Multiple Collectors + +### The Single-Writer Principle + +The Single-Writer Principle is a design principle that ensures a single logical writer for a particular resource. +Concurrent access from multiple applications that modify or report on the same data can lead to data +loss or, at least, degraded data quality. In gateway collector deployments, applying this principle guards against sending +inconsistent data to the backend. + +#### Context in OTLP + +All metric data streams within OTLP must have a [single writer](https://opentelemetry.io/docs/specs/otel/metrics/data-model/#single-writer) +This is because gateway collector deployments can involve multiple collectors in the same system. +It is possible in this case for instances to receive and process data from the same resources, +which is a violation of the Single-Writer principle. A potential consequence of this may be that +these data streams overwrite each other, leading to inconsistent time series data. + +#### Example Scenario + +Consider the following configuration: + +There is a gateway deployment configured to handle all traffic for three other collectors in the same system. +If the collectors are not uniquely identified and the SDK fails to distinguish between them, they may +send identical data to the gateway collector from different points in time. In this scenario, +it would be possible to observe inconsistent values for the same series. + + +#### Detection + +There are patterns in the data that may provide some insight into whether this is happening or not. +For example, upon visual inspection, a series with unexplained gaps or jumps in the same series may be a clue that +multiple collectors are sending the same samples. Unexplained behavior in a time series could potentially +point to the backend scraping data from multiple sources. + + +#### Prevention + +All metric streams produced by OTel SDKs should have a globally unique [Metric Identity](https://opentelemetry.io/docs/specs/otel/metrics/data-model/#opentelemetry-protocol-data-model-producer-recommendations). +This is to lower the risk of duplication, and ensure writers are sending unique data to the backend. + + [lb-exporter]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/loadbalancingexporter [tailsample-processor]: diff --git a/content/en/docs/concepts/signals/traces.md b/content/en/docs/concepts/signals/traces.md index 1ea0b8bc4e00..ca39a029fc69 100644 --- a/content/en/docs/concepts/signals/traces.md +++ b/content/en/docs/concepts/signals/traces.md @@ -118,7 +118,7 @@ that matches the `span_id` of the `hello` span. ``` This span represents the third operation in this trace and, like the previous -one, it's a child of the `hello` span. That also makes it a sibling of the +one, it's a child of the 'hello' Span. That also makes it a sibling of the `hello-greetings` span. These three blocks of JSON all share the same `trace_id`, and the `parent_id` diff --git a/content/en/docs/languages/java/automatic/spring-boot.md b/content/en/docs/languages/java/automatic/spring-boot.md index b63ac5f20c73..e8c9d6cd3291 100644 --- a/content/en/docs/languages/java/automatic/spring-boot.md +++ b/content/en/docs/languages/java/automatic/spring-boot.md @@ -137,10 +137,10 @@ The OpenTelemetry starter uses OpenTelemetry Spring Boot ```xml - - io.opentelemetry.instrumentation - opentelemetry-spring-boot-starter - + + io.opentelemetry.instrumentation + opentelemetry-spring-boot-starter + ``` @@ -253,11 +253,11 @@ from tracing: ```xml - - io.opentelemetry.contrib - opentelemetry-samplers + + io.opentelemetry.contrib + opentelemetry-samplers 1.33.0-alpha - + ``` @@ -450,26 +450,26 @@ appender in your `logback.xml` or `logback-spring.xml` file: ```xml - - - - %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n - - - - - false - true - true - true - true - * - - - - - + + + + %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n + + + + + false + true + true + true + true + * + + + + + ``` @@ -593,10 +593,10 @@ With the datasource configuration, you need to add the following dependency: ```xml - - io.opentelemetry.instrumentation - opentelemetry-jdbc - + + io.opentelemetry.instrumentation + opentelemetry-jdbc + ``` @@ -617,14 +617,14 @@ You have to add the OpenTelemetry appender to your `log4j2.xml` file: ```xml - - - - - - - - + + + + + + + + ``` diff --git a/data/registry/collector-exporter-azure-monitor.yml b/data/registry/collector-exporter-azure-monitor.yml index d0a8e934c661..89d808161ec0 100644 --- a/data/registry/collector-exporter-azure-monitor.yml +++ b/data/registry/collector-exporter-azure-monitor.yml @@ -9,7 +9,7 @@ tags: license: Apache 2.0 description: The Azure Monitor Exporter for the OpenTelemetry Collector. authors: - - name: OpenTelemetry Authors + - name: Microsoft urls: repo: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/azuremonitorexporter createdAt: 2020-06-06 diff --git a/data/registry/exporter-js-instana.yml b/data/registry/exporter-js-instana.yml index ee653012cfe2..245a4f567c8f 100644 --- a/data/registry/exporter-js-instana.yml +++ b/data/registry/exporter-js-instana.yml @@ -15,4 +15,4 @@ createdAt: 2022-04-18 package: registry: npm name: '@instana/opentelemetry-exporter' - version: 3.7.0 + version: 3.6.0 diff --git a/data/registry/instrumentation-php-slim.yml b/data/registry/instrumentation-php-slim.yml index bf2eb88bfabe..e5b1b9001b30 100644 --- a/data/registry/instrumentation-php-slim.yml +++ b/data/registry/instrumentation-php-slim.yml @@ -15,4 +15,4 @@ createdAt: 2022-12-14 package: registry: packagist name: open-telemetry/opentelemetry-auto-slim - version: 1.0.5 + version: 1.0.4 diff --git a/static/refcache.json b/static/refcache.json index 765f2278208e..dbc1be1a4f11 100644 --- a/static/refcache.json +++ b/static/refcache.json @@ -2951,10 +2951,6 @@ "StatusCode": 200, "LastSeen": "2024-03-19T10:16:37.692028682Z" }, - "https://github.com/open-telemetry/community/issues/1971": { - "StatusCode": 200, - "LastSeen": "2024-05-03T07:21:04.501478-07:00" - }, "https://github.com/open-telemetry/community/issues/828": { "StatusCode": 200, "LastSeen": "2024-01-18T19:37:16.771654-05:00" @@ -3127,10 +3123,6 @@ "StatusCode": 200, "LastSeen": "2024-04-04T11:07:15.276911438-07:00" }, - "https://github.com/open-telemetry/opentelemetry-collector/issues/9375": { - "StatusCode": 200, - "LastSeen": "2024-05-03T08:34:29.3339-07:00" - }, "https://github.com/open-telemetry/opentelemetry-collector/pull/6140": { "StatusCode": 200, "LastSeen": "2024-01-30T05:18:24.402543-05:00" @@ -4183,10 +4175,6 @@ "StatusCode": 200, "LastSeen": "2024-01-30T05:18:13.065627-05:00" }, - "https://github.com/orgs/open-telemetry/projects/83": { - "StatusCode": 200, - "LastSeen": "2024-05-03T07:21:05.157831-07:00" - }, "https://github.com/orgs/open-telemetry/teams/technical-committee": { "StatusCode": 200, "LastSeen": "2024-01-30T16:15:03.796394-05:00" @@ -5819,14 +5807,6 @@ "StatusCode": 206, "LastSeen": "2024-03-19T10:16:59.755536357Z" }, - "https://opentelemetry.io/blog/2024/otel-collector-anti-patterns/": { - "StatusCode": 206, - "LastSeen": "2024-05-06T07:53:28.679391-07:00" - }, - "https://opentelemetry.io/blog/2024/scaling-collectors/": { - "StatusCode": 206, - "LastSeen": "2024-05-06T07:53:28.903161-07:00" - }, "https://opentelemetry.io/docs/collector": { "StatusCode": 206, "LastSeen": "2024-02-23T22:55:03.656226-05:00"