Bump otel dependencies to v0.115.0/v1.21.0 #1485

dricross · 2025-01-06T15:13:03Z

Description of the issue

We need to track and pull in changes from upstream periodically.

Major changes to lookout for:

receiver.CreateSettings renamed to receiver.Settings: Bump otel dependencies to v0.115.0/v1.21.0 #1485 (comment). Many changes like this
pipelines.Config now requires a pipelines.ID rather than component.ID
- Change the Translator / TranslatorMap generic functions to also accept another type, ID
- For component translators, this ID is component.ID. For pipeline translators, this ID is pipeline.ID.
- Component translators are then the concrete type Translator[component.Config, component.ID], which mean “given a config, generate a component.Config which is identified by a component.ID”
- Pipeline translators are the concrete type Translator[*ComponentTranslators, pipeline.ID], which means “given a config, generate a ComponentTranslator (a loose definition of an otel pipeline) which is identified by a pipeline.ID”
Upstream update to TracesConfig adds new field: open-telemetry/opentelemetry-collector@b33913b
- Generated otel.Config now has a new field under Service.Telemetry.Level which is set to Basic by default
- Add level: Basic to basically all the expected yaml files
Otel config decoder no longer supports "weakly typed" values by default
- no longer accepts “true” for local_mode setting. Must be true
- no longer accepts [] for external_labels setting. Must be {}
- This is due to a upstream change: open-telemetry/opentelemetry-collector@e477c3a
- Previously, the “weak typing” was enforced so the decoder would know to convert a string to a boolean. With “strong typing” enforced by default, the decoder will no longer accept it
New defaults for clientConfig
- open-telemetry/opentelemetry-collector@4d0b088
- fields don’t appear to be used by component directly. Used by Go’s http.Transport
- The values are pulled from go’s http.DefaultTransport
  - ReadBufferSize: previously 0, which leaves transport.ReadBufferSize as the default, which is the same thing it’s doing now
  - WriteBufferSize: previously 0, which leaves transport.WriteBufferSize as the default, which is the same thing it’s doing now
  - Headers: previously nil, now empty map. No difference: https://github.com/open-telemetry/opentelemetry-collector/blob/config/confighttp/v0.116.0/config/confighttp/confighttp.go#L216
  - MaxIdleConnsPerHost: previously nil, whichleaves transport.MaxIdleConnsPerHost as the default, which is the same thing it’s doing now
  - MaxConnsPerHost: previously nil, which leaves transport.WriteBufferSize as the default, which is the same thing it’s doing now
  - IdleConnTimeout: previously nil, which leaves transport.WriteBufferSize as the default, which is the same thing it’s doing now
New defaults for server config (similar to clientConfig)
- open-telemetry/opentelemetry-collector@0e14c22
- These new fields set the http.Server settings: https://github.com/open-telemetry/opentelemetry-collector/blob/0e14c22e45fcef9561768753ccf829f03681d11b/config/confighttp/confighttp.go#L488-L491. The change essentially manually sets the http.Server settings to the defaults. No functional change but it shows up in the config file now
debugexporterhas new field: open-telemetry/opentelemetry-collector@f47b9f2
- Setting to true maintains previous functionality
receiver/awscontainerinsightsreceiver/internal/host:
- newNodeCapacity argument changed to any function but windows tests were not updated: amazon-contributing/opentelemetry-collector-contrib@d5d6147

Description of changes

Merge OTel contrib v0.115.0/v1.21.0. Previous version was v0.103.0

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

Unit tests
contrib github workflows: https://github.com/amazon-contributing/opentelemetry-collector-contrib/actions/runs/12605148285
agent github workflows: https://github.com/aws/amazon-cloudwatch-agent/actions/runs/13207009258

Requirements

Before commit the code, please do the following steps.

Run make fmt and make fmt-sh
Run make lint

github-actions · 2025-01-17T00:11:24Z

This PR was marked stale due to lack of activity.

….115.0

lisguo · 2025-02-10T21:24:11Z

service/defaultcomponents/components.go

@@ -135,7 +134,6 @@ func Factories() (otelcol.Factories, error) {
 		awsproxy.NewFactory(),
 		entitystore.NewFactory(),
 		server.NewFactory(),
-		ballastextension.NewFactory(),


does this no longer exist anymore with the upgrade? Seems odd -- I assume customers could be using this as part of their adot config?

yeah it was deprecated in v0.92.0 of open-telemetry-collector-contrib and then completely removed in v0.108.0. It was added for adot parity. confirmed with @jefchien we should be able to remove it

lisguo · 2025-02-10T21:26:46Z

translator/tocwconfig/sampleConfig/jmx_config_linux.yaml

@@ -22,7 +22,11 @@ exporters:
            enabled: false
        http2_ping_timeout: 0s
        http2_read_idle_timeout: 0s
+        idle_conn_timeout: 1m30s


seems like the default is 30s, why is this 1m30s?

https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/d597217aa3abcb34786359d8e1d5ece05e965992/exporter/signalfxexporter/README.md?plain=1#L94

I believe the values come from http.DefaultTransport which is 90s: https://github.com/golang/go/blob/master/src/net/http/transport.go#L53. Here's the commit where this was changed: open-telemetry/opentelemetry-collector@4d0b088

lisguo · 2025-02-11T19:54:15Z

translator/tocwconfig/sampleConfig/amp_config_linux.yaml

@@ -28,7 +28,11 @@ exporters:
            enabled: false
        http2_ping_timeout: 0s
        http2_read_idle_timeout: 0s
+        idle_conn_timeout: 1m30s


is this the default? I presume this shouldn't break anything

It is the default, yeah. Upstream is being more explicit about setting the clientConfig default values to http.DefaultTransport which shows up in our config files. See open-telemetry/opentelemetry-collector@4d0b088

lisguo · 2025-02-11T20:38:37Z

pr build is failing

jefchien · 2025-02-11T20:10:02Z

.github/workflows/test-build-docker.yml

@@ -134,6 +134,7 @@ jobs:
      - uses: actions/checkout@v3
        with:
          repository: ${{env.CWA_GITHUB_TEST_REPO_NAME}}
+          ref: ab30cf84d0254506c92502ce185d25dd72701c81


What is this for?

IIRC correctly, there was a problem with the main branch of the test repo and I needed to pull a specific commit to get the tests to run. I can undo this

extension/entitystore/extension_test.go

jefchien · 2025-02-11T20:13:11Z

go.mod

 )

+replace github.com/amazon-contributing/opentelemetry-collector-contrib/extension/awsmiddleware => github.com/amazon-contributing/opentelemetry-collector-contrib/extension/awsmiddleware v0.0.0-20250207173954-4e1c56b92f1e


nit: This can just be in the dependency block. There's nothing to replace since v0.115.0 of the middleware extension doesn't exist.

jefchien · 2025-02-11T20:13:27Z

go.mod

 	github.com/gin-gonic/gin v1.10.0
 	github.com/go-kit/log v0.2.1
 	github.com/go-playground/validator/v10 v10.20.0
 	github.com/go-test/deep v1.0.2-0.20181118220953-042da051cf31
 	github.com/gobwas/glob v0.2.3
 	github.com/google/go-cmp v0.6.0
 	github.com/google/uuid v1.6.0
-	github.com/grafana/regexp v0.0.0-20221122212121-6b5c0a4cb7fd // indirect
+	github.com/grafana/regexp v0.0.0-20240518133315-a468a5bfb3bc // indirect


nit: Move to indirect block.

jefchien · 2025-02-11T21:45:29Z

receiver/adapter/factory.go

@@ -62,11 +63,10 @@ func (a Adapter) createMetricsReceiver(ctx context.Context, settings receiver.Cr

 	rcvr := newAdaptedReceiver(input, ctx, consumer, settings.Logger)

-	scraper, err := scraperhelper.NewScraper(
-		settings.ID.Type().String(),
+	scraper, err := scraper.NewMetrics(


nit: scraper variable shadows scraper package.

jefchien · 2025-02-11T21:45:48Z

service/defaultcomponents/components_test.go

@@ -47,7 +47,7 @@ func TestComponents(t *testing.T) {
 		"cumulativetodelta",
 		"deltatorate",
 		"ec2tagger",
-		"experimental_metricsgeneration",
+		"metricsgeneration",


Nice that it's no longer experimental

jefchien · 2025-02-11T21:46:23Z

translator/tocwconfig/sampleConfig/advanced_config_darwin.yaml

+        traces:
+            level: Basic


Can we change the translator so this is None?

Sure, I'll take a look at that

jefchien · 2025-02-11T21:46:49Z

translator/tocwconfig/sampleConfig/appsignals_and_eks_config.yaml

@@ -294,7 +294,7 @@ extensions:
        certificate_file_path: ""
        dialer:
            timeout: 0s
-        endpoint: 0.0.0.0:2000
+        endpoint: localhost:2000


This should not have changed. We'll need to set a default in the configuration.

jefchien · 2025-02-11T22:37:27Z

translator/translate/otel/common/common.go

+	Receivers  TranslatorMap[component.Config, component.ID]
+	Processors TranslatorMap[component.Config, component.ID]
+	Exporters  TranslatorMap[component.Config, component.ID]
+	Extensions TranslatorMap[component.Config, component.ID]


nit: Could all be ComponentTranslatorMaps.

jefchien · 2025-02-11T22:38:07Z

translator/translate/otel/common/common.go

+type ID interface {
+	String() string
+}


nit: This is the exact same interface as fmt.Stringer. Just use that instead.

jefchien · 2025-02-11T22:38:54Z

translator/translate/otel/common/common.go

 	return t.list.Len()
 }

 // NewTranslatorMap creates a TranslatorMap from the translators.
-func NewTranslatorMap[C any](translators ...Translator[C]) TranslatorMap[C] {
-	t := translatorMap[C]{
+func NewTranslatorMap[C any, ID TranslatorID](translators ...Translator[C, ID]) translatorMap[C, ID] {


Suggested change

func NewTranslatorMap[C any, ID TranslatorID](translators ...Translator[C, ID]) translatorMap[C, ID] {

func NewTranslatorMap[C any, ID TranslatorID](translators ...Translator[C, ID]) TranslatorMap[C, ID] {

jefchien · 2025-02-11T22:45:02Z

translator/translate/otel/extension/agenthealth/translator.go


-func NewTranslatorWithStatusCode(name component.DataType, operations []string, isStatusCodeEnabled bool) common.Translator[component.Config] {
+func NewTranslatorWithStatusCode(name Name, operations []string, isStatusCodeEnabled bool) common.ComponentTranslator {


Why can't we just pass in the equivalent of the component.DataType, which in this case would be pipeline.Signal?

jefchien · 2025-02-11T22:45:47Z

translator/translate/otel/extension/agenthealth/translator.go

+	MetricsName    = Name(pipeline.SignalMetrics.String())
+	LogsName       = Name(pipeline.SignalLogs.String())
+	TracesName     = Name(pipeline.SignalTraces.String())
+	StatusCodeName = Name("statuscode")


What is this for?

jefchien · 2025-02-11T22:46:51Z

translator/translate/otel/common/common.go

 }

+// ComponentTranslator is a translator that


nit: Incomplete comment

jefchien · 2025-02-11T22:47:13Z

translator/translate/otel/common/common.go

+type ComponentTranslator = Translator[component.Config, component.ID]
+type ComponentTranslatorMap = TranslatorMap[component.Config, component.ID]
+type PipelineTranslator = Translator[*ComponentTranslators, pipeline.ID]
+type PipelineTranslatorMap = TranslatorMap[*ComponentTranslators, pipeline.ID]


Could likely add convenience functions for NewTranslatorMap as well.

dricross added 14 commits December 19, 2024 15:10

Bump otel

a5fb103

Update plugins

cbb3a50

Update processors

40ab3e2

Update extensions

aa84a05

Update receivers

584a759

Large translator refactor

23bb6f4

Update exporter tests

5a80df0

Update extension tests

bb8e810

Update pipeline tests

fa0ee9b

Update processor tests

ae08afc

Update expected configs

7e0a63c

Point to updated contrib package in github

2e842e2

Attempt at fixing windows docker image

dffb8f1

Update to latest otel-contrib

58af4e7

dricross requested a review from a team as a code owner January 6, 2025 15:13

dricross added 2 commits January 7, 2025 17:24

Update to latest otel-contrib

9b14a10

Update otel-contrib

927a1da

dricross force-pushed the dricross/otel-bump-v0.115.0 branch from 73b1131 to d61236c Compare January 9, 2025 15:56

Update to latest otel contrib

f2ac15b

dricross force-pushed the dricross/otel-bump-v0.115.0 branch from d61236c to f2ac15b Compare January 9, 2025 16:06

github-actions bot added the Stale label Jan 17, 2025

Merge branch 'main' into dricross/otel-bump-v0.115.0

892e031

github-actions bot removed the Stale label Jan 28, 2025

dricross added 3 commits January 30, 2025 10:01

Merge branch 'main' into dricross/otel-bump-v0.115.0

e500798

Upgrade goccy/go-json to v0.10.4

395b4cd

Merge remote-tracking branch 'origin/main' into dricross/otel-bump-v0…

3b6fb00

….115.0

dricross changed the title ~~Dricross/otel bump v0.115.0~~ Bump otel dependencies to v0.115.0/v1.22.0 Feb 2, 2025

dricross changed the title ~~Bump otel dependencies to v0.115.0/v1.22.0~~ Bump otel dependencies to v0.115.0/v1.21.0 Feb 2, 2025

Merge remote-tracking branch 'origin/main' into dricross/otel-bump-v0…

9ba23bd

….115.0

dricross added 3 commits February 6, 2025 15:33

Remove extraneous type declaration

5fc7884

Merge branch 'main' into dricross/otel-bump-v0.115.0

4192c45

bump contrib to latest commit

d074dff

lisguo approved these changes Feb 11, 2025

View reviewed changes

jefchien reviewed Feb 11, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bump otel dependencies to v0.115.0/v1.21.0 #1485

Bump otel dependencies to v0.115.0/v1.21.0 #1485

dricross commented Jan 6, 2025 •

edited

Loading

github-actions bot commented Jan 17, 2025

lisguo Feb 10, 2025

dricross Feb 11, 2025 •

edited by jefchien

Loading

lisguo Feb 10, 2025

dricross Feb 11, 2025

lisguo Feb 11, 2025

dricross Feb 11, 2025

lisguo commented Feb 11, 2025

jefchien Feb 11, 2025

dricross Feb 11, 2025

jefchien Feb 11, 2025

jefchien Feb 11, 2025

jefchien Feb 11, 2025

dricross Feb 11, 2025

jefchien Feb 11, 2025

jefchien Feb 11, 2025

dricross Feb 11, 2025

jefchien Feb 11, 2025

dricross Feb 11, 2025

jefchien Feb 11, 2025

jefchien Feb 11, 2025

jefchien Feb 11, 2025

jefchien Feb 11, 2025

jefchien Feb 11, 2025

jefchien Feb 11, 2025

jefchien Feb 11, 2025

		)

		replace github.com/amazon-contributing/opentelemetry-collector-contrib/extension/awsmiddleware => github.com/amazon-contributing/opentelemetry-collector-contrib/extension/awsmiddleware v0.0.0-20250207173954-4e1c56b92f1e

	func NewTranslatorMap[C any, ID TranslatorID](translators ...Translator[C, ID]) translatorMap[C, ID] {
	func NewTranslatorMap[C any, ID TranslatorID](translators ...Translator[C, ID]) TranslatorMap[C, ID] {


		func NewTranslatorWithStatusCode(name component.DataType, operations []string, isStatusCodeEnabled bool) common.Translator[component.Config] {
		func NewTranslatorWithStatusCode(name Name, operations []string, isStatusCodeEnabled bool) common.ComponentTranslator {

Bump otel dependencies to v0.115.0/v1.21.0 #1485

Are you sure you want to change the base?

Bump otel dependencies to v0.115.0/v1.21.0 #1485

Conversation

dricross commented Jan 6, 2025 • edited Loading

Description of the issue

Description of changes

License

Tests

Requirements

github-actions bot commented Jan 17, 2025

Choose a reason for hiding this comment

dricross Feb 11, 2025 • edited by jefchien Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lisguo commented Feb 11, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dricross commented Jan 6, 2025 •

edited

Loading

dricross Feb 11, 2025 •

edited by jefchien

Loading