Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add validate interface in configmodels to force each component do configuration validation #2802

Merged
merged 1 commit into from
Mar 30, 2021

Conversation

mxiamxia
Copy link
Member

@mxiamxia mxiamxia commented Mar 25, 2021

Description:
The goal is to add Validate() interface in config.Receiver|Exporter|Processor|Extension so we can force each component to add validation logic on component level configuration. Only have the changes for config.Receiver in this PR.

Added CustomConfigOptions interface in config.Receiver. And CustomConfigOptions interface has embedded CustomConfigValidator interface so it can force each receiver component to implement Validate() for Config validation check. Also, CustomConfigOptions interface can be extended to add more CustomConfig options.

The reason why we should have Validate() for each component.
Currently, since we don't have a unified fashion for components to validate the configuration. The component contributors write the config validate logic in different places in the component code.
For example. in prometheusreceiver the cfg validate logic is coded in createMetricsReceiver. and in awsemfexporter it was written in here.
By doing the current implementation, all the contributors will follow the same fashion for the configuration validation. And we can fail the collector in the start time with friendly error messages.

type Receiver interface {
	NamedEntity
	CustomConfigOptions
}
// CustomConfigOptions defines the interfaces for configuration customization options
// Different custom interfaces can be added like CustomConfigValidator, CustomConfigUnmarshaller, etc.
type CustomConfigOptions interface {
	CustomConfigValidator
}
// CustomConfigValidator defines the interface for the custom configuration validation on each component
type CustomConfigValidator interface {
	Validate() error
}

Custom Validation Example for prometheusreceiver,

// Validate implements the custom validation check on configuration
func (cfg *Config) Validate() error {
	if cfg.PrometheusConfig == nil || len(cfg.PrometheusConfig.ScrapeConfigs) == 0 {
		return errNilScrapeConfig
	}
	return nil
}

Need help to review this approach before moving to the others (exporter, processor, extension)

Link to tracking Issue:
#2541
#2597

Testing:
WIP

Documentation:
N/A

@mxiamxia mxiamxia requested a review from a team March 25, 2021 17:15
config/config.go Outdated Show resolved Hide resolved
config/configmodels/receiver.go Outdated Show resolved Hide resolved
config/configmodels/receiver.go Outdated Show resolved Hide resolved
receiver/opencensusreceiver/config.go Show resolved Hide resolved
Copy link
Member

@bogdandrutu bogdandrutu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way how I see this, based on the discussion in the other comment, I think we should have (in current configmodels):

package configmodels

// Processor is the configuration of a processor. Specific processors must implement this
// interface and will typically embed ProcessorSettings struct or a struct that extends it.
type Processor interface {
	NamedEntity

	Validate() errror  // and for all interfaces Exporter, etc.
}

// In the unmarshaler PR, optional interface for configs
type Unmarshaler interface {
	Unmarshal(componentParser *config.Parser, intoCfg interface{}) error
}

type UnmarshalFunc func(componentParser *config.Parser, intoCfg interface{}) error

@mx-psi you may need #2808 before in order to avoid circular deps.

config/config.go Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Mar 30, 2021

Codecov Report

Merging #2802 (336277a) into main (2dbc66c) will increase coverage by 0.00%.
The diff coverage is 90.47%.

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #2802   +/-   ##
=======================================
  Coverage   91.76%   91.77%           
=======================================
  Files         281      286    +5     
  Lines       15149    15163   +14     
=======================================
+ Hits        13902    13916   +14     
  Misses        853      853           
  Partials      394      394           
Impacted Files Coverage Δ
config/receiver.go 100.00% <ø> (ø)
receiver/prometheusreceiver/factory.go 81.25% <ø> (-1.11%) ⬇️
receiver/opencensusreceiver/config.go 80.00% <50.00%> (+2.22%) ⬆️
component/componenttest/nop_receiver.go 100.00% <100.00%> (ø)
config/config.go 100.00% <100.00%> (ø)
internal/testcomponents/example_receiver.go 100.00% <100.00%> (ø)
receiver/hostmetricsreceiver/config.go 100.00% <100.00%> (ø)
receiver/jaegerreceiver/config.go 100.00% <100.00%> (ø)
receiver/kafkareceiver/config.go 100.00% <100.00%> (ø)
receiver/otlpreceiver/config.go 100.00% <100.00%> (ø)
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2dbc66c...336277a. Read the comment docs.

@mxiamxia
Copy link
Member Author

The way how I see this, based on the discussion in the other comment, I think we should have (in current configmodels):

package configmodels

// Processor is the configuration of a processor. Specific processors must implement this
// interface and will typically embed ProcessorSettings struct or a struct that extends it.
type Processor interface {
	NamedEntity

	Validate() errror  // and for all interfaces Exporter, etc.
}

// In the unmarshaler PR, optional interface for configs
type Unmarshaler interface {
	Unmarshal(componentParser *config.Parser, intoCfg interface{}) error
}

type UnmarshalFunc func(componentParser *config.Parser, intoCfg interface{}) error

@mx-psi you may need #2808 before in order to avoid circular deps.

Yep, made the changes for it. Thanks.

@mxiamxia mxiamxia changed the title [Draft] - Add validate interface in configmodels to force each component do configuration validation Add validate interface in configmodels to force each component do configuration validation Mar 30, 2021
Copy link
Member

@mx-psi mx-psi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The design makes sense to me and I would say we can apply it to all component types (if Bogdan gives his 👍).

config/receiver.go Outdated Show resolved Hide resolved
receiver/hostmetricsreceiver/config.go Outdated Show resolved Hide resolved
config/receiver.go Outdated Show resolved Hide resolved
Copy link
Member

@bogdandrutu bogdandrutu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge as first draft, I will followup with some cleanups.

"testing"

"github.com/stretchr/testify/assert"
)

type NopConfig struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to be public.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll move

Let's merge as first draft, I will followup with some cleanups.

@bogdandrutu , I am going to expand the same checks to all the other type of components in the next PR. Sounds good to you?

@bogdandrutu bogdandrutu merged commit 2c17034 into open-telemetry:main Mar 30, 2021
pjanotti pushed a commit to pjanotti/opentelemetry-service that referenced this pull request Apr 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants