Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pipelineImpl: de-pointerize plugin interface fields #113

Merged

Conversation

tzaffi
Copy link
Contributor

@tzaffi tzaffi commented Jun 30, 2023

Summary

Non-breaking changes:

  • Modifying the plugin fields in the pipelineImpl struct to be "concrete" interface types.

Quasi-breaking changes:

  • Rename *Builder variables and types to be *Constructor
  • Rename Constructor interface which was used for constructing Importers to be ImporterConstructor in line with other such plugin related types.

Test Plan

CI

Testing TODO's

  • run locally with the block generator and see that it still works

Results of the local block generator run

It's working just as before. Below is a screen shot comparing the run's report with what's in the database. Additionally, the conduit-log had no errors.

image

@codecov
Copy link

codecov bot commented Jun 30, 2023

Codecov Report

Merging #113 (d1f9615) into master (442791a) will increase coverage by 2.72%.
The diff coverage is 76.26%.

@@            Coverage Diff             @@
##           master     #113      +/-   ##
==========================================
+ Coverage   67.66%   70.38%   +2.72%     
==========================================
  Files          32       37       +5     
  Lines        1976     2509     +533     
==========================================
+ Hits         1337     1766     +429     
- Misses        570      648      +78     
- Partials       69       95      +26     
Impacted Files Coverage Δ
conduit/data/block_export_data.go 100.00% <ø> (+92.30%) ⬆️
conduit/metrics/metrics.go 100.00% <ø> (ø)
conduit/pipeline/metadata.go 69.11% <ø> (ø)
...duit/plugins/exporters/filewriter/file_exporter.go 81.63% <ø> (-1.06%) ⬇️
conduit/plugins/exporters/postgresql/util/prune.go 78.43% <ø> (ø)
conduit/plugins/importers/algod/metrics.go 100.00% <ø> (ø)
...ins/processors/filterprocessor/filter_processor.go 83.82% <ø> (+3.54%) ⬆️
...plugins/processors/filterprocessor/gen/generate.go 34.28% <ø> (ø)
conduit/plugins/processors/noop/noop_processor.go 64.70% <ø> (+6.81%) ⬆️
pkg/cli/internal/list/list.go 20.75% <ø> (ø)
... and 16 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@tzaffi tzaffi requested a review from a team June 30, 2023 15:49
@@ -306,11 +306,11 @@ func (p *pipelineImpl) Init() error {
if err != nil {
return fmt.Errorf("Pipeline.Init(): could not make %s config: %w", p.cfg.Importer.Name, err)
}
err = (*p.importer).Init(p.ctx, *p.initProvider, pluginConfig, importerLogger)
err = (p.importer).Init(p.ctx, *p.initProvider, pluginConfig, importerLogger)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the parens still needed?

Suggested change
err = (p.importer).Init(p.ctx, *p.initProvider, pluginConfig, importerLogger)
err = p.importer.Init(p.ctx, *p.initProvider, pluginConfig, importerLogger)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conduit/pipeline/pipeline.go Outdated Show resolved Hide resolved
@@ -574,8 +696,7 @@ func MakePipeline(ctx context.Context, cfg *data.Config, logger *log.Logger) (Pi
return nil, fmt.Errorf("MakePipeline(): could not build importer '%s': %w", importerName, err)
}

importer := importerBuilder.New()
pipeline.importer = &importer
pipeline.importer = importerBuilder.New()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change and the ones below in this file aren't needed, but seem ok.

Copy link
Contributor Author

@tzaffi tzaffi Jun 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, partially they're needed, but we don't really need to remove the extra variable assignment as I'm doing here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why this variable is named "Builder". This is the "Constructor" and "New" is the single function defined in the constructor interface. The function above should be importers.ImporterConstructorByName.

What you have here is correct, previously it had to be assigned separately so that we could get the pointer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made changes in this commit:

d1f9615

However, now the PR has evolved from a non-breaking change to a quasi-breaking change since it's possible that someone can attempt to use the renamed interfaces explicitly and their code will reference these no longer existing types.

LMK

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Up to you, I don't feel very strongly. Was just calling out a possible reason for the discrepancy because you mentioned it in your top comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went ahead and merged. We should discuss how to communicate this for our next release.

conduit/pipeline/pipeline.go Outdated Show resolved Hide resolved
conduit/pipeline/pipeline.go Outdated Show resolved Hide resolved
@tzaffi tzaffi marked this pull request as ready for review June 30, 2023 16:03
@tzaffi tzaffi requested a review from a team June 30, 2023 18:42
conduit/pipeline/pipeline.go Outdated Show resolved Hide resolved
conduit/pipeline/pipeline.go Outdated Show resolved Hide resolved
@@ -574,8 +696,7 @@ func MakePipeline(ctx context.Context, cfg *data.Config, logger *log.Logger) (Pi
return nil, fmt.Errorf("MakePipeline(): could not build importer '%s': %w", importerName, err)
}

importer := importerBuilder.New()
pipeline.importer = &importer
pipeline.importer = importerBuilder.New()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why this variable is named "Builder". This is the "Constructor" and "New" is the single function defined in the constructor interface. The function above should be importers.ImporterConstructorByName.

What you have here is correct, previously it had to be assigned separately so that we could get the pointer.

tzaffi and others added 3 commits June 30, 2023 14:45
Co-authored-by: Will Winder <wwinder.unh@gmail.com>
Co-authored-by: Will Winder <wwinder.unh@gmail.com>
@tzaffi tzaffi merged commit ba19d62 into algorand:master Jun 30, 2023
3 checks passed
@tzaffi tzaffi deleted the pipeline-impl-depointerize-interface-fields branch June 30, 2023 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants