Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creates the base files for generating go protobufs #1

Merged
merged 6 commits into from
Feb 26, 2021

Conversation

MadVikingGod
Copy link
Contributor

@MadVikingGod MadVikingGod commented Jan 27, 2021

Summary

This PR aims to start the conversation around putting the generated protobuf files into a consumable module.

There are several places where these files are generated but are done internally so that other projects can't consume them. This takes the generation code from opentelemetry-go as a base to generate the protobufs that will eventually be consumed.

Things not addressed

  • Vanity URLs. I have assumed that the use of go.opentelemetry.io/proto. This is easily changed.
  • The actually generated files. These are large and can be added after reviewing the technical parts of generation are complete.
  • Tests, e.g., that the source and the generated files match. Automation of this kind can be added after this proof of concept
  • Notification/Automation, e.g., some signal that this repository is out of date and should be updated.
  • Updating the proto generation tool. Again, this can be addressed after this is shown to be useful.

Resolves #2

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Jan 27, 2021

CLA Signed

The committers are authorized under a signed CLA.

Makefile Outdated Show resolved Hide resolved
Aaron Clawson added 2 commits January 28, 2021 09:43
This simplified generation, and means we require only docker and rsync
@punya
Copy link
Member

punya commented Jan 30, 2021

Did we consider the approach of generating language-specific packages from a Github action in opentelemetry-proto, rather than creating a separate repo that includes the protos as a submodule, needs to be updated, etc.? In the case of Go, the generated artifact would need to live in its own Github repo of course, but that repo would be purely for automated updates.

@MadVikingGod
Copy link
Contributor Author

My understanding is that the maintainers of opentelemetry-proto do not want to support N-language generation in that repository. It is up to the individual language SIGs to support this kind of effort.

@MrAlias
Copy link
Contributor

MrAlias commented Feb 2, 2021

Did we consider the approach of generating language-specific packages from a Github action in opentelemetry-proto, rather than creating a separate repo that includes the protos as a submodule, needs to be updated, etc.? In the case of Go, the generated artifact would need to live in its own Github repo of course, but that repo would be purely for automated updates.

@punya yes we have discussed this. Please see open-telemetry/opentelemetry-proto#79 for context.

go.mod Show resolved Hide resolved
The final import is now going to be like `go.opentelemetry.io/proto/otlp/trace/v1`
Copy link
Contributor

@MrAlias MrAlias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we checkout the proto submodule into an internal package so external dependencies on that code are made here?

README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
@MadVikingGod
Copy link
Contributor Author

Fixed the readme. What is the ask with the proto files?

Do you want the submodule which holds the .proto files in its own module?
Is this about the code that exists in gen/? I wasn't expecting to check that in, but it could be generated in an internal directory or module just to make sure its dependencies aren't accidentally checked into the root go.mod.

@MrAlias
Copy link
Contributor

MrAlias commented Feb 11, 2021

What is the ask with the proto files?

Do you want the submodule which holds the .proto files in its own module?
Is this about the code that exists in gen/? I wasn't expecting to check that in, but it could be generated in an internal directory or module just to make sure its dependencies aren't accidentally checked into the root go.mod.

Looking a bit more into the ask I'm not sure it is valid, but here's a description of what I was thinking. By creating the opentelemetry-proto submodule at a top-level users will be able to import go.opentelemetry.io/proto/opentelemetry-proto/....

I took another look at the proto repo and it doesn't look like there is any Go code in the repo anymore so this is probably not a concern. However, it might make things a bit cleaner if instead of instantiating the opentelemetry-proto submodule at a path of ./opentelemetry-proto we create it at ./internal/opentelemetry-proto so there isn't any confusion in the future if that repo starts to contain Go code (probably overkill, so feel free to disregard).

@MadVikingGod
Copy link
Contributor Author

Just to add to that, even if there were go code its mod file wouldn't match that directory, and you would get an import mismatch.

Playing around I moved opentelemetry-proto and gen into a subfolder, I called proto and added a mod with a non-canonical name so it couldn't be imported. So we then had proto\opentelemetry-proto and proto\gen. This made the structure a bit more obvious, but I also don't like the name proto for this purpose.

I don't think it matters for this, as this is all non-imported or persisted code. I also think that we would be better served by a CI check that the root go.mod doesn't change. I.E. there shouldn't be anything imported by the root level, as there is no code.

@MrAlias
Copy link
Contributor

MrAlias commented Feb 12, 2021

All good points and ideas. Sounds like we can disregard my comment about moving the path. I think we can include that check you described at a later time as well so as to not have the scope of this PR creep too much.

@MrAlias
Copy link
Contributor

MrAlias commented Feb 12, 2021

@bogdandrutu any other thoughts before we merge this?

@bogdandrutu bogdandrutu merged commit 20b11b7 into open-telemetry:main Feb 26, 2021
@MadVikingGod MadVikingGod deleted the code-generation branch March 4, 2021 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create minimal code generation tooling
4 participants