Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate structs from schema #159

Merged
merged 20 commits into from
Jun 13, 2022

Conversation

pedromss
Copy link
Contributor

@pedromss pedromss commented Jun 10, 2022

@nrwiersma here's a sample of what the generation feature could be, let me know what you think.

Considerations:

  • I avoided touching any existing code, so there might be somethings that are are better placed elsewhere or already exist
  • RefSchema isn't handled yet, understanding it first
  • Pulled in dst for some comodity when printing

If the core looks ok, we could discuss which options the generator should support (other tags like json or different casing for example) and where to place it. Usually code generation runs as part of my build phase, so I'd expect a command line utility to be the interface for this feature, open to suggestions.

The generation can be done in a separate repo, but keeping them together might reduce the possibility of a change in the core lib (some change in the Schema interface for example) breaking the generator.

You can run TestGenFromRecordSchema and see the result as that test is currently printing the Go code

TODO:

  • Casing in package name
  • Adding the imports to the generated file
  • Support RefSchema

Future possible improvements

  • Generating functions to register types (I think this one is interesting as it would remove the need to manually call avro.Register)

Thoughts on avro use cases not covered in the test are welcome (aside from the RefSchema)

#119 (comment)

@nrwiersma
Copy link
Member

Wow, this looks awesome. I am hoping to have some time to dig into it this weekend.

To answer one question off the top of my head, I think there should be a cmd but also it should be usable as a lib. I have personally used both types in code, and in the CI when getting the cmd can be a pain and an extra step, using the lib from an internal cmd is much simpler.

pedromss added 2 commits June 10, 2022 19:44
Dropped dave/dst because it was not printing the imports. Tried using the DecoratorWithImports but eventually it was just faster to write a printer than to figure out how to deal with dst and ast.FileSet type.
RefSchemas are just references so we need only extract the type of the field. The actual type declaration will eventually be generated.
@pedromss
Copy link
Contributor Author

Thank you.

After you've looked into this, it looks good, I think the step to move out of draft could be:

  • Figure out which options make sense in GenConf
  • Implement an behaviour required for the options
  • Implement the CLI (would need to know where you would like the cmd as I don't see a cmd folder which is where I'm used to see binaries

If after you've looked into this you feel like its not worth adding this to hamba/avro also no worries, feel free to dismiss :)

I had thought about generating the avro.Register calls, but I think that could come in another PR.

I've never used "internal cmds" I think, but if that means we can defer the CLI to a separate PR, maybe that's better, up to you

Copy link
Member

@nrwiersma nrwiersma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Firstly, awesome work 🎉 . I have added some suggestions and thoughts. In general I think this is on an amazing track.

gen.go Outdated Show resolved Hide resolved
gen.go Outdated Show resolved Hide resolved
gen.go Outdated Show resolved Hide resolved
gen.go Outdated Show resolved Hide resolved
gen.go Outdated Show resolved Hide resolved
@pedromss
Copy link
Contributor Author

Added a few commits to address the comments. Thinking on adding now:

  1. Option to add more tags, like json and their casing
  2. CLI interface

But let me know if you have more things to be addressed first.

@pedromss pedromss requested a review from nrwiersma June 13, 2022 08:21
@pedromss
Copy link
Contributor Author

Noticed the actions failed. I was trying to run make build, test, lint locally but I get Makefile:1: github.com/hamba/make/golang.mk: No such file or directory. I've never used make with include so I'm not sure if I need to install something else for it to be able to resolve the include directive.

@nrwiersma
Copy link
Member

It is using mmake for the include (https://github.com/tj/mmake). Honestly I dont use it so much any more. Think i need to get rid of it.

@pedromss
Copy link
Contributor Author

Installed gofumpt and fixed the failures + warnings in the action, in case you want to run it again

@nrwiersma
Copy link
Member

I would personally be happy to merge this here and do the rest as additional PRs. What do you think?

@pedromss
Copy link
Contributor Author

Sounds good. I'll add a few tests for the failure cases and that should make coveralls happy

@nrwiersma
Copy link
Member

After this merge I will sort out the Makefile and rebase on master.

gen/gen.go Outdated
)

// Conf exposes the options available for the code generation.
type Conf struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nitpick: Could this perhaps be Config rather?

@pedromss
Copy link
Contributor Author

When ready to merge let me know if you prefer I squash all commits into 1

@nrwiersma nrwiersma marked this pull request as ready for review June 13, 2022 11:14
Copy link
Member

@nrwiersma nrwiersma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🎉

@nrwiersma
Copy link
Member

Merging now as is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants