Add support for pydantic dataclasses

[SamuelYvon]

Pydantic dataclasses are a drop-in replacement for builtin dataclasses
to offer a stronger type validation. This PR adds support for
generating pydantic dataclasses instead of the builtin variant.

This enable usage of protobuf classes in API endpoint (like FastAPI)
which supports parsing pydantic objects but not builtin dataclasses.

[SamuelYvon]

I am pitching this as an additional feature, feel free to let me know what you think. I will maintain my own fork otherwise; I do feel like this is a trivial improvement to implement.

edit
It was rather foolish to think it was trivial, it is not;

[SamuelYvon]

I'll put as draft for now. Some issues to note:

• Does not play well with oneOf
• ForwardRefs need to be updated

Right now the imports are done through the template, but it might be cleaner to do the pydantic imports through the builtin import system.

ForwardRefs are an easy fix, oneOf can be partially achieved through validators and overriding __post_init__ to set to None the fields that aren't set (otherwise a validation error from pydantic occurs)

[SamuelYvon]

Plan for the oneOf (with pydantic dataclasses):

• Set each field part of a group to Optional
• Set the message_field optional attribute to True
• Add a pydantic validator (for the entire dataclass) that checks that at least one of the field is defined

    @root_validator()
    def check_oneof(cls, values):
        meta = cls._betterproto_meta.oneof_field_by_group

        for group, field_set in meta.items():
            if not any([values[field.name] is not None for field in field_set]):
                raise ValueError(f"Group {group} has no value; all fields are None")

        return values

[SamuelYvon]

... that checks that at least one of the field is defined

Changed the validator to ensure that exactly one field is defined.

[SamuelYvon]

Further tasks

• Only add the call to update the forward refs if the message has other messages as fields
• Add tests

[SamuelYvon]

Hey @danielgtaylor. I've completed these changes a while back. This code has been in production at Garda World for well over two months now without any specific incident. I just completed the unit test portion for windows (generation part). Let me know if this is something you are interested in having in the mainline version of python-betterproto. I did not add any specific test to protobuf but instead relied on use-cases; all previous tests were also ported so they are running on the protobuf version (insuring no regression)

[cetanu]

Looks fine to me and seems like a safe addition, since the generation is simply controlled by an option.

Probably want to hear from another member to see what they think

[Kludex]

Hi there 👋

Anything I can do to help here? 🙏

[Gobot1234]

I'll try and take a look at this over the weekend

[Gobot1234]

Looks good so far, it'd be nice if there was some documentation for this in the README.md instead of having to source dive for this

[Kludex]

@SamuelYvon are you still interested in this? If not, I can take over.

[SamuelYvon]

@Kludex Ill finish the requested changes today :) We are actively using this at Garda so I have allocated time to finish this feature.

[Kludex]

Thinking a bit ahead... Would there be a way to create validators and not lose them when regenerating again?

[SamuelYvon]

Thinking a bit ahead... Would there be a way to create validators and not lose them when regenerating again?

Ah! Great question. I personally think this is out of scope for a project like this, since the goal is to translate protobuf to python. The validation is enforced by the underlying protobuf model since this is a 1:1 translation (mainly to enforce types and groups). If I had other business logic to validate, I would add methods dynamically in a __init__.py file to perform the pydantic validation.

[SamuelYvon]

@Gobot1234 Fixed all suggestions and added text in the README. Double checking that everything still works since I merged w/master.

[Gobot1234]

Looks pretty much ready to ship

[SamuelYvon]

@Gobot1234 All is good on my side; the merge w/master did not affect anything. If this ever need maintenance or issues pop-up, feel free to @ me and I'll be right there.

[Gobot1234]

Thanks for all your work on this