service implementation stub generation

[w4rum]

This implements #19. An example compiler output can be found here.

When compiling protobuf definitions, a service implementation stub is generated along the existing client stub:

class ExampleServiceImplementation(betterproto.ServiceImplementation):
    async def example_unary_unary(
        self, example_string: str, example_integer: int
    ) -> "ExampleResponse":
        raise grpclib.GRPCError(grpclib.const.Status.UNIMPLEMENTED)

    async def __rpc_example_unary_unary(self, stream):
        request = await stream.recv_message()

        request_kwargs = {
            "example_string": request.example_string,
            "example_integer": request.example_integer,
        }

        await self._call_rpc_handler_server_unary(
            self.example_unary_unary,
            stream,
            request_kwargs,
        )

    def __mapping__(self) -> Dict[str, grpclib.const.Handler]:
        return {
            "/example_service.ExampleService/ExampleUnaryUnary": grpclib.const.Handler(
                self.__rpc_example_unary_unary,
                grpclib.const.Cardinality.UNARY_UNARY,
                ExampleRequest,
                ExampleResponse,
            ),
        }

The user is then supposed to create their own class that inherits from and overrides the methods of this generated implementation stub:

class ExampleService(ExampleServiceImplementation):
    async def example_unary_unary(
        self, example_string: str, example_integer: int
    ) -> "ExampleResponse":
        return ExampleResponse(
            example_string=example_string,
            example_integer=example_integer,
        )

Note that the generated implementation stub includes a wrapper for each defined service (__rpc_example_unary_unary). This wrapper and the utility functions in the new abstract base class ServiceImplementation take care of translating grpclib.server.Stream's recv_message and send_message from and to yield and return statements and parameters. I tried to match this to how the generated client stub is used.

Finally, the generated implementation stub can be used as a grpclib service handler because of the __mapping__ function. An example of this is included as a new test case (example_service). I'm not sure if it's good practice to bind ports during tests but I didn't know how else to link the server and client. I also think it's not ideal to test the betterproto server against the betterproto client. Maybe it makes sense to include gRPC in output_reference so that we can create a reference client.

EDIT: Updated gist and snippets to reflect changes made in 207233b.

[w4rum]

Whoops. I seem to have misunderstood the test framework. My test doesn't work. I'll get on fixing that.

[w4rum]

Hmm, I don't know how my test could be incorporated into the existing test framework. My test doesn't have a message or service called Test and renaming my service to Test doesn't help because the generated class is called TestImplementation. I could write a custom test but I don't see how I could still have the test framework compile my protobuf sources then.

[cetanu]

For me, this looks good, but I will wait for others to weigh in.
I think opening a socket during tests is not the worst thing ever, but there could be a proper way to do it.

[w4rum]

I removed __service_name__ and __rpc_methods__. I realized that that was an unnecessary indirection. The __mapping__ method necessary to register the implementation stub as a grpclib handler is now part of the generated class.

I also cut down on some of the duplicate code in the generated class. The code for handling the grpclib stream is now part of the ABC.

The original post has been updated.

It appears that the CI is broken. Correct me if I'm wrong.

[nat-n]

Thanks for picking this up, it'll be good to have a solution.

I still need to properly look at the code generation part when I get some time, but I found a points worth looking at for now.

[w4rum]

On a side note: Why does the CI suddenly work again? This branch does not have the changes from #172. Is it using the action config from master because the PR is targeting master? If so, why did the initial CI run on #172 succeed even though it was targeting master with the old action config?

[nat-n]

On a side note: Why does the CI suddenly work again? This branch does not have changes from #172. Is it using the action config from master because the PR is targeting master? If so, why did the initial CI run on #172 succeed even though it was targeting master with the old action config?

I'm as surprised as you 🤷

[w4rum]

I'm not sure what the issue with the test framework is. Specifically, I don't see why example_service fails but service doesn't. This error occurred after I removed the Test message and named the service Test.

[nat-n]

I'm not sure what the issue with the test framework is. Specifically, I don't see why example_service fails but service doesn't. This error occurred after I removed the Test message and named the service Test.

I think maybe you need to register the service in tests/inputs/config.py

[w4rum]

Thank you, that seems to have fixed it.

[nat-n]

Almost good to go, I just spotted a couple more minor issue.

[nat-n]

looks like this isn't used either actually?

[w4rum]

Not anymore since 5bbe19a. Nice catch. Good thing you're looking over this. I completely forgot.

[nat-n]

Ship it!