Is it possible to connect BentoML serving service via C# gRPC client?

[rick-jun]

Hello all,
I am writing a program in C# and testing BentoML to serve my model.
At first I tried with Http Client to connect to BentoML and it was quite straightforward.
So now, I want to test gRPC client to check the performance but it is not as easy as I expected....
And after searching for a while, I am not sure whether it is even possible or not.

So is there anyone who tried and succeeded in connecting C# gRPC client to BentoML serving service?
if yes, could you share how you setup proto for C# gRPC client?

I am trying to connect below types of BentoML services with C# client.
(the code is from https://github.com/bentoml/BentoML/blob/main/examples/custom_runner/torch_hub_yolov5/service.py)

//-------------------------------------------------------------------------------
@svc.api(input=Image(), output=PandasDataFrame())
async def invocation(input_img):
batch_ret = await yolo_v5_runner.inference.async_run([input_img])
return batch_ret[0]

@svc.api(input=Image(), output=Image())
async def render(input_img):
batch_ret = await yolo_v5_runner.render.async_run([input_img])
return batch_ret[0]
//-------------------------------------------------------------------------------

Thank you for your time,
Rick

[ssheng]

Are you starting the server with gRPC, see https://docs.bentoml.org/en/latest/guides/grpc.html?

[rick-jun]

Hello ssheng, thank you for your reply~
yes I started the server with gRPC option so with Python, I can conned to the server using gRPC.
And due to some technical requirements, I need to connect to the server with C# clients using gRPC and I have no idea how to create proto file for C# client... and C# is not in the BentoML gRPC client page so I wonder whether I can use C# client or not....

[aarnphm]

Hi there, with regarding to your question, Yes. It is possible to write any client in any languages that gRPC supports.

For C# client refer to https://github.com/grpc/grpc-dotnet.

[aarnphm]

You can generate the generated files from protobuf from https://github.com/bentoml/BentoML/blob/main/src/bentoml/grpc/v1/service.proto. Note that we use https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/struct.proto and https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/wrappers.proto so make sure to follow the grpc-dotnet instruction to see how to use these files.

[rick-jun]

Hello aarnphm, really thank you for your help!!!
I will follow your instruction and let you know the result.
Have a wonderful day~