Configure options for gRPC (Triton server)

[amirhmk]

Hello! I am trying to play around with the configs for gRPC for the triton server.

I’m using the docker-compose setup, so not sure if the CLI will work for my usecase (perhaps passing them as env variables would work?)

For instance, I’d like to set some variables like this:

('grpc.max_send_message_length', 512 * 1024 * 1024), ('grpc.max_receive_message_length', 512 * 1024 * 1024)]

Is this possible currently? I’m getting an error from gRPC that my payload is more than the limit (8MB instead of 4MB…)

#2

[ainoam]

Hi @amirhmk,

At this time this is not possible, though it definitely should be 🙂.
An upcoming v1.3.0 should have an environment override mechanism e.g. use CLEARML_GRPC_max_send_message_length.

In the meantime, you can try changing here:

channel = self._ext_grpc.aio.insecure_channel(triton_server_address,
 options=dict([('grpc.max_send_message_length', 512 * 1024 * 1024), ('grpc.max_receive_message_length', 512 * 1024 * 1024)]))

[pollfly]

Hey @amirhmk! v1.3.0 is now out supporting gPRC channel configuration

[jkhenning]

@amirhmk I'm closing the issue. Please reopen if required.