Python client proto output makes it difficult to compile new protos for use in clients #6376
Labels
bug
Something isn't working
grpc
python-client
release blocker
A bug/behavior that puts is below the "good enough" threshold to release.
triage
Milestone
The
pydeephaven
package currently provides the generated protoc output in thepydeephaven.proto
namespace, despite the proto files having originated in adeephaven/proto/
directory - care was taken to rewrite python references so that we could relocate these, and avoid confusion betweendeephaven
(server side module) andpydeephaven
(client side module that uses grpc+proto to communicate).Unfortunately, this makes it difficult for downstream projects to define new .proto files and compile them to python, and successfully have them reference the types in pydeephaven (for example Ticket in
pydeephaven.proto.ticket_pb2
), because the actual contents of the.proto
(including its directory structure?) is serialized into the generated python files and is expected to match when loaded. The assumption by protoc is that python packaging follows directory structure and no additional nesting/vendoring/renaming would ever happen. In our case, this renaming is necessary, asdeephaven
has a module which will fail if a JVM is not already started in the current process - so any venv with bothpydeephaven
anddeephaven-core
installed cannot be used purely as a client.protocolbuffers/protobuf#17663
protocolbuffers/protobuf#19212
protocolbuffers/protobuf#1491 esp protocolbuffers/protobuf#1491 (comment)
protocolbuffers/protobuf#7470
Rather than rename every language to use
pydeephaven/proto
as the directory structure (JS for example is another language that uses directory structure in its output), we're renaming the parent directories of our proto files to bedeephaven_core/proto/
, as this is language agnostic, and still leaves some space for more than one package. The deephaven python client then will also include this namespace package.To futureproof any possible deephaven-core protobuf usage, no
__init__.py
should be added todeephaven_core/
or the nestedproto/
directory.The text was updated successfully, but these errors were encountered: