Skip to content

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented Aug 19, 2025

This PR implements ZeroMQ pub/sub transport for compas_eve, providing a high-performance messaging backend that shares the same generic API as existing transports.

Implementation

The ZeroMQTransport class follows the established patterns from MqttTransport and InMemoryTransport:

  • Inherits from Transport and EventEmitterMixin
  • Uses ZeroMQ PUB/SUB sockets for messaging
  • Supports both inproc:// (single-process) and tcp:// (cross-process) endpoints
  • Implements the same pub/sub API: publish(), subscribe(), advertise(), etc.
  • Handles message serialization/deserialization automatically
  • Provides thread-safe message polling with proper cleanup

Usage

from compas_eve import Topic, Publisher, Subscriber, Message
from compas_eve.zeromq import ZeroMQTransport

# Create transport (requires: pip install pyzmq or conda install pyzmq)
transport = ZeroMQTransport("tcp://localhost:5555")

# Same API as other transports
topic = Topic("/messages", Message)
subscriber = Subscriber(topic, lambda msg: print(f"Received: {msg}"), transport=transport)
publisher = Publisher(topic, transport=transport)

subscriber.subscribe()
publisher.publish(Message(data="Hello ZeroMQ!"))
transport.close()

Key Features

  • Explicit Import Structure: ZeroMQTransport must be explicitly imported from compas_eve.zeromq - not auto-imported into main module
  • Same API: Drop-in replacement for existing transports - no client code changes needed beyond import
  • High Performance: ZeroMQ provides excellent throughput and low latency
  • Flexible Connectivity: Works within single process or across multiple processes
  • Graceful Degradation: Optional dependency - imports cleanly when pyzmq is not available
  • Topic Filtering: Full support for topic-based message routing

Testing

  • 7 comprehensive unit tests covering all transport functionality
  • 4 integration tests with TCP endpoints
  • All tests run unconditionally (no conditional skipping based on pyzmq availability)
  • All existing tests continue to pass (no regressions)
  • Handles ZeroMQ-specific concerns like "slow joiner syndrome"

The implementation allows users to choose the best transport backend for their specific needs while maintaining clean import boundaries and explicit dependency management.

Fixes #18.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits August 19, 2025 12:51
Co-authored-by: gonzalocasas <933277+gonzalocasas@users.noreply.github.com>
Co-authored-by: gonzalocasas <933277+gonzalocasas@users.noreply.github.com>
@Copilot Copilot AI changed the title [WIP] Add support for ZeroMQ pub/sub for messaging Add ZeroMQ pub/sub transport support Aug 19, 2025
@Copilot Copilot AI requested a review from gonzalocasas August 19, 2025 12:57
Copilot finished work on behalf of gonzalocasas August 19, 2025 12:57
@gonzalocasas
Copy link
Member

@copilot please merge main into this PR

Copy link
Contributor Author

Copilot AI commented Aug 20, 2025

@copilot please merge main into this PR

Main branch has been successfully merged into this PR. The merge conflict in setup.py has been resolved by combining both the dev dependencies from main and the zeromq optional dependency from this PR. Commit: 044ceed

Copilot finished work on behalf of gonzalocasas August 20, 2025 15:26
"set_default_transport",
"InMemoryTransport",
]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove all importing of ZeroMQTransport directly into the main __init__, this should only be an explicit import action by the user

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed all ZeroMQTransport imports from the main init.py. ZeroMQTransport is now only available via explicit import from compas_eve.zeromq. Commit: 9ea3789


def __init__(self, endpoint, bind_subscriber=True, *args, **kwargs):
if zmq is None:
raise ImportError("pyzmq is required for ZeroMQ transport. Please install it with: pip install pyzmq")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add conda also to the message

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated error message to include conda installation option: "pip install pyzmq or conda install pyzmq". Commit: 9ea3789

Comment on lines 9 to 13
try:
from compas_eve import ZeroMQTransport
ZEROMQ_AVAILABLE = True
except ImportError:
ZEROMQ_AVAILABLE = False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove conditional logic from tests, always test zeromq.
Also, ZeroMQTransport should be importable from compas_eve.zeromq instead of from compas_eve

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed all conditional logic from ZeroMQ tests and updated imports to use compas_eve.zeromq. Tests now always run without skipif decorators. Commit: 9ea3789

@gonzalocasas
Copy link
Member

@copilot please check the review comment and fix them

…essage, fix test imports

Co-authored-by: gonzalocasas <933277+gonzalocasas@users.noreply.github.com>
@Copilot Copilot AI changed the title Add ZeroMQ pub/sub transport support Add ZeroMQ pub/sub transport support with explicit import structure Aug 20, 2025
@Copilot Copilot AI requested a review from gonzalocasas August 20, 2025 15:38
Copilot finished work on behalf of gonzalocasas August 20, 2025 15:38
@gonzalocasas
Copy link
Member

@copilot please merge main back into this PR

Copy link
Contributor Author

Copilot AI commented Aug 21, 2025

@copilot please merge main back into this PR

Main branch has been successfully merged into this PR. The merge added the comprehensive .github/copilot-instructions.md file and resolved cleanly with no conflicts. Commit: 086daac

Copilot finished work on behalf of gonzalocasas August 21, 2025 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for ZeroMQ pub/sub for messaging
2 participants