Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LIMS-1354: Migrate from ActiveMQ to RabbitMQ #826

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

JPHall-DLS
Copy link
Collaborator

JIRA ticket: LIMS-1354

Summary:
SynchWeb sends messages to ActiveMQ to trigger MX reprocessing and AlphaFold model prediction jobs. A single ActiveMQ server proved unreliable so a RabbitMQ cluster was established in early 2021. An ActiveMQ to RabbitMQ bridge service was implemented as a temporary solution while applications migrated to RabbitMQ. The failure of the bridge service in mid-2024 highlighted the need to update SynchWeb to send messages direct to RabbitMQ.
SynchWeb uses the Stomp (Simple Text Oriented Message Protocol) PHP client to send messages to ActiveMQ. This ticket updates SynchWeb to use an AMQP (Advanced Message Queuing Protocol) client to send messages to RabbitMQ. The php-amqplib client is the de facto standard and detailed in the RabbitMQ PHP tutorial.

Changes:

  • Update composer.json to replace stomp-php dependency with php-amqplib.
  • Add global variables for RabbitMQ configuration that reflect RabbitMQ nomenclature.
  • Update Queue class and Queue::send method to use AMQPStreamConnection and AMQPMessage. Remove redundant ActiveMQ header information e.g. synchweb.host and synchweb.user.
  • Update Process class to use new global variable names.
  • Update Page class to use new global variable names, Queue class signature, and Queue::send method signature.

To test:

  • Reprocess MX data via the gearwheel icon on the Data Collection page. In the database, SynchWeb will create a new record in the ProcessingJob table. SynchWeb will send a message to RabbitMQ with this ProcessingJobId. Zocalo will read the message from RabbitMQ and create a corresponding record in the AutoProcProgram table. A new processing job will soon be shown under Auto Processing on the Data Collection page.
  • AlphaFold model prediction jobs are triggered when proteins are added or updated with a sequence. Model prediction does not currently work as AlphaFold itself was compressed in the great GPFS02 purge of 2024.

@JPHall-DLS JPHall-DLS marked this pull request as ready for review September 5, 2024 19:28
api/src/Queue.php Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants