Schedule Cron Tasks Post-Startup

[dradux]

Is there a way to add cron tasks post-startup? The docs show how to add a cron job on startup via cron_jobs of WorkerSettings but I dont see a way to add jobs after startup of the worker.

My scenario:

• fastapi app
• arq worker
  My app needs to create cron jobs in arq. For example if a new user is created then create a cron job that updates their feeds every 10 minutes.

[dradux]

For those who have a similar scenario to mine (needing a dynamic scheduler rather than a static scheduler defined on startup of arq) - I solved my issue by using apscheduler. This is a common path to address the issue; however, I have a circumstance which is worth noting. I wanted to leverage the async nature and scalabiilty of arq; however, this is not simple with apscheduler as you need a single apscheduler and apscheduler needs to use its AsyncIOScheduler which is difficult if you need to use apscheduler's RPC (RPyC) server.

My solution was to create a fastapi + apscheduler (using AsyncIOScheduler) - fastapi exposes apscheduler as an API and apscheduler in turn creates arq tasks. Hope the info is useful to others out there looking for a more robust scheduling mechanism to arq.

FWIW I do not think this logic should be in arq, using an external service works well and provides its own benefits. RQ has a similar design with rq-scheduler providing scheduling features in much the same way. If anyone needs code on this let me know as I'd be happy to share.

[mtsewrs]

@dradux Looking to do this as well, would love for you to share the code!

[wackazong]

@dradux I would be eager to see that code as well if possible.

[dradux]

A full project using apscheduler + arq + fastapi can be seen here: https://gitlab.com/drad/radicchio. You'll likely be interested in things in the api directory. Of particular interest is _scheduler where apscheduler logic is located and _worker where the arq worker is located. Let me know if there are any questions - be happy to explain further.