Is it possible to have the worker run in a separate process or is this a non-goal?

[LasseJacobs]

River works great but if I try enqueue a job that has a worker running in a separate process (and thus using a different River client) I get a:

job kind is not registered in the client's Workers bundle: <job kind>

I dug into the code a little and indeed it checks for workers that were registered on that same client:

if _, ok := c.config.Workers.workersMap[args.Kind()]; !ok {
     return &UnknownJobKindError{Kind: args.Kind()}
}

I would like to run some workers in a separate process because some of the jobs require a lot of memory and these processes are more likely to get OOM killed (or potentially have CPU starvation). I don't want these jobs to disrupt the response time or reliability of the main process which serves HTTP requests.

Is this something that will be supported in the future or a non-goal for the project at the moment?

[bgentry]

Hi @LasseJacobs, the workers can indeed run in a different process and in general they should. However you do need to have the same workers defined on both the enqueueing side and the worker side if you want to be able to enqueue and work jobs. The Workers config is used to map your Worker definition to its Kind() based on the corresponding JobArgs definition. When enqueueing, this value is encoded into the database and used on the downstream Client to know what Worker code to run for that particular job.

I hope that makes sense! We’ll think about what we can do to improve the docs in this regard.

[LasseJacobs]

@bgentry that makes sense. Thank you for the quick reply.