daemonic processes are not allowed to have children

[yannpom]

Hello,

First, thank you for this project, Celery was too complex for my use case and I wanted to use Django's database as the broker. So far I'm very happy with django-q.

But I got one problem: I have a task spawning some child processes using multiprocessing.Process but it fails to run asynchronously using django-q with the following error:

daemonic processes are not allowed to have children

This is because the workers are started with daemon = True in cluster.py:148

If I add daemon = False as shown here:

    def spawn_process(self, target, *args):
        """
        :type target: function or class
        """
        p = Process(target=target, args=args)
        p.daemon = True
        if target == worker:
            p.daemon = False         # <=== LINE ADDED HERE ======
            p.timer = args[2]
            self.pool.append(p)
        p.start()
        return p

Everything works and I can start child processes in my task.

So I was wondering what are the consequences? I could not notice any change in the behaviour of django-q, it may be related to process termination and orphaned children as the multiprocessing documentation would suggest HERE.
But I tried to stop my qcluster while a task was ongoing, and everything works as expected. The qcluster stops after the task has terminated.

In what scenario changing Daemon to False would cause a problem?

[Koed00]

I set it up so when the stop signal is sent, each worker is politely asked to finish its work and then exit. This will safely deal with the child processes you are spawning.
They would be orphaned however if the any of the main processes are terminated without being able to complete the stop sequence. ie. forced termination by the server.

I guess we could add an option to un-daemonize the workers , for those that require it and understand the consequences.

[yannpom]

Thanks for your reply. I see how that could result in orphaned child process, but I'm willing to take the limited risk.
I proposed the pull request #212
But I won't feel offended if you don't accept it don't worry, I'm already using my fork.