Skip to content

Commit

Permalink
doc: update parallel-computing for ClusterManager
Browse files Browse the repository at this point in the history
  • Loading branch information
nlhepler committed Aug 11, 2013
1 parent 3342e30 commit ba2a297
Showing 1 changed file with 35 additions and 0 deletions.
35 changes: 35 additions & 0 deletions doc/manual/parallel-computing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -589,3 +589,38 @@ required, since the threads are scheduled cooperatively and not
preemptively. This means context switches only occur at well-defined
points: in this case, when ``remotecall_fetch`` is called.


ClusterManagers
---------------

Julia worker processes can also be spawned on arbitrary machines,
enabling Julia's natural parallelism to function quite transparently
in a cluster environment. The ``ClusterManager`` interface provides a
way to specify a means to launch and manage worker processes, for
example::

immutable SSHManager <: ClusterManager
launch::Function
manage::Function
machines::AbstractVector

SSHManager(; machines=[]) = new(launch_ssh_workers, manage_ssh_workers, machines)
end

function launch_ssh_workers(cman::SSHManager, np::Integer, config::Dict)
...
end

function manage_ssh_workers(id::Integer, config::Dict, op::Symbol)
...
end

where ``launch_ssh_workers`` is responsible for instantiating new
Julia processes and ``manage_ssh_workers`` provides a means to manage
those processes, e.g. for sending interrupt signals. New processes can
then be added at runtime using ``addprocs``::

addprocs(5, cman=LocalManager())

which specifies a number of processes to add and a ``ClusterManager`` to
use for launching those processes.

0 comments on commit ba2a297

Please sign in to comment.