diff --git a/docs/make.jl b/docs/make.jl index 5156bc55..0523a659 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -17,6 +17,7 @@ makedocs(; "ThreadPinning" => "index.md", "Examples" => [ "Pinning Julia Threads" => "examples/ex_pinning_julia_threads.md", + "Pinning Julia Tasks" => "examples/ex_pinning_tasks.md", "Pinning BLAS Threads" => "examples/ex_blas.md", "External Affinity Mask" => "examples/ex_affinity.md", "MPI and Hybrid" => "examples/ex_mpi.md", diff --git a/docs/src/examples/ex_pinning_tasks.md b/docs/src/examples/ex_pinning_tasks.md new file mode 100644 index 00000000..e963e88d --- /dev/null +++ b/docs/src/examples/ex_pinning_tasks.md @@ -0,0 +1,44 @@ +# Pinning Julia Tasks + +## Task-based multithreading +It is important to note that Julia implements **task-based multithreading**: `M` dynamically created user tasks get scheduled onto `N` Julia threads. By default, task scheduling is dynamic and is handled by Julia's built-in scheduler. Similar to how the operating system's scheduler can freely move Julia threads between CPU threads, Julia's scheduler can move tasks between Julia threads. Consequently, before pinning, a user cannot reliably predict on which Julia thread a task will run and on which CPU thread a Julia thread will run (see the visualization below). + +
+ +![tasks_threads_cores](tasks_threads_cores.svg) + +
+ +The primary purpose of ThreadPinning.jl is to allow you to pin Julia threads to CPU-threads. In this sense, it enables you to supersede the dynamic OS scheduler (right part in the image above). However, the dynamic scheduling of Julia tasks (left part in the image above) remains as is. + +## Static scheduling and *sticky* tasks + +If you want to opt-out of Julia's dynamic task scheduling and want to "pin" Julia tasks to specific Julia threads, you will often need to use tools from external libraries (such as ThreadPinning.jl), as support for this in base Julia is sparse. The only official API for static scheduling of *sticky* tasks (i.e. tasks that stay on the Julia thread they've been spawned on) is [`Threads.@threads :static`](https://docs.julialang.org/en/v1/base/multi-threading/#Base.Threads.@threads). In code like + +```julia +Threads.@threads for i in 1:Threads.nthreads() + do_something_on_thread(i) +end +``` + +it is guaranteed that each Julia thread will run precisely one *sticky* task that corresponds to one iteration of the loop. Hence, `i` may be interpreted as a Julia thread index. In fact, it is even guaranteed that the first (default) Julia thread will run the first task (iteration), the second (default) Julia thread will run the second task (iteration), and so on. + +!!! warning + Beware, the indices of the *default* Julia threads (as given by `Threads.threadid()` if called on them) actually only start at 1 in the absence of *interactive* threads. + +!!! warning + Under the hood, `Threads.@threads` always splits up the iteration regime into `Threads.nthreads()` many tasks, irrespective of the length of the iteration range. In the example above, there is a one-to-one correspondence between iterations and tasks but for general iterations (say, `1:N` where `N > Threads.nthreads()`) a task - and consequently a Julia thread - will take care of more than one iteration. + +The static scheduling option `Threads.@threads :static` is the only official built-in way to get sticky tasks. In particular, there is no sticky pendant of `@spawn` for manually creating tasks. + +ThreadPinning.jl aims to fill this gap by providing [`ThreadPinning.@spawnat`](@ref api_stabletasks) and a few other tools. As the name suggests, this macro allows you to spawn a *sticky* task on a specific Julia thread, e.g. `ThreadPinning.@spawnat 3 println("Hello from thread 3")`. + +Using `ThreadPinning.@spawnat` we can rewrite the code above as + +```julia +@sync for i in 1:Threads.nthreads() + ThreadPinning.@spawnat i do_something_on_thread(i) +end +``` + +Both the task-iteration mapping and the task-thread assignment are explicitly and immediately visible here. \ No newline at end of file diff --git a/docs/src/examples/tasks_threads_cores.svg b/docs/src/examples/tasks_threads_cores.svg new file mode 100644 index 00000000..41c80985 --- /dev/null +++ b/docs/src/examples/tasks_threads_cores.svg @@ -0,0 +1,1494 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/src/index.md b/docs/src/index.md index 86af5898..5e2767bf 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -5,11 +5,6 @@ Most notably, [ThreadPinning.jl](https://github.com/carstenbauer/ThreadPinning.j * to pin Julia threads to specific CPU-threads ("hardware threads") with [`pinthreads`](@ref pinthreads) and * to obtain a visual overview of the system topology with [`threadinfo`](@ref threadinfo). -!!! note - Note that Julia implements **task-based multithreading**: `M` user tasks get scheduled onto `N` Julia threads. - While this package allows you to pin Julia threads to CPU-threads, it is generally not - safe to assume that a computation (started with `Threads.@spawn` or `Threads.@threads`) will run on or stay on(!) a certain Julia thread (see [this discourse post](https://discourse.julialang.org/t/julia-1-7-says-it-can-switch-the-thread-your-task-is-on-how-often-does-that-happen-and-how-can-it-be-disabled/75373/4?u=carstenbauer) for more information). If you want this guarantee, you can use tools like `Threads.@threads :static` or [`ThreadPinning.@spawnat`](@ref api_stabletasks). - ## What is this about? (10 minutes) Check out my lightning talk that I gave as part of [JuliaCon 2023](https://juliacon.org/2023/) at MIT.