Task ringbuffer

[PeterTh]

This is a performance refactoring replacing the unordered_map in the task_manager with a ringbuffer.

Far more importantly than the difference in data structure (though that is also measurable) this completely eliminates any locking contention for accessing tasks by id.

It also introduces a new microbenchmark for task handling, in 2 variants, one with pure task handling, and one in a more realistic scenario with another thread accessing tasks:

benchmark name                       samples       iterations    estimated
                                     mean          low mean      high mean
                                     std dev       low std dev   high std dev
------------------------------------------------------------------------------- Master
generating and deleting tasks                  100             1    590.461 ms 
                                        5.74591 ms    5.67021 ms    5.80049 ms 
                                        324.677 us    251.264 us    397.061 us 
generating and deleting tasks with                                             
access thread                                  100             1     3.78352 s 
                                        37.0397 ms    35.3406 ms    38.6955 ms 
                                        8.59095 ms    7.65626 ms    9.70456 ms 
------------------------------------------------------------------------------- This Branch
generating and deleting tasks                  100             1    464.589 ms 
                                        4.56318 ms    4.50633 ms    4.60211 ms 
                                        237.555 us    174.134 us    308.134 us 
generating and deleting tasks with                                             
access thread                                  100             1    878.879 ms 
                                        8.39569 ms    8.29195 ms    8.48651 ms 
                                        495.973 us    419.964 us    580.127 us 

[fknorr]

This is exciting! My major concern with the implementation is, however, that an allocation-free implementation like this is closely tied to stalling the main thread on submission, but that goal is only partially addressed by spin-waiting inside reserve_new_tid (We explicitly want to throttle submission to avoid overwhelming MPI as discussed earlier).

Moving task deletion to the scheduler thread (inside notify...) unnecessarily introduces a second mutator to the ring buffer. We should really merge #86 first, which gives us the epoch_monitor that allows the task manager to block on a horizon being executed (and becoming the new effective epoch) if it would otherwise overrun the ring buffer. reserve_new_tid would then return an optional, and wait_for_available_slot becomes unnecessary.

Also, a couple of smaller things:

• task_ring_buffer should rigorously document which member functions can be invoked only by the producer thread (main / task manager) and which ones are thread-safe.
• I don't believe we need to separate next_active_tid from next_task_id if we extend has_task to check for != nullptr on the task pointer. Then we can also make the API of emplace() less restrictive. Edit: On second thought, that is not true, since we need the release-store on next_active_id to make the non-atomic insertion into the unique_ptr visible.
• Once the task manager thread becomes the only mutator, number_of_deleted_tasks does not need to be atomic any more. Also, all atomic accesses can be either acquire or release, no need for seq_cst anywhere.

[PeterTh]

This latest version now uses the epoch monitor for synchronization. The performance impact of that compared to spinning is measurable but seems ok:

There are now also unit tests for the borderline scenarios of the ringbuffer (running out of buffer space due to long-running active tasks but eventually recovering; and running into a situation which would deadlock).

[fknorr]

Thanks! Some general remarks in addition to the line comments:

• I would perfer to avoid implicitly-seq_cst atomic operations in favor of those with explicit memory ordering. Even though the former is always correct, it keeps the reader in the dark about when atomics are used to synchronize other memory accesses. E.g. most loads in the task manager thread can be relaxed.
• task_ring_buffer should rigorously document which member functions can be invoked only by the producer thread (main / task manager) and which ones are thread-safe.

[psalz]

LGTM

[fknorr]

LGTM!