[IDAG] Switch to Instruction-Graph Scheduling #265
Conversation
|
Check-perf-impact results: (877795252c9a57f7b343e4747db6ca4f)
Relative execution time per category: (mean of relative medians)
|
Pull Request Test Coverage Report for Build 10213951594Details
💛 - Coveralls |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Fixes a bug with SYCL instant submission around reductions.
|
Check-perf-impact results: (f2e639c8a97550e58528a410c1b8586d)
Relative execution time per category: (mean of relative medians)
Edit: We inadvertently disabled mimalloc. All hail the benchmark suite! |
|
Check-perf-impact results: (2908f97f836fd2def14c3429cd4d61ac)
Relative execution time per category: (mean of relative medians)
|
Switches to the new IDAG-based runtime and drops all newly unused legacy components. - runtime now manages multiple devices, and the distr_queue API has been updated to reflect the fact. - buffer_manager, reduction_manager and host_object_manager are now gone. ID assignment for these types is now handled by the runtime directly, and all components interacting with buffers, reductions and host objects track relevant state themselves. lifetime_extending_state is removed. - scheduler now generates both the command- and the instruction graph in the same thread, and maintains ownership of both structures. The CDAG is pruned at generation time (since commands never leave the scheduler thread), and the IDAG is pruned once the scheduler is notified of epoch completion. Command serialization is gone, and with it, the command is_flushed marker. - runtime destruction is now delayed until the last buffer / queue / host object is destroyed. ~distr_queue will continue to epoch-synchronize. - runtime asserts that non-thread-safe functions are only called from the application thread, which will trigger onaccidental value-captures of buffers / host objects into host tasks. - Reductions are now available on all SYCL implementations since hipSYCL has added support. - log_context, which was only used by worker_job, is removed. - vendor/ctpl, which was only used by host_queue, is removed.
This is the final PR in the IDAG series. It switches to the new IDAG-based runtime and drops all newly unused legacy components.
runtimenow manages multiple devices, and thedistr_queueAPI has been updated to reflect the fact.buffer_manager,reduction_managerandhost_object_managerare now gone. ID assignment for these types is now handled by the runtime directly, and all components interacting with buffers, reductions and host objects (graph generators, executor and recorders) track the relevant state themselves as instructed bynotify_*_created/_destroyedintroduced in Explicitly manage buffer / host object lifetimes in graph generation #246. As a result, tasks do not need to keep strong references to buffers and host objects around anymore (lifetime_extending_state).schedulernow generates both the command- and the instruction graph in the same thread, and maintains ownership of both structures. The CDAG is pruned at generation time (since commands never leave the scheduler thread), and the IDAG is pruned once the scheduler is notified of epoch completion. Command serialization is gone, and with it, the commandis_flushedmarker.runtimenow useslive_executor(replacinglegacy_executorandworker_job) together with acommunicatorandbackendinstance to execute instructions.communicatortogether withreceive_arbiterreplacebuffer_transfer_manager.backendimplementations replacelegacy_backend,host_queueanddevice_queue.~distr_queuewill continue to epoch-synchronize.runtimeasserts that non-thread-safe functions are only called from the application thread, which will trigger onaccidental value-captures of buffers / host objects into host tasks.#ifdefsin tests and frontend code.log_context, which was only used byworker_job, is removed.vendor/ctpl, which was only used byhost_queue, is removed.Since one node now addresses multiple GPUs, scheduling becomes more expensive (IDAG generation is maybe ~4x as expensive as CDAG generation). This will be visible in benchmark results.