Add a new memory type: Hierarchy #227

zhuofan1123 · 2024-10-08T07:52:45Z

This PR adds a new memory type called Hierarchy, which exploits power-law degree distribution in graph to reduce inter-node communication.

Hierarchy WholeMemory has the same storage pattern as Distributed WholeMemory and only optimizes the gather function.
Hierarchy WholeMemory can achieve 1.5x-2.0x speedup in multi-node gather, compared with Distributed.
For intra-node host memory location, Hierarchy can still achieve 1.5x-2.0x speedup.

copy-pr-bot · 2024-10-08T07:52:49Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

BradReesWork · 2024-10-09T12:50:43Z

/okay to test

linhu-nv

Seems good to me

BradReesWork · 2024-10-10T13:31:52Z

/okay to test

BradReesWork · 2024-10-10T18:05:07Z

/merge

linhu-nv and others added 5 commits October 8, 2024 05:36

add wholememory_hierarchy_type, new gather func needed

18e61d8

support obtaining local_comm and cross_comm for hierarchy memory type

1d61a49

implement hierarchy gather function

7680289

quick fix in creating sub-communicator for hierarchy memory type

5aed03c

set a knob for sort-unique

2d4fceb

BradReesWork added improvement Improves an existing functionality non-breaking Introduces a non-breaking change labels Oct 9, 2024

support different rank sizes for heirarchy memory type

cba73a2

zhuofan1123 force-pushed the hierarchy_rebase branch from 42eaf9e to cba73a2 Compare October 9, 2024 14:57

linhu-nv approved these changes Oct 10, 2024

View reviewed changes

BradReesWork approved these changes Oct 10, 2024

View reviewed changes

rapids-bot bot merged commit 98e6272 into rapidsai:branch-24.12 Oct 10, 2024
49 checks passed

Provide feedback