P33 - Topology Aware Collective Communication based on Cyclic Shift and Recursive Exchange
Presenter
DescriptionThe cyclic shift and recursive exchange algorithms for collective communication on parallel computers were comprehensively investigated recently [1]. With suitable parameters of the schemes determined with a benchmark at installation time and a heuristic at runtime implementations with high performance for the message passing interface (MPI) can be obtained. For the collective communication patterns reduce_scatter, allgatherv, and allreduce on hybrid shared and distributed computers the topology is mostly addressed using a hierarchical approach. We show how a combination of the cyclic shift and recursive exchange algorithms can match the topology of multi CPU or GPU per node architectures. The algorithm applied is recursive exchange with higher radix and different factors for each step where the steps are performed with cyclic shift and multiple ports per node are used. Out of many algorithmic options the communication is arranged such that the largest data volumes occur for the fast shared memory while over the network smaller volumes are sent. Comparisons with the hierarchical implementation are made for persistent collective communication, but our approach is not limited to this case. [1] "An optimisation of allreduce communication in message-passing systems" A.Jocksch, N.Ohana, E.Lanti, E.Koutsaniti, V.Karakasis, L.Villard, Parallel Comput. 2021
TimeTuesday, June 289:00 - 11:00 CEST
LocationFoyer 2nd Floor
Event Type
Poster
