pylibwholegraph.torch.comm.create_group_communicator#

pylibwholegraph.torch.comm.create_group_communicator(group_size: int = -1, comm_stride: int = 1)#

Create WholeMemory Communicator. For example: 24 ranks with group_size = 4 and comm_stride = 2 will create following groups: [0, 2, 4, 6], [1, 3, 5, 7], [8, 10, 12, 14], [9, 11, 13, 15], [16, 18, 20, 22], [17, 19, 21, 23] :param group_size: Size of each group, -1 means to use all ranks