pylibwholegraph.torch.tensor.create_wholememory_tensor_from_filelist#

pylibwholegraph.torch.tensor.create_wholememory_tensor_from_filelist(comm: WholeMemoryCommunicator, memory_type: str, memory_location: str, filelist: Union[List[str], str], dtype: dtype, last_dim_size: int = 0, last_dim_strides: int = -1, tensor_entry_partition: Optional[List[int]] = None)#

Create WholeMemory Tensor from list of binary files. :param comm: WholeMemoryCommunicator :param memory_type: WholeMemory type, should be continuous, chunked or distributed :param memory_location: WholeMemory location, should be cpu or cuda :param filelist: list of binary files :param dtype: data type of the tensor :param last_dim_size: 0 for create 1-D array, positive value for create matrix column size :param last_dim_strides: stride of last_dim, -1 for same as size of last dim. :param tensor_entry_partition: rank partition based on entry; tensor_entry_partition[i] determines the entry count of rank i and shoud be a positive integer; the sum of tensor_entry_partition should equal to total entry count; entries will be equally partitioned if None :return: WholeMemoryTensor