pylibwholegraph.torch.tensor.create_wholememory_tensor_from_filelist

pylibwholegraph.torch.tensor.create_wholememory_tensor_from_filelist(comm: WholeMemoryCommunicator, memory_type: str, memory_location: str, filelist: List[str] | str, dtype: dtype, last_dim_size: int = 0, last_dim_strides: int = -1, tensor_entry_partition: List[int] | None = None)

Create WholeMemory Tensor from list of binary files. :param comm: WholeMemoryCommunicator :param memory_type: WholeMemory type, should be continuous, chunked or distributed :param memory_location: WholeMemory location, should be cpu or cuda :param filelist: list of binary files :param dtype: data type of the tensor :param last_dim_size: 0 for create 1-D array, positive value for

create matrix column size

Parameters:

• last_dim_strides – stride of last_dim, -1 for same as size of last dim.

• tensor_entry_partition – rank partition based on entry; tensor_entry_partition[i] determines the entry count of rank i and shoud be a positive integer; the sum of tensor_entry_partition should equal to total entry count; entries will be equally partitioned if None

Returns:

WholeMemoryTensor