Tensors#

pylibwholegraph.torch.WholeMemoryTensor(...)

WholeMemory Tensor

pylibwholegraph.torch.create_wholememory_tensor(...)

Create empty WholeMemory Tensor. Now only support dim = 1 or 2 :param comm: WholeMemoryCommunicator :param memory_type: WholeMemory type, should be continuous, chunked or distributed :param memory_location: WholeMemory location, should be cpu or cuda :param sizes: size of the tensor :param dtype: data type of the tensor :param strides: strides of the tensor :param tensor_entry_partition: rank partition based on entry; tensor_entry_partition[i] determines the entry count of rank i and shoud be a positive integer; the sum of tensor_entry_partition should equal to total entry count; entries will be equally partitioned if None :return: Allocated WholeMemoryTensor.

pylibwholegraph.torch.create_wholememory_tensor_from_filelist(...)

Create WholeMemory Tensor from a list of files. :param comm: WholeMemoryCommunicator :param memory_type: WholeMemory type, should be continuous, chunked or distributed :param memory_location: WholeMemory location, should be cpu or cuda :param filelist: list of files :param dtype: data type of the tensor :param last_dim_size: 0 creates a 1-D array and a positive value creates a matrix with that column count. Required for binary input and inferred from Parquet metadata when omitted. :param last_dim_strides: stride of last_dim, -1 for same as size of last dim. :param tensor_entry_partition: rank partition based on entry; tensor_entry_partition[i] determines the entry count of rank i and shoud be a positive integer; the sum of tensor_entry_partition should equal to total entry count; entries will be equally partitioned if None :param file_format: file format, one of binary, parquet, or auto :param expected_entry_count: optional expected number of rows. An error is raised before allocation when the files contain a different row count. :param expected_shape: optional expected 1-D or 2-D shape. A one-column Parquet file defaults to 1-D unless (N, 1) is specified here. :param fail_on_dtype_mismatch: raise an error instead of warning and converting when Parquet column dtypes differ from dtype. :return: WholeMemoryTensor.

pylibwholegraph.torch.destroy_wholememory_tensor(...)

Destroy allocated WholeMemory Tensor :param wm_tensor: WholeMemory Tensor :return: None