ai.rapids.cudf.nvcomp

Class BatchedZstdCompressor

java.lang.Object

ai.rapids.cudf.nvcomp.BatchedCompressor

ai.rapids.cudf.nvcomp.BatchedZstdCompressor

public class BatchedZstdCompressor
extends BatchedCompressor

Multi-buffer ZSTD compressor

Constructor Summary

Constructors

Constructor and Description
BatchedZstdCompressor(long chunkSize, long maxIntermediateBufferSize) Construct a batched ZSTD compressor instance

Method Summary

Modifier and Type	Method and Description
protected void	batchedCompressAsync(long devInPtrs, long devInSizes, long chunkSize, long batchSize, long tempPtr, long tempSize, long devOutPtrs, long compressedSizesOutPtr, long stream) Asynchronously compress a batch of buffers.
protected long	batchedCompressGetTempSize(long batchSize, long maxChunkSize) Get the temporary workspace size required to perform compression of an entire batch.

Methods inherited from class ai.rapids.cudf.nvcomp.BatchedCompressor

compress

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BatchedZstdCompressor

public BatchedZstdCompressor(long chunkSize,
                             long maxIntermediateBufferSize)

Construct a batched ZSTD compressor instance

Parameters:

chunkSize - maximum amount of uncompressed data to compress as a single chunk. Inputs larger than this will be compressed in multiple chunks.

maxIntermediateBufferSize - desired maximum size of intermediate device buffers used during compression.

Method Detail

batchedCompressGetTempSize

protected long batchedCompressGetTempSize(long batchSize,
                                          long maxChunkSize)

Description copied from class: BatchedCompressor

Get the temporary workspace size required to perform compression of an entire batch.

Specified by:

batchedCompressGetTempSize in class BatchedCompressor

Parameters:

batchSize - number of chunks in the batch

maxChunkSize - maximum size of an uncompressed chunk in bytes

Returns:

The size of required temporary workspace in bytes to compress the batch.

batchedCompressAsync

protected void batchedCompressAsync(long devInPtrs,
                                    long devInSizes,
                                    long chunkSize,
                                    long batchSize,
                                    long tempPtr,
                                    long tempSize,
                                    long devOutPtrs,
                                    long compressedSizesOutPtr,
                                    long stream)

Description copied from class: BatchedCompressor

Asynchronously compress a batch of buffers. Note that compressedSizesOutPtr must point to pinned memory for this operation to be asynchronous.

Specified by:

batchedCompressAsync in class BatchedCompressor

Parameters:

devInPtrs - device address of uncompressed buffer addresses vector

devInSizes - device address of uncompressed buffer sizes vector

chunkSize - maximum size of an uncompressed chunk in bytes

batchSize - number of chunks in the batch

tempPtr - device address of the temporary workspace buffer

tempSize - size of the temporary workspace buffer in bytes

devOutPtrs - device address of output buffer addresses vector

compressedSizesOutPtr - device address where to write the sizes of the compressed data written to the corresponding output buffers. Must point to a buffer with at least 8 bytes of memory per output buffer in the batch.

stream - CUDA stream to use