Package ai.rapids.cudf.nvcomp

Class BatchedLZ4Compressor

java.lang.Object
ai.rapids.cudf.nvcomp.BatchedCompressor
ai.rapids.cudf.nvcomp.BatchedLZ4Compressor
public class BatchedLZ4Compressor extends BatchedCompressor
Multi-buffer LZ4 compressor

  • Constructor Summary

    Constructors
    Constructor
    Description
    BatchedLZ4Compressor(long chunkSize, long maxIntermediateBufferSize)
    Construct a batched LZ4 compressor instance

  • Method Summary

    Modifier and Type
    Method
    Description
    protected void
    batchedCompressAsync(long devInPtrs, long devInSizes, long chunkSize, long batchSize, long tempPtr, long tempSize, long devOutPtrs, long compressedSizesOutPtr, long stream)
    Asynchronously compress a batch of buffers.
    protected long
    batchedCompressGetTempSize(long batchSize, long maxChunkSize)
    Get the temporary workspace size required to perform compression of an entire batch.

    Methods inherited from class ai.rapids.cudf.nvcomp.BatchedCompressor

    compress

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • BatchedLZ4Compressor

      public BatchedLZ4Compressor(long chunkSize, long maxIntermediateBufferSize)
      Construct a batched LZ4 compressor instance
      Parameters:
      chunkSize - maximum amount of uncompressed data to compress as a single chunk. Inputs larger than this will be compressed in multiple chunks.
      maxIntermediateBufferSize - desired maximum size of intermediate device buffers used during compression.

  • Method Details

    • batchedCompressGetTempSize

      protected long batchedCompressGetTempSize(long batchSize, long maxChunkSize)
      Description copied from class: BatchedCompressor
      Get the temporary workspace size required to perform compression of an entire batch.
      Specified by:
      batchedCompressGetTempSize in class BatchedCompressor
      Parameters:
      batchSize - number of chunks in the batch
      maxChunkSize - maximum size of an uncompressed chunk in bytes
      Returns:
      The size of required temporary workspace in bytes to compress the batch.

    • batchedCompressAsync

      protected void batchedCompressAsync(long devInPtrs, long devInSizes, long chunkSize, long batchSize, long tempPtr, long tempSize, long devOutPtrs, long compressedSizesOutPtr, long stream)
      Description copied from class: BatchedCompressor
      Asynchronously compress a batch of buffers. Note that compressedSizesOutPtr must point to pinned memory for this operation to be asynchronous.
      Specified by:
      batchedCompressAsync in class BatchedCompressor
      Parameters:
      devInPtrs - device address of uncompressed buffer addresses vector
      devInSizes - device address of uncompressed buffer sizes vector
      chunkSize - maximum size of an uncompressed chunk in bytes
      batchSize - number of chunks in the batch
      tempPtr - device address of the temporary workspace buffer
      tempSize - size of the temporary workspace buffer in bytes
      devOutPtrs - device address of output buffer addresses vector
      compressedSizesOutPtr - device address where to write the sizes of the compressed data written to the corresponding output buffers. Must point to a buffer with at least 8 bytes of memory per output buffer in the batch.
      stream - CUDA stream to use