Package ai.rapids.cudf

Class ChunkedPack

java.lang.Object
ai.rapids.cudf.ChunkedPack
All Implemented Interfaces:
AutoCloseable
public class ChunkedPack extends Object implements AutoCloseable
JNI interface to cudf::chunked_pack. ChunkedPack has an Iterator-like API with the familiar `hasNext` and `next` methods. `next` should be used in a loop until `hasNext` returns false. However, `ChunkedPack.next` is special because it takes a `DeviceMemoryBuffer` as a parameter, which means that the caller can call `next` giving any bounce buffer it may have previously allocated. No requirement exists that the bounce buffer be the same each time, the only requirement is that their sizes are all the same, and match the size that was passed to `Table.makeChunkedPack` (which instantiates this class). The user of `ChunkedPack` must close `.close()` when done using it to clear up both host and device resources.

  • Constructor Details

    • ChunkedPack

      public ChunkedPack(long nativePtr)
      This constructor is invoked by `Table.makeChunkedPack` after creating a native `cudf::chunked_pack`.
      Parameters:
      nativePtr - pointer to a `cudf::chunked_pack`

  • Method Details

    • getTotalContiguousSize

      public long getTotalContiguousSize()
      Get the final contiguous size of the table we are packing. This is the size that the final buffer should be, just like if the user called `cudf::pack` instead.
      Returns:
      the total number of bytes for the table in contiguous layout

    • hasNext

      public boolean hasNext()
      Method to be called to ensure that `ChunkedPack` has work left. This method should be invoked followed by a call to `next`, until `hasNext` returns false.
      Returns:
      true if there is work left to be done (`next` should be called), false otherwise.

    • next

      public long next(DeviceMemoryBuffer userPtr)
      Place the next contiguous chunk of our table into `userPtr`. This method throws if `hasNext` is false.
      Parameters:
      userPtr - the bounce buffer to use for this iteration
      Returns:
      the number of bytes that we were able to place in `userPtr`. This is at most `userPtr.getLength()`.

    • buildMetadata

      public PackedColumnMetadata buildMetadata()
      Generates opaque table metadata that can be unpacked via `cudf::unpack` at a later time.
      Returns:
      a `PackedColumnMetadata` instance referencing cuDF packed table metadata

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable