ai.rapids.cudf

Class MultiBufferDataSource

java.lang.Object

ai.rapids.cudf.DataSource

ai.rapids.cudf.MultiBufferDataSource

All Implemented Interfaces:

AutoCloseable

public class MultiBufferDataSource
extends DataSource

This is a DataSource that can take multiple HostMemoryBuffers. They are treated as if they are all part of a single file connected end to end.

Constructor Summary

Constructors

Constructor and Description
MultiBufferDataSource(HostMemoryAllocator allocator, HostMemoryBuffer... buffers) Create a new data source backed by multiple buffers.
MultiBufferDataSource(HostMemoryBuffer... buffers) Create a new data source backed by multiple buffers.

Method Summary

Modifier and Type	Method and Description
void	close()
long	deviceRead(long offset, DeviceMemoryBuffer dest, Cuda.Stream stream) Read data from the source at the given offset into dest.
long	getDevReadBytes()
long	getDevReads()
long	getHostReadBytes()
long	getHostReads()
long	hostRead(long offset, HostMemoryBuffer dest) Read data from the source at the given offset into dest.
HostMemoryBuffer	hostRead(long offset, long amount) Read data from the source at the given offset.
long	size() Get the size of the source in bytes.
boolean	supportsDeviceRead() Return true if this supports reading directly to the device else false.

Methods inherited from class ai.rapids.cudf.DataSource

getDeviceReadCutoff, onHostBufferDone

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MultiBufferDataSource

public MultiBufferDataSource(HostMemoryBuffer... buffers)

Create a new data source backed by multiple buffers.

Parameters:

buffers - the buffers that will back the data source.

MultiBufferDataSource

public MultiBufferDataSource(HostMemoryAllocator allocator,
                             HostMemoryBuffer... buffers)

Create a new data source backed by multiple buffers.

Parameters:

allocator - the allocator to use for host buffers, if needed.

buffers - the buffers that will back the data source.

Method Detail

size

public long size()

Description copied from class: DataSource

Get the size of the source in bytes.

Specified by:

size in class DataSource

hostRead

public HostMemoryBuffer hostRead(long offset,
                                 long amount)

Description copied from class: DataSource

Read data from the source at the given offset. Return a HostMemoryBuffer for the data that was read.

Specified by:

hostRead in class DataSource

Parameters:

offset - where to start reading from.

amount - the maximum number of bytes to read.

Returns:

a buffer that points to the data.

hostRead

public long hostRead(long offset,
                     HostMemoryBuffer dest)

Description copied from class: DataSource

Read data from the source at the given offset into dest. Note that dest should not be closed, and no reference to it can outlive the call to hostRead. The target amount to read is dest's length.

Specified by:

hostRead in class DataSource

Parameters:

offset - the offset to start reading from in the source.

dest - where to write the data.

Returns:

the actual number of bytes written to dest.

supportsDeviceRead

public boolean supportsDeviceRead()

Description copied from class: DataSource

Return true if this supports reading directly to the device else false. The default is no device support. This cannot change dynamically. It is typically read just once.

Overrides:

supportsDeviceRead in class DataSource

deviceRead

public long deviceRead(long offset,
                       DeviceMemoryBuffer dest,
                       Cuda.Stream stream)

Description copied from class: DataSource

Read data from the source at the given offset into dest. Note that dest should not be closed, and no reference to it can outlive the call to hostRead. The target amount to read is dest's length.

Overrides:

deviceRead in class DataSource

Parameters:

offset - the offset to start reading from

dest - where to write the data.

stream - the stream to do the copy on.

Returns:

the actual number of bytes written to dest.

close

public void close()

Specified by:

close in interface AutoCloseable

Overrides:

close in class DataSource

getHostReads

public long getHostReads()

getHostReadBytes

public long getHostReadBytes()

getDevReads

public long getDevReads()

getDevReadBytes

public long getDevReadBytes()