Package ai.rapids.cudf
Class MultiBufferDataSource
java.lang.Object
ai.rapids.cudf.DataSource
ai.rapids.cudf.MultiBufferDataSource
- All Implemented Interfaces:
AutoCloseable
This is a DataSource that can take multiple HostMemoryBuffers. They
are treated as if they are all part of a single file connected end to end.
-
Constructor Summary
ConstructorsConstructorDescriptionMultiBufferDataSource
(HostMemoryAllocator allocator, HostMemoryBuffer... buffers) Create a new data source backed by multiple buffers.MultiBufferDataSource
(HostMemoryBuffer... buffers) Create a new data source backed by multiple buffers. -
Method Summary
Modifier and TypeMethodDescriptionvoid
close()
long
deviceRead
(long offset, DeviceMemoryBuffer dest, Cuda.Stream stream) Read data from the source at the given offset into dest.long
long
long
long
hostRead
(long offset, long amount) Read data from the source at the given offset.long
hostRead
(long offset, HostMemoryBuffer dest) Read data from the source at the given offset into dest.long
size()
Get the size of the source in bytes.boolean
Return true if this supports reading directly to the device else false.Methods inherited from class ai.rapids.cudf.DataSource
getDeviceReadCutoff, onHostBufferDone
-
Constructor Details
-
MultiBufferDataSource
Create a new data source backed by multiple buffers.- Parameters:
buffers
- the buffers that will back the data source.
-
MultiBufferDataSource
Create a new data source backed by multiple buffers.- Parameters:
allocator
- the allocator to use for host buffers, if needed.buffers
- the buffers that will back the data source.
-
-
Method Details
-
size
public long size()Description copied from class:DataSource
Get the size of the source in bytes.- Specified by:
size
in classDataSource
-
hostRead
Description copied from class:DataSource
Read data from the source at the given offset. Return a HostMemoryBuffer for the data that was read.- Specified by:
hostRead
in classDataSource
- Parameters:
offset
- where to start reading from.amount
- the maximum number of bytes to read.- Returns:
- a buffer that points to the data.
-
hostRead
Description copied from class:DataSource
Read data from the source at the given offset into dest. Note that dest should not be closed, and no reference to it can outlive the call to hostRead. The target amount to read is dest's length.- Specified by:
hostRead
in classDataSource
- Parameters:
offset
- the offset to start reading from in the source.dest
- where to write the data.- Returns:
- the actual number of bytes written to dest.
-
supportsDeviceRead
public boolean supportsDeviceRead()Description copied from class:DataSource
Return true if this supports reading directly to the device else false. The default is no device support. This cannot change dynamically. It is typically read just once.- Overrides:
supportsDeviceRead
in classDataSource
-
deviceRead
Description copied from class:DataSource
Read data from the source at the given offset into dest. Note that dest should not be closed, and no reference to it can outlive the call to hostRead. The target amount to read is dest's length.- Overrides:
deviceRead
in classDataSource
- Parameters:
offset
- the offset to start reading fromdest
- where to write the data.stream
- the stream to do the copy on.- Returns:
- the actual number of bytes written to dest.
-
close
public void close()- Specified by:
close
in interfaceAutoCloseable
- Overrides:
close
in classDataSource
-
getHostReads
public long getHostReads() -
getHostReadBytes
public long getHostReadBytes() -
getDevReads
public long getDevReads() -
getDevReadBytes
public long getDevReadBytes()
-