Runtime Settings

Compatibility Mode KVIKIO_COMPAT_MODE

When KvikIO is running in compatibility mode, it doesn’t load libcufile.so. Instead, reads and writes are done using POSIX. Notice, this is not the same as the compatibility mode in cuFile. It is possible that KvikIO performs I/O in the non-compatibility mode by using the cuFile library, but the cuFile library itself is configured to operate in its own compatibility mode. For more details, refer to cuFile compatibility mode and cuFile environment variables .

The environment variable KVIKIO_COMPAT_MODE has three options (case-insensitive):

  • ON (aliases: TRUE, YES, 1): Enable the compatibility mode.

  • OFF (aliases: FALSE, NO, 0): Disable the compatibility mode, and enforce cuFile I/O. GDS will be activated if the system requirements for cuFile are met and cuFile is properly configured. However, if the system is not suited for cuFile, I/O operations under the OFF option may error out.

  • AUTO: Try cuFile I/O first, and fall back to POSIX I/O if the system requirements for cuFile are not met.

Under AUTO, KvikIO falls back to the compatibility mode:

  • when libcufile.so cannot be found.

  • when running in Windows Subsystem for Linux (WSL).

  • when /run/udev isn’t readable, which typically happens when running inside a docker image not launched with --volume /run/udev:/run/udev:ro.

This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name compat_mode.

Thread Pool KVIKIO_NTHREADS

KvikIO can use multiple threads for IO automatically. Set the environment variable KVIKIO_NTHREADS to the number of threads in the thread pool. If not set, the default value is 1.

This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name num_threads.

Task Size KVIKIO_TASK_SIZE

KvikIO splits parallel IO operations into multiple tasks. Set the environment variable KVIKIO_TASK_SIZE to the maximum task size (in bytes). If not set, the default value is 4194304 (4 MiB).

When opportunistic Direct I/O read is enabled (KVIKIO_AUTO_DIRECT_IO_READ=1), this value should be a multiple of page size (typically 4 KiB) so that parallel tasks start at page-aligned file offsets, avoiding buffered I/O fallback.

This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name task_size.

GDS Threshold KVIKIO_GDS_THRESHOLD

In order to improve performance of small IO, .pread() and .pwrite() implement a shortcut that circumvent the threadpool and use the POSIX backend directly. Set the environment variable KVIKIO_GDS_THRESHOLD to the minimum size (in bytes) to use GDS. If not set, the default value is 16384 (16 KiB).

This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name gds_threshold.

Size of the Bounce Buffer KVIKIO_BOUNCE_BUFFER_SIZE

KvikIO might have to use intermediate host buffers (one per thread) when copying between files and device memory. Set the environment variable KVIKIO_BOUNCE_BUFFER_SIZE to the size (in bytes) of these “bounce” buffers. If not set, the default value is 16777216 (16 MiB).

This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name bounce_buffer_size.

HTTP Retries KVIKIO_HTTP_STATUS_CODES, KVIKIO_HTTP_MAX_ATTEMPTS

The behavior when a remote I/O read returns an error can be controlled through the KVIKIO_HTTP_STATUS_CODES, KVIKIO_HTTP_MAX_ATTEMPTS, and KVIKIO_HTTP_TIMEOUT environment variables.

KvikIO will retry a request should any of the HTTP status code in KVIKIO_HTTP_STATUS_CODES is received. The default values are 429, 500, 502, 503, 504. This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name http_status_codes.

The maximum number of attempts to make before throwing an exception is controlled by KVIKIO_HTTP_MAX_ATTEMPTS. The default value is 3. This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name http_max_attempts.

The maximum duration of each HTTP request is controlled by KVIKIO_HTTP_TIMEOUT. The default value is 60, which is the duration in seconds to allow. This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name http_timeout.

HTTP Verbose KVIKIO_REMOTE_VERBOSE

For debugging HTTP requests, you can enable verbose output that shows detailed information about HTTP communication including headers, request/response bodies, connection details, and SSL handshake information.

Set the environment variable KVIKIO_REMOTE_VERBOSE to true, on, yes, or 1 (case-insensitive) to enable verbose output. Otherwise, verbose output is disabled by default.

Warning

This may show sensitive contents from headers and data.

Remote I/O Backend KVIKIO_REMOTE_IO_BACKEND

KvikIO supports two backends for remote (HTTP/S3/WebHDFS) reads, selected at process startup via the environment variable KVIKIO_REMOTE_IO_BACKEND. The accepted values (case-insensitive) are:

  • EASY_THREADPOOL (default): Libcurl easy API running in the KvikIO thread pool. Each sub-range of a kvikio.RemoteFile.pread() is dispatched to a worker thread that blocks in curl_easy_perform() until its transfer completes. Concurrency is bounded by the thread pool size: one busy thread per in-flight transfer.

  • MULTI_POLL: Libcurl multi API driven by N reactor threads, each of which blocks in curl_multi_poll(). A single reactor multiplexes many in-flight easy handles concurrently, so the number of simultaneous transfers is bounded by KVIKIO_REMOTE_IO_MAX_CONCURRENT_REQUESTS rather than by the reactor count.

The MULTI_POLL backend honors three additional settings, described in the sections below: KVIKIO_REMOTE_IO_NUM_REACTORS, KVIKIO_REMOTE_IO_REACTOR_DISPATCH, and KVIKIO_REMOTE_IO_MAX_CONCURRENT_REQUESTS. They have no effect under EASY_THREADPOOL.

Remote I/O Reactor Count KVIKIO_REMOTE_IO_NUM_REACTORS

Number of reactor threads used by the MULTI_POLL backend. The default value is 1. Each reactor owns one CURLM* handle and serializes its libcurl-multi calls. Increase beyond 1 to spread the in-callback memcpy cost across cores when one reactor’s CPU is the bottleneck. This setting has no effect under EASY_THREADPOOL.

Remote I/O Reactor Dispatch KVIKIO_REMOTE_IO_REACTOR_DISPATCH

Controls how the sub-ranges of a single kvikio.RemoteFile.pread() are distributed across reactor threads when MULTI_POLL is active. When only one reactor is used, both modes are equivalent. This setting has no effect under EASY_THREADPOOL. The accepted values (case-insensitive) are:

  • PER_CHUNK (default): Sub-ranges are routed to reactors round-robin, independently of which kvikio.RemoteFile.pread() they belong to. This maximizes load balance across reactors. Trade-off: two sub-ranges of the same file may land on different reactors, each with its own libcurl connection cache, so they may not share an established TCP/TLS connection.

  • PER_PREAD: All sub-ranges of a single kvikio.RemoteFile.pread() are submitted to the same reactor (the reactor is itself chosen round-robin per kvikio.RemoteFile.pread() call). The sub-ranges then share that reactor’s libcurl connection cache, allowing an established TCP/TLS connection to be reused. Best for HTTPS, where the TLS handshake cost is non-trivial.

Remote I/O Concurrency Cap KVIKIO_REMOTE_IO_MAX_CONCURRENT_REQUESTS

Upper bound on the number of HTTP range requests the MULTI_POLL backend keeps in flight at once, summed across all reactor threads. The default value is 256. It must be a non-negative integer, and 0 means unlimited. The value is ignored when the active backend is not MULTI_POLL (EASY_THREADPOOL is already bounded by KVIKIO_NTHREADS).

The global budget is divided into an equal private share per reactor (KVIKIO_REMOTE_IO_MAX_CONCURRENT_REQUESTS divided by KVIKIO_REMOTE_IO_NUM_REACTORS), so each reactor enforces its own cap against its own inbox with no cross-reactor synchronization. Integer division rounds the per-reactor share down when the budget is not a multiple of the reactor count, and a floor of 1 rounds it up when the budget is smaller than the reactor count (a computed share of 0 would be a reactor that can never admit a request). The effective total is therefore only approximate.

The even split assumes sub-ranges are spread across reactors, which holds under PER_CHUNK. Under PER_PREAD all sub-ranges of one large kvikio.RemoteFile.pread() land on a single reactor, so that read is effectively limited to one reactor’s share while the others stay idle.

CA bundle file and CA directory CURL_CA_BUNDLE, SSL_CERT_FILE, SSL_CERT_DIR

The Certificate Authority (CA) paths required for TLS/SSL verification in libcurl can be explicitly specified using the following environment variables in order of overriding priority:

  • CURL_CA_BUNDLE (also used in the curl program) or SSL_CERT_FILE (also used in OpenSSL): Specifies the CA certificate bundle file location.

  • SSL_CERT_DIR (also used in OpenSSL): Specifies the CA certificate directory.

When neither is specified, KvikIO searches several standard system locations for the CA file and directory, and if the search fails falls back to the libcurl compile-time defaults.

Opportunistic POSIX Direct I/O operations KVIKIO_AUTO_DIRECT_IO_READ, KVIKIO_AUTO_DIRECT_IO_WRITE

Overview

By default, POSIX I/O operations perform buffered I/O using the OS page cache. However, Direct I/O (bypassing the page cache) can significantly improve performance in certain scenarios, such as writes and cold page-cache reads.

Traditional Direct I/O has strict requirements: The buffer address must be page-aligned, the file offset must be page-aligned, and the transfer size must be a multiple of page size (typically 4096 bytes). kvikio.CuFile provides the feature of opportunistic Direct I/O, which removes these restrictions by automatically handling alignment. Specifically, KvikIO can split a POSIX I/O operation into unaligned and aligned segments and apply buffered I/O and direct I/O respectively.

Configuration

Set the environment variable KVIKIO_AUTO_DIRECT_IO_READ / KVIKIO_AUTO_DIRECT_IO_WRITE to true, on, yes, or 1 (case-insensitive) to enable opportunistic Direct I/O.

export KVIKIO_AUTO_DIRECT_IO_READ=1
export KVIKIO_AUTO_DIRECT_IO_WRITE=1

Set them to false, off, no, or 0 to disable this feature and use buffered I/O.

export KVIKIO_AUTO_DIRECT_IO_READ=0
export KVIKIO_AUTO_DIRECT_IO_WRITE=0

If not set, the default setting is buffered I/O for POSIX read (KVIKIO_AUTO_DIRECT_IO_READ=0) and Direct I/O for POSIX write (KVIKIO_AUTO_DIRECT_IO_WRITE=1).

Programmatic Access

These settings can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name auto_direct_io_read and auto_direct_io_write.

Example:

import kvikio.defaults

# Check current settings
print(kvikio.defaults.get("auto_direct_io_read"))
print(kvikio.defaults.get("auto_direct_io_write"))

# Enable Direct I/O for reads, and disable it for writes
kvikio.defaults.set({"auto_direct_io_read": True, "auto_direct_io_write": False})

Over-read Alignment KVIKIO_AUTO_DIRECT_IO_READ_OVERREAD

When opportunistic Direct I/O is enabled for reads, unaligned prefix and suffix portions of each transfer fall back to buffered I/O. Setting KVIKIO_AUTO_DIRECT_IO_READ_OVERREAD=1 forces all disk reads to use Direct I/O by aligning offsets down and sizes up to page boundaries, discarding the extra bytes after the read. This only affects the device memory read path (disk to bounce buffer to GPU). Host memory reads are unaffected. Defaults to disabled.

export KVIKIO_AUTO_DIRECT_IO_READ_OVERREAD=1

Logging KVIKIO_LOG_LEVEL, KVIKIO_LOG_FILE

By default, logging is disabled and no output is produced.

Set the environment variable KVIKIO_LOG_LEVEL to enable logging (case-insensitive). From most to least verbose:

  • TRACE

  • DEBUG

  • INFO

  • WARN

  • ERROR

  • CRITICAL

Each level includes all messages from less verbose levels.

If not set or set to any other value, logging is disabled.

By default, log output are written to the standard error stream. To write log output to a file, set the environment variable KVIKIO_LOG_FILE to a file path. The file is overwritten on each process start. If the file cannot be opened (e.g. the parent directory does not exist), KvikIO falls back to the standard error with a warning. KVIKIO_LOG_FILE has no effect when logging is disabled.