JSON

class pylibcudf.io.json.JsonReaderOptions

The settings to use for read_json

For details, see :cpp:class:`cudf::io::json_reader_options

Methods

allow_nonnumeric_numbers(self, bool val)
allow_numeric_leading_zeros(self, bool val)
allow_unquoted_control_chars(self, bool val)
builder(SourceInfo source)	Create a JsonReaderOptionsBuilder object
enable_dayfirst(self, bool val)
enable_experimental(self, bool val)
enable_keep_quotes(self, bool keep_quotes)	Set whether the reader should keep quotes of string values.
enable_lines(self, bool val)	Set whether to read the file as a json object per line.
enable_mixed_types_as_string(self, ...)	Set whether to parse mixed types as a string column.
enable_normalize_single_quotes(self, bool val)
enable_normalize_whitespace(self, bool val)
enable_prune_columns(self, bool prune_columns)	Set whether to prune columns on read, selected based on the set_dtypes option.
set_byte_range_offset(self, size_t offset)	Set number of bytes to skip from source start.
set_byte_range_size(self, size_t size)	Set number of bytes to read.
set_delimiter(self, str val)
set_dtypes(self, list types)	Set data types for columns to be read.
set_na_values(self, list vals)
set_source(self, SourceInfo src)	Set a new source info location.
set_strict_validation(self, bool val)

allow_nonnumeric_numbers(self, bool val) → void

allow_numeric_leading_zeros(self, bool val) → void

allow_unquoted_control_chars(self, bool val) → void

static builder(

SourceInfo source,

) → JsonReaderOptionsBuilder

Create a JsonReaderOptionsBuilder object

For details, see cudf::io::json_reader_options::builder()

Parameters:

sinkSourceInfo

The source to read the JSON file from.

Returns:

JsonReaderOptionsBuilder

Builder to build JsonReaderOptions

enable_dayfirst(self, bool val) → void

enable_experimental(self, bool val) → void

enable_keep_quotes(self, bool keep_quotes) → void

Set whether the reader should keep quotes of string values.

Parameters:

keep_quotesbool

Boolean value to indicate whether the reader should keep quotes of string values

Returns:

None

enable_lines(self, bool val) → void

Set whether to read the file as a json object per line.

Parameters:

valbool

Boolean value to enable/disable the option to read each line as a json object

Returns:

None

enable_mixed_types_as_string(

self,

bool mixed_types_as_string,

) → void

Set whether to parse mixed types as a string column. Also enables forcing to read a struct as string column using schema.

Parameters:

mixed_types_as_stringbool

Boolean value to enable/disable parsing mixed types as a string column

Returns:

None

enable_normalize_single_quotes(

self,

bool val,

) → void

enable_normalize_whitespace(self, bool val) → void

enable_prune_columns(

self,

bool prune_columns,

) → void

Set whether to prune columns on read, selected based on the set_dtypes option.

Parameters:

prune_columnsbool

When set as true, if the reader options include set_dtypes, then the reader will only return those columns which are mentioned in set_dtypes. If false, then all columns are returned, independent of the set_dtypes setting.

Returns:

None

set_byte_range_offset(self, size_t offset) → void

Set number of bytes to skip from source start.

Parameters:

offsetsize_t

Number of bytes of offset

Returns:

None

set_byte_range_size(self, size_t size) → void

Set number of bytes to read.

Parameters:

sizesize_t

Number of bytes to read

Returns:

None

set_delimiter(self, str val) → void

set_dtypes(self, list types: list[DataType] | list[NameAndType]) → void

Set data types for columns to be read.

Parameters:

typeslist

List of dtypes or a list of tuples of column names, dtypes, and list of tuples (to support nested column hierarchy)

Returns:

None

set_na_values(self, list vals: list[str]) → void

set_source(self, SourceInfo src) → void

Set a new source info location.

Parameters:

srcSourceInfo

New source information, replacing existing information.

Returns:

None

set_strict_validation(self, bool val) → void

class pylibcudf.io.json.JsonReaderOptionsBuilder

Builder to build options for read_json.

Methods

build(self)	Create a JsonReaderOptions object
byte_range_offset(self, size_t byte_range_offset)	Set number of bytes to skip from source start.
byte_range_size(self, size_t byte_range_size)	Set number of bytes to read.
compression(self, compression_type compression)	Sets compression type.
dayfirst(self, bool val)	Set whether the reader should parse dates as DD/MM versus MM/DD.
delimiter(self, str delimiter)	Set delimiter character separating records in JSON lines inputs
dtypes(self, list types)	Set data type for columns to be read
experimental(self, bool val)	Set whether to enable experimental features.
keep_quotes(self, bool val)	Set whether the reader should keep quotes of string values.
lines(self, bool val)	Set whether to read the file as a json object per line.
mixed_types_as_string(self, bool val)	Set whether to parse mixed types as a string column.
na_values(self, list vals)	Sets additional values to recognize as null values.
nonnumeric_numbers(self, bool val)	Set whether unquoted number values should be allowed NaN, +INF, -INF, +Infinity, Infinity, and -Infinity.
normalize_single_quotes(self, bool val)	Sets whether to normalize single quotes around strings.
normalize_whitespace(self, bool val)	Sets whether to normalize unquoted whitespace characters
numeric_leading_zeros(self, bool val)	Set whether leading zeros are allowed in numeric values.
prune_columns(self, bool val)	Set whether to prune columns on read, selected based on the @ref dtypes option.
recovery_mode(self, ...)	Specifies the JSON reader's behavior on invalid JSON lines.
strict_validation(self, bool val)	Set whether strict validation is enabled or not
unquoted_control_chars(self, bool val)	Set whether in a quoted string should characters greater than or equal to 0 and less than 32 be allowed without some form of escaping.

build(self) → JsonReaderOptions

Create a JsonReaderOptions object

byte_range_offset(

self,

size_t byte_range_offset,

) → JsonReaderOptionsBuilder

Set number of bytes to skip from source start.

Parameters:

byte_range_offsetsize_t

Number of bytes of offset

Returns:

Self

byte_range_size(

self,

size_t byte_range_size,

) → JsonReaderOptionsBuilder

Set number of bytes to read.

Parameters:

byte_range_sizesize_t

Number of bytes to read

Returns:

Self

compression(

self,

compression_type compression,

) → JsonReaderOptionsBuilder

Sets compression type.

Parameters:

compressionCompressionType

The compression type to use

Returns:

Self

dayfirst(

self,

bool val,

) → JsonReaderOptionsBuilder

Set whether the reader should parse dates as DD/MM versus MM/DD.

Parameters:

valbool

Boolean value to indicate whether the reader should enable/disable DD/MM parsing

Returns:

Self

delimiter(

self,

str delimiter,

) → JsonReaderOptionsBuilder

Set delimiter character separating records in JSON lines inputs

Parameters:

delimiterstr

Character to be used as delimiter separating records

Returns:

Self

dtypes(

self,

list types,

) → JsonReaderOptionsBuilder

Set data type for columns to be read

Parameters:

typeslist

List of dtypes or a list of tuples of column names, dtypes, and list of tuples (to support nested column hierarchy)

Returns:

Self

experimental(

self,

bool val,

) → JsonReaderOptionsBuilder

Set whether to enable experimental features. When set to true, experimental features, such as the new column tree construction, utf-8 matching of field names will be enabled.

Parameters:

valbool

Boolean value to enable/disable experimental features

Returns:

Self

keep_quotes(

self,

bool val,

) → JsonReaderOptionsBuilder

Set whether the reader should keep quotes of string values.

Parameters:

valbool

Boolean value to indicate whether the reader should keep quotes of string values

Returns:

Self

lines(

self,

bool val,

) → JsonReaderOptionsBuilder

Set whether to read the file as a json object per line.

Parameters:

valbool

Boolean value to enable/disable the option to read each line as a json object

Returns:

Self

mixed_types_as_string(

self,

bool val,

) → JsonReaderOptionsBuilder

Set whether to parse mixed types as a string column. Also enables forcing to read a struct as string column using schema.

Parameters:

valbool

Boolean value to enable/disable parsing mixed types as a string column

Returns:

Self

na_values(

self,

list vals,

) → JsonReaderOptionsBuilder

Sets additional values to recognize as null values.

Parameters:

valslist

Vector of values to be considered to be null

Returns:

Self

nonnumeric_numbers(

self,

bool val,

) → JsonReaderOptionsBuilder

Set whether unquoted number values should be allowed NaN, +INF, -INF, +Infinity, Infinity, and -Infinity. Strict validation must be enabled for this to work.

Parameters:

valbool

Boolean value to indicate whether leading zeros are allowed in numeric values

Returns:

Self

normalize_single_quotes(

self,

bool val,

) → JsonReaderOptionsBuilder

Sets whether to normalize single quotes around strings.

Parameters:

valbool

Boolean value to enable/disable the option to normalize single quotes around strings

Returns:

Self

normalize_whitespace(

self,

bool val,

) → JsonReaderOptionsBuilder

Sets whether to normalize unquoted whitespace characters

Parameters:

valbool

Boolean value to enable/disable the option to normalize unquoted whitespace characters

Returns:

Self

numeric_leading_zeros(

self,

bool val,

) → JsonReaderOptionsBuilder

Set whether leading zeros are allowed in numeric values. Strict validation must be enabled for this to work.

Parameters:

valbool

Boolean value to indicate whether leading zeros are allowed in numeric values

Returns:

Self

prune_columns(

self,

bool val,

) → JsonReaderOptionsBuilder

Set whether to prune columns on read, selected based on the @ref dtypes option. When set as true, if the reader options include @ref dtypes, then the reader will only return those columns which are mentioned in @ref dtypes. If false, then all columns are returned, independent of the @ref dtypes setting.

Parameters:

valbool

Boolean value to enable/disable column pruning

Returns:

Self

recovery_mode(

self,

json_recovery_mode_t recovery_mode,

) → JsonReaderOptionsBuilder

Specifies the JSON reader’s behavior on invalid JSON lines.

Parameters:

recovery_modejson_recovery_mode_t

An enum value to indicate the JSON reader’s behavior on invalid JSON lines.

Returns:

Self

strict_validation(

self,

bool val,

) → JsonReaderOptionsBuilder

Set whether strict validation is enabled or not

Parameters:

valbool

Boolean value to indicate whether strict validation is to be enabled

Returns:

Self

unquoted_control_chars(

self,

bool val,

) → JsonReaderOptionsBuilder

Set whether in a quoted string should characters greater than or equal to 0 and less than 32 be allowed without some form of escaping. Strict validation must be enabled for this to work.

Parameters:

valbool

Boolean value to indicate whether unquoted control chars are allowed

Returns:

Self

class pylibcudf.io.json.JsonWriterOptions

The settings to use for write_json

For details, see cudf::io::json_writer_options

Methods

builder(SinkInfo sink, Table table)	Create a JsonWriterOptionsBuilder object
set_compression(self, compression_type comptype)	Sets compression type to be used
set_false_value(self, str val)	Sets string used for values == 0
set_rows_per_chunk(self, size_type val)	Sets string to used for null entries.
set_true_value(self, str val)	Sets string used for values != 0

static builder(

SinkInfo sink,

Table table,

) → JsonWriterOptionsBuilder

Create a JsonWriterOptionsBuilder object

Parameters:

sinkSinkInfo

The sink used for writer output

tableTable

Table to be written to output

Returns:

JsonWriterOptionsBuilder

Builder to build JsonWriterOptions

set_compression(

self,

compression_type comptype,

) → void

Sets compression type to be used

Parameters:

comptypeCompressionType

Compression type for sink

Returns:

None

set_false_value(self, str val) → void

Sets string used for values == 0

Parameters:

valstr

String to represent values == 0

Returns:

None

set_rows_per_chunk(self, size_type val) → void

Sets string to used for null entries.

Parameters:

valsize_type

String to represent null value

Returns:

None

set_true_value(self, str val) → void

Sets string used for values != 0

Parameters:

valstr

String to represent values != 0

Returns:

None

class pylibcudf.io.json.JsonWriterOptionsBuilder

Builder to build options for write_json.

Methods

build(self)	Create a JsonWriterOptions object
compression(self, compression_type comptype)	Sets compression type of output sink.
include_nulls(self, bool val)	Enables/Disables output of nulls as 'null'.
lines(self, bool val)	Enables/Disables JSON lines for records format.
metadata(self, TableWithMetadata tbl_w_meta)	Sets optional metadata (with column names).
na_rep(self, str val)	Sets string to used for null entries.
utf8_escaped(self, bool val)	Sets whether to write UTF-8 characters in string fields without escaping them.

build(self) → JsonWriterOptions

Create a JsonWriterOptions object

compression(

self,

compression_type comptype,

) → JsonWriterOptionsBuilder

Sets compression type of output sink.

Parameters:

comptypeCompressionType

Compression type used

Returns:

Self

include_nulls(

self,

bool val,

) → JsonWriterOptionsBuilder

Enables/Disables output of nulls as ‘null’.

Parameters:

valbool

Boolean value to enable/disable

Returns:

Self

lines(

self,

bool val,

) → JsonWriterOptionsBuilder

Enables/Disables JSON lines for records format.

Parameters:

valbool

Boolean value to enable/disable

Returns:

Self

metadata(

self,

TableWithMetadata tbl_w_meta,

) → JsonWriterOptionsBuilder

Sets optional metadata (with column names).

Parameters:

tbl_w_metaTableWithMetadata

Associated metadata

Returns:

Self

na_rep(

self,

str val,

) → JsonWriterOptionsBuilder

Sets string to used for null entries.

Parameters:

valstr

String to represent null value

Returns:

Self

utf8_escaped(

self,

bool val,

) → JsonWriterOptionsBuilder

Sets whether to write UTF-8 characters in string fields without escaping them.

Parameters:

valbool

If False, disables escaping of UTF-8 characters in output

Returns:

Self

pylibcudf.io.json.chunked_read_json(

JsonReaderOptions options,

int chunk_size=100000000,

stream: CudaStreamLike | None = None,

DeviceMemoryResource mr=None,

) → tuple

Reads chunks of a JSON file into a TableWithMetadata.

Parameters:

optionsJsonReaderOptions

Settings for controlling reading behavior

chunk_sizeint, default 100_000_000 bytes.

The number of bytes to be read in chunks. The chunk_size should be set to at least row_size.

streamStream | None

CUDA stream used for device memory operations and kernel launches

Returns:

tuple

A tuple of (columns, column_name, child_names)

pylibcudf.io.json.read_json(

JsonReaderOptions options,

stream: CudaStreamLike | None = None,

DeviceMemoryResource mr=None,

) → TableWithMetadata

Read from JSON format.

The source to read from and options are encapsulated by the options object.

For details, see read_json().

Parameters:

options: JsonReaderOptions

Settings for controlling reading behavior

streamStream | None

CUDA stream used for device memory operations and kernel launches

Returns:

TableWithMetadata

The Table and its corresponding metadata (column names) that were read in.

pylibcudf.io.json.read_json_from_string_column(

Column input,

Scalar separator,

Scalar narep,

list dtypes=None,

compression_type compression=compression_type.NONE,

json_recovery_mode_t recovery_mode=json_recovery_mode_t.RECOVER_WITH_NULL,

stream: CudaStreamLike | None = None,

DeviceMemoryResource mr=None,

) → TableWithMetadata

Joins a column of JSON strings into a device buffer and reads it into a table using the JSON reader.

The source to read from is a string column of JSON records.

For details, see join_strings() and read_json().

Parameters:

input: Column

String column with json-like strings as rows

separator: Scalar

String scalar used to join the input strings

narep: Scalar

String scalar used to replace null values during join

dtypes: List

Set data types for columns to be read.

compression: CompressionType

Set compression type of the string column contents

recovery_mode: JSONRecoveryMode

Set recovery option for corrupted JSON input in string column

streamStream | None

CUDA stream used for device memory operations and kernel launches

Returns:

TableWithMetadata

The Table and its corresponding metadata (column names)

pylibcudf.io.json.write_json(

JsonWriterOptions options,

stream: CudaStreamLike | None = None,

) → void

Writes a set of columns to JSON format.

Parameters:

optionsJsonWriterOptions

Settings for controlling writing behavior

stream: Stream

CUDA stream used for device memory operations and kernel launches

Returns:

None