In-memory engine#

The in-memory engine (engine=pl.GPUEngine(executor="in-memory")) is the only non-streaming path in cudf-polars. It materializes the whole query in device memory on a single GPU.

For most workflows, prefer a streaming engine. Use the in-memory engine when:

  • The data comfortably fits in device memory and you want minimum setup.

  • You need LazyFrame.profile (see Profiling and Tracing).

  • You are debugging and want the simpler, non-streaming execution path.

result = query.collect(engine=pl.GPUEngine(executor="in-memory"))

This is the path documented in Polars’ own GPU support guide. By contrast, engine="gpu" (or engine=pl.GPUEngine()) selects the default streaming path on a single GPU (see Default engine="gpu"). That default accepts no options, so for anything beyond a quick script, construct an explicit engine.

Configuration#

The in-memory engine does not accept StreamingOptions. Pass keyword arguments to pl.GPUEngine(...) directly:

import polars as pl

engine = pl.GPUEngine(executor="in-memory", parquet_options={"chunked": True})

See the Polars GPU support guide for the full in-memory usage story.