Other Engines

The examples in Usage use RayEngine. The pages below cover other ways to run cudf-polars:

• Dask runs on a Dask distributed cluster with one Dask worker per GPU. Use this when you already have a Dask deployment or a preferred Dask launcher.

• SPMD is single program, multiple data: the same script runs once per GPU, typically launched with rrun. Single-rank mode needs no external cluster at all.

• Default engine="gpu" documents what engine="gpu" does under the hood when no engine is constructed explicitly. Useful to understand, but for any non-trivial workflow we recommend constructing an explicit engine so you can pass StreamingOptions.

• In-memory engine (engine=pl.GPUEngine(executor="in-memory")) is the only non-streaming path. Suitable for small queries (data that fits in device memory), debugging, or when you specifically need LazyFrame.profile.

See Engines for the conceptual comparison with RayEngine (cluster model, runtime dependencies, typical use), and Configuration Options for the shared StreamingOptions configuration (the in-memory engine does not accept StreamingOptions).