Skip to content

Install on Ray

The Ray deployment runs Polars Nanook on a Ray cluster you already operate, next to your other Ray workloads. The polars-nanook-ray Python package starts the scheduler and the workers as Ray actors, each running the polars-nanook-compute package of a bare-metal node, and configures them from Python instead of a TOML file. A Ray cluster connects to your workspace with a service account, or runs air-gapped with an Enterprise license.

Python environment

The polars-nanook-ray package from PyPI needs Python 3.11 or later. It installs ray, the polars-nanook client and, on Linux, the polars-nanook-compute package with the polars release it is pinned to. Install it in two places:

  • Every node of the Ray cluster that can host an actor: the actors are defined in the package, and under the subprocess runtime they start the polars-nanook-compute command of the same environment. Install the packages your queries use, such as plugins, there too.
  • Where you start the cluster and run queries, the driver in Ray's terms.

Ray needs the same ray and Python versions on the driver and the nodes, and clients need the cluster's Python release down to the patch version (see Connect to a cluster you run). Install the same environment everywhere, for example from one requirements.txt file:

uv venv --python 3.12 && source .venv/bin/activate
uv pip install polars-nanook-ray -r requirements.txt

The cluster keeps its data in directories under /tmp/polars by default: stored results, the license, the query history, shuffle data and temporary data. Under the subprocess runtime, create them on every node, or point the configuration elsewhere:

mkdir --parents /tmp/polars/{anonymous-results,license,observatory,shuffle-data,temporary-data}

Connect to your workspace

A Ray cluster needs a license to run. A connected cluster registers with your workspace through a service account: a client ID and secret for the workspace. Set it, with the workspace ID, in the license field of the cluster configuration:

from polars_nanook_ray.config import (
    PolarsRayClusterConfig,
    PolarsSchedulerConfig,
    PolarsServiceAccountLicenseConfig,
    PolarsWorkerConfig,
)

config = PolarsRayClusterConfig(
    cluster_id="polars-cluster",
    num_workers=4,
    license=PolarsServiceAccountLicenseConfig(
        workspace_id="<WORKSPACE_ID>",
        client_id="<CLIENT_ID>",
        client_secret="<CLIENT_SECRET>",
    ),
    scheduler=PolarsSchedulerConfig(),
    worker=PolarsWorkerConfig(),
)

The cluster registers with your workspace the first time the scheduler starts, and keeps the compute token it gets in cert_dir, /tmp/polars/license by default. Keep that directory across restarts, or the cluster registers again.

Start and stop the cluster

PolarsRayCluster starts the scheduler and worker actors, and RayClusterContext does the same and is also the cluster context queries run on. Attach to the Ray cluster first, in the namespace of the Polars cluster:

import ray

from polars_nanook_ray.cluster import PolarsRayCluster

ray.init(address="auto", namespace=config.cluster_id)

cluster = PolarsRayCluster(config)
cluster.start()

start() waits until the scheduler and every worker are ready, and logs Engine cluster ready: scheduler at <ADDRESS>, 4 workers. The actors are detached: they keep running after the process that started them exits. Calling start() again with the same cluster_id, from any process attached to the namespace, reconnects to them instead of starting new ones. stop() stops the scheduler and the workers, as does leaving the with block of a RayClusterContext.

Actor placement

Ray places the actors on nodes with the resources they request:

  • The scheduler requests a share of a custom head resource, so it runs on the node started with ray start --head --resources='{"head":1}'. The autoscaling actor and the license server do the same.
  • Each worker requests worker.cpus_hint CPUs and worker.memory_hint bytes of memory. Ray uses them to choose a node, not to limit the worker: see Worker resources and memory.
  • The workers of one cluster listen on the same ports, so under the subprocess runtime two workers that Ray places on the same node collide. single_host_cluster=True offsets the ports of each worker, so several workers fit on one machine, as in the quickstart. Containers have ports of their own, so the Docker runtime needs no offset.

The cluster_id is also the Ray namespace of the actors, so several Polars clusters can share one Ray cluster as long as their cluster_id and ports differ. It may only contain letters, digits, dots, dashes and underscores, and starts with a letter or digit.

Runtimes

The runtime field of the configuration sets how each actor runs polars-nanook-compute.

PolarsSubprocessConfig, the default, runs the polars-nanook-compute command of the node's Python environment as a subprocess of the actor, with the environment of the Ray node. binary_path runs another executable of the command instead, such as one you built.

from polars_nanook_ray.config import PolarsSubprocessConfig

runtime = PolarsSubprocessConfig()

PolarsDockerConfig runs polars-nanook-compute in a Docker container that the actor starts with the docker CLI. The nodes need Docker and the image, but not the polars-nanook-compute package, so actors also run on hosts the package is not released for, such as macOS. Docker enforces cpus_hint and memory_hint as limits on each container.

Build an image that runs the command, with the Python release of your clients:

FROM python:<PYTHON_VERSION>-slim

RUN pip install --no-cache-dir "polars-nanook-compute==<VERSION>"

ENTRYPOINT ["polars-nanook-compute"]
docker build --tag polars-nanook-compute:<VERSION> .

Build or pull the image on every node, then name it in the configuration:

from polars_nanook_ray.config import PolarsDockerConfig

runtime = PolarsDockerConfig(image="polars-nanook-compute:<VERSION>")

The containers run as the user of the Ray node and join the polars-nanook-ray Docker network. They see the paths the configuration mentions, such as the license file and the storage locations, at the same path as on the host; list any other local path that queries scan or sink in extra_volume_mounts. The scheduler and observatory ports are published on the loopback interface of the head node only, so clients run on that node.

Environment variables

The actors pass the environment of the Ray node to polars-nanook-compute, so set these variables before ray start on each node. NANOOK__<SECTION>__<KEY> overrides a key of the config file that the configuration classes do not set, such as the worker extras for HDFS. The other variables are listed on the Bare-metal environment variables page. PolarsRayClusterConfig sets PLC_LOG_LEVEL from log_level, and POLARS_EULA_ACCEPTED from accept_eula.

export NANOOK__worker__extras__hdfs__enabled=true
ray start --address=<HEAD_NODE_ADDRESS>:6379

Production configuration

The main settings to review for production: