Install on bare-metal
The bare-metal deployment runs Polars Nanook on your own servers, without Kubernetes. The
polars-nanook-compute Python package runs the scheduler, worker, observatory and monitoring
components, configured through one TOML file. A bare-metal cluster connects to your workspace with a
service account, or runs air-gapped with an Enterprise license.
Python environment
The polars-nanook-compute package from PyPI
needs Python 3.11 or later. It installs the polars-nanook-compute command together with
cloudpickle and the polars release it is pinned to, listed per version on the
Releases page. The command runs inside that Python environment, so install the
packages your queries use, such as plugins, into the same
environment. This example installs them from a requirements.txt file. Clients run the same Python
release as this environment, down to the patch version (see
Connect to a cluster you run).
uv venv --python 3.12 && source .venv/bin/activate
uv pip install polars-nanook-compute -r requirements.txt
Connect to your workspace
A bare-metal cluster needs a license to run. A connected cluster registers with your workspace
through a service account: a client ID and secret for
the workspace. Set it, with the workspace ID, in the [license.on_prem] section of the
leader node:
cluster_id = "polars-cluster"
instance_id = "node-0"
[license.on_prem]
cert_dir = "./certs"
workspace_id = "<WORKSPACE_ID>"
client_id = "<CLIENT_ID>"
client_secret = "<CLIENT_SECRET>"
The cluster registers with your workspace the first time the service starts, and the leader node
keeps the compute token it gets in cert_dir. Keep that directory across restarts, or the cluster
registers again. The [license] section of the config file
reference lists every key.
Start the service on each node with this command, which reads the variables listed on the Environment variables page:
polars-nanook-compute service --config-path /etc/polars/config.toml
Production configuration
The main settings to review for production:
- Network addresses and static leader: the addresses of the services on each node, and the leader node that runs the scheduler and observatory.
- Example configurations: complete files for a single-node and a multi-node cluster.
- Result storage: where stored results go.
- Shuffle storage: where intermediate data between workers is stored.
- Worker resources and memory: the memory limit of each node, and running the service in a cgroup.
- Checkpointing.
- Monitoring and query history: persist the observatory database.
- Scaling service: the service you run to scale the worker pool of an autoscaled cluster.