Example configurations

The configuration files below use the default ports and connect the cluster to your workspace with the [license.on_prem] section from Install on bare-metal. Network addresses and static leader describes the addresses and how to change the ports, and the Config file reference lists every key. Create the /opt/db directory on the leader first.

Single node

Every component runs on node-0:

cluster_id = "polars-cluster"
instance_id = "node-0"

[license.on_prem]
cert_dir = "/etc/polars/certs"
workspace_id = "<WORKSPACE_ID>"
client_id = "<CLIENT_ID>"
client_secret = "<CLIENT_SECRET>"

[scheduler]
enabled = true

[worker]
enabled = true
task_service.public_addr = "127.0.0.1"
shuffle_service.public_addr = "127.0.0.1"
shuffle_location.local.path = "/mnt/storage/polars/shuffle"

[observatory]
enabled = true
max_metrics_bytes_total = 104857600
database_path = "/opt/db/observatory.db"

[monitoring]
enabled = true

[static_leader]
leader_instance_id = "node-0"
observatory_service.public_addr = "127.0.0.1"
scheduler_service.public_addr = "127.0.0.1"

Multi-node

The leader node-0 (192.168.1.2) runs the scheduler and observatory, and node-1 (192.168.1.1) runs a worker. Only the leader node has the [license.on_prem] section.

Leader node:

cluster_id = "polars-cluster"
instance_id = "node-0"

[license.on_prem]
cert_dir = "/etc/polars/certs"
workspace_id = "<WORKSPACE_ID>"
client_id = "<CLIENT_ID>"
client_secret = "<CLIENT_SECRET>"

[scheduler]
enabled = true

[observatory]
enabled = true
max_metrics_bytes_total = 104857600
database_path = "/opt/db/observatory.db"

[monitoring]
enabled = true

[static_leader]
leader_instance_id = "node-0"
observatory_service.public_addr = "192.168.1.2"
scheduler_service.public_addr = "192.168.1.2"

Worker node:

cluster_id = "polars-cluster"
instance_id = "node-1"

[worker]
enabled = true
task_service.public_addr = "192.168.1.1"
shuffle_service.public_addr = "192.168.1.1"
shuffle_location.local.path = "/mnt/storage/polars/shuffle"

[monitoring]
enabled = true

[static_leader]
leader_instance_id = "node-0"
observatory_service.public_addr = "192.168.1.2"
scheduler_service.public_addr = "192.168.1.2"