Skip to main content

Configuration

The full coordinator.toml schema for zisk-coordinator, documenting every table ([service], [server], [coordinator], [metrics], [logging], [backend]) and its settings and defaults, including the coordinator-core tuning file.

The TOML file describes zisk-coordinator's long-lived configuration. Every table is optional; if you omit a key, the built-in default applies. Every table is parsed with deny_unknown_fields: an unknown or misplaced key stops the coordinator at start-up, for example Error: unknown field `bogus`, expected one of `host`, `port`, `shutdown_timeout_seconds` for key `server` for a stray key under [server]. CLI flags and ZISK_COORDINATOR_* environment variables override values set here (for the log level, RUST_LOG wins over both). See CLI & environment for the full flag list.

coordinator.toml
[service]
name = "ZisK Coordinator"
environment = "production" # development | staging | production

[server]
host = "0.0.0.0"
port = 7000 # client-facing gRPC
shutdown_timeout_seconds = 30

[coordinator]
port = 50051 # worker-facing gRPC

[metrics]
enabled = true
host = "0.0.0.0"
port = 9090 # Prometheus + /health

[logging]
level = "info" # trace | debug | info | warn | error
format = "pretty" # pretty | json | compact

[backend]
mode = "coordinator"

[service]​

Coordinator identity. Used for log labels and status output; no behavioural effect on the proving pipeline.

SettingDefaultNotes
name"ZisK Coordinator"Shown in logs and status output.
environmentdevelopmentOne of development, staging, production.

[server]​

The public API plane. Clients talk to this port to submit jobs and retrieve proofs.

SettingDefaultNotes
host0.0.0.0Bind address. Bind to a specific interface to restrict access.
port7000Client gRPC port. CLI: --api-port, env: ZISK_COORDINATOR_API_PORT.
shutdown_timeout_seconds30Drain window after SIGTERM. Jobs that don't complete in this window are abandoned at forced exit.

[coordinator]​

The cluster plane. Workers dial this port to register and stream their assignments and proofs.

SettingDefaultNotes
port50051Worker gRPC port. CLI: --cluster-port, env: ZISK_COORDINATOR_CLUSTER_PORT.
config_file(unset)Path to a coordinator-core tuning file (worker pool size, phase timeouts, heartbeat intervals, job TTL, webhook URL).

Coordinator-core tuning

The config_file exposes internals the main TOML doesn't: how many workers can pile onto a single proof job, how long each proving phase is allowed to run, and how quickly a disconnected worker fails. The upstream production baseline lives at distributed/crates/coordinator/config/prod.toml and lays these settings out under a [coordinator] table:

coordinator-core.toml
[coordinator]
max_workers_per_job = 20
max_total_workers = 5000
phase1_timeout_seconds = 600 # 10 min
phase2_timeout_seconds = 1200 # 20 min
reconnect_grace_period_ms = 500

The core file is parsed with deny_unknown_fields too: a key in the wrong table stops the coordinator at start-up. The file may also carry [service], [server] (host, port, shutdown_timeout_seconds) and [logging] tables, but zisk-coordinator does not use them. The worker port comes from [coordinator].port in the main file, and the drain window and logging come from the main file's [server] and [logging].

The defaults below are the built-in ones, used when a key is absent. The prod.toml values above are a production baseline, not the defaults.

SettingDefaultNotes
max_workers_per_job10Accepted but has no effect in 1.4.0-alpha: the coordinator parses it and never reads it, so it doesn't cap the workers on a job.
max_total_workers1000Cap on registered workers in the cluster. The coordinator refuses new registrations past this limit.
phase1_timeout_seconds300Maximum duration of Phase 1 (Partial contributions). Workers that don't return their partial challenges in this window fail the job.
phase2_timeout_seconds600Maximum duration of Phase 2 (Prove). Workers that don't return their partial proofs in this window fail the job.
reconnect_grace_period_ms500Window after a worker's stream drops before the job it is computing fails. A worker that reconnects within the window keeps computing. Larger values absorb transient network blips; smaller values fail fast.
execution_timeout_seconds300Maximum duration of an execute-only job.
phase3_timeout_seconds100Maximum duration of Phase 3 (Aggregation).
distributed_aggregationtrueSpread the Phase 3 fold tree across the job's workers. false keeps every fold on one worker.
heartbeat_interval_seconds30Expected interval between worker heartbeats.
heartbeat_max_missed3Missed heartbeats before a computing worker is considered dead and its job aborted. The dead threshold is heartbeat_interval_seconds × heartbeat_max_missed.
job_monitor_interval_seconds10How often the background monitor sweeps for timeouts and stale heartbeats.
stale_disconnected_threshold_seconds300Seconds a worker can stay Disconnected before it is removed from the pool.
stuck_recovery_threshold_seconds600Seconds before a worker stuck in recovery after a cancelled or failed job is unregistered. 0 disables the sweep.
job_ttl_seconds3600Seconds a finished job (Completed, Failed, Cancelled) stays in memory so clients can still query it. 0 evicts it on the next monitor sweep.
webhook_url(unset)URL the coordinator POSTs to when a job completes or fails. A {$job_id} placeholder is replaced with the job ID; without one, /<job_id> is appended.
default_compute_units0Compute units a job gets when the request does not set any. 0 means all the capacity available now.
min_compute_units0Minimum compute units a job needs to start. Jobs are rejected with CLUSTER_UNAVAILABLE when the ready capacity is below it. 0 means the whole set-up fleet (ready, recovering and busy workers).
allow_client_random_seedfalseAccept a host randomness seed sent by the client (replays, CI). When false, a job that carries a seed is rejected with CLIENT_RANDOM_SEED_NOT_ALLOWED (1006). See Random seed gate.
allow_job_metadatatrueAccept caller-defined key/value metadata on job requests. When false, a job that carries metadata is rejected with JOB_METADATA_NOT_ALLOWED (1007).

Setting any of the four timeouts (execution_timeout_seconds and phase1, phase2, phase3_timeout_seconds) to 0 turns that timeout off. A phase that exceeds its timeout fails the job and cancels every worker on it.

warning

Keep allow_client_random_seed = false when clients are not trusted. The seed decides the values the guest reads from zkvm_random_u64, so a client that also writes the input could pick a seed that makes that input expensive (for example hash collisions). See Host randomness.

Point [coordinator].config_file at this TOML to load the tuning:

coordinator.toml
[coordinator]
port = 50051
config_file = "/etc/zisk/coordinator-core.toml"

[metrics]​

A plain HTTP server bound alongside the gRPC ports. GET /metrics returns a Prometheus text payload (cluster / worker / job counters); GET /health returns 200 OK while the binary is alive and is the canonical liveness probe.

SettingDefaultNotes
enabledtrueSet false to turn off the HTTP server. /metrics and /health both go away.
host0.0.0.0Listen address for the scrape endpoint.
port9090Scrape port. CLI: --metrics-port, env: ZISK_COORDINATOR_METRICS_PORT.
caution

enabled = false also turns off GET /health. The HTTP server serves both paths, and with metrics off it does not start. If the port can't be bound, the coordinator logs a warning and keeps running without either endpoint. For a probe that doesn't depend on this server, use the standard gRPC health service (grpc.health.v1.Health) on the client-facing [server].port, for example with grpc_health_probe -addr=:7000.

warning

Never expose the metrics port to the public Internet. The /metrics endpoint leaks operational detail and the binary performs no authentication on it.


[logging]​

SettingDefaultNotes
levelinfotrace, debug, info, warn, error. --log-level overrides it, and RUST_LOG takes precedence over both.
formatprettypretty, json, compact. Accepted but has no effect in 1.4.0-alpha: all three print the same plain-text lines.
file_path(unset)Also write logs to a file that rotates daily (the date is appended to the name). A relative name goes under ./logs/ in the working directory. Leave the key out on systemd hosts; journald captures stdout. An empty string still writes files, so omit the key instead of setting "".

[backend]​

Determines the role the binary plays at runtime. Keep the default coordinator for a real cluster.

SettingDefaultNotes
modecoordinatorcoordinator runs the embedded coordinator core and the worker-facing cluster port. mock is also accepted: it serves the client API from an in-memory mock backend for testing, opens no cluster port and proves nothing.