Usage
How to invoke zisk-worker, the stateless proving process that dials the coordinator and proves assigned segments. Covers its three configuration sources and override precedence, the operationally relevant defaults, signal handling, compute-unit sizing, and worked examples.
A worker is the proving process. It dials the coordinator, registers a compute capacity, and proves the segments it's assigned. Workers are stateless across jobs — add, remove, or restart them at will. A worker host needs ~32 GB RAM and, optionally, a CUDA-capable GPU.
zisk-worker [OPTIONS]
Configuration
zisk-worker reads its settings from three sources, plus
built-in defaults for anything you don't set:
- TOML config file passed via
--config <PATH>. The canonical home for long-lived settings: coordinator URL, advertised capacity, connection tuning, telemetry. The full schema is documented on Configuration. - Environment variables. The binary reads only
ZISK_WORKER_CONFIG_PATH(the config file when--configis unset), plusZISK_HOME/ZISK_CACHE_DIRfor its paths. It ignoresRUST_LOG. The otherZISK_WORKER_*variables are read by the install script, which turns them into flags. Full mapping on CLI & environment. - Command-line flags for identity overrides, proving keys,
backend selection, performance tuning, and diagnostics. The
narrowest layer; use for one-off testing, container
entrypoints, or the systemd unit's
ExecStart. Full flag list on CLI & environment.
Override precedence
Settings are resolved in three layers, with later sources winning:
- Built-in defaults.
- TOML config file (
--config, elseZISK_WORKER_CONFIG_PATH). - Command-line flags.
Example: a TOML with [worker].compute_capacity.compute_units = 10
is shifted to 32 by --compute-capacity 32. Setting
ZISK_WORKER_COMPUTE_CAPACITY=16 in the worker's environment has
no effect: only the install script reads it, and it writes it into
the service as --compute-capacity 16. Neither RUST_LOG nor
[logging] changes the worker's log level; only -v (debug) and
-vv (trace) do.
Defaults
If you start zisk-worker with no flags and no config file, the
built-in defaults take over. The full schema is on
Configuration; the most operationally relevant
defaults are:
| Setting | Default | Override |
|---|---|---|
| Coordinator URL | http://127.0.0.1:50051 | [coordinator].url / --coordinator-url |
| Compute units advertised | 10 | [worker].compute_capacity / --compute-capacity |
| Worker identifier | random UUID | [worker].worker_id / --worker-id |
| STARK proving key path | ${ZISK_HOME}/provingKey ($HOME/.zisk/provingKey; /opt/zisk/provingKey for a system install) | --proving-key / ZISK_HOME |
| SNARK proving key path | ${ZISK_HOME}/provingKeySnark, read only with --snark | --proving-key-snark / ZISK_HOME |
| Final-SNARK jobs | off | --snark |
| Backend | Assembly (unless --emulator is set; on macOS the emulator is selected automatically) | --asm <PATH> / --emulator / --gpu |
| Inputs folder | . (current dir; under the install script's unit, its working directory /var/lib/zisk-worker) | [worker].inputs_folder |
| Reconnect interval | 5s | [connection].reconnect_interval_seconds |
| Telemetry | on, every 10s | [telemetry].enabled / --no-telemetry, [telemetry].interval_seconds |
| Log level | info | -v (debug) / -vv (trace) |
Signals
| Signal | Effect |
|---|---|
SIGTERM / SIGINT / SIGUSR1 | Stop now. The worker cancels the job in flight, kills its ASM services, unlinks their shared memory and semaphores, and exits with status 0, within about 5 s. Past 5 s it exits anyway and the next start unlinks what is left. Every MPI rank stops on its own. |
SIGKILL | Forced exit with no clean-up. Shared memory stays in /dev/shm until the worker starts again. |
A stopped worker doesn't finish its work. If it was computing, its
stream drops and, once reconnect_grace_period_ms passes, the
coordinator fails the job with Worker <id> connection dropped
and parks the job's other workers until they recover. The
coordinator doesn't reassign the job; the client has to submit it
again. Stop workers between jobs when you can.
Under MPI
mpirun can't pass SIGTERM on: it treats it as an abort, sends
the ranks SIGTERM and then SIGKILL, and exits with status 1.
Send SIGUSR1 to mpirun instead. It forwards the signal to every
rank, and with --mca orte_allowed_exit_without_sync 1 (Open MPI
4) it exits with status 0 once they all have. The service unit
the install script writes does both:
ExecStart=<path to mpirun> ... --mca orte_allowed_exit_without_sync 1 ... /usr/local/bin/zisk-worker ...
KillMode=mixed
KillSignal=SIGUSR1
TimeoutStopSec=60
A unit written by an older install script doesn't. Re-run the install script to pick up the new unit.
Exit status
| Status | Meaning |
|---|---|
0 | Stopped by a signal (see above). |
75 | EX_TEMPFAIL: the worker ended itself because it can't recover in-process and needs a restart. It happens when a cancellation handshake fails, when recovery takes more than 300 s, or when a panic hits a worker with several MPI ranks. Every rank is aborted. |
| other | A start-up error, such as a bad config or a missing proving key. |
Before exiting with 75 the worker logs, at ERROR and on stderr,
<reason>; aborting the worker so that it restarts. On its next
start it reports the reason to the coordinator (see
Telemetry). Run the worker under a
supervisor that restarts it on failure: the systemd unit uses
Restart=on-failure and the launchd plist KeepAlive. A worker
that keeps exiting with 75 points to a deeper problem.
Compute units explained
Compute units are an abstract weight the coordinator uses when
splitting a job. A worker advertising compute_units = 10 gets
roughly twice the load of one advertising 5.
Examples
Run with the config installed by the deploy script:
zisk-worker --config /etc/zisk/worker.toml
Connect to a remote coordinator with a stable worker id and a larger capacity than the default:
zisk-worker --config /etc/zisk/worker.toml \
--coordinator-url http://coord.lan:50051 \
--worker-id worker-a-gpu1 --compute-capacity 32
Use the GPU build with four concurrent streams:
zisk-worker --config /etc/zisk/worker.toml --gpu --max-streams 4
Serve final-SNARK jobs. --snark is required; the SNARK key must
be on disk (for example installed with --with-snark), and every
worker in the cluster must use a key of the same protocol:
zisk-worker --config /etc/zisk/worker.toml \
--snark --preload-snark \
--proving-key-snark /opt/zisk/provingKeySnark