Fontana CLI
The fontana CLI is the single operator entry point for a Fontana host. You declare desired state in fontana.yaml and reconcile the box with fontana apply. The CLI manages the platform k3d cluster (welcome gateway, shared services) and one k3d cluster per workspace tenant (the full Fontana app stack). It ships inside the verified deploy bundle that your host pulls from GHCR (or from your private registry mirror).
Install
Section titled “Install”export FONTANA_GHCR_PAT=… # read:packages token from Fontanacurl -fsSL <pages-host>/cli/install.sh | sudo -E bash # Linuxcurl -fsSL <pages-host>/cli/install.sh | bash # macOS (no sudo)The installer links fontana into your PATH and seeds fontana.yaml from the bundle template. All CLI-owned state (releases, snapshots, kubeconfigs) lives under one FONTANA_HOME directory (/opt/fontana on Linux production hosts, ~/.fontana on macOS).
Configuration
Section titled “Configuration”fontana.yaml is the sole declarative install config for the host. It is per-host and not checked into your own git (like .env.local). The installer copies a starter template on first install.
| Key | Purpose |
|---|---|
baseDomain | Platform hostname (localhost for lab installs; your production domain otherwise). Tenants serve at <name>.<baseDomain>. |
source.kind | ghcr (pull bundle + images from GHCR or your private registry mirror). |
source.tag | Release tag; pin an immutable sha-… tag in production. |
snapshots.retain | How many full-stack snapshots to keep per tenant. |
platform.docling.disabled | When true, skip platform-cluster Docling (saves RAM; PDF preload fails hard at use time). Omitted defaults to enabled. |
platform.opensandbox.disabled | When true, skip OpenSandbox charts and image imports (sandbox MCP fails hard at use time). Omitted defaults to enabled. |
tenants | Desired workspace list. fontana apply creates missing tenants and upgrades existing ones. |
Secrets never belong in fontana.yaml. At apply time, export:
FONTANA_GHCR_PAT(and optionallyFONTANA_GHCR_USER) so the CLI can pull the release bundle and images- Convex instance secrets are auto-generated in-cluster by the vault-seed hook
Override the config path with --config <path> or FONTANA_CONFIG.
Example production config:
version: 1baseDomain: flow.example.comsource: kind: ghcr tag: sha-abc123def456snapshots: retain: 3tenants: demo: {}Light local installs (RAM-constrained hosts)
Section titled “Light local installs (RAM-constrained hosts)”On a laptop or workstation with limited RAM, you can omit heavy platform-cluster singletons that you do not need for day-to-day Flow work. Add a platform: block to fontana.yaml (both keys optional; omitted means enabled, matching production):
platform: docling: disabled: true # skips Docling (~2Gi request); PDF preload fails hard if used opensandbox: disabled: true # skips OpenSandbox control plane; sandbox MCP fails hard if usedfontana apply logs disabled services, skips their image import and Helm deploy, and reconciles away workloads that were previously running. Re-enable a service by setting disabled: false or removing the key, then run fontana apply again.
fontana apply
Section titled “fontana apply”fontana apply is the day-to-day reconcile command. It:
- Ensures the platform k3d cluster is running and deploys the gateway + shared charts (Docling, OpenSandbox, and related platform services when not disabled in
fontana.yaml). - For each tenant in
fontana.yaml: creates or upgrades the tenant cluster and deploys the Fontana platform Helm release. - Refreshes Caddy edge routes and the managed
/etc/hostsblock once at the end (not mid-tenant, so routes stay stable during maintenance). - Runs fail-closed post-apply health checks (clusters, workloads, HTTPS probes).
Tenants that exist on the box but are absent from YAML are warned only. Removal always requires an explicit fontana tenant destroy <name>.
Upgrade pipeline (per tenant)
Section titled “Upgrade pipeline (per tenant)”When the bundle release version differs from what is deployed in-cluster, the CLI takes a pre-upgrade snapshot (stops the cluster briefly, archives the data volume), then redeploys. While upgrading, Caddy serves a static maintenance page for that tenant’s hostnames; auth hostnames stay proxied so OIDC discovery keeps working.
Gates you will see in logs:
| Gate | Why |
|---|---|
cluster_wait_ready | Blocks kubectl/helm until server-0, serverlb, and API /readyz succeed (required after snapshot stop/start). |
| Vault unseal | Vault seals on cluster restart; deploy fails closed until unsealed. |
| Image import verify | After k3d image import, the CLI confirms the image ref is present on the node. |
apply_status_check | Exits non-zero if any component or HTTPS route is unhealthy. |
Skip pre-upgrade snapshots when re-applying the same release with FONTANA_SKIP_SNAPSHOT=1.
Command reference
Section titled “Command reference”Top-level
Section titled “Top-level”| Command | Description |
|---|---|
fontana apply | Reconcile platform + all YAML tenants (create or upgrade). |
fontana status | Component health checks; exits non-zero on failure. |
fontana audit | Compare live k3d clusters and edge hostnames to fontana.yaml. |
fontana resources <tenant> | List HTTPS URLs and operator login hints (Flow, Gatus, Convex dashboard, ImmuDB, HyperDX). |
fontana secrets <tenant> | Print deploy credentials for .env.local (Convex, Zitadel, Vault keys, ImmuDB WORM audit, automation PAT), plus an Operator surfaces block with Gatus and HyperDX browser logins. |
Platform
Section titled “Platform”| Command | Description |
|---|---|
fontana platform create | Install platform cluster + gateway (alias: add). |
fontana platform upgrade | Redeploy platform charts from the current release. |
fontana platform destroy | Delete the platform cluster (tenants untouched). |
fontana platform start / stop / status / list | Cluster lifecycle and listing. |
Tenant
Section titled “Tenant”| Command | Description |
|---|---|
fontana tenant add <name> | Create tenant cluster + deploy the app stack. |
fontana tenant upgrade <name> | Snapshot (when release changed), then redeploy. |
fontana tenant destroy <name> | Final snapshot (unless skipped), delete cluster + data volume + routes. |
fontana tenant start / stop / status <name> | Cluster lifecycle. |
fontana tenant list | List tenants with app and auth hostnames. |
Snapshots and rollback
Section titled “Snapshots and rollback”| Command | Description |
|---|---|
fontana snapshot create <name> | Full-stack snapshot (release metadata + data volume tarball). |
fontana snapshot list [name] | List snapshots for a tenant. |
fontana rollback <name> [snapshot] | Restore data volume + redeploy pinned release (default: newest snapshot). |
Credentials and URLs
Section titled “Credentials and URLs”| Command | Description |
|---|---|
fontana resources <name> | HTTPS URLs for Flow, APIs, Gatus, Convex dashboard, HyperDX (observe-<tenant>), plus operator login hints (ImmuDB, Gatus, HyperDX). |
fontana secrets <name> | Full .env.local credential block for devcontainers (Convex, Zitadel client id, ImmuDB, service keys), followed by the Operator surfaces block (Gatus basic-auth and HyperDX admin logins from workspace Vault). |
Use fontana resources when you need browser URLs and sign-in details for operator consoles. Use fontana secrets when wiring a local dev loop or copying keys into .env.local.
State on disk
Section titled “State on disk”All CLI-owned paths sit under FONTANA_HOME:
| Path | Contents |
|---|---|
releases/ / current | Downloaded GHCR release bundles for the pinned tag. |
snapshots/<tenant>/ | Snapshot tarballs + meta.json (release version, image pins). |
kubeconfigs/ | Per-cluster kubeconfig files (not ~/.config/k3d). |
cluster-ports/<cluster> | Recorded API + HTTP port pairs for recreate after Docker cleanup. |
image-import-stamps/<cluster>/ | Content-hash stamps for skipped tarball imports. |
Each k3d cluster mounts a named Docker volume fontana-data-<cluster> at the k3s storage path. That volume is the unit for snapshots and rollback.
Host ports are discovered from the live serverlb container (no separate clusters.json registry). During snapshot stop, edge routing uses HostConfig.PortBindings so Caddy keeps stable :808x mappings while containers are stopped.
Recovery after manual Docker cleanup
Section titled “Recovery after manual Docker cleanup”If you remove k3d containers or images outside fontana (Docker Desktop prune, manual docker rm, and similar), k3d can lose its node records while the data volume still holds PVC data. Symptoms include:
No nodes found for given clusterduringfontana applyor after a snapshot restart- Docling or tarball import errors such as
content digest … not found - Edge HTTPS failing after a partial apply
The CLI is designed to recover automatically when the data volume remains:
- Snapshots can proceed even when k3d node records are missing.
- When a cluster fails to start but
fontana-data-<cluster>still exists, the CLI recreates the k3d cluster on the same host ports and re-attaches the data volume. - Port mappings are read from the live cluster or from
${FONTANA_HOME}/cluster-ports/<cluster>if containers are gone. fontana applytreats a tenant with a data volume but no k3d cluster as an upgrade (not a fresh create), so you do not lose tenant data.- Image imports: after recreate, the CLI re-pulls images from your registry and verifies each ref is present on the node before deploy continues.
What you should do:
export FONTANA_GHCR_PAT=… # required if Docker pruned cached imagesfontana applyVault seals when a cluster is recreated; the deploy pipeline unseals it before hook jobs run. If apply still fails, check fontana status and logs. A full reset is fontana tenant destroy <name> then fontana apply (destroys in-cluster state unless you restore from a snapshot).
Environment variables
Section titled “Environment variables”| Variable | Purpose |
|---|---|
FONTANA_CONFIG | Path to fontana.yaml (alternative to --config). |
FONTANA_HOME | Root for releases, snapshots, kubeconfigs, port records. |
FONTANA_GHCR_PAT / FONTANA_GHCR_USER | GHCR pull credentials at apply time. |
FONTANA_SKIP_SNAPSHOT=1 | Skip pre-upgrade and pre-destroy snapshots. |
FONTANA_SKIP_MAINTENANCE=1 | Skip maintenance-page Caddy routes during upgrade. |
FONTANA_LOG_FORMAT=json | Structured JSON logs (OTel-friendly). |
FONTANA_K8S_CLUSTER_READY_TIMEOUT | Seconds to wait for API /readyz (default 180). |
Logging
Section titled “Logging”Install and deploy logs use a unified format:
2026-06-30T06:11:48Z INFO [tenant-demo] apply: tenant 'demo': exists - upgrading2026-06-30T06:12:38Z INFO [tenant-demo] cluster: recreating k3d cluster (api=127.0.0.1:6446 http=8082, data volume preserved)Cluster context appears in brackets (platform, tenant-demo). Components include apply, cluster, snapshot, source, deploy, and edge.
Related documentation
Section titled “Related documentation”- Architecture: platform and workspace clusters
- Backup and restore: snapshots, rollback, Admin configuration backup
- Cloud deployment: managed cloud and customer VPC
- Self-hosted deployment: on-premises and private registry mirrors
- Deployment overview: deployment models and release pinning
- Observability: HyperDX onboarding during
fontana apply