Skip to content

Instantly share code, notes, and snippets.

@sini

sini/refresh_daemon_spec.md

Created August 20, 2026 21:48
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save sini/83f4ea5255d37b096a2b6c4031e5146a to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save sini/83f4ea5255d37b096a2b6c4031e5146a to your computer and use it in GitHub Desktop.
Download ZIP
OPKSSH Token Refresh Daemon — design specification (rev 2, post-adversarial-review; supersedes the draft discussed in openpubkey/opkssh#606)
Raw
refresh_daemon_spec.md

OPKSSH Token Refresh Daemon — Design Specification

Successor to the draft reviewed in openpubkey/opkssh#606 (gist version). This revision is the ratified design; §2 lists every material change from that draft with rationale.

1. Overview

opkssh daemon is a standalone, user-started background process that owns OIDC authentication and token refresh end-to-end. When it is running, every opkssh login delegates to it: the daemon runs the OIDC flow, holds all secrets, mints certificates, loads them into ssh-agent, and keeps them fresh by refreshing tokens and rotating agent entries in place. The CLI is purely the user interface — it launches or prints the browser URL, streams progress, and exits.

When no daemon is running, opkssh login behaves exactly as today, and opkssh login --auto-refresh runs the same refresh engine in the foreground. One engine, two hosts.

┌─────────────┐  start-login / status / drop   ┌──────────────────────────┐
│ opkssh CLI  ├───────────────────────────────►│      opkssh daemon       │
│  (UI only)  │◄───────────────────────────────┤                          │
└──────┬──────┘  auth-URL / progress events    │  ┌────────────────────┐  │
       │                                       │  │ Engine             │  │
       │ opens browser                         │  │  session workers   │  │
       ▼                                       │  │  refresh + rotate  │  │
┌─────────────┐   OIDC redirect callback       │  └─────────┬──────────┘  │
│   Browser   ├───────────────────────────────►│            │             │
└─────────────┘                                └────────────┼─────────────┘
                                                            ▼
                                               ┌──────────────────────────┐
                                               │        ssh-agent         │
                                               │  [opkssh:google:alice@…] │
                                               │  [opkssh:azure:alice@…]  │
                                               └──────────────────────────┘

2. Changes from the #606-reviewed draft

Draft (gist) Ratified Why
Login performs auth; sessions handed to daemon Daemon owns the auth flow; CLI relays the browser URL Secrets are born in the daemon — no refresh-token extraction, no secret handoff, no upstream openpubkey changes needed
login --auto-refresh --daemon spawns the daemon Daemon is user-started, never spawned by login Clean lifecycle ownership; aligns with restart/rehydration flow and later service management
--auto-refresh gates daemon involvement Daemon manages every login when running; --auto-refresh kept only for daemonless backward compat "Start the daemon once, then just opkssh login"
File-free re-hydration recovers sessions Rehydration recovers monitoring only; lost daemon = lost refresh capability Refresh tokens are not (and must never be) recoverable from certs or the agent; see §10
prune_stale_keys scan Deleted PR 1 (#611) always sets agent key lifetimes; the agent self-cleans
Desktop notifications, service install, Windows transports Documented follow-ups, out of the core PR Reviewability; each is separable
Mint refreshed certs with CertTimeInfinity Refresh-capable sessions' certs carry ValidBefore = token exp; non-refreshable and CI/CD logins keep PR-1 semantics Ratified by maintainer in #606: exp-binding is correct when refreshing keeps the token live — so it applies exactly to the sessions where refreshing does
full_scan_interval: 30s agent_sync_frequency: 60s, one merged reconcile/refresh pass Single ticker drives repair + refresh + sleep/wake detection

3. Design principles

  1. Secrets are born where they live. The OIDC keypair, refresh token, and access token come into existence inside the daemon and never leave it. No key material, tokens, or minted credentials cross the IPC socket, enter a certificate, or touch disk (core). The one non-derived secret that may transit the socket is a client_secret inside a full provider spec the user passed on the CLI — their own config value, over their own same-UID socket.
  2. File-free session state. All observable state is re-derivable from ssh-agent contents plus client config — no session-state files. (Refresh capability is deliberately not re-derivable; the optional persistence follow-up adds a credential file per identity; see §12.)
  3. Lost daemon = lost refresh capability. Certs in the agent remain valid until their exp/lifetime; a restarted daemon reports them honestly as monitor-only until re-login.
  4. Reconcile, don't hope. Desired state (sessions in daemon memory) is continuously reconciled against observed state (agent contents). Agent restarts and manual key removals are repaired within one sync interval.
  5. The client cannot know server policy. Certificate expiry and agent lifetimes are client-side bounds, not statements about server acceptance (§8).

4. Architecture and repository placement

One refresh engine, two hosts. Placement follows the repo's audited conventions: domain subsystem as a top-level flat package, commands as structs in commands/ with cobra wiring in main.go, platform splits by file suffix, config types in commands/config.

Unit Purpose
daemon/engine.go Session supervisor: one merged sync pass (repair, refresh, rotate), per-session workers, backoff, injected clock
daemon/session.go Per-identity state: provider, OpkClient, signer, current PK token + cert blob, timing, state machine
daemon/agent.go (+agent_unix.go/agent_windows.go) Agent verbs: dial (DialAgent, reused by login's PR-1 path), Add with lifetime + comment, Remove by blob, List
daemon/discovery.go Startup/foreign-cert scan: openpubkey-pkt extension → identity, exp (FreshIDToken-aware) → monitor-only entries
daemon/ipc.go / daemon/ipc_client.go (+sock_unix.go/sock_windows.go) ndjson protocol server/client, socket lifecycle, flock'd singleton lockfile
daemon/flow.go Daemon-side login flow: keypair, provider with OpenBrowser: false + SetLoginURIHook + SetDefaultWriters (the library prints the login URI and warnings even when hooked — those writers must feed the daemon log/event stream, not vanish), redirect listener, token exchange, session creation
commands/daemon.go, commands/status.go DaemonCmd, StatusCmd per the LogoutCmd idiom
commands/config/client_config.go daemon: block beside agent_lifetime

The engine consumes refresh capability as a one-method interface satisfied by *client.OpkClient; the foreground host feeds it the login's own client, the daemon host feeds it flow-born clients. No upstream openpubkey changes are required: providers.SetLoginURIHook + OpenBrowser: false capture the auth URL for relay, and each session's OpkClient.Refresh() runs as shipped.

5. Login flows

Daemon present (the normal path once the daemon is adopted):

  1. CLI resolves config, finds the socket, sends start-login in one of three forms: a provider alias, a full provider spec (the --provider flag's value — may carry a client_secret, see §3.1), or chooser (no provider named): the daemon builds the browser-provider list from its own config at that moment and serves the chooser page. Also carried: principals, key type/path, send-access-token flag.
  2. Daemon generates the keypair, builds the provider (OpenBrowser: false), binds the localhost redirect listener, captures the auth URL via SetLoginURIHook, and streams it to the CLI.
  3. CLI opens the browser or prints the URL (--disable-browser-open, remote-redirect, and webchooser flows unchanged — the daemon serves the chooser page; the CLI only launches URLs).
  4. OP redirects to the daemon's listener → token exchange → CIC signing → PK token assembled. Refreshability is determined here, per session: the provider must implement RefreshableOpenIdProvider and the token response must actually include a refresh token (generic-issuer configs build a plain StandardOp, which is not refreshable; a refresh-class provider without offline-access scopes grants no refresh token). Refresh-capable sessions mint ValidBefore = token exp, agent lifetime = exp + grace, and get a worker. Non-refreshable sessions mint with PR-1 semantics (CertTimeInfinity, agent_lifetime bound), get no worker, and appear in status as non-refreshable.
  5. Completion event → CLI prints the identity summary (including a warning if the CLI's own SSH_AUTH_SOCK differs from the agent socket the daemon manages) and exits.

Flow hygiene: one flow at a time — a second start-login is refused; its CLI exits non-zero with "another login is in progress" and a --no-daemon hint. A 5-minute flow timeout releases the redirect ports; CLI disconnect (Ctrl-C) cancels the flow.

Session replacement: sessions are keyed by (issuer, client_id, sub) — the same identity-continuity pair the library itself enforces across refreshes (SameIdentity); email is display-only and may be absent or reassigned. The identity is known only at flow completion; a match replaces the existing session — new cert added, old worker stopped, old blob removed (Add-before- Remove, so the identity never has zero usable keys). Agent comments are likewise display-only: opkssh:<alias-or-issuer-host>:<email-or-sub> — discovery always identifies certs by the openpubkey-pkt extension, never by comment.

Daemon absent: today's CLI flow. Plain login = one-shot (PR-1 semantics: CertTimeInfinity, agent_lifetime bound). --auto-refresh = the engine in the foreground with exp-bound certs, exactly the daemon's refresh behavior minus the daemon.

Flag semantics: --auto-refresh is backward compat, not a gate — with a daemon running it delegates like any login and returns; its promise ("keys stay fresh") is honored by the daemon. --no-daemon forces the daemonless path. --print-key, --configure, --create-config always bypass the daemon (--print-key's contract is key-to-stdout, which is never shipped over a socket). CI/CD provider logins (GitHub Actions, GitLab CI, Forgejo) bypass the daemon by design — machine identities in ephemeral jobs have no browser, no human, and no use for background refresh; they keep today's CLI path unconditionally.

6. Refresh and reconciliation

A single ticker per daemon (agent_sync_frequency, default 60s) drives one merged pass — this is also the sleep/wake and clock-jump detector, since every pass recomputes from the wall clock:

  1. agent.List() once. For each registered session:
    • cert missing from the agent → re-add from memory (agent restart, ssh-add -D; reconcile-authority: the sanctioned removal is opkssh logout, which drops the session too);
    • refresh due with one-tick lookahead: now + agent_sync_frequency ≥ exp − 1m → refresh now. (Without the lookahead, sampling a ≥ exp − 1m condition every 60s starts the refresh just before — or with a larger sync frequency, after — expiry, an outage window every single cycle.)
  2. Certs with no matching session → monitor-only entries (daemon restarted under a surviving agent, or daemonless logins); status shows "re-login to enable refresh". No pruning: every opkssh-added key carries a lifetime and the agent evicts it itself.

Refresh step: session.client.Refresh(ctx) → new exp read from FreshIDToken (never pkt.Payload, which keeps the original claims forever) → re-mint cert with ValidBefore = new exp, re-embedding the refreshed access token for send-access-token sessions (Refresh updates it; dropping it would break userinfo-based policy mid-session) → disk sync → agent rotation: Add new blob, then Remove old blob (entries are keyed by full blob; brief coexistence, never a gap). The keypair never changes across refreshes — the key binding lives in the original ID token by construction.

Failure handling — retries are independent of exp. Refresh tokens outlive the ID token's exp by design, and the oidc_refreshed server policy exists precisely to accept a certificate whose original token expired (§8) — so a post-exp refresh still succeeds and restores service with no re-login. Transient errors (network, timeouts, 5xx) back off 10s → 30s → 1m → capped at backoff_max and keep retrying indefinitely; a session whose cert expires while refresh is failing enters degraded: the agent entry evicts at its lifetime as usual, the worker keeps retrying at the backoff cap, and a later success re-mints and re-adds. Only terminal errors (invalid_grant, revocation) tombstone the session: worker exits, entry left to lifetime eviction, reason kept visible in status. Unknown errors are treated as transient. Agent unreachable → retry next pass (the existing cert is still valid; rotation is idempotent). Disk write failure → warn and continue (the agent has the fresh cert; disk retried next rotation).

7. FreshIDToken semantics

A PK token's key binding lives in the original ID token (its nonce commits to the user's public key). Refresh yields a new ID token without that commitment; it rides alongside as FreshIDToken — the latest OP-signed proof of session liveness. Consequences baked into this design:

  • exp for scheduling, ValidBefore, and discovery/status is read from FreshIDToken when present, falling back to the original payload;
  • the keypair is fixed for a session's lifetime (rotation = same key, new cert);
  • a refreshed cert embeds the compact PK token including FreshIDToken, which is what makes it verifiable under oidc_refreshed server policy.

The exp-from-FreshIDToken rule also fixes a live upstream bug: today's LoginWithRefresh reads bytes.Split(compactPkt, ".")[1] as "the payload", but the compact format is colon-joined with the fresh ID token dot-appended — so that index is the fresh token's protected header, which has no exp; the tracked expiry never updates after the first refresh and the loop hot-spins refresh requests against the OP (verified against pktoken/compact.go and commands/login.go:payloadFromCompactPkt).

8. Expiration honesty

Server-side acceptance is governed by the per-provider expiration policy in /etc/opk/providers (verifier/expiration.go):

  • oidc_refreshed — original exp, falling back to FreshIDToken exp. This is the policy the daemon serves: refresh keeps certs verifiable indefinitely.
  • max-age policies (default 24h) — enforce iat + maxAge on the original token; refreshing does not reset this clock. Under these policies a refreshed session still dies at original-auth + maxAge, and only a full re-login restarts it. The daemon cannot see or influence this; the PR and docs state it plainly.

Client-side bounds: refresh-capable sessions (daemon-managed or foreground-refresh) carry ValidBefore = token exp with agent lifetime = exp + 5m grace (internal constant). Non-refreshable sessions and daemonless one-shot logins keep PR-1 semantics (CertTimeInfinity + agent_lifetime, default 24h). Precedence with PR 1's flag: an explicit --lifetime always wins for the agent entry's lifetime in every mode, preserving #611's flag contract (ValidBefore is unaffected — an early-evicted entry is simply re-added by reconcile rule 1); the agent_lifetime config value applies only to non-refresh sessions.

9. IPC protocol

Newline-delimited JSON over a unix domain socket — the project's only wire format is JSON (JOSE stack, policy-plugin protocol); there is no gRPC/protobuf precedent and no appetite for that dependency tree in a security tool.

  • Envelope: {"v":1,"type":...}; unknown types tolerated (forward compat).
  • Messages: start-login, event stream (auth_url, progress, complete, error), status, drop, sync (reconcile poke).
  • Zero-credential invariant: nothing derived from key material or tokens crosses the socket beyond the public auth URL and status text.
  • Socket: sock_path config; default $XDG_RUNTIME_DIR/opkssh/daemon.sock, fallback os.TempDir()/opkssh-<uid>/ whenever XDG_RUNTIME_DIR is unset (macOS, headless Linux, bare ssh sessions); dir 0700 — same-UID trust model as ssh-agent.
  • Singleton authority is an flock'd lockfile beside the socket, not the socket itself — this closes three holes a socket-only singleton has: (a) the takeover race (two starters both see a stale socket, the second unlinks the first's freshly bound live socket → split-brain): lock is taken before bind, and unlink happens only while holding it; (b) a hung-but- alive daemon (probe timeout, distinct from connection-refused) is treated as running — no takeover; (c) XDG_RUNTIME_DIR cleanup at desktop-session end can delete the socket under a live daemon (tmux survivors) — every reconcile pass stats its own socket path and re-binds or exits loudly if it vanished, so the CLI never silently falls back to daemonless against a live-but-unreachable daemon.
  • Deadlines: 5s read/write everywhere except the event-driven login-flow wait, which runs under the flow timeout.

10. Security considerations

  • Refresh tokens never enter the certificate or the agent (considered and rejected): ssh presents the cert to every server attempted, and agent.List() returns cert blobs to any process with socket access — a long-lived, remotely usable, silent bearer credential must not be broadcast. The already-shipped precedent is openpubkey-act: a far weaker credential, still opt-in with a security warning. Encrypting into the cert is dominated by the keyring follow-up (same local threat model, plus broadcast ciphertext). Rotation litter (OPs rotate refresh tokens per use, some with grace windows) makes it worse.
  • Secrets born in the daemon never cross a process boundary in the core design. The persistence follow-up (§12) is the sole, explicit exception.
  • Socket trust = filesystem permissions (0700 dir), identical to ssh-agent.
  • --print-key bypasses the daemon by construction.
  • Daemon shutdown does not remove agent keys — daemon death is not logout; keys self-evict at their lifetimes.

11. Lifecycle, config, CLI

Lifecycle: foreground process. SIGTERM/SIGINT → graceful (workers stopped, socket unlinked, agent untouched). SIGHUP ignored. Worker panics are recovered and tombstone only their own session. Logging via log/slog to stderr; log_file redirects; levels per log_level.

Config (daemon: block in the client config; all fields strings with total accessor defaults; durations accept 60s or raw seconds):

daemon:
  ssh_auth_sock: ""         # agent socket; named for and defaulting to $SSH_AUTH_SOCK
  agent_sync_frequency: ""  # merged reconcile/refresh pass; default 60s
  backoff_max: ""           # refresh retry cap; default 5m
  log_level: ""             # debug|info|warn|error; default info
  log_file: ""              # default stderr
  sock_path: ""             # IPC socket; default $XDG_RUNTIME_DIR/opkssh/daemon.sock

CLI: opkssh daemon (foreground, --config-path only) · opkssh status (human output; --json; exit 0 iff daemon up) · opkssh login (no new flags except --no-daemon) · opkssh logout (daemon drop + direct agent purge by openpubkey-pkt extension, then disk cleanup as today).

opkssh daemon: running (pid 12345, uptime 2h13m)
agent: /run/user/1000/keyring/ssh
[google]  alice@company.com  active        refreshed 12m ago · next in 48m · cert expires 17:32
[azure]   alice@corp.org     backoff (3)   next attempt in 40s · cert expires 16:10
[gitlab]  bob@example.com    monitor-only  re-login to enable refresh · cert expires 18:00

status prints the agent socket the daemon manages, and both status and login completion warn when the CLI's own SSH_AUTH_SOCK differs from it — otherwise an agent restarted at a new path leaves the daemon faithfully maintaining keys no ssh client consults, with everything reporting "active". Session states: active / refreshing / backoff / degraded (expired, still retrying) / non-refreshable / monitor-only / expired (terminal, tombstoned) / login in progress.

12. Delivery sequencing

  1. Predicate PR (separate design discussion, before increment 2): XDG base-directory support ($XDG_CONFIG_HOME/opk, $XDG_STATE_HOME/opk, with ~/.opk fallback) and per-provider key-file naming replacing the single id_ecdsa. This is a correctness prerequisite: multi-provider sessions write disk keys for several identities and today's fixed path clobbers. Needs its own migration/back-compat design (existing paths, configureSSH IdentityFile references).
  2. Daemon PR — increment 1: engine + foreground host (extract and harden LoginWithRefresh: merged tick, backoff, FreshIDToken exp, exp-bound certs, rotation). No dependencies.
  3. Daemon PR — increment 2: daemon process, IPC, daemon-owned auth, reconcile loop, status, logout integration.
  4. Documented follow-ups (explicitly out of the core PR):
    • Cross-restart persistence: per-identity credential files (0600, under the predicate-PR layout; OS keyring where real). Purely additive inside the daemon. Requires refresh-token extraction — the small upstream GetRefreshToken()/WithRefreshTokens() accessor (or the interface-mirroring provider-wrapper fallback) moves to this item's dependency list.
    • Windows transports: named-pipe agent dial + IPC (_windows.go files exist from day one, returning a clear unsupported error).
    • Desktop notifications (shell-out, no new dependency).
    • Service manifests: opkssh service install (systemd user unit, launchd plist, Task Scheduler).
    • Key-binding refresh with agent-backed signer (#606 stretch goal): the daemon-side client construction is where an agent-backed crypto.Signer plugs in.

13. Testing strategy

Deterministic engine tests via an injected clock (now + tick source — the one deliberate testability seam), mock provider (providers.NewMockProvider, which implements RefreshTokens), and the in-process recording agent pattern from PR 1 (agent.ServeAgent over a temp unix socket).

  • Engine: refresh due with one-tick lookahead, exp from FreshIDToken; Add-before-Remove order, blob change, comment format, lifetime = exp+grace and --lifetime override; agent-wipe repair; single-key re-add; backoff ladder + cap + reset; terminal-failure tombstone (no re-add); expiry-while-failing → degraded → post-exp refresh success recovers (re-mint + re-add); refreshability determination (refresh-class provider without refresh token → non-refreshable, PR-1 mint, no worker); same-identity (issuer, client_id, sub) replacement incl. absent-email collisions; worker panic isolation.
  • IPC: envelope/version tolerance, deadlines, stale-socket takeover, second-instance rejection, single-flight login, disconnect-cancels-flow.
  • Daemon flow: mock-provider end-to-end in-process (start-login → session → agent → status); URL-relay leg at integration tier via the existing fakeop + gosubmit machinery.
  • CLI/config: OutWriter golden output, --json shape, exit codes; daemon: parsing and defaults per PR-1-style tables.
  • Integration (tagged): real ssh-agent + daemon processes, fake OP, actual SSH to containerized sshd via the agent; agent-kill repair; daemon-kill monitor-only rehydration; SIGTERM hygiene. Suite-wide SSH_AUTH_SOCK isolation already in TestMain; each test sets its own agent explicitly. Race detector in CI.
  • Not covered in the core PR: real-OP browser dances (manual checklist), macOS launchd behavior, Windows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment