The public reports hub for the Intent Eval Platform — it verifies signed evidence before it renders a single row, and refuses to let one number lie about many predicates.
intent-eval-dashboard is the 6th member of the Intent Eval Platform and its satellite consumer — not one of the 5 convergence repos. It ingests DSSE-signed Evidence Bundles from the platform repos, re-verifies every one (OIDC subject, Rekor inclusion proof, DSSE signature, kernel schema) before rendering, and publishes the results as a static hub at labs.intentsolutions.io. Methodology-first by design: the eval-set browser ships before any results browser, so readers see what was measured before they see how it scored.
Every eval dashboard you've seen lies to you a little. It shows a big green "94% PASS" number that averages a security check against a formatting check against a behavioral eval — three completely different things measured against three completely different bars — into one meaningless percentage. It carries yesterday's green forward into today's empty slot so the wall of squares stays reassuring even when a data source went dark hours ago. And it renders whatever landed in the manifest without re-checking that the signature is real, the transparency-log entry exists, or the schema still matches.
For a platform whose entire pitch is trustworthy, signed evidence about AI behavior, a dashboard that fudges any of those things isn't a nice-to-have bug. It's the product contradicting itself in public. If the hub that displays the evidence can't be trusted, the evidence can't be trusted.
The pain is specific: you need a public place to show gate results and adoption signals across six repos and outside consumers, but you cannot let that place invent a rollup, back-fill a stale bucket, or render an unverified attestation — because the moment it does, every claim on the site becomes suspect.
intent-eval-dashboard treats "don't render anything you haven't verified" as a structural invariant, not a code-review reminder. A six-worker supervision tree ingests signed Evidence Bundles from each source repo. Every worker verifies four things per row — the OIDC issuer/subject/workflow_ref, the Rekor inclusion proof, the DSSE signature, and schema validation against the @intentsolutions/core kernel — before a single row reaches the renderer. A worker that fails to verify crashes with a structured reason; the supervisor marks the snapshot stale and the site renders the last known-good state with a loud stale_since badge. It never guesses.
Two integrity locks are built into the type system, not bolted on. First, there is no aggregate PASS% across heterogeneous predicates — the "C3" gate is a cross-predicate-aware scanner wired into CI, and the per-skill signals surface has no representable rollup field at all, so a combined score is unwritable, not just discouraged. Second, predicate URIs never appear at labs.* — they live only at evals.intentsolutions.io, enforced by test assertions on the URL host. Absence is always shown loudly: an hour with no verified data is colored as loudly as a failure and is never carried forward.
The site itself is deliberately boring plumbing — self-contained single-file HTML, zero build step, deployed by rsync to Caddy on a VPS. Anyone can curl a page and read the whole thing in view-source. The sophistication is in the verify-before-render pipeline, not a JavaScript framework.
| Aspect | Details |
|---|---|
| Who | Platform operators, eval-methodology readers, and anyone auditing the Intent Eval Platform's public claims; a separate tailnet-only view serves internal operators |
| What | A static reports hub — eval-set browser, gate-results browser, per-skill adoption/trust signals, freshness strip, retraction protocol, and ops-lite alerting — over verify-before-render ingest |
| Where | labs.intentsolutions.io, served by Caddy on a Contabo VPS; an inverse operator-internal view is tailnet-gated (Tailscale identity, no basicauth) |
| When | Whenever you need to show signed eval evidence in public without fudging a rollup, back-filling a stale window, or rendering an unverified attestation |
| Why | It is the only dashboard in the ecosystem that re-verifies every bundle at ingest and makes a lying aggregate score structurally unrepresentable |
| Layer | Technology | Purpose |
|---|---|---|
| Contracts | @intentsolutions/core@^0.9.0 |
Canonical kernel — Evidence Bundle / gate-result/v1 / retraction/v1 / dashboard-render/v1 schemas + UsageEvent/HumanReview entities; consumed as a pure integrator, never vendored |
| Ingest + render | TypeScript (Node 20+) | Six-worker verify-before-render supervision tree, view-models, generators |
| Signing verification | @sigstore/bundle, @sigstore/verify, sigstore |
DSSE signature + Rekor inclusion-proof checks at ingest |
| Validation | Zod 4 | Runtime schema validation against kernel schemas |
| Site | Single-file HTML + one shared style.css |
Zero build step; curl-inspectable pages; rsync deploy |
| Serving | Caddy on Contabo VPS | Public labs.intentsolutions.io origin; reload, never restart |
| CI | GitHub Actions (ingest-ci.yml, deploy.yml) |
C3 gate + uptime-claim guard + arm-symmetry + audit-harness verify as required checks |
| Tests | Vitest | Integrity bindings proven by synthetic fixtures |
- Verify-before-render, structurally — a six-worker supervision tree re-checks OIDC + Rekor + DSSE + kernel schema on every row; failure crashes the worker and shows a loud stale badge, never a guess.
- A lying aggregate is unwritable — the C3 no-aggregate-PASS gate is enforced in CI and the per-skill signals surface has no rollup field on its type, so a cross-predicate score cannot be represented.
- Absence is loud — an empty freshness bucket is colored as loudly as a failure and is never carried forward, inferred, or blanked.
- Honest retraction — Rekor entries can't be un-logged, so a retracted result returns 410 Gone with a tombstone disclosing the reason class, not a 404 that pretends it never existed.
curl-inspectable, zero-build — single-file HTML deployed by rsync; the whole page is readable in view-source, with no hidden framework complexity.- Pure kernel consumer — imports every type and schema from
@intentsolutions/core, eating the ecosystem's own dog food exactly like an external integrator would.
intent-eval-dashboard is the public-facing reports hub for the Intent Eval Platform. Its job is to take DSSE-signed Evidence Bundles produced by the five platform repos (plus selected external consumers), re-verify each one at ingest, and render the results as a static site at labs.intentsolutions.io. It is explicitly a satellite consumer — it produces zero canonical artifacts of its own and vendors nothing; it imports every contract from the @intentsolutions/core kernel exactly as an outside integrator would.
What's built and committed: the six-worker verify-before-render ingest supervision tree; the gate-results browser (gate-result/v1 rows from verified snapshots); the public per-skill adoption + human-trust signals surface; the freshness/decision-mix strip and /status USE-method view; the operator-internal (tailnet-only) inverse view; the retraction protocol with its Caddy-410 kill-switch; ops-lite ntfy alerting; and the Phase A.0 symmetric-render structural-diff gate. The internal teaching-dashboard testing lane under site-internal/internal/testing/ is also built. Data-wise the site is honest about being early: most sources currently render a loud no-data state because upstream emit-evidence isn't fully wired, and that emptiness is shown, never faked.
The technology foundation is deliberately minimal. Ingest and render are TypeScript on Node 20+. The site is self-contained single-file HTML with one shared stylesheet — no framework, no build step — deployed by rsync to Caddy on a Contabo VPS. Signature and transparency-log verification use the sigstore libraries; runtime schema validation uses Zod 4 against the kernel schemas.
The biggest risks are operational, not architectural, and they're known. Several surfaces (the tailnet-only operator hostname + Tailscale-gated Caddy block, the real ntfy push, the retraction Caddy reload) are deliberately human-gated VPS steps not wired into repo automation — until an operator does that wiring there is no live route to those outputs. The Astro migration is genuinely deferred; the site stays single-file HTML. Version is 0.1.0-pre — pre-release, private package.
| Category | Technology | Version | Purpose |
|---|---|---|---|
| Runtime | Node.js | ≥20.0.0 | Ingest workers, view-models, generators |
| Language | TypeScript | ^5.7 | All ingest + render code |
| Contracts kernel | @intentsolutions/core |
^0.9.0 | Evidence Bundle / gate-result/v1 / retraction/v1 / dashboard-render/v1 schemas + UsageEvent/HumanReview |
| Signing verification | sigstore / @sigstore/verify / @sigstore/bundle |
^5 / ^4 / ^4 | DSSE signature + Rekor inclusion proof at ingest |
| Validation | Zod | ^4.4 | Runtime schema validation |
| Testing | Vitest | ^2.1 | Unit + integrity-binding tests, synthetic fixtures |
| Lint/format | ESLint 9 + Prettier 3 | ^9 / ^3 | Static gates |
| Harness | @intentsolutions/audit-harness |
^1.1.5 | Hash-pinned policy verification (IS Testing SOP) |
| Package manager | pnpm | 9.15.0 | Dev deps (no framework deps at v0.1.0) |
| Serving | Caddy | — | labs.intentsolutions.io on Contabo VPS |
SIGNED EVIDENCE (from 6 sources: iec · iel · iah · iaj · iar · ccp)
│ DSSE-signed Evidence Bundles + Rekor log entries
▼
┌─────────────────────────────────────────────────────────┐
│ 6-worker ingest supervision tree (src/ingest/) │
│ per row, per worker — VERIFY BEFORE RENDER: │
│ 1 OIDC issuer/subject/workflow_ref (pinned allowlist) │
│ 2 Rekor inclusion proof 3 DSSE signature │
│ 4 kernel schema (@intentsolutions/core) │
│ fail → worker crashes w/ reason → supervisor marks │
│ last_known_good_stale_since; renderer uses prior │
│ snapshot + loud stale_since badge │
└───────────────┬─────────────────────────────────────────┘
│ RenderInput (verified snapshots only)
┌───────┼───────────────┬──────────────┬───────────┐
▼ ▼ ▼ ▼ ▼
┌────────┐┌────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│results/││skills/ │ │freshness/│ │retraction│ │alerting/ │
│gate- ││per-dim ││ │24-bucket │ │410 + tomb│ │7d-silent │
│result/ ││signals ││ │+ /status │ │ -stone │ │→ ntfy │
│v1 rows ││(no roll││ │USE view │ │ │ │prod-alert│
└───┬────┘└───┬────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘
│ C3 gate │ (no aggregate PASS% across predicates) │
▼ ▼ ▼ ▼ ▼
┌─────────────────────────────┐ ┌──────────────────────────┐
│ site/ (public HTML) │ │ site-internal/ (tailnet) │
│ served by Caddy → │ │ ALL tiers + WHY-public │
│ labs.intentsolutions.io │ │ badge; refuses to write │
│ (single-file HTML, rsync) │ │ into site/ │
└─────────────────────────────┘ └──────────────────────────┘
│ predicate URIs only ever RENDERED, pointed at
▼ evals.intentsolutions.io — NEVER declared at labs.*
| Decision | Chosen | Over | Why | Revisit When |
|---|---|---|---|---|
| Site format | Single-file HTML | Astro / SPA framework | Zero build step, curl-inspectable, no hidden framework complexity, plays to Claude's single-file HTML strength |
Interactive surfaces arrive at Phase 2 |
| Aggregate scoring | Per-predicate counts only | One rollup PASS% | A cross-predicate average is a lie; making it unrepresentable is the integrity binding | Never — this is a hard refusal |
| Stale data | Loud no-data (colored as fail) | Carry-forward last-known-good in-bucket | Silently filling a bucket makes a dark source look healthy | Never — carry-forward has no code path |
| Deploy wiring | Human-gated VPS steps | Full automation of Caddy/Tailscale/ntfy | Tailnet + kill-switch + paging seams are operator-authority, not CI-authority | An operator explicitly wires the VPS block |
| Kernel contracts | Import from @intentsolutions/core |
Vendor local types/schemas | Eat the ecosystem's own dog food; single source of truth | Never — vendoring is banned here |
| Retraction | 410 Gone + tombstone | 404 (pretend it never existed) | Rekor is append-only; honesty requires disclosing the retraction, not hiding it | Never — append-only-honesty binding |
The single-file-HTML choice trades interactive richness for total inspectability and a zero-dependency deploy. The per-predicate-only choice trades a satisfying single number for the platform's entire credibility. The human-gated-deploy choice trades one-command convenience for keeping tailnet exposure and paging under explicit operator control.
intent-eval-dashboard/
├── src/
│ ├── ingest/ # 6-worker verify-before-render supervision tree; renderer seam
│ ├── results/ # gate-result/v1 browser: visibility gate, row-model,
│ │ # render-html, C3 scanner; render-internal (operator view)
│ ├── skills/ # per-skill signals — 3 independent dimensions, NO rollup field
│ ├── freshness/ # 24-bucket decision-mix + USE-method /status view
│ ├── retraction/ # denylist, signed retraction/v1 statement, Caddy 410, tombstone
│ └── alerting/ # 7-day-silence evaluator + ntfy formatter + no-uptime guard
├── scripts/ # generate-{results,skills,status,internal,retractions}.ts,
│ # lint-no-aggregate-pass.ts (C3), check-uptime-claims.ts
├── site/ # PUBLIC generated HTML — served by Caddy (public origin only)
├── site-internal/ # TAILNET-ONLY generated HTML incl. internal/testing/ lane
├── deploy/ # retractions.snippet (Caddy 410 blocks)
├── 000-docs/ # runbooks (retraction 4h SLO), decision records
└── .github/workflows/ # ingest-ci.yml, deploy.yml (paths: site/**)
| Capability | Command | Notes |
|---|---|---|
| Install deps | pnpm install |
pnpm 9.15.0; Node ≥20 |
| Run full gate | pnpm run check |
format + lint + typecheck + test + build + all generators + C3 (public & internal) + uptime + arm-symmetry |
| Run tests | pnpm test |
Vitest; integrity bindings proven by synthetic fixtures |
| Generate results | pnpm run generate:results |
Writes site/results/* from verified snapshots |
| Generate operator view | pnpm run generate:internal |
Writes site-internal/; refuses to write into site/ |
| C3 gate | pnpm run lint:c3 / :c3:internal |
Cross-predicate aggregate-PASS scanner; exit code is load-bearing (never pipe through tee) |
| Uptime-claim guard | pnpm run lint:uptime |
Fails if any uptime-SLA claim appears in site/ |
| Liveness alert pass | pnpm run check:liveness |
Pages only on >7-day silence; default transport is no-op |
| Harness verify | pnpm run harness:verify |
Hash-pinned policy check before commits |
| Deploy | GitHub Actions → Tailscale OIDC → SSH → rsync → caddy reload |
Triggers on site/**; NO build step; reload never restart |
- Six-worker verify-before-render ingest tree, results browser, per-skill signals surface, freshness strip +
/status, operator-internal view, retraction protocol, and ops-lite alerting are all built and committed (beads puxu.6/.7/.9/.10/.11/.12, ig4h.6). - Integrity bindings are enforced in code and test: C3 no-aggregate-PASS (structural + CI scanner), predicate-URI-at-
evals.*-only (URL-host assertions), no-carry-forward freshness (25h-silent-worker test), 7-day-only paging (boundary tests), no-uptime-claim guard (self-checking fixture). - Consumes
@intentsolutions/core@^0.9.0as a pure integrator — no vendored types, schemas, or validators. - CI carries the C3 gate, uptime guard, arm-symmetry lint, and audit-harness verify as required checks; Greptile + Gemini are advisory reviewers, CodeQL owns security.
- Live at
labs.intentsolutions.io, served by Caddy on the Contabo VPS via zero-build rsync deploy.
- Medium — Most sources currently render a loud no-data state because upstream emit-evidence isn't fully wired; the emptiness is honest, but the site has little real data to show yet.
- Medium — Several surfaces (tailnet-only operator hostname + Tailscale-gated Caddy block, real ntfy push, retraction Caddy reload) are deliberately human-gated VPS steps not in repo automation; until wired, there is no live route to those outputs.
- Low — Astro migration is deferred; the site stays single-file HTML until interactive surfaces arrive at Phase 2.
- Low — Package is
0.1.0-preand private; no published release artifact.
- Repo: https://github.com/jeremylongshore/intent-eval-dashboard
- Live site: https://labs.intentsolutions.io
- CI: GitHub Actions (
ingest-ci.yml,deploy.yml) — C3 + uptime + arm-symmetry + harness verify required - License: Apache-2.0
- Last Release: unreleased (
0.1.0-pre, private) - Governance: ratified by ISEDC Session 8 (DR-035 on
intent-eval-lab)