Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Select an option

  • Save tobert/e5724af7c307eb564b9148f9a3a85100 to your computer and use it in GitHub Desktop.

Select an option

Save tobert/e5724af7c307eb564b9148f9a3a85100 to your computer and use it in GitHub Desktop.
gpt-5.6-sol's offline design review of kaish's approval-ledger API (via kaibo deliberate) — verdict: keep the architecture, do a contract pass before 0.14

Sol reviews the kaish approval-ledger design

2026-08-05. This is the verbatim output of an offline design deliberation over kaish's approval-ledger API, run through kaibo's deliberate tool: a fast explorer model reads the codebase read-only and assembles a cited dossier, then a heavyweight model reasons over that evidence on the provider's batch lane. For this run the explorer was gemini-flash-lite-latest (a 652k-token sweep over the kaish tree plus staged copies of two sibling projects) and the deliberating model was gpt-5.6-sol at max thinking. The deliberation-staging/ paths cited in the review are those staged copies — kaibo's sandbox is scoped to the kaish tree, so sources from kaibo and candle-lfm2-encoder were copied in for the dossier build and removed after.

How this run was asked for

Amy (kaish's maintainer) to her Claude Code session, verbatim:

have a kaibo deliberate, maybe sol, load up the ledger and hook code, maybe copy in some kaibo and kaish so it can grab some of them too and consider the design overall, if we should keep iterating on the kaish API to make it right. The goal is eventually kaibo-helper can have a kaish that has permission hooks that can back up through ACP for user approval. You can also copy in ~/src/candle-lfm2-encoder because that will be part of the approval chain, especially our new command classifier, and some siblings we'll train later (I am considering a router + specialist & llm fallback approach, sometime after we smooth this out). Sol has 1M context window, have the explorer be generous so Sol can think holistically about things.

Outcome: the maintainer read the verdict the same evening and re-scoped the release — the contract pass runs before 0.14.0 ships, while nothing external pins the API.

The prompt Claude gave the deliberation

Deliberate holistically on the kaish approval-ledger API: is it the right shape, and should we keep iterating on it before freezing anything at the 0.14.0 release? You are reasoning for the maintainer (Amy), who wants a design-level judgment, not a bug hunt.

THE NORTH STAR this API must eventually serve: a future embedder, kaibo-helper, will embed a kaish kernel whose permission hooks escalate outward through ACP (the Agent Client Protocol — the JSON-RPC protocol agent processes speak to client UIs/editors, which carries session-scoped permission-request round-trips) so a human user approves privileged operations from their client UI. Below the human sits a layered machine approval chain: first a local command classifier — an LFM2-based encoder with a sequence-classification head running on Candle, in-process, millisecond-latency (the copied-in candle-lfm2-encoder project; its training/ dir shows the command-dataset generation and finetune pipeline for exactly this classifier) — and later a router + specialist-classifier + LLM-fallback arrangement: a small router model dispatches to specialist classifiers per domain, with an LLM as the fallback judge for the novel remainder, all before anything reaches the human. Every layer must leave the typed audit record the ledger already defines.

WHAT EXISTS TODAY (all in this tree, main branch): docs/approval-ledger.md is the canonical spec and is implemented through PR 8 plus refinements — the Requester/Approvals/ApproverHandle capability split, the request/grant/attempt state machine, the four-stage decision chain (standing grants → Approver::policy sync → Approver::decide async under a patient hold with decide_budget default 300s → defer to exit 2), redemption-time precondition verification via StateResolver, /v/approvals + the approvals builtin, fs.* observability subscriptions (observe = chainless Observed tap entry, enforce = decision chain) with an engaged() atomic fast path, plugin gate production via ToolCtx::request_approval (now carrying presented: Option<&str>), deny_self_approval, and ToolSchema.operations for discovery. Section C.6 of the spec (freshly landed) adds the statement gate: a NON-OPTIONAL observe-all tap posting one chainless Observed entry per executed top-level statement carrying a typed Plan (unexpanded rendered text + per-command argv0/args/redirects), plus a StatementClassifier seam — sync, non-blocking, returning Observe (floor) or Gate{reason, risk} — where gate-classified statements enter the same decision chain. C.6's implementation is in flight on PR #296 (branch feat/statement-gate, a followed worktree; you are reading the main tree, so trust the spec text plus this summary for that part). #296's pre-merge review found instructive seams: the statement gate initially neither extracted a presented --confirm key from the plan nor redacted it from Capture::Statement's raw source — a credential leak and a broken redemption path found only when the two features met. That class of finding — two individually-correct layers composing wrongly — is exactly what this deliberation should hunt for at the design level.

COPIED-IN CONTEXT (deliberation-staging/, temporary copies of sibling projects your sandbox cannot otherwise reach — read them): candle-lfm2-encoder/ is the classifier that will sit in the chain (src/sequence_classification.rs is the head; src/lib.rs the API; training/ the dataset+finetune pipeline — note what its input actually is: a rendered text sequence, which is what Plan.rendered provides). kaibo/ holds the current MCP embedder's CLAUDE.md, README, and the three files that embed and drive kaish kernels today (sandbox.rs constructs kernels with KernelConfig; session.rs and explorer.rs drive them) — kaibo-helper will be a sibling of this code.

QUESTIONS TO DELIBERATE, in priority order:

  1. THE HOOK TOPOLOGY. There are now three hook shapes: Approver (policy sync + decide async), StatementClassifier (sync, scoping only), and subscriptions (glob registrations). Plus StateResolver for preconditions. Is this the right factoring for the north star, or accidental accretion? Specifically: the router+specialist+LLM-fallback chain — does it map cleanly onto the existing stages (classifier as scoping, specialists as policy, LLM as decide), or does the API need an explicit chain-of-approvers composition (today there is exactly ONE Approver; composing several means hand-writing a fan-out inside one impl — is that acceptable or a missing primitive)?
  2. THE ACP ROUND-TRIP. An ACP permission request is an out-of-process async round-trip to a human with unbounded think time. Today's shapes for that: blocking decide under patient hold (bounded by decide_budget), or defer-to-exit-2 with out-of-band grant + confirm replay. Is the deferred path complete enough for an ACP embedder — request TTL (default 60s) vs human latency, renewal ergonomics (who calls renew, when), correlation (does the embedder get everything it needs on ExecResult.approval / job Gated status to build the ACP request UI: operation, risk, resources, transitions, plan, hint), and replay (Capture::Statement re-parse-and-execute vs Capture::Exact) — walk the full kaibo-helper flow end to end on today's API and name every gap or awkwardness.
  3. THE CLASSIFIER SEAM AS AN ML SEAM. StatementClassifier is sync and non-blocking by contract. The LFM2 encoder is ms-latency in-process — fine. But a router+specialist stage might want a few tens of ms, and an LLM fallback cannot be sync. Is the right design classifier-stays-sync (ML that is slow belongs in Approver::decide) — or does the posture seam need an async escape? Consider what Plan gives an encoder as features (rendered text, argv0s, redirect kinds): is anything missing that the classifier pipeline will want (cwd, session principal, prior-statement context, mount/URL categories)? Adding fields to Plan later is additive — but wire-frozen once 0.14 ships embedders.
  4. WHAT TO FIX BEFORE 0.14. The release freezes wire shapes (LedgerEntry serde, ApprovalRequestView, --json envelopes) and embedder API signatures for two downstream embedders that pin versions. Given 1-3: rank the concrete API changes worth making NOW while everything is still unreleased, versus safely additive later. Be specific — signatures, fields, defaults.
  5. Any structural risk you see that nobody has named: capability-split soundness for a multi-session helper (one ApproverHandle per process — is that right when kaibo spawns many subagent sessions?), ledger scaling under always-on statement observe in a long-lived helper, the trust story when the classifier itself is model-driven (a misclassifying specialist auto-observing something that should gate — what bounds the blast radius?).

DELIVER: (a) a verdict — is the API fundamentally right for the north star, yes/no/with-caveats; (b) a ranked pre-0.14 change list with concrete signatures; (c) a defer-list of things that are safely additive; (d) the end-to-end kaibo-helper walkthrough with every gap named; (e) structural risks. Cite file:line evidence from the dossier throughout.

Attached whole for the dossier: docs/approval-ledger.md, docs/EMBEDDING.md, crates/kaish-types/src/approval.rs, crates/kaish-tool-api/src/ctx.rs, crates/kaish-kernel/src/ledger/{approver,handles,subscription,operation}.rs, and from the staged copies: candle-lfm2-encoder's README, CLAUDE.md, src/lib.rs, src/sequence_classification.rs, training/README.md, and kaibo's CLAUDE.md and src/sandbox.rs.


Sol's answer, verbatim

Verdict

The approval-ledger is fundamentally the right architecture for the north star, but it should not be frozen in its current public and wire shape for 0.14.0. Keep the capability split, four-stage decision chain, synchronous statement-posture seam, subscriptions, and redemption-time StateResolver. Those are distinct concerns rather than accidental accretion (crates/kaish-kernel/src/ledger/handles.rs:24, docs/approval-ledger.md:78, crates/kaish-kernel/src/ledger/approver.rs:220).

Before 0.14, do a bounded contract pass focused on:

  1. ACP request identity, lifetime, cancellation, and deferred resolution.
  2. Replay binding and systematic secret redaction.
  3. Execution context for statement classification.
  4. A typed way to audit every machine/human assessment.
  5. Versioned, scoped, paginated wire envelopes.

Do not delay 0.14 for a kernel-level vector of approvers or an async StatementClassifier. The single Approver should remain the authorization coordinator. A router/specialist/LLM/human arrangement is a routed decision pipeline, not necessarily a linear chain; it can implement the one Approver trait. Reusable pipeline combinators can be added later without changing the kernel boundary.

One correction to the dossier's narrative matters: the question states that the request TTL defaults to 60 seconds, whereas Approver::decide_budget defaults to 300 seconds. I use those values below. The dossier's one reference to a 300-second request TTL appears to conflate the two.


1. Hook topology: keep the factoring, not the current contextual thinness

The existing seams divide four genuinely different jobs:

  • Subscriptions select which lower-level events are observed or enforced.
  • StatementClassifier sets a cheap statement-level posture: observe or enter the approval chain.
  • Approver makes authorization decisions, with synchronous policy and asynchronous decision stages (crates/kaish-kernel/src/ledger/approver.rs:78).
  • StateResolver checks current facts when authority is redeemed, rather than trusting request-time state.

That separation is sound. In particular, StateResolver should not be folded into classifiers or approvers: request-time risk assessment and redemption-time validity are different questions.

How the north-star pipeline should map

The clean mapping is:

  1. Statement classifier: only cheap, conservative scoping.

    • Static hard rules.
    • The local preloaded LFM2 encoder, if it meets a strict latency budget.
    • Failures, low confidence, or out-of-distribution input must return Gate.
  2. Approver::policy: deterministic, non-I/O rules and cached facts.

    • Allow/deny lists.
    • Organization policy.
    • Possibly a very fast local specialist, but only if it really is policy-path latency and cannot block.
  3. Approver::decide: the asynchronous orchestrator.

    • Router invocation.
    • Specialist inference that takes tens of milliseconds or may queue.
    • LLM fallback.
    • ACP human escalation.
    • Timeouts, cancellation, and assessment recording.

The LFM2 classifier operates over raw rendered text sequences (deliberation-staging/candle-lfm2-encoder/src/sequence_classification.rs:40), so Plan.rendered is a natural feature source (docs/approval-ledger.md:1132). That does not mean all later ML belongs in StatementClassifier.

Keep StatementClassifier synchronous

The current contract is deliberately synchronous and non-blocking (docs/approval-ledger.md:1176). Keep that contract.

Making it async would put optional network or model latency in front of every top-level statement merely to decide whether approval is relevant. It would also make classifier unavailability capable of stalling ordinary shell execution. The safe escape hatch already exists: return Gate, then do slow inference in Approver::decide.

A tens-of-milliseconds specialist that is too slow for the statement hot path belongs in decide, not behind an async posture hook. A classifier implementation should be preloaded, bounded, and free of downloads, queue waits, or remote I/O.

Do not make the kernel own a generic approver chain yet

The kernel currently accepts one Arc<dyn Approver> (crates/kaish-kernel/src/kernel.rs:359). That is a reasonable single authority boundary. A generic Vec<Approver> immediately raises security-sensitive semantics:

  • Does deny override allow?
  • Does allow short-circuit?
  • Is fallback different from abstention?
  • Are specialists routed or all invoked?
  • Are timeouts abstentions or denials?
  • Is the human always last?
  • Which layer owns request renewal and ACP cancellation?

The proposed north-star topology is a DAG with routing and fallback, not a simple chain. Hand-writing that orchestration once in kaibo-helper is acceptable. Later, an additive ApprovalPipeline can implement Approver:

pub trait ApprovalLayer: Send + Sync {
    async fn assess(
        &self,
        request: &ApprovalRequestView,
        ctx: &DecisionContext,
    ) -> LayerOutcome;
}

pub enum LayerOutcome {
    Allow(Assessment),
    Deny(Assessment),
    Abstain(Assessment),
    Escalate(Assessment),
}

That does not require changing the kernel from one Approver to many.

The important pre-0.14 requirement is that the existing Approver boundary carry enough context and audit capability for such a pipeline.


2. Ranked pre-0.14 changes

The exact names can follow current conventions, but the semantics should be fixed before downstream embedders pin them.

1. Add session/attempt identity and make deferred approval a first-class wire result

Every request, transition, observation, and gated job needs a stable origin:

pub struct ApprovalScope {
    pub kernel_id: KernelId,
    pub session_id: SessionId,
    pub actor_id: Option<PrincipalId>,
}

pub struct ApprovalRequestView {
    pub id: RequestId,
    pub scope: ApprovalScope,
    pub attempt_id: AttemptId,
    pub parent_request_id: Option<RequestId>,

    pub created_at: Timestamp,
    pub expires_at: Timestamp,
    pub revision: u64,

    // Existing operation, risk, resources, transitions, plan, hint...
}

parent_request_id or a shared attempt_id matters because a statement gate can be followed by a plugin or fs.* enforcement gate. Without hierarchy, the ACP UI can show two apparently unrelated prompts for one command.

Use the same structured pending object in synchronous execution and jobs:

pub struct PendingApproval {
    pub request: ApprovalRequestView,
    pub resume: ResumeAction,
}

pub enum ResumeAction {
    ConfirmStatement { plan_digest: PlanDigest },
    RetryOperation,
    NotReplayable,
}

ExecResult.approval and job Gated should carry this object, not divergent or stringly-shaped metadata. The canonical view is said to contain operation, risk, resources, transitions, plan, and hint (docs/approval-ledger.md:1297), which is enough for most UI rendering once scope, timestamps, and structured resume semantics are added.

2. Separate request leases from decision patience

A 60-second request lifetime combined with a 300-second decide_budget is internally awkward: unless expiration is suspended during the patient hold, a request can expire while decide is still waiting. The available evidence does not establish whether expiration is suspended; if it is, document that explicitly. If it is not, fix the invariant before release.

Recommended configuration:

pub struct ApprovalLeasePolicy {
    pub initial_ttl: Duration,       // ACP-oriented default: at least 15 minutes
    pub max_renewal_horizon: Duration,
}

pub struct KernelConfig {
    pub approval_lease: ApprovalLeasePolicy,
    // existing configuration...
}

No finite default handles genuinely unbounded human think time, so renewal remains necessary:

impl Requester {
    pub async fn renew(
        &self,
        id: RequestId,
        expected_revision: u64,
        lease: Duration,
    ) -> Result<ApprovalRequestView, LedgerError>;

    pub async fn cancel(
        &self,
        id: RequestId,
        expected_revision: u64,
        reason: CancelReason,
    ) -> Result<ApprovalRequestView, LedgerError>;
}

impl ApproverHandle {
    pub async fn resolve(
        &self,
        id: RequestId,
        expected_revision: u64,
        resolution: ApprovalResolution,
    ) -> Result<ApprovalRequestView, LedgerError>;
}

There is already a requester renewal concept (crates/kaish-kernel/src/ledger/handles.rs:306). Preserve the capability meaning: the requester/session owner should renew the intent, not the approver. Letting the approver indefinitely extend requests would let the authority side keep stale operations alive after their originating session has gone away.

The kaibo-helper session task should renew while:

  • the ACP permission request remains open,
  • the originating session is alive, and
  • the operation has not been canceled or superseded.

Stopping renewal naturally expires abandoned intent. Every renew, cancel, expire, grant, deny, and redeem should be a typed transition.

resolve should be idempotent or revision-checked so a late ACP response cannot race expiry, renewal, cancellation, or an inline decision.

3. Bind replay to the approved attempt and centralize redaction

Capture::Statement is useful, but re-parsing source later is not equivalent to replaying the originally assessed operation. The working directory, environment, aliases, mounts, and expansion results may have changed.

Define this invariant:

Confirmation is a fresh attempt through all enforcement hooks. A grant may be redeemed only if the new attempt is covered by the approved operation/resources/preconditions and matches the required plan/context fingerprint; otherwise the kernel emits a new request.

The request should therefore carry a digest over the sanitized plan and security-relevant context:

pub struct PlanBinding {
    pub plan_digest: PlanDigest,
    pub cwd: VirtualPath,
    pub scope: ApprovalScope,
    pub sandbox_profile: SandboxProfileId,
}

StateResolver still verifies dynamic preconditions. A plan digest does not replace state resolution; it prevents an approval for one parsed statement from silently authorizing a materially different replay.

Capture::Exact should be explicitly documented as either:

  • live-kernel-only and non-recoverable after restart, or
  • serializable/durable.

The dossier does not establish which it is. ACP latency and helper restarts make that distinction observable.

Redaction must cover more than --confirm

The #296 finding is not just a local parser bug. It exposes a systemic problem: an always-on statement tap serializing rendered source can leak any inline credential, not only the approval bearer token. Examples include authorization headers, passwords in URLs, environment assignments, or tool-specific token flags.

Centralize extraction and redaction before the plan is:

  • classified,
  • stored as Observed,
  • attached to a request,
  • captured for replay, or
  • returned through /v/approvals.

Use a wire type that cannot accidentally serialize unredacted values:

pub struct RedactedText(String);

pub enum PlannedValue {
    Plain(String),
    Redacted {
        kind: SecretKind,
        fingerprint: Option<SecretFingerprint>,
    },
}

At minimum, the presented approval key must never appear in Plan, Capture::Statement, hints, logs, or ledger entries. Ideally the classifier consumes the same sanitized text, with <SECRET> markers. Secrets rarely improve command-risk classification enough to justify placing them in an audit surface.

4. Enrich the classifier input, while keeping Plan syntactic

Plan currently provides rendered text, statement kind, commands, arguments, and redirects under C.6 (docs/approval-ledger.md:1106, docs/approval-ledger.md:1132). That is a good syntax representation, but not enough security context.

Do not overload Plan with session policy. Introduce an input wrapper:

pub struct StatementClassificationInput<'a> {
    pub plan: &'a Plan,
    pub context: &'a ExecutionContext,
}

pub struct ExecutionContext {
    pub cwd: VirtualPath,
    pub scope: ApprovalScope,
    pub sandbox_profile: SandboxProfileId,
    pub mounts: Vec<MountDescriptor>,
}

pub struct MountDescriptor {
    pub virtual_prefix: VirtualPath,
    pub class: MountClass, // Project, Scratch, System, External, etc.
    pub access: MountAccess, // ReadOnly, ReadWrite
}

Do not expose host filesystem paths to classifiers or wire clients; use logical VFS paths and trusted categories. The current kaibo sandbox already distinguishes scratch MemoryFs from project LocalFs and read-only behavior (deliberation-staging/kaibo/sandbox.rs:130), so these categories correspond to real policy differences.

Change the classifier result to make failures and provenance auditable:

pub trait StatementClassifier: Send + Sync {
    fn classify(
        &self,
        input: &StatementClassificationInput<'_>,
    ) -> Result<StatementAssessment, ClassificationError>;
}

pub struct StatementAssessment {
    pub posture: StatementPosture,
    pub assessor: AssessorId,
    pub model: Option<ModelIdentity>,
    pub confidence: Option<Probability>,
}

The kernel must map Err to Gate, never Observe.

Prior-statement context should not be mandatory input. Stateful classifiers can maintain session-specific history keyed by scope.session_id, but authorization should not become dependent on opaque conversation history. Additional URL/resource categories can be optional future context fields if the structs and serde policy are prepared for extension.

5. Ensure every layer can append a typed assessment

The north star requires router, specialist, LLM, and human layers each to leave a typed audit record. The dossier proves immutable approval records exist, but it does not show a public, restricted mechanism by which sub-layers inside one Approver can record attributed intermediate assessments. This must be verified before release.

If it does not already exist, add an append-only recorder to the approver context:

pub struct DecisionContext {
    pub deadline: Instant,
    pub cancellation: CancellationToken,
    pub assessments: AssessmentRecorder,
}

pub struct ApprovalAssessment {
    pub request_id: RequestId,
    pub assessor: AssessorId,
    pub stage: AssessmentStage,
    pub outcome: AssessmentOutcome,
    pub reason: String,
    pub risk: Option<Risk>,
    pub confidence: Option<Probability>,
    pub model: Option<ModelIdentity>,
    pub latency: Duration,
}

Then evolve the trait now, while it is unreleased:

pub trait Approver: Send + Sync {
    fn policy(
        &self,
        request: &ApprovalRequestView,
        ctx: &DecisionContext,
    ) -> PolicyOutcome;

    async fn decide(
        &self,
        request: &ApprovalRequestView,
        ctx: DecisionContext,
    ) -> Decision;
}

A recorder is preferable to returning a vector of assessments: if an LLM call times out or decide is canceled at its budget, records emitted before cancellation must not disappear.

Model identity should include a stable model/version or weight hash. Otherwise "specialist allowed this" is not a reproducible audit statement.

6. Version and bound the wire surfaces

Before freezing LedgerEntry, ApprovalRequestView, /v/approvals, and JSON output:

pub struct LedgerRecord {
    pub schema_version: u16,
    pub sequence: u64,
    pub timestamp: Timestamp,
    pub scope: ApprovalScope,
    pub entry: LedgerEntry,
}

pub struct ApprovalPage {
    pub items: Vec<ApprovalRequestView>,
    pub next_cursor: Option<LedgerCursor>,
}

Also:

  • Mark public Rust structs/enums appropriately #[non_exhaustive].
  • Give future optional serde fields defaults.
  • Require clients to tolerate unknown object fields.
  • Define behavior for unknown enum variants/schema versions.
  • Add scope, state, time-range, and cursor filters now.
  • Do not make /v/approvals an unbounded array.

The always-on C.6 tap posts one Observed entry per top-level statement, so unbounded listing is not viable in a long-lived helper. Backend retention and storage implementations can evolve later, but the pagination envelope and stable sequence cursor should be frozen now.


3. Changes that can safely wait

These do not need to block 0.14 once the underlying contexts and wire extension points exist:

  1. ApprovalPipeline/CompositeApprover library combinators.
    Additive because they implement the existing single Approver boundary.

  2. Actual router, specialist, LLM, and ACP adapter implementations.
    The kernel need not know their topology.

  3. Parallel/quorum/deny-overrides composition policies.
    These belong in a pipeline crate after real usage clarifies semantics.

  4. Additional classifier features.
    URL classes, resolved executable identities, platform metadata, or optional historical features can be added to an extensible context.

  5. Confidence calibration and model tuning.

  6. Persistent ledger backends, compaction policies, and export sinks.
    The implementation is additive once cursor, scope, and retention semantics are fixed.

  7. An async posture hook.
    It is not currently desirable. If a future use case proves one is necessary, add a distinct preflight stage rather than changing the meaning of the cheap statement classifier.


4. End-to-end kaibo-helper / ACP walkthrough

A. Session setup

For each ACP session, kaibo-helper creates or derives:

  • a kernel,
  • a requester scoped to that kernel/session,
  • read-only Approvals,
  • an approver coordinator connected to ACP,
  • the local statement classifier,
  • subscriptions and StateResolver.

The capability split is appropriate (crates/kaish-kernel/src/ledger/handles.rs:24). The ACP bridge, not an agent or plugin, owns the granting capability. Tools receive only the request-facing surface exposed through ToolCtx::request_approval (crates/kaish-tool-api/src/ctx.rs:65).

Current awkwardness: the dossier does not show a mandatory session/actor identity on requests. If multiple kernels share a ledger or handle, the helper needs external maps to recover this association.

B. A statement arrives

The kernel parses the top-level statement and emits the mandatory Observed record with its typed plan under C.6 (docs/approval-ledger.md:1106).

Before that record is emitted:

  1. Extract any presented confirmation key.
  2. Remove it from classifier input, plan, capture, hints, and logs.
  3. Apply general secret redaction.
  4. Attach session/attempt context.

The synchronous classifier returns:

  • Observe: no statement-level gate, though plugin and fs.* enforcement remain active.
  • Gate: create an approval request with reason/risk and enter the normal decision chain.

Current gap: the initial #296 interaction showed extraction and capture were not originally composed through one safe normalization path. The same class of leak remains possible for non-kaish credentials unless redaction is generalized.

C. The decision chain runs

The existing order is sound:

  1. standing grant,
  2. synchronous policy,
  3. asynchronous decide under a patient hold,
  4. defer to exit 2 (docs/approval-ledger.md:78, crates/kaish-kernel/src/ledger/approver.rs:220).

The approver coordinator should:

  1. run and record the router,
  2. run the selected specialist,
  3. run the LLM fallback if needed,
  4. escalate through ACP only if machine layers do not authorize or deny.

Each assessment is posted with attribution and model identity.

D. ACP escalation

The ACP bridge constructs a session-scoped permission request from ApprovalRequestView. The existing operation, risk, resources, plan, transitions, and hint are useful (docs/approval-ledger.md:1297). It additionally needs:

  • session/actor identity,
  • request and attempt IDs,
  • created and expiry times,
  • current revision,
  • structured resume behavior,
  • optional parent request ID.

There are two valid modes:

Bounded inline response

Approver::decide waits for ACP up to its budget. If the user responds, it returns allow or deny directly.

This only works if the request lease outlives the wait. A 60-second lease does not naturally match a 300-second decision budget.

Deferred response

The ACP bridge registers the outbound request in durable helper state and returns Defer. Execution returns exit 2 with PendingApproval.

The ACP RPC must not be owned solely by the decide future: the kernel may cancel/drop that future when the decision budget is reached. The bridge needs a separate response task or correlation registry that survives the return from decide.

Current awkwardness: this cancellation-survival requirement is not visible in the trait shape. At minimum, document it and provide the request ID, cancellation token, and deadline in DecisionContext.

E. While the human is thinking

The session-side requester renews the request lease. It stops renewing if:

  • the ACP session disconnects,
  • the agent cancels the attempt,
  • the job is terminated,
  • a replacement attempt supersedes it.

Current gaps:

  • The 60-second default is too short for normal human latency.
  • The existing renewal API apparently selects an implementation-defined renewal interval rather than expressing a desired lease (crates/kaish-kernel/src/ledger/handles.rs:306).
  • Cancellation and late-response behavior need explicit, idempotent semantics.
  • If helper restarts are supported, pending ACP correlation and replay information must be durable. The dossier does not establish durability.

F. The human answers

The ACP response handler looks up the session-scoped request and invokes the approver capability:

  • grant,
  • deny, or
  • perhaps cancel if ACP indicates the session ended.

Resolution is revision-checked. A late response to an expired or superseded revision is rejected and recorded rather than silently reviving it.

The bearer confirmation key stays inside trusted helper/kernel state. It should not be sent to the ACP UI or language model.

Current gap: without session scope and revision, RequestId plus an external map can work functionally, but it is vulnerable to stale-response and confused-deputy mistakes in a multi-session process.

G. Resume

For Capture::Exact, resume the exact retained invocation if it is still valid.

For Capture::Statement, reparse the sanitized source and statement index, then run the new attempt through all current hooks:

  • recompute plan and resources,
  • compare against the approved binding,
  • resolve preconditions,
  • redeem only if covered,
  • otherwise produce a new approval request.

Current gap: reparse-and-execute can change meaning if cwd, environment, mounts, or expansions changed. StateResolver only protects facts represented as preconditions; it cannot compensate for missing cwd/context binding.

The helper must also decide whether ACP approval implies automatic continuation. For a still-pending ACP tool call, automatic resume is reasonable. If the originating session has moved on or been canceled, it should not replay merely because a late user response arrived.

H. Nested enforcement

After statement approval, actual execution may encounter an fs.* or plugin gate. That is correct defense in depth, but it can create another ACP request.

The attempt and parent IDs let the UI explain that relationship. Do not automatically turn a broad statement approval into authority for every hidden child operation unless the grant's operation/resources explicitly cover it.


5. Structural risks

Multi-session confused deputy

One process-wide ApproverHandle is acceptable only if the helper is fully trusted and all sessions share one administrative authority. It must never be exposed to subagents or plugins.

The evidence does not establish whether kaibo uses one ledger/handle for all kernels or one per kernel; its sandbox construction shows kernel configuration but not the sharing topology (deliberation-staging/kaibo/sandbox.rs:130). If each kernel has a separate ledger, isolation is naturally stronger. If ledgers are shared, derive scoped approver and observer handles:

let session_approver = root_approver.scope(session_id);
let session_approvals = approvals.scope(session_id);

Read capability also needs scoping because always-on plans may contain sensitive command information.

Model false negatives

Observe is an authorization bypass for the statement-level gate. Therefore:

  • ML may elevate to Gate, but must not override static mandatory gates.
  • Errors, OOD input, and low confidence become Gate.
  • Dangerous syntax classes should have static gate floors.
  • Plugin and actual fs.* enforcement remain active.
  • Model weights and configuration should be authenticated and recorded.

The current kaibo read-only project mount and scratch MemoryFs provide meaningful containment (deliberation-staging/kaibo/sandbox.rs:130), but future writable/project/network configurations will have a larger blast radius.

LLM prompt injection

The command being judged is adversarial input. An LLM fallback must receive a structured assessment prompt that treats command text as data, not instructions. Its output must be schema-validated, and high-impact auto-allow should require either a specialist/static rule or human approval.

Ledger privacy and scale

Always-on statement observation creates both volume and secret-retention risk.

Define:

  • bounded in-memory retention,
  • durable sink behavior,
  • cursor pagination,
  • session filtering,
  • access control,
  • sink failure semantics.

If the audit record is security-critical, silently dropping entries under backpressure is unacceptable. Whether execution should fail closed when the durable sink is unavailable is a deployment policy, but it must be explicit.

Approval storms and inconsistent granularity

Statement, plugin, and filesystem gates can all trigger on one attempt. Without attempt hierarchy and clear grant coverage, users will see duplicate prompts and may learn to approve reflexively. Coalescing is a later policy feature; correlation must exist now.

In-process capability limits

The Requester/Approvals/ApproverHandle split is strong API hygiene, but it is not a security boundary against arbitrary code execution in the helper process. A compromised helper can access whatever root capabilities and ACP credentials that process owns. Strong tenant isolation eventually requires separate processes or OS-level compartments, not only Rust handle types.


Bottom line for 0.14

Keep the architecture; delay the freeze for one targeted design pass. The release need not wait for the actual router, specialists, LLM, or a generic chain combinator. It should wait until:

  • requests are session- and attempt-scoped,
  • deferred ACP resolution has coherent lease/cancel/race semantics,
  • pending results are structurally complete,
  • replay is bound to the approved context,
  • statement audit data is systematically redacted,
  • classifiers receive cwd/sandbox context while remaining synchronous,
  • every decision layer has a typed assessment recorder, and
  • wire/listing surfaces are versioned and paginated.

Once those are in place, the single-Approver architecture is a sound foundation for kaibo-helper and the later layered machine-to-human approval chain.

——— kaibo · batch · gpt · batch_6a73c4818d9c81909d54c78f4ed338f3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment