Radius request sequence diagram

radius-request-sequence.md

Radius message request sequence

sequenceDiagram
    autonumber
    actor User
    participant Pi as pi CLI/TUI
    participant Ext as Radius provider extension<br/>(pi-provider.ts)
    participant Gateway as Radius Gateway Worker<br/>radius.pi.dev
    participant Catalog as Gateway Catalog<br/>(DO/storage)
    participant Ledger as AccountLedger Durable Object
    participant Store as Durable Object Storage
    participant PiDev as Radius UI / pi.dev<br/>radius-ui.pi.dev
    participant Upstream as Upstream Model Provider<br/>(OpenAI/Anthropic/etc.)
    participant Analytics as Analytics Pipeline / R2

    Note over User,Analytics: Prerequisite: pi already has a Radius access token and model config.

    rect rgb(245, 248, 255)
    Note over User,Analytics: First message request for this account/model

    User->>Pi: Send prompt
    Pi->>Ext: streamSimple(model, context, options)
    Ext->>Gateway: HTTP POST /v1/messages<br/>Authorization: Bearer access_token<br/>Accept: text/event-stream

    Gateway->>Gateway: Validate request body<br/>Resolve logical Radius model route
    Gateway->>Catalog: Load provider/model runtime status
    Catalog-->>Gateway: Catalog snapshot
    Gateway->>Gateway: Verify signed Radius access token

    Gateway->>Ledger: RPC selectDestinationAndReserve(route, candidates, estimated cost)
    Ledger->>Store: Load ledger state, sticky route, reservation
    Store-->>Ledger: No local grant and no sticky route yet
    Ledger->>Ledger: Choose upstream provider from candidates
    Ledger->>PiDev: HTTP POST /api/gateway/accounting/grants<br/>request local grant / usage-limit check
    PiDev-->>Ledger: Approved grant + account_id
    Ledger->>Store: Persist local grant, spend reservation,<br/>sticky route decision, alarm
    Ledger-->>Gateway: Selected upstream provider + spend lease

    alt Credit-backed model
        Gateway->>Gateway: Resolve configured provider API key
    else BYOK model
        Gateway->>PiDev: HTTP POST /api/gateway/credentials/resolve
        PiDev-->>Gateway: User's encrypted upstream credential
        Gateway->>Gateway: Decrypt upstream API key
    end

    Gateway->>Upstream: HTTP streaming request<br/>provider-native API
    Gateway-->>Ext: HTTP 200 text/event-stream<br/>x-pi-gateway-upstream-provider/model

    loop Upstream stream events
        Upstream-->>Gateway: Provider SSE/chunk
        Gateway->>Gateway: Convert provider event to pi-messages event
        Gateway->>Ledger: RPC recordUsage(usage delta)
        Ledger->>Store: Update reservation usage
        Gateway-->>Ext: SSE assistant event
        Ext-->>Pi: AssistantMessageEvent
        Pi-->>User: Render streamed output
    end

    Upstream-->>Gateway: Terminal done/error event
    Gateway-->>Ext: SSE done/error event
    Ext-->>Pi: Terminal event
    Gateway->>Analytics: waitUntil send analytics events
    Gateway->>Ledger: RPC finalize(status, final usage, analytics)
    Ledger->>Store: Move reservation to pending accounting event<br/>refund unused local grant
    opt Flush now or on Durable Object alarm
        Ledger->>PiDev: HTTP POST /api/gateway/accounting/batches
        PiDev-->>Ledger: Batch accepted
        Ledger->>Store: Delete flushed pending events
    end
    end

    rect rgb(246, 255, 246)
    Note over User,Analytics: Second message request for the same account/model shortly after

    User->>Pi: Send next prompt
    Pi->>Ext: streamSimple(model, updated context, options)
    Ext->>Gateway: HTTP POST /v1/messages<br/>Authorization: Bearer access_token<br/>Accept: text/event-stream

    Gateway->>Gateway: Validate request body<br/>Resolve same logical model route
    Gateway->>Catalog: Load provider/model runtime status
    Catalog-->>Gateway: Catalog snapshot
    Gateway->>Gateway: Verify signed Radius access token

    Gateway->>Ledger: RPC selectDestinationAndReserve(route, candidates, estimated cost)
    Ledger->>Store: Load ledger state and sticky route
    Store-->>Ledger: Fresh sticky route + active local grant
    Ledger->>Ledger: Reuse previous upstream provider
    alt Local grant has enough remaining balance
        Ledger->>Store: Persist new spend reservation<br/>No pi.dev grant network request
    else Local grant is expired or insufficient
        Ledger->>PiDev: HTTP POST /api/gateway/accounting/grants
        PiDev-->>Ledger: Additional approved grant
        Ledger->>Store: Persist updated grant + reservation
    end
    Ledger-->>Gateway: Same upstream provider + new spend lease

    Gateway->>Gateway: Resolve upstream API key<br/>(configured key or BYOK credential)
    Gateway->>Upstream: HTTP streaming request<br/>same sticky provider
    Gateway-->>Ext: HTTP 200 text/event-stream

    loop Upstream stream events
        Upstream-->>Gateway: Provider SSE/chunk
        Gateway->>Ledger: RPC recordUsage(usage delta)
        Ledger->>Store: Update second reservation usage
        Gateway-->>Ext: SSE assistant event
        Ext-->>Pi: AssistantMessageEvent
        Pi-->>User: Render streamed output
    end

    Upstream-->>Gateway: Terminal done/error event
    Gateway-->>Ext: SSE done/error event
    Gateway->>Analytics: waitUntil send analytics events
    Gateway->>Ledger: RPC finalize(status, final usage, analytics)
    Ledger->>Store: Pending accounting event + refund unused grant
    opt Batched accounting flush
        Ledger->>PiDev: HTTP POST /api/gateway/accounting/batches<br/>may include first and second request events
        PiDev-->>Ledger: Batch accepted
        Ledger->>Store: Delete flushed pending events
    end
    end

Loading