Radius current request flow

radius-current-request-flow.md

Radius current request flow

sequenceDiagram
    autonumber
    participant C as Client / Pi session
    participant G as Gateway Cloudflare Worker
    participant L as Account Ledger Durable Object
    participant U as Selected upstream provider

    Note over C: The client owns the conversation transcript.<br/>sessionId is correlation/stickiness input, not a live upstream connection.

    rect rgb(235, 245, 255)
        Note over C,U: Turn 1
        C->>G: POST /v1/messages<br/>sessionId + full context + "Investigate this bug"

        opt Multi-provider route or sticky auto-route
            G->>L: getRouteDecisionState(route/conversation key)
            L-->>G: current decision, if any
            G->>L: recordSelectedDestination(...)
            L-->>G: selected destination confirmed
        end

        G->>L: reserveLease(requestId, model, estimated cost)
        L-->>G: lease

        Note right of G: The Worker creates one upstream iterator<br/>bound to the selected destination.
        G->>U: Direct streaming provider request<br/>(context, provider credential, abort signal)
        U-->>G: Stream events / usage / done

        G-->>C: SSE response events

        loop Each changed usage observation
            G->>L: recordUsage(requestId, cumulative usage)
            L-->>G: allow / deny
        end

        G->>L: finalize(requestId, completed, final usage)
        L-->>G: reservation → pending settlement
        Note over L: The DO later delivers settlement to<br/>the control plane. It never proxies model traffic.
    end

    rect rgb(240, 255, 240)
        Note over C,U: Turn 2
        C->>G: POST /v1/messages<br/>same sessionId + prior transcript + "Fix that bug"

        Note right of G: This is a new Worker request and a new<br/>upstream stream—not a continuation of Turn 1's HTTP stream.

        opt Multi-provider route or sticky auto-route
            G->>L: getRouteDecisionState(route/conversation key)
            L-->>G: prior selected destination
            G->>L: recordSelectedDestination(...)
            L-->>G: selection confirmed
        end

        G->>L: reserveLease(new requestId, estimated cost)
        L-->>G: lease

        G->>U: New direct streaming provider request<br/>(complete updated context)
        U-->>G: New stream events / usage / done
        G-->>C: SSE response events

        G->>L: recordUsage(...) / finalize(...)
        L-->>G: accounting decisions
    end

Loading