Skip to content

Instantly share code, notes, and snippets.

@artydev
Last active June 15, 2026 08:00
Show Gist options
  • Select an option

  • Save artydev/fb3ac5c583b5db05f63d45ab00b47027 to your computer and use it in GitHub Desktop.

Select an option

Save artydev/fb3ac5c583b5db05f63d45ab00b47027 to your computer and use it in GitHub Desktop.
Using Local LLM (Ollama) through websocketd

Ollama × websocketd

#https://claude.ai/chat/45bf346a-27c9-4a8d-91ae-9cebd76f2f07

A minimal, zero-dependency browser chat interface for local Ollama models, built on websocketd and a bare-metal C# streaming bridge.

Browser ←── WebSocket ──→ websocketd ←── stdin / stdout ──→ C# bridge ←── HTTP/2 ──→ Ollama

No Node.js. No Python server. No framework. Just a static HTML file, a compiled .NET binary, and websocketd wiring them together.


What it does

  • Streams tokens from Ollama to the browser in real time, word by word, as they are generated
  • Maintains full conversation history across turns — the browser accumulates the message array and sends it whole on every request, so the model always has context
  • Reconnects automatically after each turn (websocketd spawns a fresh process per connection; the UI handles the reconnect transparently)
  • Lets you clear the conversation history at any time with a single button, starting a fresh context without reloading the page

Architecture

Layer File Role
Browser UI index.html Manages history array, sends JSON over WebSocket, renders streaming tokens
Process bridge websocketd Spawns one C# process per WebSocket connection, pipes stdin ↔ stdout
Streaming bridge OllamaWsBridge.cs Deserializes history from stdin, forwards to Ollama /api/chat, streams tokens to stdout
Inference Ollama Local LLM server, any model

How history works

websocketd spawns a fresh process for every connection, so the C# bridge is completely stateless. The browser owns the conversation history — a plain JS array of {role, content} objects. On each turn:

  1. The user's new message is appended to the array
  2. The entire array is serialized as JSON and sent over the WebSocket
  3. The C# bridge deserializes it and forwards it verbatim to Ollama's messages field, with a system prompt prepended
  4. When streaming finishes, the assistant's response is appended to the array in the browser
  5. The socket closes (process exits), the browser reconnects silently for the next turn

Why Console.Out.Flush() matters

By default, .NET's stdout is line-buffered when piped. Without an explicit flush after each token, the browser receives nothing until the process exits — which destroys the streaming effect. Every Console.Write(token) is immediately followed by Console.Out.Flush().

The __DONE__ sentinel

websocketd closes the WebSocket when the process exits, but the close event and the last message can arrive in either order. The bridge writes __DONE__ as the final stdout line so the browser can finalize the stream reliably before the socket drops. The onClose handler also calls finishStream() as a fallback, guarded against double-execution.

Zero-allocation token parsing

The C# bridge avoids heap allocation on the hot path:

  • ArrayPool<byte>.Shared provides a reusable 4 KB read buffer, returned in a finally block
  • ReadOnlySpan<byte> slices lines from the raw buffer without copying
  • Utf8JsonReader parses JSON directly on the stack
  • Utf8JsonWriter builds the Ollama request body into a MemoryStream using UTF-8 string literals ("model"u8) to skip encoding overhead

Requirements


Setup

1. Install websocketd

# macOS
brew install websocketd

# Linux
wget https://github.com/joewalnes/websocketd/releases/latest/download/websocketd-linux-amd64.zip
unzip websocketd-linux-amd64.zip && sudo mv websocketd /usr/local/bin/

# Windows — download the .exe from the releases page and add it to your PATH

2. Pull a model in Ollama

ollama pull llama3
# or whichever model you want to use

3. Build the bridge

dotnet new console -n OllamaWsBridge
cp OllamaWsBridge.cs OllamaWsBridge/Program.cs
cd OllamaWsBridge
dotnet build

Edit ModelName at the top of OllamaWsBridge.cs to match your model before building.

4. Launch

# From the folder containing index.html and the built binary:
websocketd --port=9090 --staticdir=. dotnet ".\bin\Debug\net8.0\OllamaWsBridge.dll"

--staticdir=. tells websocketd to also serve index.html as a static file, so you only need one port for everything.

5. Open the browser

Navigate to http://localhost:9090 — the status pill in the top right turns green when the WebSocket is live. Type a message and press Enter.


Configuration

What Where
Model name const string ModelName at the top of OllamaWsBridge.cs
System prompt const string SystemPrompt at the top of OllamaWsBridge.cs
Temperature writer.WriteNumber("temperature"u8, 0.7f) in StreamChatAsync
Ollama URL BareMetalOllamaClient constructor, default http://localhost:11434
WebSocket port --port flag passed to websocketd, mirrored in the ws:// field in the UI

Notes

One process per connection is websocketd's model and works fine for personal use. For a shared or production deployment you would replace websocketd with a persistent WebSocket server (e.g. ASP.NET Core with System.Net.WebSockets) and move history storage server-side.

CORS: if you open index.html from a different origin, add --origin='*' to the websocketd command.

Windows paths: use absolute paths or backslashes when pointing websocketd at the DLL on Windows, since relative ./ paths can resolve incorrectly depending on the working directory.

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Ollama / websocketd</title>
<style>
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
:root {
--bg: #0d0d0f;
--surface: #16161a;
--surface2: #1e1e24;
--border: #2a2a32;
--text: #e8e8f0;
--muted: #6b6b80;
--accent: #7c6ff7;
--accent-lo: #7c6ff718;
--green: #4ade80;
--yellow: #facc15;
--red: #f87171;
--font-mono: "JetBrains Mono", "Fira Code", "Cascadia Code", ui-monospace, monospace;
--font-ui: system-ui, -apple-system, sans-serif;
--radius: 10px;
}
body {
background: var(--bg); color: var(--text);
font-family: var(--font-ui);
height: 100dvh; display: flex; flex-direction: column; overflow: hidden;
}
header {
display: flex; align-items: center; gap: 10px;
padding: 14px 20px; border-bottom: 1px solid var(--border); flex-shrink: 0;
}
.logo {
width: 28px; height: 28px; border-radius: 7px; background: var(--accent);
display: flex; align-items: center; justify-content: center;
font-size: 13px; font-weight: 700; color: #fff;
}
header h1 { font-size: 14px; font-weight: 600; flex: 1; }
.header-right { display: flex; align-items: center; gap: 8px; }
#clear-btn {
padding: 3px 10px; background: transparent;
border: 1px solid var(--border); border-radius: 5px; color: var(--muted);
font-family: var(--font-mono); font-size: 11px; cursor: pointer;
transition: border-color 0.2s, color 0.2s;
}
#clear-btn:hover { border-color: var(--red); color: var(--red); }
.status-pill {
display: flex; align-items: center; gap: 6px;
font-size: 12px; font-family: var(--font-mono); color: var(--muted);
background: var(--surface); border: 1px solid var(--border);
border-radius: 20px; padding: 4px 10px;
}
.dot { width: 7px; height: 7px; border-radius: 50%; background: var(--red); transition: background 0.3s; }
.dot.connected { background: var(--green); }
.dot.connecting { background: var(--yellow); animation: pulse 1s infinite; }
@keyframes pulse { 0%,100%{opacity:1} 50%{opacity:0.3} }
#thread {
flex: 1; overflow-y: auto; padding: 24px 20px;
display: flex; flex-direction: column; gap: 20px; scroll-behavior: smooth;
}
#thread::-webkit-scrollbar { width: 4px; }
#thread::-webkit-scrollbar-thumb { background: var(--border); border-radius: 4px; }
#empty {
flex: 1; display: flex; flex-direction: column;
align-items: center; justify-content: center;
gap: 10px; color: var(--muted); font-size: 13px; text-align: center;
}
#empty svg { opacity: 0.25; }
.msg { display: flex; flex-direction: column; gap: 4px; max-width: 700px; }
.msg.user { align-self: flex-end; align-items: flex-end; }
.msg.ollama { align-self: flex-start; align-items: flex-start; }
.msg-label {
font-size: 11px; font-family: var(--font-mono);
color: var(--muted); text-transform: uppercase; letter-spacing: 0.08em;
}
.bubble {
padding: 10px 14px; border-radius: var(--radius);
font-size: 14px; line-height: 1.65; white-space: pre-wrap; word-break: break-word;
}
.msg.user .bubble { background: var(--accent); color: #fff; border-bottom-right-radius: 3px; }
.msg.ollama .bubble {
background: var(--surface); border: 1px solid var(--border);
color: var(--text); border-bottom-left-radius: 3px;
font-family: var(--font-mono); font-size: 13px;
}
.cursor::after {
content: "▋"; animation: blink 0.8s step-end infinite;
color: var(--accent); margin-left: 1px;
}
@keyframes blink { 0%,100%{opacity:1} 50%{opacity:0} }
/* history counter badge */
.turn-count {
font-size: 11px; font-family: var(--font-mono); color: var(--muted);
padding: 2px 8px; background: var(--surface2);
border: 1px solid var(--border); border-radius: 20px;
align-self: center; flex-shrink: 0;
}
footer { padding: 12px 20px 16px; border-top: 1px solid var(--border); flex-shrink: 0; }
.config-bar {
display: flex; align-items: center; gap: 12px; padding: 0 0 10px;
font-size: 12px; font-family: var(--font-mono); color: var(--muted);
}
.config-bar label { display: flex; align-items: center; gap: 5px; }
.config-bar input[type="text"] {
background: var(--surface2); border: 1px solid var(--border);
color: var(--text); font-family: var(--font-mono); font-size: 12px;
padding: 3px 8px; border-radius: 5px; width: 180px; outline: none;
}
.config-bar input[type="text"]:focus { border-color: var(--accent); }
#reconnect-btn {
margin-left: auto; padding: 3px 10px; background: transparent;
border: 1px solid var(--border); border-radius: 5px; color: var(--muted);
font-family: var(--font-mono); font-size: 11px; cursor: pointer;
transition: border-color 0.2s, color 0.2s;
}
#reconnect-btn:hover { border-color: var(--accent); color: var(--accent); }
.input-row {
display: flex; gap: 8px; align-items: flex-end;
background: var(--surface); border: 1px solid var(--border);
border-radius: var(--radius); padding: 8px 8px 8px 14px; transition: border-color 0.2s;
}
.input-row:focus-within { border-color: var(--accent); box-shadow: 0 0 0 3px var(--accent-lo); }
textarea {
flex: 1; background: transparent; border: none; outline: none; resize: none;
color: var(--text); font-family: var(--font-ui); font-size: 14px; line-height: 1.5;
max-height: 140px; min-height: 24px; overflow-y: auto;
}
textarea::placeholder { color: var(--muted); }
#send {
flex-shrink: 0; width: 34px; height: 34px; border-radius: 7px;
background: var(--accent); border: none; cursor: pointer;
display: flex; align-items: center; justify-content: center;
transition: opacity 0.15s, transform 0.1s;
}
#send:hover:not(:disabled) { opacity: 0.85; }
#send:active:not(:disabled) { transform: scale(0.95); }
#send:disabled { opacity: 0.35; cursor: default; }
</style>
</head>
<body>
<header>
<div class="logo">Ol</div>
<h1>Ollama via websocketd</h1>
<div class="header-right">
<button id="clear-btn">clear history</button>
<div class="status-pill">
<div class="dot" id="dot"></div>
<span id="status-text">disconnected</span>
</div>
</div>
</header>
<div id="thread">
<div id="empty">
<svg width="38" height="38" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.2">
<path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"/>
</svg>
<span>No messages yet. Say something.</span>
</div>
</div>
<footer>
<div class="config-bar">
<label>ws:// <input type="text" id="ws-host" value="localhost:9090" /></label>
<button id="reconnect-btn">reconnect</button>
</div>
<div class="input-row">
<textarea id="prompt" rows="1" placeholder="Ask anything… (Enter to send, Shift+Enter for newline)"></textarea>
<button id="send" aria-label="Send">
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="#fff" stroke-width="2.2" stroke-linecap="round" stroke-linejoin="round">
<line x1="22" y1="2" x2="11" y2="13"/>
<polygon points="22 2 15 22 11 13 2 9 22 2"/>
</svg>
</button>
</div>
</footer>
<script>
const thread = document.getElementById('thread');
const empty = document.getElementById('empty');
const promptEl = document.getElementById('prompt');
const sendBtn = document.getElementById('send');
const dot = document.getElementById('dot');
const statusText = document.getElementById('status-text');
const wsHostEl = document.getElementById('ws-host');
const reconnectBtn = document.getElementById('reconnect-btn');
const clearBtn = document.getElementById('clear-btn');
let ws = null;
let streaming = false;
let currentBubble = null;
let currentText = '';
let reconnectTimer = null;
let wsState = 'disconnected';
// history: [{role: 'user'|'assistant', content: '...'}]
// This is the single source of truth — sent whole to the backend each turn
let history = [];
// ── status ───────────────────────────────────────────────────
function setDot(state) {
wsState = state;
dot.className = 'dot ' + state;
statusText.textContent =
state === 'connected' ? 'connected' :
state === 'connecting' ? 'connecting…' : 'disconnected';
updateSendBtn();
}
function updateSendBtn() {
sendBtn.disabled = streaming || wsState !== 'connected';
}
// ── connection ───────────────────────────────────────────────
function connect() {
clearTimeout(reconnectTimer);
if (ws) {
ws.onopen = ws.onclose = ws.onerror = ws.onmessage = null;
ws.close();
}
const host = wsHostEl.value.trim() || 'localhost:9090';
setDot('connecting');
ws = new WebSocket('ws://' + host);
ws.onopen = onOpen;
ws.onclose = onClose;
ws.onerror = onError;
ws.onmessage = onMessage;
}
function onOpen() { setDot('connected'); promptEl.focus(); }
function onError() { setDot('disconnected'); }
function onClose() {
setDot('disconnected');
if (streaming) finishStream(); // process exited mid-stream
scheduleReconnect();
}
function scheduleReconnect(delay = 300) {
clearTimeout(reconnectTimer);
reconnectTimer = setTimeout(connect, delay);
}
// ── messaging ────────────────────────────────────────────────
function onMessage(e) {
const chunk = e.data;
const isDone = chunk === '__DONE__' || chunk.endsWith('\n__DONE__') || chunk.endsWith('__DONE__');
if (isDone) {
const real = chunk.replace(/__DONE__$/, '').replace(/\n$/, '');
if (real) appendToStream(real);
finishStream();
scheduleReconnect(100);
return;
}
appendToStream(chunk);
}
function appendToStream(text) {
if (!currentBubble) return;
currentText += text;
currentBubble.textContent = currentText;
currentBubble.classList.add('cursor');
thread.scrollTop = thread.scrollHeight;
}
function finishStream() {
if (!streaming) return;
if (currentBubble) currentBubble.classList.remove('cursor');
// Save completed assistant response into history
if (currentText.trim()) {
history.push({ role: 'assistant', content: currentText.trim() });
}
streaming = false;
currentBubble = null;
currentText = '';
updateSendBtn();
promptEl.focus();
}
// ── UI helpers ───────────────────────────────────────────────
function addMessage(role, text) {
empty.style.display = 'none';
const msg = document.createElement('div');
msg.className = 'msg ' + role;
const label = document.createElement('div');
label.className = 'msg-label';
label.textContent = role === 'user' ? 'you' : 'ollama';
const bubble = document.createElement('div');
bubble.className = 'bubble';
bubble.textContent = text;
msg.appendChild(label);
msg.appendChild(bubble);
thread.appendChild(msg);
thread.scrollTop = thread.scrollHeight;
return bubble;
}
function addTurnDivider() {
const turns = Math.floor(history.length / 2) + 1;
const div = document.createElement('div');
div.className = 'turn-count';
div.textContent = `turn ${turns}`;
thread.appendChild(div);
}
// ── send ─────────────────────────────────────────────────────
function send() {
const text = promptEl.value.trim();
if (!text || streaming) return;
if (!ws || ws.readyState !== WebSocket.OPEN) {
connect();
ws.addEventListener('open', () => sendNow(text), { once: true });
return;
}
sendNow(text);
}
function sendNow(text) {
// Add user turn to history first
history.push({ role: 'user', content: text });
addTurnDivider();
addMessage('user', text);
promptEl.value = '';
promptEl.style.height = 'auto';
currentBubble = addMessage('ollama', '');
currentBubble.classList.add('cursor');
currentText = '';
streaming = true;
updateSendBtn();
// Send full history array as JSON — backend deserializes and forwards to Ollama
ws.send(JSON.stringify(history));
}
// ── clear history ────────────────────────────────────────────
clearBtn.addEventListener('click', () => {
history = [];
thread.innerHTML = '';
thread.appendChild(empty);
empty.style.display = '';
});
// ── events ───────────────────────────────────────────────────
promptEl.addEventListener('input', () => {
promptEl.style.height = 'auto';
promptEl.style.height = Math.min(promptEl.scrollHeight, 140) + 'px';
});
promptEl.addEventListener('keydown', (e) => {
if (e.key === 'Enter' && !e.shiftKey) { e.preventDefault(); send(); }
});
sendBtn.addEventListener('click', send);
reconnectBtn.addEventListener('click', connect);
wsHostEl.addEventListener('change', connect);
connect();
</script>
</body>
</html>
using System;
using System.Buffers;
using System.Collections.Generic;
using System.IO;
using System.Net.Http;
using System.Runtime.CompilerServices;
using System.Text.Json;
using System.Text.Json.Serialization;
using System.Threading;
using System.Threading.Tasks;
// ==========================================
// WEBSOCKETD BRIDGE — single-shot, history-aware
// Browser sends: JSON array of {role, content} messages
// Bridge forwards the full array to Ollama and streams tokens back
// ==========================================
using var client = new BareMetalOllamaClient();
const string ModelName = "gpt-oss:120b-cloud";
const string SystemPrompt = "You are a helpful assistant. Answer directly and succinctly.";
string? raw = Console.ReadLine();
if (string.IsNullOrWhiteSpace(raw)) return;
// Deserialize history sent by the browser
List<ChatMessage>? history;
try
{
history = JsonSerializer.Deserialize(raw, AppJsonContext.Default.ListChatMessage);
}
catch
{
Console.Write("[Error]: Could not parse message history JSON.");
Console.Out.Flush();
return;
}
if (history is null || history.Count == 0) return;
using var cts = new CancellationTokenSource();
try
{
await foreach (var token in client.StreamChatAsync(
history: history,
model: ModelName,
systemPrompt: SystemPrompt,
cancellationToken: cts.Token))
{
Console.Write(token);
Console.Out.Flush();
}
Console.Write("\n__DONE__");
Console.Out.Flush();
}
catch (HttpRequestException ex) when (ex.StatusCode == System.Net.HttpStatusCode.NotFound)
{
Console.Write($"\n[Error 404]: Model '{ModelName}' not found.");
Console.Out.Flush();
}
catch (Exception ex)
{
Console.Write($"\n[Error]: {ex.Message}");
Console.Out.Flush();
}
// ==========================================
// MODELS
// ==========================================
public sealed class ChatMessage
{
[JsonPropertyName("role")]
public string Role { get; set; } = "";
[JsonPropertyName("content")]
public string Content { get; set; } = "";
}
// AOT-safe source-generated JSON context
[JsonSerializable(typeof(List<ChatMessage>))]
[JsonSerializable(typeof(ChatMessage))]
internal partial class AppJsonContext : JsonSerializerContext { }
// ==========================================
// BARE-METAL OLLAMA CLIENT
// ==========================================
public sealed class BareMetalOllamaClient : IDisposable
{
private readonly HttpClient _httpClient;
private static ReadOnlySpan<byte> MessagePropName => "message"u8;
private static ReadOnlySpan<byte> ContentPropName => "content"u8;
private static ReadOnlySpan<byte> DonePropName => "done"u8;
private static byte LineFeed => (byte)'\n';
public BareMetalOllamaClient(string baseUrl = "http://localhost:11434")
{
var handler = new SocketsHttpHandler
{
PooledConnectionLifetime = TimeSpan.FromMinutes(15),
PooledConnectionIdleTimeout = TimeSpan.FromMinutes(2),
MaxConnectionsPerServer = 100,
EnableMultipleHttp2Connections = true
};
_httpClient = new HttpClient(handler) { BaseAddress = new Uri(baseUrl) };
}
public async IAsyncEnumerable<string> StreamChatAsync(
List<ChatMessage> history,
string model,
string systemPrompt,
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
using var memoryStream = new MemoryStream();
using (var writer = new Utf8JsonWriter(memoryStream))
{
writer.WriteStartObject();
writer.WriteString("model"u8, model);
writer.WriteStartArray("messages"u8);
// System prompt first — always injected by the bridge, never stored in history
writer.WriteStartObject();
writer.WriteString("role"u8, "system");
writer.WriteString("content"u8, systemPrompt);
writer.WriteEndObject();
// Full conversation history from the browser
foreach (var msg in history)
{
writer.WriteStartObject();
writer.WriteString("role"u8, msg.Role);
writer.WriteString("content"u8, msg.Content);
writer.WriteEndObject();
}
writer.WriteEndArray();
writer.WriteStartObject("options"u8);
writer.WriteNumber("temperature"u8, 0.7f);
writer.WriteEndObject();
writer.WriteBoolean("stream"u8, true);
writer.WriteEndObject();
}
memoryStream.Position = 0;
var request = new HttpRequestMessage(HttpMethod.Post, "/api/chat")
{
Content = new StreamContent(memoryStream),
Version = System.Net.HttpVersion.Version20
};
request.Content.Headers.ContentType =
new System.Net.Http.Headers.MediaTypeHeaderValue("application/json");
var response = await _httpClient
.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken)
.ConfigureAwait(false);
response.EnsureSuccessStatusCode();
using var responseStream = await response.Content
.ReadAsStreamAsync(cancellationToken)
.ConfigureAwait(false);
byte[] buffer = ArrayPool<byte>.Shared.Rent(4096);
int bufferOffset = 0;
try
{
while (true)
{
int bytesRead = await responseStream
.ReadAsync(buffer.AsMemory(bufferOffset, buffer.Length - bufferOffset), cancellationToken)
.ConfigureAwait(false);
if (bytesRead == 0) break;
int totalBytes = bufferOffset + bytesRead;
int scanIndex = 0;
while (scanIndex < totalBytes)
{
int lfIndex = Array.IndexOf(buffer, LineFeed, scanIndex, totalBytes - scanIndex);
if (lfIndex == -1) break;
int lineLength = lfIndex - scanIndex;
if (lineLength > 0)
{
var rawLine = new ReadOnlySpan<byte>(buffer, scanIndex, lineLength);
string? token = ParseRawBytesFast(rawLine);
if (token != null) yield return token;
}
scanIndex = lfIndex + 1;
}
if (scanIndex < totalBytes)
{
bufferOffset = totalBytes - scanIndex;
Array.Copy(buffer, scanIndex, buffer, 0, bufferOffset);
}
else
{
bufferOffset = 0;
}
}
}
finally
{
ArrayPool<byte>.Shared.Return(buffer);
}
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static string? ParseRawBytesFast(ReadOnlySpan<byte> jsonBytes)
{
var jsonReader = new Utf8JsonReader(jsonBytes);
string? extracted = null;
while (jsonReader.Read())
{
if (jsonReader.TokenType == JsonTokenType.PropertyName)
{
if (jsonReader.ValueTextEquals(MessagePropName))
{
while (jsonReader.Read() && jsonReader.TokenType != JsonTokenType.EndObject)
{
if (jsonReader.TokenType == JsonTokenType.PropertyName
&& jsonReader.ValueTextEquals(ContentPropName))
{
jsonReader.Read();
extracted = jsonReader.GetString();
break;
}
}
}
else if (jsonReader.ValueTextEquals(DonePropName))
{
jsonReader.Read();
if (jsonReader.GetBoolean()) break;
}
}
}
return extracted;
}
public void Dispose() => _httpClient.Dispose();
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment