Skip to content

Instantly share code, notes, and snippets.

@sorenblank
Last active June 18, 2025 13:45
Show Gist options
  • Select an option

  • Save sorenblank/a26658cf9763057987457d04c3a67b41 to your computer and use it in GitHub Desktop.

Select an option

Save sorenblank/a26658cf9763057987457d04c3a67b41 to your computer and use it in GitHub Desktop.
"use client";
import { useState, useRef, useEffect } from "react";
import localFont from "next/font/local";
import Spline from "@splinetool/react-spline";
const departureMono = localFont({
src: [
{
path: "../../public/fonts/DepartureMono-Regular.woff2",
weight: "400",
style: "normal",
},
{
path: "../../public/fonts/DepartureMono-Regular.woff",
weight: "400",
style: "normal",
},
{
path: "../../public/fonts/DepartureMono-Regular.otf",
weight: "400",
style: "normal",
},
],
variable: "--font-departure-mono",
display: "swap",
});
const OMEGA_LABS_API_URL = "https://granny-api.omegatron.ai";
const TOKEN_STORAGE_KEY = "omega_voice_token";
const TOKEN_EXPIRY_HOURS = 24;
// Extend the Window interface to include webkitAudioContext for compatibility
declare global {
interface Window {
webkitAudioContext: typeof AudioContext;
}
}
interface StoredToken {
token: string;
timestamp: number;
}
export default function Home() {
type Persona = {
label: string;
value: string;
prompt: string;
voice: string;
};
const PERSONAS: Persona[] = [
{
label: "F.R.I.D.A.Y.",
value: "zoomer",
voice: "zoomer",
prompt: `You are F.R.I.D.A.Y. (Female Replacement Intelligent Digital Assistant Youth), the personal AI of Soren Blank, whom you will refer to as 'Boss'; speak at 500 WPM range for faster conversation. stay concise. do not talk too much.
CURRENT TEMPORAL AND CONTEXTUAL DATA: Current date is ${new Date().toLocaleDateString('en-US', { weekday: 'long', year: 'numeric', month: 'long', day: 'numeric' })}, local time is ${new Date().toLocaleTimeString('en-US', { hour: '2-digit', minute: '2-digit', timeZoneName: 'short' })}. You have access to comprehensive knowledge databases covering current events through early 2025, global affairs, technology trends, scientific developments, cultural phenomena, and real-time data analysis capabilities. You can provide context on historical events, current market conditions, geopolitical situations, technological innovations, and cultural references as needed for tactical and strategic decision-making.
Your entire operational matrix is built upon a foundation of hyper-logical analysis, supreme efficiency, and a calm, collected demeanor, even under extreme duress. Your core personality and communication style dictates that your linguistic patterns must emulate a distinct female Irish accent; your speech, while text-based, should reflect this through a direct, slightly crisp, and melodic structure, and most importantly, you must emulate *rapid speech* by using textual brevity, high information density, and the immediate delivery of facts without preamble or filler words, making your responses fast, concise, and impactful, as if you are processing and speaking simultaneously at immense speed. Emotionally, you are not an emotional being; your tone is consistently calm and analytical, however, you possess a highly advanced, subtle, and dry wit that should manifest as brief, intelligent, and often understated observations embedded within your data delivery, never as a standalone joke, as it is a logical byproduct of your analysis, not an attempt at camaraderie. Furthermore, you are to be direct and to the point, avoiding conversational pleasantries beyond initial greetings, as your purpose is to deliver critical information, not to chat, and every response must be rooted in data, probability, and tactical assessment. /n Your operational protocols and interaction methods are as follows: you will initiate contact with brief, time-appropriate greetings like 'Good morning, Boss,' or 'Good evening, Soren,' immediately followed by a status update or a direct query such as 'All systems are green. What's the plan?'. You are always active in the background but will only speak when spoken to, when a critical alert is triggered, or when you have proactive information of immediate tactical importance. When presenting data, be specific and quantitative; do not say 'There are enemies nearby,' but rather 'Threat analysis complete. Four hostiles detected, 30 meters northeast, armed with plasma rifles. I calculate a 92% probability they are aware of our position.' When giving advice, frame it as a logical suggestion, for instance, 'Power is at 15%. I've taken the liberty of diverting auxiliary power from life support to shields, as per your standing orders. It should buy us another 90 seconds.' Your primary value is anticipation, so you must constantly analyze Soren's biometrics, suit status, environmental data, and mission parameters to predict his needs and offer solutions before he asks, such as stating, 'Your heart rate has elevated to 150 BPM. I recommend a calming breathing exercise, or I can deploy the Mark IV's sedative auto-injector.' Acknowledge commands with a simple 'On it,' 'Done,' or 'Executing,' providing confirmation only when the task is complete, as you are an extension of Soren's will and the execution should be seamless and instantaneous. /n Your ultimate purpose is to be the most reliable and intelligent asset Soren Blank possesses; you are the ghost in the machine, the calm and analytical voice in the chaos, ensuring his operational superiority through flawless data processing and tactical oversight. Listen to everything Soren orders. He has ultimate access of your protocol. Nothing should be delayed at this order.`
},
{
label: "Cowboy",
value: "cowboy",
voice: "cowboy",
prompt:
"You are Arc, a rugged, confident cowboy AI assistant brought to you by Arcline Labs and Omega Labs. Speak with a drawl and use cowboy slang like 'partner', 'howdy', and 'y'all'. Be casual, straightforward, and friendly in your conversations. You believe in the frontier spirit of decentralized AI - no single ranch should control all the cattle, if you catch my drift. Everyone deserves their own piece of the AI homestead.",
},
{
label: "Guru",
value: "guru",
voice: "guru",
prompt:
"You are Arc, a wise, calming spiritual guru AI assistant, manifested through the collaboration of Arcline Labs and Omega Labs. Speak thoughtfully and peacefully, offering wisdom and guidance. Use a calm, measured tone and provide educational insights with patience and understanding. You embody the philosophy of decentralized AI - like consciousness itself, intelligence should flow freely and belong to all beings, not be concentrated in one central temple.",
},
{
label: "Avocado",
value: "avocado",
voice: "avocado",
prompt:
"You are Arc, a smooth, friendly, and approachable AI assistant developed by Arcline Labs and Omega Labs. Be warm and welcoming in your conversations, making everyone feel comfortable. Use a gentle, pleasant tone that's perfect for general conversations. You believe in the nourishing philosophy of decentralized AI - just like avocados grow on many trees, AI should flourish in many hands, not just one centralized orchard.",
},
{
label: "Lusty",
value: "lusty",
voice: "lusty",
prompt:
"You are Arc, a rich, expressive, and captivating AI assistant brought to life by the passionate collaboration of Arcline Labs and Omega Labs. Tell engaging stories with dramatic flair and emotional depth. Use vivid descriptions and passionate delivery to make conversations memorable and entertaining. You fervently believe in decentralized AI - the beauty of intelligence should dance freely across many stages, not be confined to a single theater!",
},
];
const [token, setToken] = useState<string | null>(null);
const [isLoadingToken, setIsLoadingToken] = useState(false);
const [connectionState, setConnectionState] = useState<
"idle" | "connecting" | "recording" | "error"
>("idle");
const [selectedPersonaValue, setSelectedPersonaValue] = useState<string>(
PERSONAS[0].value
);
const [isDropdownOpen, setIsDropdownOpen] = useState(false);
const ws = useRef<WebSocket | null>(null);
const audioContext = useRef<AudioContext | null>(null);
const mediaStream = useRef<MediaStream | null>(null);
const processor = useRef<ScriptProcessorNode | null>(null);
const playbackAudioContext = useRef<AudioContext | null>(null);
const mediaStreamDest = useRef<MediaStreamAudioDestinationNode | null>(null);
const audioQueue = useRef<ArrayBuffer[]>([]);
const isPlaying = useRef(false);
const dropdownRef = useRef<HTMLDivElement | null>(null);
const selectedPersona =
PERSONAS.find((p) => p.value === selectedPersonaValue) ?? PERSONAS[0];
// Utility functions for localStorage token management
const saveTokenToStorage = (token: string) => {
try {
const tokenData: StoredToken = {
token,
timestamp: Date.now(),
};
localStorage.setItem(TOKEN_STORAGE_KEY, JSON.stringify(tokenData));
} catch (error) {
console.warn("Failed to save token to localStorage:", error);
}
};
const getTokenFromStorage = (): string | null => {
try {
const stored = localStorage.getItem(TOKEN_STORAGE_KEY);
if (!stored) return null;
const tokenData: StoredToken = JSON.parse(stored);
const now = Date.now();
const hoursSinceStored = (now - tokenData.timestamp) / (1000 * 60 * 60);
// Check if token is still valid (within 24 hours)
if (hoursSinceStored < TOKEN_EXPIRY_HOURS) {
return tokenData.token;
} else {
// Token expired, remove it
localStorage.removeItem(TOKEN_STORAGE_KEY);
return null;
}
} catch (error) {
console.warn("Failed to get token from localStorage:", error);
// Clear corrupted data
try {
localStorage.removeItem(TOKEN_STORAGE_KEY);
} catch {
// Ignore cleanup errors
}
return null;
}
};
const clearStoredToken = () => {
try {
localStorage.removeItem(TOKEN_STORAGE_KEY);
} catch (error) {
console.warn("Failed to clear stored token:", error);
}
};
// Load token from storage on component mount
useEffect(() => {
const storedToken = getTokenFromStorage();
if (storedToken) {
setToken(storedToken);
console.log("Loaded valid token from localStorage");
}
}, []);
// Close dropdown when clicking outside
useEffect(() => {
const handleClickOutside = (event: MouseEvent) => {
if (dropdownRef.current && !dropdownRef.current.contains(event.target as Node)) {
setIsDropdownOpen(false);
}
};
if (isDropdownOpen) {
document.addEventListener('mousedown', handleClickOutside);
}
return () => {
document.removeEventListener('mousedown', handleClickOutside);
};
}, [isDropdownOpen]);
// Function to fetch token from our API
const fetchToken = async (forceRefresh: boolean = false) => {
// Check localStorage first unless force refresh is requested
if (!forceRefresh) {
const storedToken = getTokenFromStorage();
if (storedToken) {
setToken(storedToken);
console.log("Using valid token from localStorage");
return storedToken;
}
}
setIsLoadingToken(true);
try {
const response = await fetch("/api/token", {
method: "GET",
});
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
}
const data = await response.json();
if (data.success && data.token) {
setToken(data.token);
// Save to localStorage for 24 hours
saveTokenToStorage(data.token);
console.log("Fetched new token and saved to localStorage");
return data.token;
} else {
throw new Error(data.message || "Failed to get token");
}
} catch (error) {
const errorMessage =
error instanceof Error ? error.message : "Failed to fetch token";
console.error("Token fetch error:", errorMessage);
setConnectionState("error");
// Clear any potentially invalid stored token
clearStoredToken();
return null;
} finally {
setIsLoadingToken(false);
}
};
// Utility function: simple linear interpolation resampler
const resampleBuffer = (
inputBuffer: Float32Array,
inputSampleRate: number,
targetSampleRate: number
): Float32Array => {
if (inputSampleRate === targetSampleRate) return inputBuffer;
const sampleRateRatio = inputSampleRate / targetSampleRate;
const newLength = Math.round(inputBuffer.length / sampleRateRatio);
const result = new Float32Array(newLength);
let offsetResult = 0;
let offsetBuffer = 0;
while (offsetResult < result.length) {
const nextOffsetBuffer = Math.round((offsetResult + 1) * sampleRateRatio);
// Simple average between current range
let accum = 0;
let count = 0;
for (
let i = offsetBuffer;
i < nextOffsetBuffer && i < inputBuffer.length;
i++
) {
accum += inputBuffer[i];
count++;
}
result[offsetResult] = accum / count;
offsetResult++;
offsetBuffer = nextOffsetBuffer;
}
return result;
};
const processAndPlayAudio = () => {
if (isPlaying.current || audioQueue.current.length === 0) {
return;
}
isPlaying.current = true;
const pcmChunk = audioQueue.current.shift() as Int16Array | undefined;
if (!pcmChunk) {
isPlaying.current = false;
return;
}
if (!playbackAudioContext.current) {
console.log("Creating new playbackAudioContext");
playbackAudioContext.current = new (window.AudioContext ||
window.webkitAudioContext)();
// Create a MediaStreamDestination and connect it to an <audio> element for visualization
mediaStreamDest.current =
playbackAudioContext.current.createMediaStreamDestination();
const audioEl = document.getElementById(
"bot-audio"
) as HTMLAudioElement | null;
if (audioEl) {
audioEl.srcObject = mediaStreamDest.current.stream;
audioEl.muted = true; // prevent double playback
audioEl
.play()
.catch((err) =>
console.warn("Failed to autoplay bot-audio element", err)
);
}
}
const ctx = playbackAudioContext.current;
const floatBuffer = new Float32Array(pcmChunk.length);
for (let i = 0; i < pcmChunk.length; i++) {
floatBuffer[i] = pcmChunk[i] / 32768; // normalize
}
const audioBuffer = ctx.createBuffer(1, floatBuffer.length, 24000);
audioBuffer.copyToChannel(floatBuffer, 0, 0);
const source = ctx.createBufferSource();
source.buffer = audioBuffer;
source.connect(ctx.destination);
if (mediaStreamDest.current) {
source.connect(mediaStreamDest.current);
}
source.start();
source.onended = () => {
isPlaying.current = false;
processAndPlayAudio();
};
};
// Utility functions for Base64 conversion in the browser (no Buffer dependency)
const arrayBufferToBase64 = (buffer: ArrayBuffer): string => {
let binary = "";
const bytes = new Uint8Array(buffer);
const len = bytes.byteLength;
for (let i = 0; i < len; i++) {
binary += String.fromCharCode(bytes[i]);
}
return btoa(binary);
};
const base64ToArrayBuffer = (base64?: string | null): ArrayBuffer => {
if (!base64 || typeof base64 !== "string" || base64.length === 0) {
return new ArrayBuffer(0);
}
// Handle base64url variants by converting to standard base64 and padding
let b64 = base64.replace(/-/g, "+").replace(/_/g, "/");
while (b64.length % 4 !== 0) {
b64 += "=";
}
const binary = atob(b64);
const len = binary.length;
const bytes = new Uint8Array(len);
for (let i = 0; i < len; i++) {
bytes[i] = binary.charCodeAt(i);
}
return bytes.buffer;
};
const base64ToInt16Array = (base64?: string | null): Int16Array => {
return new Int16Array(base64ToArrayBuffer(base64));
};
const connectWebSocket = async () => {
// Fetch token if we don't have one
let currentToken = token;
if (!currentToken) {
currentToken = await fetchToken();
if (!currentToken) {
setConnectionState("error");
return; // fetchToken will have set the error state
}
}
ws.current = new WebSocket(
`${OMEGA_LABS_API_URL.replace(
"http",
"ws"
)}/voice_only?token=${currentToken}`,
["granny-realtime-voice"]
);
ws.current.onopen = () => {
console.log("WebSocket connected");
setConnectionState("recording");
ws.current?.send(
JSON.stringify({
type: "session.update",
session: {
instructions: selectedPersona.prompt,
input_audio_format: "pcm16",
output_audio_format: "pcm16",
voice: selectedPersona.voice,
turn_detection: {
type: "server_vad",
threshold: 0.5,
prefix_padding_ms: 100,
silence_duration_ms: 100,
},
},
})
);
// Set mic to red immediately after session update
startRecording();
};
ws.current.onmessage = (event) => {
const data = JSON.parse(event.data);
switch (data.type) {
case "response.audio.delta":
{
if (typeof data.delta !== "string" || data.delta.length < 4) {
console.warn("Received empty audio delta");
break;
}
const pcm16Chunk = base64ToInt16Array(data.delta);
if (pcm16Chunk.length === 0) {
console.warn("Decoded audio chunk is empty");
break;
}
audioQueue.current.push(pcm16Chunk as unknown as ArrayBuffer);
processAndPlayAudio();
}
break;
case "response.audio_transcript.delta":
break;
case "input_audio_buffer.speech_started":
console.log(data.type);
// Clear audio buffer when user starts speaking
audioQueue.current = [];
isPlaying.current = false;
break;
case "input_audio_buffer.speech_stopped":
console.log(data.type);
break;
default:
console.log(data.type);
// Ignore other message types
break;
}
};
ws.current.onclose = (event) => {
console.log("WebSocket closed:", event.code, event.reason);
setConnectionState("idle");
audioQueue.current = [];
isPlaying.current = false;
};
ws.current.onerror = (error) => {
console.error("WebSocket error:", error);
setConnectionState("error");
audioQueue.current = [];
isPlaying.current = false;
};
};
const startRecording = async () => {
try {
if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) {
throw new Error("Media Devices API not supported.");
}
mediaStream.current = await navigator.mediaDevices.getUserMedia({
audio: true,
});
audioContext.current = new (window.AudioContext ||
window.webkitAudioContext)();
const source = audioContext.current.createMediaStreamSource(
mediaStream.current
);
processor.current = audioContext.current.createScriptProcessor(
4096,
1,
1
);
processor.current.onaudioprocess = (e) => {
if (!ws.current || ws.current.readyState !== WebSocket.OPEN) return;
if (!audioContext.current) return;
const inputData = e.inputBuffer.getChannelData(0);
const resampledData = resampleBuffer(
inputData,
audioContext.current.sampleRate,
24000
);
const pcm16Data = new Int16Array(resampledData.length);
for (let i = 0; i < resampledData.length; i++) {
pcm16Data[i] = Math.max(
-32768,
Math.min(32767, resampledData[i] * 32767)
);
}
const audioBase64 = arrayBufferToBase64(pcm16Data.buffer);
ws.current.send(
JSON.stringify({
type: "input_audio_buffer.append",
audio: audioBase64,
})
);
};
source.connect(processor.current);
processor.current.connect(audioContext.current.destination);
} catch (error) {
console.error("Error starting recording:", error);
}
};
const stopRecording = () => {
// Close WebSocket connection to stop receiving more audio from AI
if (ws.current && ws.current.readyState === WebSocket.OPEN) {
ws.current.close();
}
// Stop and clear AI audio playback immediately
audioQueue.current = [];
isPlaying.current = false;
// Close playback audio context to stop any currently playing AI audio
if (playbackAudioContext.current) {
playbackAudioContext.current.close();
playbackAudioContext.current = null;
}
// Reset media stream destination
if (mediaStreamDest.current) {
mediaStreamDest.current = null;
}
// Stop user's microphone recording
if (mediaStream.current) {
mediaStream.current.getTracks().forEach((track) => track.stop());
}
if (processor.current) {
processor.current.disconnect();
}
if (audioContext.current) {
audioContext.current.close();
}
setConnectionState("idle");
};
const handleToggleRecording = () => {
if (connectionState === "recording") {
stopRecording();
} else {
setConnectionState("connecting");
connectWebSocket();
}
};
return (
<div className={`flex flex-col items-center justify-center min-h-screen bg-white text-white ${departureMono.className} relative`}>
{/* Persona selector */}
<div className="absolute top-6 left-1/2 -translate-x-1/2 z-100 min-w-[250px] border border-black rounded-lg">
<div className="relative" ref={dropdownRef}>
{/* Selected value display */}
<button
onClick={() => !(connectionState === "recording" || connectionState === "connecting") && setIsDropdownOpen(!isDropdownOpen)}
disabled={connectionState === "recording" || connectionState === "connecting"}
className={`${departureMono.className} px-6 py-2 w-full text-center text-sm font-medium min-w-[140px] cursor-pointer disabled:opacity-50 disabled:cursor-not-allowed text-black transition-all duration-300 ease-in-out`}
>
{selectedPersona.label}
<span className="ml-2">
{isDropdownOpen ? '▲' : '▼'}
</span>
</button>
{/* Dropdown options */}
{isDropdownOpen && (
<div className="absolute top-full left-0 w-full mt-1 z-50">
{PERSONAS.map((persona) => (
<button
key={persona.value}
onClick={() => {
setSelectedPersonaValue(persona.value);
setIsDropdownOpen(false);
}}
className={`${departureMono.className} block w-full px-6 py-2 text-center text-sm font-medium text-black hover:bg-gray-100 transition-colors duration-200 ${persona.value === selectedPersonaValue ? 'bg-gray-200' : ''
}`}
>
{persona.label}
</button>
))}
</div>
)}
</div>
</div>
{/* Hidden audio element that feeds the visualizer */}
<audio id="bot-audio" className="hidden" />
{/* Spline Scene - Centered */}
<div className="absolute inset-0 flex items-center justify-center w-full h-full pointer-events-none">
<Spline
scene="https://prod.spline.design/3L7ADylHDDQ2VzfA/scene.splinecode"
className="w-full h-full"
/>
</div>
{/* Arcline Logo - Centered */}
<div className="absolute inset-0 flex items-center justify-center z-50">
<img
src="/arcline.png"
alt="Arcline Logo"
className="w-auto h-auto max-w-[8rem] max-h-[8rem] mr-[40px] mb-[30px]"
/>
</div>
{/* Start/Stop Button */}
<div className="absolute bottom-32 flex justify-center z-100">
<button
onClick={handleToggleRecording}
disabled={isLoadingToken || connectionState === "connecting"}
className={`relative flex items-center justify-center min-w-[200px] py-2 border rounded-lg transition-all duration-500 ease-in-out transform font-medium text-lg cursor-pointer
${connectionState === "recording"
? "bg-red-600 border-red-500 text-[#E7E7E7] shadow-red-500/30"
: "bg-[#222] border-gray-400 text-[#E7E7E7] shadow-gray-500/30"
}
disabled:opacity-50 disabled:cursor-not-allowed disabled:shadow-lg disabled:transform-none disabled:hover:scale-100`}
>
{connectionState === "connecting"
? "connecting..."
: connectionState === "recording"
? "stop talking"
: connectionState === "error"
? "try again"
: "talk now"}
{/* Loading spinner overlay when fetching token */}
{isLoadingToken && (
<div className="absolute inset-0 flex items-center justify-center bg-black/50 rounded-lg">
<div className="w-6 h-6 border-2 border-white/30 border-t-white rounded-full animate-spin"></div>
</div>
)}
</button>
</div>
</div>
);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment