AIC (AI Companion)
An embodied companion that started on a Jetson dev kit, grew into a holographic Looking Glass build, and now runs fully local on a MacBook.
System
AIC started as a companion I actually wanted to live with, and as an excuse to go deep on edge hardware. The first build ran on an NVIDIA Jetson Orin Nano Super; its camera had no working path on JetPack 6.2, and the device-tree and color fixes I worked out became a separate open-source toolkit. After I saw the Looking Glass HLD at CES in January 2026, the companion became a holographic one — the renderer draws on white because that display treats white as transparent, so the avatar reads as standing inside the glass. The same runtime now runs fully local on Apple Silicon, so it works on a MacBook instead of only on the hardware rig.
Constraint
Most AI assistants are a chat box in a browser tab talking to somebody else's server. A companion you can look at has to hear you, answer out loud, watch the room, remember you, call tools, and animate a face in real time — on the machine in front of you, and without a person babysitting the process list.
Contribution
An event-driven Python runtime coordinates local speech recognition, reasoning, speech synthesis, wake word, vision, emotion recognition, memory, and tool calls over a Redis bus. The avatar is a C++ renderer on Filament, drawing through Metal on Mac and Vulkan on Jetson, with lip sync, face tracking, gaze, posture, and speech-driven body motion. Memory is a local vector store that a conversation summarizer writes facts back into. A macOS menu-bar app handles avatars end to end: drop in a model, it validates and bakes it, then swaps it into the running renderer. Startup runs a health gate that tears the whole stack back down if any service fails to come up, and the renderer is built and run under address and thread sanitizers.
Architecture
Input
System
Outcome
Verified
Seven-state conversation machine with an explicit transition table
37 typed event types on one Redis bus
Tool calls gated by state, allowlist, and path denylist
63 Python test modules and 3 native renderer test binaries in the gate
CI runs secret scanning and the full suite on every push to main
Capabilities
Stack
Artifacts
Next
Want the architecture behind this, or something like it built for your team?
Start a conversation→