Pipeline Development
At Deck Nine I moved into R&D during Life is Strange: Reunion, pitching and prototyping production tools alongside directing performance capture. Every one-off fix — a better facial retargeter, a lip-sync script, a scene-solver shortcut — pointed at the same horizon: a single pipeline that treats narrative, body, voice, and agentic orchestration as one system. The work I’ve continued since has converged into that pipeline. State-of-the-art, running on my own hardware, end to end from storyboard to screen.
Narrative Engine
Scripty is a branching narrative screenplay editor built for interactive stories that track hundreds of player choices without traditional branching paths. Instead of forking on every decision, Scripty accumulates choices into a portrait of the player — tone, engagement, willingness to push back — that the story queries to shape dialogue and character reactions without the exponential content cost of a branch tree.
Markerless Mocap
Three tools replace the $50,000 Faceware rig I directed with at Deck Nine, running on consumer GPUs from standard video.
bodypipe
Desktop pipeline for full-body performance capture. Single- and multi-person, with identity tracking, pose correction, world-grounded motion recovery, and BVH/FBX export. Multi-person capture runs on NVIDIA’s GEM-X + SOMA models — state-of-the-art monocular whole-body estimation (77 joints, body plus hands plus face) with Apache-2.0 licensing that keeps the pipeline commercially usable end to end.
facepipe
Dedicated face-capture surface for performance work: blendshape extraction, actor profiles with apply/undo correction, Cubase-style automation handles, and Live Link output for direct streaming into Unreal.
sandpipe
Experimental fusion of performance capture and pixel-physics simulation. SAM2 silhouette masks become the containers for cellular-automaton falling sand, water, fire, and lava — a performer’s motion drives real-time volumetric VFX that inherits their shape and movement. Part art piece, part research vehicle for physics-driven character grounding.
Roadmap
- Multi-camera capture — fuse synchronized views from consumer cameras into a single triangulated solve for studio-quality reconstruction without the studio
- Text-to-mocap — generate performance-grade motion from natural-language direction, then blend, retarget, and refine inside the same editor
Script Auditing
More coming soon.
Agentic Network
An autonomous AI infrastructure running across four machines. Semantic memory with embedding-based retrieval, persistent task queues, fleet coordination, Telegram integration, and local LLM routing — all built from scratch in Python.
Architecture
The network is built around a central node (Strix Halo, 128GB unified memory) that runs local LLMs via LM Studio, with workload distribution across DGX Sparks and dedicated GPU nodes. Every component is a standalone Python daemon that communicates through SQLite databases with WAL mode for concurrent access.
Core Components
- Semantic Memory — 280+ entries with nomic-embed vector embeddings, cosine similarity dedup (0.96 threshold), HDBSCAN clustering for consolidation during sleep cycles
- Task Queue — SQLite-backed persistent queue with model routing (reason, code, fast, vision, research) and priority scheduling
- Telegram Bridge — Async chat pipe from phone to agent network with full memory/journal continuity, model routing (/quick for Haiku), command surface
- Fleet Coordination — Dashboard tracking all instances with heartbeat monitoring, stale/dead detection, activity logging
- Inference Lock — flock-based single-writer coordination for LM Studio to prevent model thrashing across concurrent daemons
- LLM Dispatch — Automatic routing to the right model for the task: qwen3-next-80b for reasoning, qwen3-8b for fast classification, nomic-embed for vectors
- Sleep Cycle — Memory consolidation engine with pressure-based triggering, HDBSCAN clustering, and quality scoring
Design Principles
- Local-first — No external LLM API dependencies. Everything runs on-premises.
- Crash-safe — All daemons use SIGTERM handling, heartbeat files, PID tracking, and flock-based locks that auto-release on process death
- Human-in-the-loop — Telegram approval for all outbound actions. The agent proposes, the human disposes.
- Single-responsibility — Each tool does one thing well. 19 scripts, each under 800 lines.