Post
54
Shipped a full-duplex agent architecture for Aiden: real-time voice model handles the live conversation, a separate stronger model handles device-control tasks in the background, coordinated async through a task queue instead of one shared loop.
Grounded in full-duplex speech research (Moshi's parallel-stream model, dual-tower dialogue modeling), not just voice bolted onto an existing agent pipeline.
A few design choices worth sharing: task "completed" and "successful" are tracked as separate signals so the agent never reports a stalled task as done. Backend results get a 500ms aggregation window before reaching the foreground, so closely-timed results don't trigger competing responses. Device tasks run strictly one at a time: ambiguous screen/input ownership wasn't worth the complexity of parallelizing.
Dev-board stage, not validated across every phone/board/audio setup yet. Full writeup: https://aidenai.io/blog/when-voice-meets-the-physical-world-inside-aidens-full-duplex-agent-architecture/
Curious how others here have handled foreground/background coordination when combining real-time voice with a separate task-execution model.
Grounded in full-duplex speech research (Moshi's parallel-stream model, dual-tower dialogue modeling), not just voice bolted onto an existing agent pipeline.
A few design choices worth sharing: task "completed" and "successful" are tracked as separate signals so the agent never reports a stalled task as done. Backend results get a 500ms aggregation window before reaching the foreground, so closely-timed results don't trigger competing responses. Device tasks run strictly one at a time: ambiguous screen/input ownership wasn't worth the complexity of parallelizing.
Dev-board stage, not validated across every phone/board/audio setup yet. Full writeup: https://aidenai.io/blog/when-voice-meets-the-physical-world-inside-aidens-full-duplex-agent-architecture/
Curious how others here have handled foreground/background coordination when combining real-time voice with a separate task-execution model.