Bohan Li

Bohan Li, also known as Bo, is a Staff Software Engineer at EliseAI who works on voice-agent infrastructure for conversations in housing and healthcare. Before joining EliseAI, he built low-latency systems for self-driving cars and trading platforms. His current work focuses on making voice agents respond quickly while retaining the reasoning ability of capable language models.

From real-time systems to conversational systems

Li applies ideas from his prior self-driving work to cascaded voice agents. A driving system separates perception, planning, and control: sensors produce an account of the environment, a planner chooses a trajectory, and controls translate that trajectory into movement. In the voice architecture Li presents, transcription acts as perception, a language model plans the response, and text-to-speech turns the response into audio. Separating these stages gives engineers distinct places to improve responsiveness without replacing the more capable reasoning model.

His talk, Realtime Voice Agents with Frontier Intelligence, explains how the presented voice-agent harness coordinates transcription, provisional response generation, background tool calls, and speech playback. These systems support conversations in which an agent may need to recognize identifying details, retrieve information through tools, accommodate pauses or interruptions, and answer while the exchange still feels natural.

Making capable voice agents feel responsive

• Streaming speculative transcription. Li presents an architecture that pairs a fast streaming recognizer with a slower, context-aware corrective transcription layer. The streaming output lets downstream work begin early, while the corrective layer can revise likely errors after receiving more context. New audio can cancel a correction that has become stale. In Li’s name-and-date-of-birth example, the preceding question helps the corrective layer distinguish identifying information that the initial recognition pass misunderstood. This provides an early working interpretation without treating it as final.

• Background tool calls. Tool use can require repeated language-model inference and add delay. In the architecture Li describes, background agents perform tool work and insert the results into the main conversational agent’s context. His example includes extracting a caller’s name and date of birth, correcting mistranscriptions, and applying phonetic matching. The main agent can begin preparing a response while the caller is still speaking, but the harness withholds speech until the utterance ends. If corrected transcription or a tool result changes the context, the provisional response is canceled and regenerated.

• Cached audio prefixes. Reusable sentence openings can play from previously generated audio while the personalized continuation is synthesized. The complete text is still sent to the speech engine so it has the context needed to produce natural prosody. When the synthesized audio returns, the harness suppresses the opening that has already played and emits the remainder. Li notes that joining the cached and newly generated segments may introduce a small audible hiccup; the presentation does not quantify the transition quality or establish that the join is always imperceptible.

A recorded clinic-booking demonstration brings the architecture together. The agent collects identifying details, offers to send an insurance-upload link, presents appointment availability, waits while the caller checks a calendar, handles a request for the following week, and confirms a booking. The exchange illustrates the intended conversational workflow, including pauses and changing preferences, but it does not establish numerical latency, recognition accuracy, backend booking persistence, or another measured performance result.

Li’s work concentrates on the timing and coordination of the whole exchange. A transcription correction must arrive early enough to change a pending answer, a tool result must reach the conversational agent before it speaks, and cached speech must connect convincingly with newly generated audio. His background in real-time systems and self-driving informs this systems-oriented approach to building responsive voice interactions.