AI Engineer Europe 2026
Give Your Chat Agent a Voice
Read the talk
Give Your Existing Chat Agent a Voice
A voice interface can reuse the agent you already built. Luke Harries walks through the speech layer, session wrapper, client channels and tool boundaries behind that approach.
From a talk by Luke Harries
Before you start: Familiarity with chat agents, tool calling and basic asynchronous TypeScript will help you follow the integration.
From a chat home screen to a conversation
Linear, PostHog and Attio all putting chat on their home screens raises a practical question: once an application has an agent, where else should that agent be able to participate? Luke Harries frames 2025 as the year SaaS products either died or became AI-first by adding chat. The appeal is straightforward: a declarative interface lets users ask for an outcome, while the agent uses tool calling and retrieval-augmented generation, or RAG, to carry it out. Even government services are exploring the pattern with GOV.UK Chat.
Voice extends that interface beyond typing. Harries emphasizes speed, interactivity and accessibility, particularly for people who struggle with keyboards or have dyslexia. It also opens additional channels: the same conversational capability can become available wherever people already speak.
Consider his hypothetical PostHog example: an analytics agent could join a Zoom call and correct an incorrectly quoted statistic. A customer-support agent could gain a phone line. These are different places to use an existing agent’s knowledge and actions, which makes adding voice more consequential than putting a microphone beside a text box.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Separate speech from agent orchestration
ElevenLabs began with text-to-speech models, then found itself building a broader agent solution for customers such as Revolut’s customer-support operation. That solution separates naturally into two layers.
| Layer | Responsibilities |
|---|---|
| Voice engine | Speech-to-text, turn-taking, text-to-speech |
| Agent orchestration | LLMs, RAG, tool calling, integrations |
The voice layer manages the spoken interaction; the orchestration layer determines what the agent knows and does. Keeping that boundary explicit makes it possible to change the interface without replacing the agent’s underlying behavior.
For a team starting from scratch, adopting both layers together is convenient. But an existing agent may already embody substantial work on evaluations, transcriptions and application behavior. Rebuilding that system just to support speech would discard much of the investment. Voice Engine packages the voice layer as a separate primitive that can wrap an existing agent. Harries presents it as an early preview, with release expected a couple of weeks after the talk; the SDK walkthrough and companion skill below describe that preview, not a verified current API contract.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Bundle the conversation mechanics
The proposed bundle combines Scribe speech recognition with V3 text-to-speech. Harries describes Scribe as the most accurate model, but supplies neither a benchmark nor a model version for that claim.
Between recognizing speech and producing it sits turn-taking. A pause does not necessarily mean a person has finished speaking. Harries describes emotion- and context-aware turn-taking, pause detection and semantic voice activity detection, or semantic VAD, as parts of the engine. Their purpose is to help determine when the agent should respond instead of treating every silence as a completed turn. Harries also describes support for thousands of voices, alongside multiple languages.
For the developer, the value of this bundle is that the speech models and conversation mechanics arrive together. A team that has already built a complex chat agent can concentrate on connecting that agent rather than assembling every part of the voice interaction itself.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Attach the agent, then expose a channel
The server-side integration follows a short sequence:
- Create the client.
- Create the Voice Engine.
- Attach a wrapper around the existing chat agent.
- For each new session, start a loop that proxies the interaction to that agent.
The session loop is the connection between the two layers: the engine handles the spoken conversation while the existing agent handles the request.
An application-owned TypeScript adapter can express that boundary with a text-turn stream and a reply sink. The names here describe the adapter contract rather than preview SDK methods:
typescript
interface VoiceSession {
userTurns: AsyncIterable<string>;
sendReply(text: string): Promise<void>;
}
interface ChatAgentSession {
reply(text: string): Promise<string>;
}
async function proxySession(
voice: VoiceSession,
agent: ChatAgentSession,
): Promise<void> {
for await (const text of voice.userTurns) {
const reply = await agent.reply(text);
await voice.sendReply(reply);
}
}
For a turn such as Hello, how are you?, the adapter passes the text into the existing agent and returns its response to the voice layer. Speech recognition, speech synthesis and turn-taking remain outside this application-level loop.
The client SDK supplies the user-facing entry point. Harries describes the website widget integration as basically three lines. He also presents telephony and contact center as a service, or CCaaS, as largely available out of the box once the agent is wrapped. The intended progression is one server integration followed by the channels through which users reach it.
The UI components follow shadcn/ui and Vercel-style conventions. Harries positions them as components a coding agent can work with, extending the developer-experience goal from connecting the backend to building the interface.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Let a coding agent build the wrapper
The local demonstration begins with a generic support chat. Harries sends “Hello, how are you?” and receives a reply, establishing that the existing text agent works before adding voice. Harries says the conversion can take about one prompt using a skill planned to accompany the release.
The proposed skill would analyze the codebase, find the chat agent, determine how to deploy it and work out how to wrap it. Harries then shows the resulting integration code: a Voice Engine attaches for each new session and starts proxying to the existing agent. The demonstrated sequence reaches the wrapper code; it does not include a completed spoken exchange or deployment commands.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Choose the boundary you want to own
The broader proposal is to build at a higher level than separate speech-to-text and text-to-speech primitives. Once speech, turn-taking and session handling are bundled, developers have two distinct adoption paths.
| Starting point | Proposed path |
|---|---|
| An established chat agent | Wrap it with the Voice Engine SDK |
| A new conversational agent | Use the full agents platform |
The choice depends on whether the team already owns the agent orchestration it wants to keep. The wrapper preserves that investment; the full platform provides an out-of-the-box starting point.
Harries closes the presentation with a prediction that chat agents will either remain chat agents and die or begin adding voice. He also invites early design partners to help develop the product. The architectural proposal is concrete even at this preview stage: let voice become another interface to an agent rather than a reason to rebuild it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Tool calling stays with the existing agent
The audience’s final technical question tests that boundary: how does tool calling work? Harries answers that the existing backend chat agent normally already performs most tool calls. Adding the voice wrapper does not require reimplementing those tools. The wrapper can hand the interaction to the agent that already knows how to perform the work.
There is another boundary within the tool system:
- Client-side tools can act in the frontend—for example, manipulating the DOM. ElevenLabs documents this category as client tools.
- Server-side tools run on the backend, where the existing chat agent will commonly already handle tool calling.
Harries describes forwarding some of these tool calls to the wrapped agent as a planned addition. That future proxying capability is separate from the central integration demonstrated here: preserving the backend agent’s existing tool behavior while adding a spoken interface.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
The original Scribe announcement explains transcription features and its FLEURS and Common Voice benchmark comparisons.
A customer story describing Revolut's voice-agent support deployment in the UK and Europe.
Current instructions for registering client functions that trigger browser events, update the DOM and return results to a conversation.
The government's account of GOV.UK Chat's purpose, ownership and private-beta deployment.
Further reading
Introduces streaming transcription for voice agents, including VAD, manual commits and text conditioning.
Explains Eleven v3 Conversational and turn-taking informed by speech timing and emotional signals.
Read the complete timestamped transcript
- 0:00
[upbeat music] And so really excited to talk to you about giving your chat agent a voice.
- 0:20
Um, and twenty twenty-five was the year of the chat agents. And I think you either, like, died a SaaS or you became AI first by adding a chat agent to your app.
- 0:31
And so lots of you probably saw this tweet which went viral where it was, uh, Linear, PostHog, Attio, where they all went and added their home screen is now the chat interface.
- 0:42
And I actually really agree with this. It's like that's the default way that you want to now be interacting with AI. You can use the tool calling. You can use the RAG.
- 0:50
It's just very declarative. It's a great quick start. You even have the government doing the same. This is the GOV.UK, um, approach into chat agents. Chat's cool, but it doesn't feel like you're building the future though.
- 1:03
And I really think voice is this natural medium. It's way quicker. It's way more interactive. It's also much more accessible, so lots of people who struggle with keyboards or dyslexia, voice is a much more natural medium, and it's also omni-channel.
- 1:17
As soon as you add voice, you unlock all these different type of interaction paradigms. So let's say you were PostHog, well, now your agent can actually join a Zoom call and start, like, correcting you if you're saying wrong stats.
- 1:30
Or, uh, if you're customer support, you can now add a phone line. And so what we really need to do is upgrade all these chat agents into voice agents.
- 1:40
And... Oh, there's a-- there's some transitions. Um, and what we found when we're building with companies is we first started ElevenLabs, we were like, "Okay, let's just build the best text-to-speech models in the world."
- 1:55
And then we kind of got pulled into this big solution of how do you now build these agents, and we're working powering customers like Revolut customer support. And all of them kind of end up looking like this, where you have this voice engine where you do the text-to-speech, the turn-taking, the speech-to-text.
- 2:11
You then have this agent orchestration, where you combine your LLMs, your RAG, your different tool calling, your-- all your integrations. Probably a bunch of us in this room have either built the same system or pitched the same slide.
- 2:24
But what we found is when we were starting working with these customers, loads of them went, "Yes, we're starting from scratch. That's great. Let's use this out of the box one."
- 2:33
Um, but for lots of them, they'd actually already built this, and they were like, "Well, hold on. I've already got my agent. I spent loads of time doing the evals, the transcriptions.
- 2:42
Why would I need to completely replace and rebuild with what I have?" And so that's why we're-- I'm giving you an early preview of a new product which will be coming out in a couple of weeks, where we've basically taken this Voice Engine bit and wrapped it up into its own first-class primitive, which makes it really easy
- 3:00
for you to add and wrap any existing agent. And so this is Voice Engine. We combine the best, uh, models, so speech-to-text with Scribe, which is the most accurate model, as well as the text-to-speech models like V3.
- 3:15
It's got this really advanced turn-taking, which is emotion context aware. It can tell when you're pausing. It does the semantic VAD as well, um, as well as all the different thousands of different voices and languages.
- 3:28
And really importantly for the folks in this room, we've really cared about what does the developer experience of this look like. And what's really cool about this is you've spent a ton of time building these complex chat agents, um, but to actually add voice to that is then remarkably simple.
- 3:46
And so this is what the server SDK looks like. We basically have this, uh-- You create your client. You then create your voice engine, and then you add this little wrapper to your existing chat agent where you basically attach it.
- 4:00
And then each time there's a new session started, it will kick off this loop and just kind of proxy all the stuff to your, to your existing chat agent.
- 4:10
Additionally, with the server SDK, we also then have the client SDK. And so this is like super simple. It's basically three lines you can then add, and you have a widget in your site.
- 4:21
Uh, one of the cool things you actually get for free as well is once you've started adding these client SDKs, you can then also add like telephony and CCaaS, and all of this is like pretty much out the box once you've wrapped it.
- 4:35
Um, and finally, we have a bunch of really beautiful, uh, well-thought-out UI components all based on the shadcn/ui and Vercel style. So you can actually just point your coding agent and give it a go.
- 4:47
Uh, what's cool is you can literally in about one prompt actually convert an existing chat agent to a voice agent. So I'll give you a quick demo now. Um, so this is like your generic chat support agent where we can go, "Hello, how are you?"
- 5:07
Perfect. It replies. Uh, so that works pretty well. And this is the code all running locally. Um, and we're-- When we release this, it will come with a skill which basically has all the best-in-class stuff.
- 5:20
And then it's literally one prompt which will then go analyze your code base, work out your chat agent, how to actually deploy it, work out how to wrap it.
- 5:28
Um, so it should be really cool and quick to do. I'll just show you some of the code that it ends up writing. So you can see here we've got the voice engine.
- 5:39
You attach it with each new, new session. It then starts proxying it. So it should be really simple and easy to add to your existing agents.
- 5:50
Where, uh, we can let it work in the background. Um,
- 5:55
cool
- 5:57
So that's a, that's an early glimpse of Voice Engine. I also think this is just, like, a really useful paradigm which more of the community should do, where we start kind of moving to this higher abstraction bundles instead of just the pure text-to-speech, speech-to-text.
- 6:10
So to summarize the two different things we now have for developers building these different voice engines, um, or voice agents. You've either got Voice Engine, which is fantastic. You spend all your time building these excellent chat agents.
- 6:24
You can just do this little wrapper with a nice SDK. Or if you want to use a full agents platform for conversational, we've got that out of the box.
- 6:32
It's very easy, very quick to prompt. Uh, end with a, um,
- 6:39
a, a prediction. I think these chat agents will either die chat agents or start adding voice. Um, I'm excited to work with lots of people in the room. We're also looking for some design partners, so if you want to be some of the first people to do this, we'd love to chat.
- 6:53
But thanks so much. [audience applauding] Cool. Yeah.
- 7:01
Um, how do you handle tool calling?
- 7:04
Yeah. The cool thing about this is, um, your chat agent actually normally does the majority of tool calling. So it's actually already built out on the back end here.
- 7:15
And so you can have this wrapper without needing to deal with any of the issues of tool calling.
- 7:22
We also, um, at ElevenLabs have the concept of either client side s- uh, tools and server side tools. So you can do some pretty cool stuff where you then, like, expose the tools on the very front end to, like, manipulate the DOM, and we're gonna add in a way where you can proxy some of these tool calls
- 7:38
to the wrapped agent. But most folks will already have all the tool calling already handled with the chat agents. But...
- 7:46
Thank you.
- 7:48
Cool. Nice. Thank you so much.
- 7:52
Thank you. [audience applauding] [upbeat music]