Giving a Voice to AI Agents

Talk transcript

1. 0:00

   [on-hold music] Uh, hey, everybody. If you don't know about Deepgram, Deepgram is a company that does, uh, audio AI.

2. 0:18

   So we've been around for nine years now, um, so ancient in AI years. Um, but, uh, s- we, we brought, uh, end-to-end deep learning to, uh, speech recognition, and, uh, our most recent product is a TTS product that we released back in, uh, December.

3. 0:37

   And so you can use our API, uh, on-prem, you can use it in the cloud. You can, um, adapt models to a specific acoustic environment. I just came from a drive-through that has all sorts of crazy things happening in it, like cars running and Amazon trucks bre- backing up and all of that, and we, uh, uh, like

4. 0:56

   automatically adapt models for that kind of thing, et cetera. Um, so anyway, if, if, if you don't know about us, that's what we do. Uh, also, we're a research-led company.

5. 1:05

   W-- Uh, so what that means is we figure out fundamental things through our research team, and then we bring them to, uh, the market as soon as possible through our product team.

6. 1:14

   So, uh, yeah, it's, it's, uh, everything is moving at breakneck speed, but a lot of the things that, uh, folks enjoy now with like really fast, accurate, real-time speech recognition, if you have any devices, um, out there, I was just talking with a founder a couple minutes ago, uh, uh, uh, most of them use Deepgram behind the

7. 1:34

   scenes. So, uh, just, uh, that's, that's who we are, and, uh, we help companies build truly conversational experiences. And the bar that we set for ourselves and we set from the beginning, um, but it takes a long time to get there, is, uh, what can a human do?

8. 1:50

   And we ask this all the time in our product meetings, et cetera, and our research team. Um, and that helps, uh, guide us toward, uh, what's possible, um, because neural networks really do work in a way that is similar to a human.

9. 2:03

   They can learn by example, et cetera. And so you have to think about how can we formulate, uh, this problem in a way that, uh, this, this machine can actually learn from it.

10. 2:12

    And so we spend all of our, all of our time thinking that way. And, uh, so arguments about like how many neurons does it have? Is it equivalent to a mouse or a cat or a person or whatever?

11. 2:20

    We're talking about that kind of thing all the time. Um, and it's amazing to see, um, the progress that has happened over the last decade, and we're at a, uh, an amazing time right now.

12. 2:29

    So, um, one thing to keep in mind is that there was like a previous version of voice AI, and, uh, it was kinda, you know, voice AI 1.0. Think Siri, think things like that, where it was slow, it wasn't super accurate, um, there wasn't really a real-time feel to it.

13. 2:46

    Um, and it, it-- you could try to ask it anything, but it wouldn't really, um... It'd only answer in a very specific domain. So now with the, the next version of voice AI, there's, uh, it's open-ended, and a lot of what's driving that is LLMs.

14. 3:01

    You can put any text into it, and, uh, depending on how the model is trained, you can output any text. And so if you're having a, uh, voice AI conversation, you can take the audio, turn it, using speech-to-text software, turn it into text, inject it into an LLM model, get text out the other side, and then use

15. 3:17

    a text-to-speech model in order to, uh, produce audio. And, uh, part of the, this now voice AI 2.0 is how fast can you do that and in how smart of a way can you do that so that the agent actually like does what it's supposed to, and then how expressive does it sound?

16. 3:33

    And so we'll talk about that a little bit, but, you know, this is, uh, the, the type of process speech... Uh, like speech-to-text to an LLM, I would just internally call that text-to-text.

17. 3:43

    Who cares if it's a, if it's a transformer model or whatever it is, but, uh, you transform speech to text, and then text to text, then text to speech, and you keep that loop going.

18. 3:52

    Um, and I would say, uh, what we're, what we're, what we're seeing right now, and we, we might, you know, the folks out there might say there's a big hype cycle.

19. 4:02

    We're at the peak of it or something. I would say, uh, hang on a second. This is more like 1910. You fee-- You see the first cars in the street or something like that.

20. 4:11

    Um, really, it's just a tiny thing that's happening right now, and there's gonna be a massive hundred x, uh, explosion in the next like decade for, uh, voice AI, text, et cetera.

21. 4:20

    It's, uh, AI's gonna be everywhere. The way I would think about this and, you know, encourage others to think about it this way is there's, there was like an agricultural revolution that took like a thousand, two thousand years to happen.

22. 4:31

    Then there was an industrial revolution that took like maybe two hundred fifty, three hundred years to happen. Then there was an information revolution that took like seventy-five years to happen.

23. 4:39

    See the trend? [chuckles] The intelligence revolution is going to take like twenty-five, maybe thirty years to happen. And so if you thought tech companies were fast before, AI companies have to move three times faster.

24. 4:49

    Um, so, uh, anyway, this is how it works today. There's a, um, speech-to-text system. You can see in there, there's an, an LLM or a text-to-text system, and, uh, then a text-to-speech system.

25. 5:03

    And each of these systems works fairly independently, um, but the state-of-the-art works very quickly, and they have high efficacy or accuracy. So speech-to-text now from, from even just five or, you know, eight years ago, it used to be maybe seventy-five percent accuracy or somewhere around there.

26. 5:22

    Uh, now it's like over ninety, and that over ninety, uh, really makes a big difference. Also, speech-to-text used to be maybe

27. 5:29

    two to five-second delay for real-time. Now it's like a hundred milliseconds or maybe two hundred milliseconds, um, with the high accuracy. And also you can run it on-prem, and you can co-locate all these services together.

28. 5:42

    Uh, actually, the, the founder of Daily, I just saw him roaming around. They just came out with a blog post showing that you can do the entire, uh, voice AI round-trip conversation in le-- in less than five hundred milliseconds using Deepgram for the speech-to-text, Llama for the TTT, and then, uh, Deepgram for the text-to-speech.

29. 5:58

    Uh, but nevertheless, that's what a human responds in. It's between like four hundred to six hundred milliseconds in turn-taking. So, um, you can, you can do all that here, but there is a piece that's missing, which is, um, if you...

30. 6:11

    These are all, like I just said, speech-to-text models, text-to-text models, text-to-speech. They're not passing along any context throughout the conversation. So the-- What, what this ends up with is a few, a few spots that you're like, "Hang on a second, um..."

31. 6:26

    It's not really getting exactly what I'm saying, um, and maybe that only happens ten percent of the interactions or twenty percent of the interactions, or what I really mean is like ten or twenty percent of the turns.

32. 6:38

    Um, but the way that you combat that is by, uh, adding in context. And so, um, the-- that's, that's what this, this view is right here. Uh, it looks like a really subtle change, but instead of there being speech-to-text models now where it just takes in audio and it puts out text, it will take in audio and,

33. 6:56

    and context. So think promptable, but not necessarily text promptable. It could be promptable with anything. It could be promptable with other audio. It could be promptable with images. It could be promptable with documents.

34. 7:07

    Um, it could be promptable with the previous turn of the conversation. Um, but what that gives you is it gives that speech-to-text model context. Um, when you send something to a speech-to-text model right now, it actually has to...

35. 7:19

    It's kind of amazing what it's able to do. It knows nothing about the conversation, and then it's just thrown into, like, a basketball game, and it has to transcribe everything, you know, quickly and, and with ju- with just, like, a few seconds of context and then do a really good job.

36. 7:32

    Um, w- what happens when you give that model the entire context of the conversation up until that point, it gets way more accurate. Uh, but it's not just about the accuracy because the next step in that is once you, uh, once you pa- you can pass that context along, you can pass the original input context along, but

37. 7:48

    you can also have your speech-to-text model output context as well, and that can output text that is human-readable. It can output audio. It could output images. It could output just embeddings, um, which people are now familiar with as, like, just a, a vector embedding.

38. 8:03

    Um, and so that can carry the state of the conversation, um, throughout the entire thing. I just wanna point out, this is not how systems are built right now.

39. 8:10

    Um, but in the next year, this is how they're going to be built, and this is when things are going to flip into, like, holy shit, this is-- this feels like human.

40. 8:18

    And because once that text-to-text is contextual from the audio, it knows it's hearing an angry person or a happy person. It knows that the conversation is flowing quickly or slowly.

41. 8:29

    It knows that there's, you know, light music playing in the background. It knows all this stuff, right? And so that text-to-text model can now generate the appropriate response. But it's not just about the text that it generates.

42. 8:39

    It ge- it generates its own context, right? So it can say to the text-to-speech model, "Hey, I need you to say this softly. I need you to say it slowly."

43. 8:47

    You know, I'm speaking very quickly right now, right? But, "Uh, I need you to say it slowly. I need you to say it in an authoritative tone." You know, that kind of thing.

44. 8:56

    And, uh, then when the text-to-speech model generates that audio, it will say, "I tr- I tried to generate it this way. I sounded like this. I think I did a good job," et cetera.

45. 9:05

    That's context that's gonna get passed to the next turn in the conversation. And so all this context is gonna be passed around. We call it contextual AI internally [REDACTED:username].

46. 9:14

    But, uh, this is what the next generation of these models is going to look like, and this is actually the innovation that's going to make it feel like a human because the speed part is taken care of, the accuracy part is taken care of.

47. 9:24

    Now it's all about context. Um, I know there's, uh, to preempt any follow-up questions [chuckles] about, uh, like, a mu- a multimodal model or a speech-to-speech model. Sure. Absolutely. It's, uh, you may mold or, like, meld some of these together.

48. 9:39

    You may put them all together. The problem with putting them all together is it's not as controllable. So we, uh, we are the largest, uh, speech-to-text API in the world now, but that's mostly because of businesses using us to power, like, Spotify, to power food ordering, to power, um, call centers, and that kind of thing,

49. 9:59

    and they need controllability in, in what they're doing. So if you just give, like, an open-ended prompt to a speech-to-speech model and just say, like, "Go to town," that's not the kind of experience that, like, a bank wants.

50. 10:11

    You know, [chuckles] they want a little more control. They want to maybe put a whole bunch of compute power in the speech-to-text to make sure that they get everything, uh, precisely right, and then the text-to-text doesn't actually have to be all that big because they're just doing a few...

51. 10:25

    They're, they're, they're just doing a few things like helping them reset their password or something like that, and then they want the text-to-speech to be, like, really expressive but, like, calm and only a single voice.

52. 10:33

    And so these are all gonna be kinda compartmentalized, um, because that brings us to the COGS conversation, the cost of goods sold. Um, right now a lot of people probably feel that, you know, AI is kind of expensive, but, uh, it, it doesn't have to be if you use the right tools and you use the right services

53. 10:48

    that focus on, uh, cost of goods sold, and so, um, and if you, if you choose the right size for each component in the stack.

54. 10:56

    So, um, uh, the, so in the future, that's what it's going to look like. Um, I'm trying to hurry through this because I, uh, wanna leave room for questions.

55. 11:05

    It's already been, like, fourteen minutes, um, or maybe twelve minutes or so. But, uh, uh, I'll j- I'll give you just a flash of, like, what the future will look like.

56. 11:13

    But, um, one thing that we're doing as, you know, a platform [REDACTED:username], um, we think, hey, if you, if you want... If you're doing anything in audio, you should be thinking about Deepgram.

57. 11:23

    Maybe you don't use this for every piece of it, but you should be thinking, "Hey, if I want low latency or real-time speech-to-text," you should definitely be thinking about Deepgram.

58. 11:31

    If you wanna be, uh, if you wanna be using low latency text-to-speech, uh, definitely at least talk to us. Um, that's a new product for us, but you know, the next version will be even, even more expressive.

59. 11:41

    But right now it's, I would say, uh, better than, like, Amazon, uh, Microsoft, et cetera, than their neural models. Not quite as good as Ele- ElevenLabs. Um, but, uh, but anyway, the next product coming out for us, which I want to give everybody the chance to, uh, try out or apply to for our preview, is our voice

60. 12:01

    AI agent, uh, which is a full, full stack where we put everything together. So y- you could, if you want, you can use your own API keys and use your own, um, LLMs.

61. 12:10

    Um, but, uh, also you could have it all put together with Deepgram, and this helps with reducing that latency, so you get your, um, turn-taking down to a very short, you know, three hundred milliseconds, five hundred milliseconds, six hundred milliseconds rather than, like, eight hundred or fifteen hundred or something if you tried to piece them together yourself.

62. 12:27

    Um, and if you wanna get access to this, uh, Voice AI Agent API, we have a QR code here. Um, and we have some folks in the back too.

63. 12:36

    If you saw a workshop, I think two days ago with Damien, he ga- gave an awesome workshop. He's in the back. You could talk to him about this. Um, but also just feel free to screenshot this or go to the link now or whatever it is.

64. 12:48

    Um, also, uh, [REDACTED:username], [REDACTED:username], we, um, give out, uh, two hundred and fifty dollars in credit, so anybody can, uh, try it out. Thanks, everyone. [outro music]