Tom Redman - Building an AI assistant that makes phone calls
![](https://aie-cms-uploads.s3.us-west-1.amazonaws.com/AIE_Summit_2024_Day_2_WEB_27_d4b20f0e04.jpg)
![](https://aie-cms-uploads.s3.us-west-1.amazonaws.com/AIE_Summit_2024_Day_2_WEB_31_9aae6609ec.jpg)
![](https://aie-cms-uploads.s3.us-west-1.amazonaws.com/AIE_Summit_2024_Day_2_WEB_28_a997f74ee0.jpg)
![](https://aie-cms-uploads.s3.us-west-1.amazonaws.com/AIE_Summit_2024_Day_2_WEB_26_4220dc601e.jpg)
![](https://aie-cms-uploads.s3.us-west-1.amazonaws.com/AIE_Summit_2024_Day_2_WEB_25_1f50f3df15.jpg)
![](https://aie-cms-uploads.s3.us-west-1.amazonaws.com/AIE_Summit_2024_Day_2_WEB_22_4ce1d76ab3.jpg)
![](https://aie-cms-uploads.s3.us-west-1.amazonaws.com/AIE_Summit_2024_Day_2_WEB_24_bcc7867593.jpg)
![](https://aie-cms-uploads.s3.us-west-1.amazonaws.com/AIE_Summit_2024_Day_2_WEB_29_282d08eb6d.jpg)
![](https://aie-cms-uploads.s3.us-west-1.amazonaws.com/AIE_Summit_2024_Day_2_WEB_23_3b8d57e872.jpg)
![](https://aie-cms-uploads.s3.us-west-1.amazonaws.com/AIE_Summit_2024_Day_2_WEB_30_7fd18b5587.jpg)
RAG & LLM Frameworks: In this workshop, we'll use a variety of technologies to build a better AI assistant. This workshop leverages a Convex vector database to establish a knowledge base, Google Cloud speech-to-text, GPT-4 API, text-to-speech, Twilio and audio streaming. By the end, we'll have built an AI assistant that can take basic requests by voice, interpret what to do, make a phone call and interact with a human on the other end, then come back with some action to do (like adding an event to a calendar, or sending you an SMS update!).
This project is written entirely in Typescript, with Node, Express, and Next frameworks. To get started, clone this repo: https://github.com/get-convex/ai-world-fair.git
To follow along with full functionality, you will need keys for the following services. (Don't worry if you don't have them all! You can still follow along the presentation, and substitute the keys in later, or use entirely different services altogether.)
You will also need a free ngrok account if you wish to try it out locally.
See /server/.env.template for the required keys:
OPEN_AI_KEY=
GOOGLE_APPLICATION_CREDENTIALS=./service_account.json GOOGLE_OAUTH_CLIENT_ID=
TWILIO_ACCOUNT_SID= TWILIO_AUTH_TOKEN=
Tom is the Head of Developer Experience at Convex, which means he's the one to blame if your support tickets takes longer than 5 minutes to get a response. He also spends a lot of time talking with developers in order to better understand how to make Convex the fastest, most reliable and easy-to-use development platform on the planet. If you have any ideas, email him!
Tom got into software engineering as a kid, and it's a passion that never left. His free time is spent reading programming books (no joke), and playing covers of 90s pop songs on his guitar while singing extremely loudly and beautifully.
Tom is the Head of Developer Experience at Convex, which means he's the one to blame if your support tickets takes longer than 5 minutes to get a response. He also spends a lot of time talking with developers in order to better understand how to make Convex the fastest, most reliable and easy-to-use development platform on the planet. If you have any ideas, email him!
Tom got into software engineering as a kid, and it's a passion that never left. His free time is spent reading programming books (no joke), and playing covers of 90s pop songs on his guitar while singing extremely loudly and beautifully.
We have now sold out of Early Bird tickets; General Admission has also sold out.
Please join us online for the free livestream.