Building Reactive AI Apps

[upbeat music] How's everybody doing this morning?

All right? Yeah? I can't see anybody past the first two rows, so I assume you're all there. Now, I know what you're all thinking. I look way better in person than that AI-generated headshot, right?

Well, we're, we're gonna work on that. We're gonna... You know, the AI models are not perfect yet, but we'll get there. I need the clicker. Did that... Oh, oh, it's up here.

Awesome. All right. Well, today I'm super excited to talk to you about what we've been working on at Fixie for a while, which is a open source framework for building what we call reactive AI applications called AI.jsx.

So let's get into it. First of all, what's the problem we're trying to solve? Well, I think most of us here know this already, but building and deploying high quality LLM apps is still super hard.

It's a lot of pieces you gotta worry about. You got your vector databases, your context window limits, your RAG stack, your tool sets, all that stuff. Now, most of us in this room enjoy solving those problems.

That's why we're here. But we think most developers probably would rather not have to solve these problems themselves, so we wanna do this for them.

So at Fixie, we are aiming to solve this problem by inventing the future of AI application development.

Today I'm going to be announcing something that I think everyone here will agree is the most revolutionary technology in the AI development landscape. It is going to change the way we all think about building AI powered applications.

It's going to blow you all away. And so ladies and gentlemen, may I introduce to you the future of AI application development. Imagine the 2001 theme playing. [singing]

Angle brackets. This is the future. [clapping] Yes. [upbeat music] There was supposed to be sound with that. That's all right. Okay. [laughs]

It was better with the sound. All right. So this is AI JSX, and I'm gonna talk about this a bit. Think about this as elegant LLM development for, uh, you know, a more civilized age in TypeScript.

Okay. So before I get into what AI JSX is, you might be wondering why TypeScript, right? Well, we believe that there are many front end and full stack devs that are just...

They don't have good tools for building AI powered applications today. There are some stuff, right?

But we also think that the AI devs, uh, the front end devs are the ones building the AI experiences of the future. But today they're kind of like that bear staring in the window, right?

They're saying, "Hey, we want a piece of the action. We want to get involved in AI development." But it's all the Python devs that are in the back end that are having all the fun here, right?

So why should the Python devs have all the fun? There's also a lot more JavaScript developers in the world than there are Python developers. So we think that helping this community that hasn't been well served so far makes a lot of sense.

So without AI JSX, this is what an AI engineer looks like today. I think I met that guy out in the hall yesterday. But with AI JSX, we can all be Hacker Man or Hacker Woman.

Yes. All right. [clapping] So get ready. I'm gonna rock you with this whole presentation on this. Okay. So let's just show a Hello World of AI JSX. It's a very simple example.

What we're doing here is defining a complete application

that uses a chat completion component. That's the foundation of anything that might call into an LLM, and we're providing it a user message prompt in the form of, you know, write a Shakespearean son- uh, sonnet about large language models.

You might get a result something like this. It's actually pretty good.

That's a Hello World though. That's very simple. But let's talk a little bit about what AI JSX is. Think about it like React, but for building LLM apps. Uh, my colleague, uh, at Fixie pointed out to me that we actually have a page on our documentation site that says that AI JSX is not React, but clearly that's

wrong because here I am telling you it is. Anyway. So AI JSX, it's built in TypeScript, which means you get all the safety and performance of JavaScript with the exciting adventure of fucking around with your dev tooling. [laughs]

Uh, any model, any provider, uh, we can support both Anthropic and OpenAI.

Uh, full React integration. This is great for building full stack applications where you just wanna drop your AI powered stuff into your React app.

AI JSX supports RAG out of the box, uses multiple vector DBs behind the, behind the scenes. You can plug in your own as well.

You can use it to invoke custom tools and APIs. So you can use AI JSX in situations where you want to invoke an external service or an API.

One of the cool features of AI JSX is the ability to have the AI generate UI components for you. Because the AI JSX program is operating on effectively the in...

the, the DOM, if you will, as React components, you can use it to generate UI components.

And of course, it's fully programmable and extensible, so you can basically build whatever you want.

I'm gonna give you a whirlwind tour of all this, show you what's, what's possible. Um,

basic idea is you build components just like you do in React. Here I'm defining a component called MakeSetting. Might c- define a setting for a story that we might wanna write.

And this component takes in its children elements as a parameter,

and we basically say, "Write a one-paragraph description of this setting," and put the children components right there. Those children components can be anything. They can be a string or they can be the result of a different tree of JSX nodes that have been rendered and placed in line in that prompt.

So to call it, all I need to do is say, take the MakeSetting component, instantiate it, and give it the prompt that I want.

Okay. It's pretty cool, very easy to use. But you might be saying to yourself, "Come on. This is basically writing Python code with different syntax." Right? This is... The angle brackets are a little bit overplayed here.

Um, but it's not just about syntax, right? JSX defines an entire tree of nodes that are rendered as a stream asynchronously and in parallel. So instead of rendering to the DOM like React does, we're re-rendering, uh, we're rendering effectively to the LLM, if you will.

Um, so this allows us to do extremely powerful forms of composition. So here's a simple example of writing a story where I have a MakeStory component with three child components.

One is defining the character, another is defining the setting, and a third is defining the plot.

When we render this application, all three of those components are going to run in parallel, and they're all streaming in parallel. There's three concurrent LLM calls going on, and they're streaming their tokens back to the MakeStory component in real time.

And so as this is being rendered, the, all the tokens are streaming through, the MakeStory component is then streaming its output out to the result of that render, which might result in a story that looks like this.

So far, I'm just showing you some basic things with text to give you some intuition around the ideas, but of course, you can take this a lot further.

Um, one thing you might say about this tree-based structure in AI.jsx is that it allows you to break free of your chains. Sorry, I know that was a groaner.

Okay. Sorry. Okay. So, [coughs] here's another example of what you might be able to do. Um, let's wrap one component in another in order to constrain the former, uh, the latter component's output.

So we're gonna define a KidSafe component. This component takes in a system message that says, "Rewrite the following text so it's safe for kids," and the child components of that component are placed into the user message of that prompt.

Then when we just wrap any component we want in a KidSafe component, it automatically will rewrite the output to be kid-safe, right? So very, very powerful composition.

Let me show you a quick example of how you use AI.jsx to call out to tools and third-party APIs. In this case, we're going to define a record, which is a set of tools that we want to give the LLM access to.

We're just gonna define one tool here. This is a tool that calls the GitHub GraphQL API.

And we're gonna give it an English description of the tool, and there's a JavaScript function there. I've taken out the code for the JavaScript function because that's not interesting for this talk, but that's just calling, using the fetch API to call the GraphQL endpoint at GitHub.

To use the tool in an application then, all I have to do is instantiate a UseTools component, give it that set of tools, and then anything that might need to use those tools as part of the rendering process can now invoke them.

And so I can build very powerful applications in this way.

This is RAG, retrieval-augmented generation, in something like ten lines of code. In AI.jsx, we have a ChatCompletion. There's two children of that, the SystemMessage and the UserMessage. The SystemMessage says, "Use the following information to answer the user's query," and it gets that information by using a DocsQA component.

The DocsQA component is configured with a corpus of documents that you've crawled and indexed and placed in a vector database.

You provide the user's query. The DocsQA component returns the chunks that are relevant to that query,

places them right there in the SystemMessage, and then the UserMessage contains the query again, and the final result is you effectively have retrieval-augmented generation.

I think this is a lot easier to understand. It's a lot easier to manipulate. It's a lot easier to integrate with other applications when expressed this way rather than have a whole lot of different, uh, libraries that you have to invoke.

And finally, when you're done building an AI.jsx application and you wanna place it into your website or your web app or your mobile app or whatever it is, you can just drop it right in as a React component, right?

So in this case, we're showing you use the FloatingFixieEmbed component that when you instantiate this, gives you a fully rendered UI for your AI.jsx application with a chat window, with session management, markdown, uh, rendering, custom UI, all the things that you might want out of such a thing in effectively one line of code.

I've been talking a lot about AI.jsx as an open source project. Of course, I'm standing in front of you as a founder of a startup, so we gotta make money somehow [laughs].

And so we're talking about the Fixie platform as a really effective way to take AI.jsx applications and host them and run them and manage them in the cloud.

So we make it really easy to build and deploy these things. The Fixie cloud service has a fully managed RAG pipeline that does document ingestion, chunking, embedding, vector database, document storage, and all of the things that you need there.

It fully manages the conversational state between the end user and the agent that you've built so that you can have full context as part of that interaction with the user.

We provide interfaces to all the popular LLMs, and those are really tuned for high performance and low latency so that we get really good performance out of this.

You can build your applications either in a no-code way completely in the web UI here, or you can use AI JSX to build a completely custom thing. And as I showed earlier, embedding it on your website is super easy.

So that's the Fixie platform. Um, right. So with that, you know, I'd encourage you all to try it out. Just go to Fixie.ai. You can sign up. Um, it's free to start out.

Uh, start building. AI JSX is open source. You can use that without using Fixie if you want. Um, so go check that out. I'd really, really love to get your feedback and hear more about what you wanna build and what kind of things you wanna do.

But one more thing. This is invoking, of course, Steve.

Another thing we built with AI JSX is the ability to support real-time voice interaction. That means bi-directional. You can speak to the agent, and it can speak back to you.

And we've done a ton of work to make the performance as just incredible, just amazing as possible. If you've seen the ChatGPT voice demos that they did, uh, last week or so, there's something like a four or five-second gap after you talk to it and it talks back to you.

I'm gonna show you a demo in a moment that I think will, uh, be fairly impressive.

So we get state-of-the-art performance, real-time bi-directional conversations, and of course, it's all grounded on your docs and your data. So I'm not foolish enough with the Wi-Fi that's here to do a live demo, so fortunately, our CTO, Justin, recorded a video of this.

But I want to assure you that nothing in this video is staged. Nothing has been trimmed out. Nothing has been sped up. This is real-time, you know, live video recording from yesterday.

And you can try it out yourself at voice.fixie.ai if you don't believe me. Just go try it on your own machine. Not now. [laughs]

Not here, please. Uh, but the idea of this demo is we have a fictional donut restaurant called Dr. Donut,

and you're at the drive-through. You drive up in your Tesla or whatever you're driving, roll down the window, and you're talking to the drive-through window at the Dr. Donut, and the AI is acting to take your order.

And so with that, I'm gonna see if this works. Hopefully, we have sound on the video, and let's take it away.

Welcome to Dr. Donut, home of the best donuts in town. How can I help you?

Yeah. Could I get two of the pumpkin spice iced donuts?

Sure. I've added two pumpkin spice iced donuts to your order. Can I assist you with anything else?

Uh, what's your best coffee?

Our pumpkin spice coffee is a popular choice this season, and our caramel macchiato is a favorite all year round. Which one would you prefer?

Yeah. Let's go with two of the macchiatos.

Great. I've added two caramel macchiatos to your order. Can I get you anything else?

Yeah. What are your thoughts on, uh, large language models?

Sir, this is a Dr. Donut.

Uh, okay. Just the donuts and the coffee then.

Sure. Your order is two pumpkin spice iced donuts and two caramel macchiatos. The total is $9.56. Please pull up to the next window.

And that was all... Like, you can go try it yourself. Um, we've done a ton of work to make the latency really low, to make the quality really high.

It's using the full Fixie platform as a back end. And of course, if you're building with Fixie and AI JSX, you can build the same thing for yourself, uh, donuts or not, right?

Okay. So that's basically all I have. I'd really love to get you all to check it out. We are, of course, hiring, and so there's our careers page. And, uh, please, uh, you know, don't forget to smash that like button.

Thank you very much. [upbeat music]