← All AI Engineer talks

AI Engineer World's Fair 2024

Convex Launch

About this talk

Convex co-founder Jamie Turner explains how a backend platform designed around React-like reactive data flow, subscribable queries, mutations, and synchronized application state became well suited to multistep generative AI applications. He illustrates workflows involving speech recognition, summaries, embeddings, and generated project plans, then describes built-in vector indexes and reusable Convex Components for sophisticated stateful backend workflows.

Chapters

  1. 0:00Why Convex aims to replace backend glue code
  2. 1:18Bringing React-style reactivity to backend queries
  3. 2:43Coordinating multistep generative AI applications
  4. 3:56Adding native vector indexes to TypeScript schemas
  5. 4:37Reusable Convex Components and closing

Talk transcript

  1. 0:00

    [upbeat music] So, uh, originally I had this very fancy title for this talk, Deterministic Workflow and, uh, I don't know.

  2. 0:19

    But what I really wanna title it is, We Accidentally Made an AI Platform and What Are We Gonna Do About It? [clears throat] Convex's true mission, my company, is to replace traditional back-end engineering, uh, all the kinda stuff that we do on back-end engineering.

  3. 0:35

    Generative AI, by the way, thinks that fate limiting is one of those things. It's kinda cool. Um, sounds ominous, but, um, it is onimus- ominous, right? So we glue things to things.

  4. 0:45

    We can fit your stuff for, uh, different systems. We map data formats constantly, and a lot of times teams are spending a lot of their time, like half their time on this stuff.

  5. 0:55

    It has nothing to do with your product. Your users don't care, and they don't benefit. Um, so we wanna replace all this stuff with a high-level API, kind of functional interface that feels native to your application, similar to something like Firebase or Parse before it.

  6. 1:10

    So if you were doing this in the twenty twenties and it was a design exercise, what would you replace all that stuff with? What would that API look like?

  7. 1:18

    Well, for us, we took heavy inspiration from React and really more generally, the way that kind of all applications are starting to have this functional reactive data flow relationship to state.

  8. 1:29

    Um, if you're not familiar with React, here's a little baby example. You can create a state variable. It has the setter. And what React really empowers is it makes sure that whenever that state changes, all the places that depend on it are updated, re-rendered, um, refreshed.

  9. 1:46

    And so in this case, our app would have, "Hi, Olivia," in all caps. The problem is this paradigm breaks down when the server gets involved. The server doesn't play the game this way.

  10. 1:55

    You still have to poll the server. You have to invalidate caches. You have to event your own push mechanisms. So Convex fixes that. So Convex has queries and mutations like other frameworks you may be familiar with.

  11. 2:06

    But in Convex's case, it completely tracks pervasively data flow and dependencies through the back end. And so it extends the reactive paradigm into the back end. Um, queries are these universally subscribable app, um, entities that applications can, uh, get updates from as soon as updates are available.

  12. 2:27

    So you might say, "What does this have to do with AI?" [chuckles]

  13. 2:31

    So what it has to do with is that some of the reacting entities are actually server-side actions. It's not just the application. This may be a kind of architecture you've thought through before or played with, so something like a note-taker.

  14. 2:43

    You know, maybe you're doing Automatic speech recognition, and then you summarize it, and you generate Embeddings, and find related notes or whatever. And along the way, th-these different checkpoints, the application sometimes needs to be brought in, show the summary, you know, show related notes, et cetera.

  15. 2:59

    Um, but in practice, we find that apps are actually a lot more sophisticated than this. This is a developer named [REDACTED:username] who's building an application on Convex that kinda like generates a first project plan given a prompt.

  16. 3:10

    So in this case, he is, uh, an app to track recipes, and when he creates-- hits Create Plan, it's running on Convex. This is sort of like, uh, let's get a bunch of, like, project names.

  17. 3:21

    Let's get first features requests, color palettes, icon ideas. All of these, as you can imagine, are kind of concurrent chains that are running in the background, um, and all of them kind of flow into the application as they have results.

  18. 3:35

    It ends up that Convex is kind of combination of, like, seamlessly syncing state between these back-end steps, and the application is incredibly useful for a lot of generative AI apps.

  19. 3:45

    And for that reason, post-ChatGPT boom, like ninety-plus percent of projects on Convex are generative AI, um, and a lot of generative AI startups. So here's what we're doing about it. [chuckles]

  20. 3:56

    So the first thing we did is we got a lot of feedback from developers that one of those steps was always vector indexing or quite often ven- vector indexing.

  21. 4:04

    So the developer said, "This is how you make a schema on Convex." It's just TypeScript, type completions, all that good stuff. They said, "Well, you already allow us to add indexes to our fields like this.

  22. 4:15

    Could you allow us to add vector indexes?" And so we said, sure. We rolled that out late last year, and it's being used very broadly now by projects on Convex.

  23. 4:24

    Uh, the second thing we just did, um, which just kind of announcing right now, is we started a Convex for Startups program. Discount program, kind of access to startup-only forums and events and stuff like that.

  24. 4:37

    Um, and the first batch, we just admitted tons and tons of generative AI companies in it. So [chuckles] again, this is sort of like, uh, the, the most engaged, excited, uh, customers right now.

  25. 4:49

    And then very soon we're releasing these kind of high-level components. We have this Convex Components framework, which kind of encapsulates whole state machines in these building blocks, so you can easily drop into your app to have your back end encompass these sophisticated workflows that we've de-- co-developed with customers, um, very easily and rapidly.

  26. 5:07

    So anyway, that's us. If you're building something cool in generative AI and you wanna sort of ship with confidence and quickly, check us out at convex.dev. Thank you. [upbeat music]