AI Engineer World's Fair 2024
Convex Launch
Read the talk
Convex: carrying reactive state through AI workflows
Convex extends dependency-driven updates from the interface into the backend, helping multistep AI applications expose results as their work progresses.
From a talk by Jamie Turner
Before you start: Basic familiarity with React state and backend APIs will help; no prior Convex experience is required.
Replacing backend glue
What would it take to replace routine backend engineering with an API that feels native to an application? Jamie Turner introduces Convex through that ambition, abandoning his planned deterministic-workflow title for a more revealing one: “We Accidentally Made an AI Platform and What Are We Gonna Do About It?”
The opening illustration includes an accidental generative-AI contribution: “fate limiting,” an ominous addition to the backend engineer’s responsibilities. The familiar work is less mysterious: gluing systems together, configuring them, and mapping data formats. Turner estimates that teams sometimes spend half their time on this integration work. Users receive little direct benefit from the plumbing itself. Convex’s proposed replacement is a high-level functional interface, with Firebase and Parse as precedents. The design question is what that interface should look like for applications built in the 2020s.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Extending React’s state model to the server
React supplies the starting point: create a state variable and a setter, then let the framework refresh the places that depend on that state. In Turner’s small example, changing a name produces an uppercase greeting for Olivia. The essential relationship can be expressed in TypeScript like this:
tsx
import { useState } from "react";
export function Greeting() {
const [name, setName] = useState("");
return (
<>
<input
aria-label="Name"
value={name}
onChange={(event) => setName(event.target.value)}
/>
<p>{`Hi ${name}`.toUpperCase()}</p>
</>
);
}
Entering Olivia calls the setter; React renders the dependent expression as HI OLIVIA. There is no separate instruction to find and update the greeting.
That relationship usually breaks at the server boundary. The application must poll for changes, invalidate caches, or arrange its own push mechanism. Convex exposes queries and mutations, but its distinguishing mechanism is pervasive tracking of backend data flow and dependencies. Queries are subscribable: when underlying state changes, applications can receive updated results. The reactive relationship extends through the backend, instead of ending at local component state.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A note-taker with visible checkpoints
The connection to AI appears when server-side work participates in the flow of state. A note-taking application needs more than a final response: it performs several operations and brings the interface back into the process at useful checkpoints. Turner sketches this sequence:
- Run automatic speech recognition on the recording.
- Summarize the transcription and show the summary in the application.
- Generate embeddings for retrieval.
- Find related notes and display them in the application.
The diagram connects ASR, summarization, embedding generation, a vector index, and related notes to the application. Each intermediate result can become useful before the whole workflow is finished.
The implementation distinction is between subscribed query results and actions that perform external work. Turner describes server-side actions as reacting entities; this does not establish that actions automatically subscribe to database changes. In the current action API, actions are invoked or scheduled, call external services, and store results through mutations. Query subscriptions then expose changed database state to the interface. That distinction makes the workflow concrete without assuming an automatic trigger for every processing step.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Concurrent results in a project planner
Real applications can have several branches running at once. Turner demonstrates webdevcody’s application for generating an initial project plan from a prompt. The example asks for an app to track recipes. Clicking Create Plan starts work on Convex, with concurrent chains producing several kinds of output:
- Project names: candidate names for the application.
- Initial feature requests: a first set of capabilities to build.
- Color palettes: possible visual directions.
- Icon ideas: suggestions for the application’s identity.
Results flow into the interface as the chains produce them. The application does not need to hold every output until one complete plan is ready.
Synchronizing state between backend steps and the application turns out to be especially useful for generative AI products. Turner reports that, after the ChatGPT boom, more than 90% of projects on Convex were generative AI projects. He does not specify the observation window or classification method; the figure describes his account of adoption at the time. That concentration of AI builders shapes the next additions to the platform.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Vector indexes in the existing schema
Vector indexing repeatedly appeared as a step developers needed. Convex already let them define schemas in TypeScript, with type completions, and add indexes to fields. Their request was to extend that same interface to vector indexes. The demonstrated foods schema adds a vector-index configuration with an embedding field, dimensions, and cuisine filtering. The extension keeps vector indexing alongside the application’s existing data definition.
Turner says vector indexing shipped late the previous year and was already broadly used by Convex projects. It is an available capability in this recording, unlike the workflow components introduced at the end.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Supporting startups and packaging workflows
The next announcement is Convex for Startups: discounts plus access to startup-only forums and events. Turner says the first admitted batch includes many generative AI companies, which he describes as the most engaged and excited customers at the time.
The final product direction raises the level of reuse from individual backend functions to whole state machines. Convex Components is intended to package sophisticated workflows, co-developed with customers, into building blocks developers can drop into an application. The slide illustrates the idea with puzzle pieces labeled RAG and Model Flow. These high-level components are forthcoming in the recording, described as arriving very soon.
The progression is from synchronizing individual state changes to packaging entire workflows that maintain state across multiple steps. Turner closes by inviting generative AI builders to try Convex, with the aim of shipping applications quickly and confidently.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
The backend platform combining a database, server functions, and reactive application updates.
Further reading
Define queries and understand caching, subscriptions, deterministic execution, and consistent database reads.
Call external services and coordinate their results with database queries, mutations, and scheduled functions.
The original vector-search announcement, with TypeScript schema and action examples and historical beta restrictions.
Current guidance for declaring vector indexes and searching embeddings in Convex.
Updates since the talk
- Components for your BackendArticle
A walkthrough of reusable backend components with persistent state, isolated APIs, and transactional behavior.
Read the complete timestamped transcript
- 0:00
[upbeat music] So, uh, originally I had this very fancy title for this talk, Deterministic Workflow and, uh, I don't know.
- 0:19
But what I really wanna title it is, We Accidentally Made an AI Platform and What Are We Gonna Do About It? [clears throat] Convex's true mission, my company, is to replace traditional back-end engineering, uh, all the kinda stuff that we do on back-end engineering.
- 0:35
Generative AI, by the way, thinks that fate limiting is one of those things. It's kinda cool. Um, sounds ominous, but, um, it is onimus- ominous, right? So we glue things to things.
- 0:45
We can fit your stuff for, uh, different systems. We map data formats constantly, and a lot of times teams are spending a lot of their time, like half their time on this stuff.
- 0:55
It has nothing to do with your product. Your users don't care, and they don't benefit. Um, so we wanna replace all this stuff with a high-level API, kind of functional interface that feels native to your application, similar to something like Firebase or Parse before it.
- 1:10
So if you were doing this in the twenty twenties and it was a design exercise, what would you replace all that stuff with? What would that API look like?
- 1:18
Well, for us, we took heavy inspiration from React and really more generally, the way that kind of all applications are starting to have this functional reactive data flow relationship to state.
- 1:29
Um, if you're not familiar with React, here's a little baby example. You can create a state variable. It has the setter. And what React really empowers is it makes sure that whenever that state changes, all the places that depend on it are updated, re-rendered, um, refreshed.
- 1:46
And so in this case, our app would have, "Hi, Olivia," in all caps. The problem is this paradigm breaks down when the server gets involved. The server doesn't play the game this way.
- 1:55
You still have to poll the server. You have to invalidate caches. You have to event your own push mechanisms. So Convex fixes that. So Convex has queries and mutations like other frameworks you may be familiar with.
- 2:06
But in Convex's case, it completely tracks pervasively data flow and dependencies through the back end. And so it extends the reactive paradigm into the back end. Um, queries are these universally subscribable app, um, entities that applications can, uh, get updates from as soon as updates are available.
- 2:27
So you might say, "What does this have to do with AI?" [chuckles]
- 2:31
So what it has to do with is that some of the reacting entities are actually server-side actions. It's not just the application. This may be a kind of architecture you've thought through before or played with, so something like a note-taker.
- 2:43
You know, maybe you're doing Automatic speech recognition, and then you summarize it, and you generate Embeddings, and find related notes or whatever. And along the way, th-these different checkpoints, the application sometimes needs to be brought in, show the summary, you know, show related notes, et cetera.
- 2:59
Um, but in practice, we find that apps are actually a lot more sophisticated than this. This is a developer named [REDACTED:username] who's building an application on Convex that kinda like generates a first project plan given a prompt.
- 3:10
So in this case, he is, uh, an app to track recipes, and when he creates-- hits Create Plan, it's running on Convex. This is sort of like, uh, let's get a bunch of, like, project names.
- 3:21
Let's get first features requests, color palettes, icon ideas. All of these, as you can imagine, are kind of concurrent chains that are running in the background, um, and all of them kind of flow into the application as they have results.
- 3:35
It ends up that Convex is kind of combination of, like, seamlessly syncing state between these back-end steps, and the application is incredibly useful for a lot of generative AI apps.
- 3:45
And for that reason, post-ChatGPT boom, like ninety-plus percent of projects on Convex are generative AI, um, and a lot of generative AI startups. So here's what we're doing about it. [chuckles]
- 3:56
So the first thing we did is we got a lot of feedback from developers that one of those steps was always vector indexing or quite often ven- vector indexing.
- 4:04
So the developer said, "This is how you make a schema on Convex." It's just TypeScript, type completions, all that good stuff. They said, "Well, you already allow us to add indexes to our fields like this.
- 4:15
Could you allow us to add vector indexes?" And so we said, sure. We rolled that out late last year, and it's being used very broadly now by projects on Convex.
- 4:24
Uh, the second thing we just did, um, which just kind of announcing right now, is we started a Convex for Startups program. Discount program, kind of access to startup-only forums and events and stuff like that.
- 4:37
Um, and the first batch, we just admitted tons and tons of generative AI companies in it. So [chuckles] again, this is sort of like, uh, the, the most engaged, excited, uh, customers right now.
- 4:49
And then very soon we're releasing these kind of high-level components. We have this Convex Components framework, which kind of encapsulates whole state machines in these building blocks, so you can easily drop into your app to have your back end encompass these sophisticated workflows that we've de-- co-developed with customers, um, very easily and rapidly.
- 5:07
So anyway, that's us. If you're building something cool in generative AI and you wanna sort of ship with confidence and quickly, check us out at convex.dev. Thank you. [upbeat music]