AI Engineer Summit 2023
GPT Web App Generator - 10,000 apps created in a month: Matija Sosic
Read the talk
Generating a full-stack to-do app with Mage
Mage turns a short app description into a working React and Node.js starter by combining Wasp scaffolding, staged code generation, targeted repairs, and selective model use.
From a talk by Matija Sosic
Before you start: Basic familiarity with React, JavaScript modules, and database records will help; no prior Wasp experience is required.
Start with a to-do app
What does it take to turn a short description of a to-do app into a working application, including authentication and a database? Mage starts with a web form and returns a downloadable codebase. Matija Sosic reports that people used it to create over 10,000 applications in one month.
The demonstration begins with an application name and a few sentences describing one main page that lists tasks. Users should be able to create tasks, edit them, and toggle their completion. A creativity setting corresponds to GPT temperature: the safer setting tends toward fewer features, while a more adventurous setting can introduce more features and more mistakes. Sosic chooses the balanced option and presses Generate. The result is a finished project with source files available to inspect.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From generated files to persistent tasks
The generated stack combines React, Node.js, and Prisma through Wasp, a full-stack framework. Its central configuration describes authentication, routes, pages, and the data model in a high-level declarative form. The application still contains ordinary React client code and Node.js functions that execute on the backend; the configuration connects those pieces.
To run the downloaded project locally, Sosic uses:
sh
wasp start
The app opens with username-and-password login. After logging in, he creates tasks, including “Become an AI engineer,” and marks them complete. He then opens the database inspector included with Wasp to check the stored results.
The inspector shows two task rows with isDone set to true. Sosic adds “Have fun,” which initially has isDone: false. Completing that task in the app changes the stored value to true. This is the decisive part of the demonstration: the generated interface is connected to persistent data, and a completion toggle updates that data.
Completing “Have fun” changes its stored state
Constructed example: Object IDs are presentation identifiers; the visual shows only the task text and completion field described in the demonstration, not a complete database row.
Have fun
Operation: Complete the task in the app, then inspect its database state.
Task text
Have fun
Have fun
isDone
false
true
Deployment follows the same full-stack structure. Wasp offers a single-command helper for Fly.io, while the underlying React and Node.js application can also be deployed elsewhere. At the time of the demonstration, support for additional providers through the helper was described as forthcoming.
Sosic describes Mage as among the early coding agents capable of generating full-stack web apps with almost no errors. That prompted two recurring questions from users: why did it work so well, and was generation cheap enough to offer for free?
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reduce the work before asking the model
Mage makes generation easier by narrowing what it has to generate. Three choices reinforce each other:
- Web apps only: Supporting React, Node.js, and Wasp lets the generator make assumptions about the application structure up front.
- High-level framework: Wasp removes boilerplate that the model would otherwise need to write and connect correctly.
- Automatic debugging: Mage detects and repairs errors before returning the final result. The restricted stack and smaller code surface also make this repair problem more manageable.
Those constraints enable step zero, which happens before the coding agent takes over. Mage produces known pieces of the project without calling the OpenAI API: configuration files, some authentication logic, global CSS, and similar shared infrastructure. The model does not have to rediscover the same starting structure for every app.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Plan, generate, then repair
After scaffolding, the generation log exposes three phases:
- Planning: Turn the app description into a list of queries, actions, entities or data models, and pages. For the to-do app, the plan includes one page.
- Generation: Implement the components listed in that plan.
- Repair: Detect common errors and attempt fixes, repeating when a fix fails.
The displayed run includes a failed repair followed by another attempt. It finishes when Mage cannot detect further errors; that stopping condition does not establish that every possible behavior is correct. Sosic reports that the demonstrated run used about 27,000 tokens.
During development, the team found recurring mistakes such as mixing up default and named imports. Some errors could be fixed with a simple heuristic, without asking GPT again. For example, a named export in task.js:
js
export function completeTask(task) {
return { ...task, isDone: true };
}
requires a named import in its consumer:
js
import { completeTask } from './task.js';
const task = { title: 'Have fun', isDone: false };
const completedTask = completeTask(task);
The braces in the import follow from the export declaration. A repair rule can address that mismatch directly instead of treating it as a fresh code-generation problem.
Sosic reports that the described fixes handled 90% of errors. He does not specify an evaluation set or denominator, so this should not be read as the percentage of applications that worked, or as a result attributable entirely to heuristics. Wasp also contributes by removing code that could otherwise contain mistakes and concentrating application structure in its high-level configuration.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Spend the stronger model on planning
Sosic reports that typical Mage apps took 2–3 minutes, used 25,000–60,000 tokens, and cost $0.10–$0.20 to generate. These are historical figures for the demonstrated model setup, not current API prices.
The cost reduction comes from assigning GPT-4 and GPT-3.5 to different stages. Sosic estimates that using GPT-4 throughout would have cost ten times more, or $1–$2 per app. The actual division of work is compact:
| Stage | Model | Reason |
|---|---|---|
| Planning | GPT-4 | Most complex and creative decisions |
| Implementation | GPT-3.5 | Faster, cheaper execution of the plan |
Once the plan exists, Wasp's web abstractions provide enough guidance for the cheaper implementation model to do useful work. The architecture reduces how much independent judgment each implementation step requires.
For comparison, Sosic reports a cost range of $0.80–$10 for another, unnamed agent to produce a similar app. That agent takes a freer approach and relies more on GPT itself. The range is an illustrative comparison, with no controlled same-task methodology supplied in the talk.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A customized CRUD starter
The useful scope is a customized CRUD starter: an application that creates, reads, updates, and deletes data, with its initial structure tailored to the user's description. At that level, Sosic reports almost no or very few errors that users can readily detect and fix. React, Node.js, and Tailwind give developers familiar pieces to continue working with after generation.
Sosic sees this as a direction for SaaS starters: generate a starting point around the actual app instead of asking every developer to adapt generic boilerplate. But the prompt still needs a useful level of specificity. Pushing toward more ambitious applications produces more errors, while underspecifying the behavior also hurts results. “Make Facebook, but yellow” supplies a recognizable reference and a color, but little concrete guidance about what the generator should build.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Beyond the initial prompt
Mage began as an experiment in generating full-stack applications with Wasp. Its simplicity also creates its main product limitation: the user has no interaction with the agent beyond the initial prompt. The proposed next step is a live debugging mode, letting users remain on the web page and request changes or error fixes after seeing the result. That would turn a single generation attempt into an iterative development conversation.
A model fine-tuned for Wasp and web development is another possibility, but its value remains uncertain. Sosic anticipates greater expense and questions how much improvement specialization would bring when Wasp's syntax is already simple and human-readable. Both live debugging and fine-tuning are proposals in this account.
Mage comes from the team behind Wasp, a fully open-source framework for React and Node.js development. Sosic closes by inviting questions and comments through the project's repository and Discord community.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
Further reading
- How Mage's code agent worksArticle
The team's original explanation of scaffolding, planning, code generation, repair and model allocation.
Examples and limitations of the initial React, Node.js, Prisma and Wasp generator.
A team-authored comparison using Pong and blogging apps, including prompts, repairs and historical generation costs.
The speaker's account of Mage's origins, launch and adoption.
Updates since the talk
Versioned instructions for generating apps through Mage or the Wasp CLI.
- Mage source archiveRepository
Archived source and setup instructions for Mage, frozen on Wasp 0.23 and no longer maintained.
Read the complete timestamped transcript
- 0:00
[upbeat music] Hey, I'm Matija, and I will show you how we created a GPT-powered full-stack web app generator and how it was used to create over ten thousand
- 0:24
applications in one month. So first, we will see what it is, and then secondly, we will check out how it works under the hood. So let's get started. So everything happens on this web page, and it's super simple.
- 0:37
First, we have to enter the name of our application. Let's say we are building a simple to-do app. Second part is describe how it works in a couple of sentences.
- 0:46
So we have a simple to-do app with one main page, uh, listing all the tasks. User can create tasks, change them, toggle them, edit them. Creativity level corresponds to GPT temperature, so we can go on the safe side and get less features, or we can go a little bit crazy, but also have more mistakes.
- 1:03
So I will stick with this balanced one. And the last thing left to do is just to hit this Generate button. Bam. Here we can see the result of the generation.
- 1:13
So we got a full-stack app in React, Node.js, Prisma, and it's all glued together with a full-stack framework, Wasp. So the secret of Wasp is that it relies on this single configuration file which describes your app in a high-level declarative manner.
- 1:29
So here, for example, we can see our auth in just a couple of lines, our routes, pages, our data model. So everything is here. And still here we can see our client code.
- 1:39
For example, here is our React, and here we have our Node.js functions which are being executed on the back end. So the last thing to do is just to download this app locally and run it with Wasp.
- 1:52
So let's do that. So I downloaded the app locally, and now we just have to run it via Wasp Start,
- 2:00
and there we have it. We can log in with username and password.
- 2:04
Now let's create a couple of tasks with Mage. Become an AI engineer. Mark it as done. And now let's check it out in the database. So we have a database inspector that also comes with Wasp.
- 2:21
And here we can see two rows, one for each task, and both are completed. Is done is true. So let's try adding another task. Have fun. And let's check it out in the database again.
- 2:33
We see it's not completed. Is done is false. But if we complete it, we can see now it's true. So there it is. We got a fully working full-stack web app in React and Node.js and Wasp.
- 2:48
You can also now deploy this app with a single CLI command.
- 2:52
It's a regular React and Node.js app, so you can deploy it virtually anywhere. But we have a CLI helper in Wasp that makes it super easy to deploy to Fly.io, and more providers are coming soon.
- 3:03
This is one of my favorite features. When we got Mage out, it was hardly the first AI coding agent, but it was among the first ones that could generate a full-stack web app with almost no errors.
- 3:14
When we released this and people started using it, we were getting two main questions: how come this works so well, and secondly, how can you offer it for free?
- 3:23
Is it that cheap? So let's answer them one by one.
- 3:28
There are three main reasons for Mage's performance. First, it is specialized only for full-stack web apps and nothing else, only React, Node.js, and Wasp. That allows us to assume a lot up front and makes everything easier and faster.
- 3:43
Second, it makes use of a high-level web framework, Wasp. That takes away a ton of boilerplate and makes it much easier for GPT to do its job. And lastly, Mage fixes the errors before it gives you the final result.
- 3:57
Again, because of the two points I mentioned previously, this is also a simpler problem than for the general AI coding agents. Let's dive a bit deeper. Let's go back to our generated app.
- 4:08
Since Mage knows we are building a full-stack web app and it's using Wasp for it, we can produce a lot of code up front without even touching the OpenAI's API and asking GPT any questions.
- 4:20
For example, some of the config files, then also some of the authentication logic, which we can see right here, and global CSS and similar. So we call this step zero.
- 4:32
Only then the code agent takes over. The code agent's work consists of three main phases: planning, generating the code, and fixing the errors. So let's expand the generation log and explore each of the cases.
- 4:46
Here, following the step zero, we can see the planning phase. Given our app description, Mage devises it needs to generate the following queries and actions, entities or data models, and one page.
- 4:58
After that, the generation step happens. Mage is actually implementing everything it planned for above. And finally, here comes the error-fixing phase. Mage can detect some of the common errors and fix it for itself.
- 5:11
Here, it failed to fix, so it had to try again. And final- finally, when it cannot detect any more errors, we are done. We can also see that all this took about twenty-seven thousand tokens.
- 5:22
The cool thing is that while developing Mage, we identified the most common errors it consistently kept making, like mixing up the default and named imports. Some of them we even ended up fixing with a simple heuristic without involving GPT.
- 5:36
That took care of ninety percent of all errors. Again, Wasp framework with its high-level configuration was of great help here since it removed a ton of code and reduced the space for errors significantly.
- 5:48
Now let's take a look at an- another question we had. How much does it all cost? Typical app we created with Mage took about two to three minutes and twenty-five to sixty thousand tokens, which comes to about ten to twenty cents.
- 6:02
But there is one trick we used. We used GPT-3.5 and GPT-4 interchangeably for different stages, and that reduced the bill a lot. If we used only GPT-4 for everything, the cost would have been 10X more, so one to two dollars per app.
- 6:21
What we did is we used GPT-4 only for the planning stage, which is the most complex step and one that requires the most creativity. For the actual implementation, we could comfortably use GPT-3.5, which is both faster and cheaper, and that worked great.
- 6:36
Again, the key here is that we provided a highly guided environment for the coding agent, given Wasp's web abstractions, and that's why this approach worked. This is also the main difference between Mage and the other coding agents.
- 6:50
We tried another popular agent that uses the more free approach and relies more on the GPT itself, and the cost to make a similar app as we did with Mage was between 80 cents and $10.
- 7:03
So what should you use Mage for, and what should you expect? Is it going to magically produce any app you imagine, or do you still have to put some work in?
- 7:12
At current stage, Mage serves as a really good and highly customized CRUD starter for full-stack web apps. At that level, it can operate with almost no or very little errors that you can easily detect and fix.
- 7:24
Most of people that tried it found it as a super easy way to get their app kick-started with the mainstream pieces of stack such as React, Node, and Tailwind, and that's how Mage got its popularity.
- 7:34
I personally believe this is what the future of SaaS starters look like- looks like, tailored to your app instead of starting out with a generic boilerplate. As you would expect, the more you push it, more errors it starts making.
- 7:47
On the other hand, not giving enough information and just saying something like, "Make Facebook, but yellow," can also be countereffective.
- 7:56
So what comes next? We created Mage as an experiment to see how well it can produce full-stack web apps with Wasp, and it worked surprisingly well. The current main limitation of Mage comes from its simplicity and the fact there is no interaction with the user beyond the initial prompt.
- 8:13
So that's something we are looking to add next, a live debugging mode where you can, while still on the web page, interact with the agent and request changes and error fixes.
- 8:23
Another thing that would be interesting to explore would be using an LLM that is fine-tuned for Wasp and web development, although that would also make it more expensive. Also, since Wasp has such simple and human-readable syntax, it's hard to predict how much benefit would fine-tuning bring.
- 8:38
Still, it would be a cool thing to try out. And that's it. We saw what Mage was, how it works, and what is the secret sauce that made it both fast and affordable to create web apps.
- 8:50
So thanks so much for watching. I had a lot of fun making this video with my helper, and I hope you also found it interesting. Please give Mage a try and let us know how it went.
- 9:00
We are the same team that created Wasp, which is a fully open source web framework that makes it super easy to develop with React and Node.js. Also, check out our repo and join our Discord for any questions and comments. [outro music]