AI Engineer World's Fair 2025
On Curiosity — Sharif Shameem, Lexica
Read the talk
On Curiosity: Discovering Model Capabilities by Making Demos
From browser-compiled JSX to failed shopping agents and recursive problem maps, Sharif Shameem explores how small demos reveal capabilities that model specifications cannot.
From a talk by Sharif Shameem
Before you start: Basic familiarity with language models, context windows, and React components will help with the implementation examples.
An idea arrives on the bus
Henri Poincaré had been stuck for weeks on a problem involving Fuchsian functions. Then, while boarding a bus, an insight arrived suddenly. In his account of the episode, the breakthrough follows sustained work but occurs away from the desk. The interesting question is what happens between deliberate effort and an idea becoming available to conscious thought.
For Sharif Shameem, founder of the image-model company Lexica, curiosity is the feeling through which subconscious pattern recognition surfaces. That is his interpretation of the creative process: experience accumulates, connections form before we can fully explain them, and an interesting possibility pulls us toward an experiment. A demo turns that feeling into something other people can inspect.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From an exciting idea to a working experiment
The demos follow a recurring sequence. An idea initially seems wonderful; implementation makes it look impossible; then a concrete workaround gets it working. Small context windows make that middle stage particularly tangible: an application that seems straightforward in your head may not fit into the model’s input at all. The eventual reward is both the pleasure of making something work and new evidence about what the model can do.
This makes demos more than presentation material. You cannot fully understand a model’s capabilities without interacting with it, and building an interesting application forces that interaction. Curiosity chooses the question; implementation exposes the constraints; the working result makes the possibility visible.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make generated code immediately visible
The first examples come from the release of GPT-3 in 2020. Shameem describes its context window as roughly 2,000 tokens. He recalls a price of about $75 per million output tokens, needing permission before sharing model outputs publicly, and restrictions that prevented shipping a chat app. These are recollections of the early API environment, not current pricing or usage rules. Despite those constraints, the model felt like a new computing primitive: software could be produced from a description.
A talk by Bret Victor supplied the design principle: creators need immediate feedback from what they create. Copying generated code into an editor, compiling it, and running it inserted several steps between a request and its visible consequence. Shameem put a JSX compiler in the browser, shortening that loop. The selected demo frame shows the result: a request for rainbow-colored buttons beside generated code and rendered rows labeled Red through Violet. He contrasts the old constraints with a Claude system prompt he describes as 25,000 tokens at the time of the talk—a striking comparison, rather than a fixed specification for Claude.
A second demo takes a request for a Google homepage. The whole application would not fit into one generation, so the implementation sends three prompts in parallel, generates three components, and joins their outputs in the background. Decomposition makes the application possible within the available context. The interface presents one request and one result even though the underlying work is split across several generations.
The idea combines two things Shameem had encountered separately: language models that could write code and a design principle about immediate feedback. His point about a person’s unique experience is concrete here. The demo did not require a wholly unprecedented ingredient; it required noticing a useful connection and giving it an interface.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A page that fits is not a task that succeeds
In a 2021 experiment, the question changes from generating an interface to operating one: could a model buy AirPods in Chrome? Shameem recalls roughly 4,000 tokens of context for this demo, while the Walmart shopping page alone occupied about 24,000 tokens. Passing the full page into the model was therefore not workable.
After several frustrating days, he wrote a custom HTML parser that reduced the page to its essential information. That solved the representation problem: the model could receive a useful description of the page within its context window. It did not solve the shopping problem. The agent became distracted by the terms of service and failed to complete the purchase.
The distinction matters. There are two separate questions: can the system expose a usable environment to the model, and can the model reliably pursue its objective within that environment? The parser addressed the first. Shameem interpreted the resulting behavior as a sign of agency latent in a model pretrained on web text, but the failed purchase also exposed how far that behavior was from dependable autonomy. In 2021, even seeing the attempt was surprising.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A basketball video suggests a different product
Farza’s basketball demo moves the exploration to Gemini 2.5 Pro. It takes footage of him playing basketball and uses it for a shot tracker, with feedback requested as though Michael Jordan were watching his gameplay. The important shift is the input and experience: a video becomes the basis for an application organized around a physical activity.
Watching a screen and commenting on code is one use of video understanding, but this demo makes another class of products easier to imagine. A concrete example expands the search space more effectively than saying that a model supports video. Shameem goes further, conjecturing that frozen model weights could support another ten years of compelling product development without additional training. That is a forecast about unexplored applications, not a measured capability horizon.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Excavation, barnacles, and a foggy pond
Shameem invokes a distinction attributed to Richard Hamming: uncertainty belongs naturally to scientific discovery, whereas engineering ordinarily requires knowing how to achieve a specified result. AI engineering mixes these activities. You may have to discover whether a capability exists before you can build a dependable product around it. In Shameem’s metaphor, the work resembles excavation: the demo is a tool, and curiosity directs the search.
That discovery does not belong exclusively to model laboratories. Shameem reports that OpenAI researchers told him they had not known GPT-3 could browse the web or produce functioning React components in the ways his demos showed. His anecdote illustrates why application builders can contribute knowledge: an unfamiliar interface or task can expose behavior that the people training the model have not explored.
Darwin’s barnacle research supplies a longer-timescale analogy. Shameem points to eight years of study and connects that work to evidence supporting evolution. The lesson is about delayed visibility: while an investigation is underway, it can be hard to distinguish consequential work from apparently aimless play. Its contribution may become clear only later.
By the talk’s 2025 vantage point, Shameem contrasts early contexts of roughly 2,000 tokens with windows reaching roughly a million tokens. More capacity expands the opportunities, but it does not reveal a complete route through them. His image is a foggy pond: take one stepping stone, inspect what becomes visible, and continue or backtrack. The next experiment depends on what the previous one reveals.
An unattributed tweet sharpens the danger of choosing too narrow a product category: marketing Claude only as a coding or reasoning model resembles selling an intergalactic spaceship as a toaster because one surface occasionally gets hot. Familiar uses can hide less obvious capabilities. Exploration and play help uncover those possibilities, while each builder’s particular experiences suggest different places to look. Shameem’s recommendation is to act on those connections even before their value is certain.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make the discovery available to others
The prepared closing reaches back to J. C. R. Licklider’s 1960 paper Man-Computer Symbiosis. Shameem contrasts the era of vacuum tubes, punch cards, and long waits to run programs with access to Claude Opus 4 on a personal computer. The slide puts Licklider’s tools beside Claude Opus 4 and Cursor in 2025. The contrast makes the abundance of contemporary tools difficult to ignore.
For Shameem, that abundance creates a moral obligation to the computing pioneers whose ambitions exceeded their tools. Exploring privately is only part of it. Sharing a demo makes a newly visible possibility available to other builders, who can adapt it, extend it, or discover something else through it. That is how an individual’s curiosity can contribute to the field.
Suggest correction
A running experiment and a broken banking app
The extended discussion immediately supplies an example of that transfer. The moderator had adapted the basketball idea to running footage and reported feedback resembling advice from a $600-per-month running coach. The exchange touches on gait and feedback at the level of individual steps, though reproducing the arrow above the athlete’s head remained unfinished. This is a personal usage anecdote, not an evaluation establishing that the model can replace a coach.
Shameem then opens another demo from a few weeks after the GPT-3 API became available. A friend’s enthusiastic claim that OpenAI had created AGI prompted him to try it; the useful discovery was more specific: it could write code. Connecting the API to a browser compiler made that ability into a back-and-forth visual editing experience, one he retrospectively compares with chatting beside code in Cursor.
The application is a simple bank balance interface. Requests add $3 or subtract $5, but the logic contains a revealing bug: a button for giving away all the money also clears a negative balance. The following JSX captures that failure mode. After adding $3 and subtracting $5, the balance is −$2; clicking the final button erases the debt because the handler unconditionally assigns zero.
jsx
import { useState } from "react";
export default function BankDemo() {
const [balance, setBalance] = useState(0);
return (
<main>
<output>Balance: ${balance}</output>
<button onClick={() => setBalance(value => value + 3)}>
Add $3
</button>
<button onClick={() => setBalance(value => value - 5)}>
Subtract $5
</button>
<button onClick={() => setBalance(0)}>
Give away all my money
</button>
</main>
);
}
A visible, interactive result makes the program’s behavior easy to discover—including behavior that violates the meaning of the operation.
Shameem describes the demo as an early form of vibe coding, severe bugs included. It encouraged people to see these systems as reasoning engines that could operate tools, rather than only as generators of prose. His metaphor of intelligent beings connected to different apparatuses is a way of thinking about tool-enabled behavior, not a claim that the banking logic was sound. The practical invitation remains modest: build something fun and simple, then share it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The demos used a base completion model
An audience question clarifies an important historical detail: these early demos used the base GPT-3 model, not an instruction-tuned assistant. Shameem recalls instruction models arriving about a year later. The interaction was prefix completion: give the model text, include a few examples of the desired pattern, and let it continue. The examples established what kind of continuation was useful. This is the mechanism behind the early demos, rather than the conversational instruction-following interface readers may now expect.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Revisit the assistant that knows your work
Asked how curiosity and the subconscious became a moral argument, Shameem describes looking back at why he built the demos. His starting point was frustration: powerful models were available, yet their capabilities remained poorly understood. Thinking about pioneers such as Licklider and Alan Kay made that frustration sharper. They imagined ambitious systems with limited computers; contemporary builders have far stronger tools but often reproduce familiar products.
The alternative is to treat older computing visions as a source of unfinished product ideas. Shameem reads Licklider’s symbiosis vision as pointing toward an assistant that knows what you are working on, maintains that context, and can help immediately. He contrasts this with ChatGPT’s New Chat workflow as he experienced it at talk time: starting over with little continuity beyond a few remembered facts. The product opportunity is sustained working context, not simply another place to type a prompt. Doing justice to those earlier ideas means trying to implement them.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Multivac makes a large problem explorable
The final demo, Multivac, returns to the small context window with a larger ambition: how can a model help reason about a problem that cannot fit into a single request? Its answer is to break the problem into digestible subproblems and expose the structure in a visual interface.
The climate-change example illustrates the interaction:
- Enter an ambitious question, such as how to address climate change.
- Generate candidate directions, such as encouraging vegetarianism, building wind turbines, or installing solar panels.
- Click a sub-idea to generate a more detailed breakdown.
The visible map shows how this can continue: encouraging vegan adoption branches into explaining its benefits and making vegan versions of familiar recipes. Each expansion gives the user a smaller piece of the problem to inspect. These are proposed directions, not executed actions or evidence that the overall problem has been solved.
The ambition is a useful reasoning assistant and, eventually, a thought partner that contributes ideas of its own. Looking back at the rudimentary interface, Shameem invites someone to rebuild it with a newer model, using the name “Opus Max.” The recording ends with an unfinished experiment worth revisiting: keep the inspectable problem structure, bring stronger capabilities to it, and see what becomes possible.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Licklider's 1960 proposal for computers that participate in human thinking, exploration, and decisions.
Further reading
The original GPT-3 paper explains few-shot prompting and the model's 2,048-token context window.
The French account of Poincaré's breakthrough while boarding an omnibus at Coutances.
- Darwin's barnacle researchArticle
Historical context and correspondence sources for Darwin's eight years studying barnacles and their contribution to his evolutionary work.
Read the complete timestamped transcript
- 0:00
[upbeat music] All right.
- 0:15
Hey, everyone. Uh, my name is Sharif. I'll be talking to you about demos and why I think demos are probably the most important thing in the world right now.
- 0:24
Um, I'm the founder of Lexica. We're working on generative models, specifically image models. Um, but I kind of wanna just talk to you about something a bit more than just models themselves.
- 0:35
Um, even more than demos, I kind of just wanna talk to you about curiosity. Um, there was a famous [REDACTED:origin] mathematician, Poincaré. He said, "At the moment when I put my foot on the step, the idea came to me."
- 0:48
He was working on this really, really esoteric field of mathematics called fusion functions, um, and he was stuck on this problem for weeks at a time. He didn't really know how to make any progress at all.
- 0:57
And he was boarding a bus one day, and suddenly it kind of just all came to him. He went from not knowing at all to having a full understanding of the problem.
- 1:05
Uh, he said something along the lines of, "The role of this unconscious work in mathematical invention appears to me as incontestable." Um, I'm gonna make one main argument to you guys today, and that's going to be that curiosity is the main force for how we pull ideas from the future into the present.
- 1:23
Um, and when we have these subconscious patterns that our brains recognize, they kind of surface as a feeling, and this feeling is what we know as curiosity.
- 1:34
So I'm gonna present you with a few demos I've worked on over the years. Um, and they've each followed a specific pattern where initially I had this really great idea.
- 1:43
I thought it was fantastic, that it was gonna change everything. And then you kind of get to implementing the specific idea, and you realize it's actually not possible at all.
- 1:51
And then, like, through sheer effort and, like, determination, you somehow find a way to make things work, even though you're working with models that have maybe a context length of, like, two thousand tokens.
- 2:00
Uh, then once you get it working, you feel this, like, really good sense of pride and joy. Um, and I think the most important thing about good demos is that, um, they're kind of a way of exploring what's possible with these models.
- 2:12
I'm c-- I kind of see these models as not necessarily things you can kind of understand fully without interacting with them. And I think the way you can best interact with them is by making really, really interesting demos.
- 2:23
And, uh, the way you make interesting demos is just by following your curiosity.
- 2:29
So this is from 2020. This was when GPT-3 was released. Um, this was pretty mind-blowing for me, and I was surprised no one was really talking about this. Uh, GPT-3, for those of you who remember, had a context length of two thousand tokens.
- 2:44
Uh, it cost, I think, seventy-five dollars per million output tokens. Um, and yeah, you had to get specific permissions from OpenAI before you shared anything about the model. Uh, you couldn't ship a chat app because that violated their terms of service, but it was a really magical time.
- 3:00
Uh, it felt like you had this new tool in this, like, in this toolkit of computing, and you could do so many things with it. And I think what was really important about this demo is that it inspired people that you could actually do things with software.
- 3:13
Um, I think the way you get really impressive ideas is actually not by doing anything special. I think each person has their own unique kind of context window, the things you've seen and experienced, and I just so happened to watch a talk by Brett Victor before making this where, um, he came up with this principle that creators
- 3:31
need immediate feedback with what they're creating. And I was really tired of copy and pasting code into my editor and then compiling it and then, like, running it and see what would happen.
- 3:40
So I decided to just put, like, a JSX compiler in the browser, and it just felt different. It felt kind of magical in a way. Um, and now today, like, Claude system prompt is twenty-five thousand tokens, which is kind of funny.
- 3:54
Here's another similar demo. This one's a bit more interactive, so you can kind of describe what you want and then... Uh, because the context windows were so small, it, it couldn't actually generate the entire application in a single prompt.
- 4:06
Uh, you actually had to do three parallel prompts and then join them in the background. Uh, this is really simple, just asking for a Google homepage, and then it generates three different components for it.
- 4:15
Um, but yeah, this is just, I think, one of the ways you can express your curiosity. Uh, you kind of look at the world around you and what you've experienced, and you kind of synthesize new ideas, and you get this subconscious feeling pulling you in a direction.
- 4:29
And I think the demo is the best way to kind of express that feeling to the world.
- 4:34
Um, here's another more interesting one. This was 2021. I think the context length's now expanded from two thousand tokens to about four thousand tokens, so we could do a bit more with this.
- 4:43
Um, I was kind of curious if these models had any sense of agency, so I decided to give it a really simple objective of buying me AirPods in Chrome.
- 4:52
And, uh, if you were to just dump a web page into the browser, into the context window, it just wouldn't work. Like, the walmart.com shopping page would be, like, twenty-four thousand tokens.
- 5:01
It was just impossible. Uh, so I was actually a bit frustrated that I couldn't get it to work for a few days. Uh, so I wrote a custom HTML parser that would parse a web page into its core essence, which was able to fit it into the tiny context window of GPT-3 in 2021.
- 5:15
Uh, it definitely failed spectacularly. It got distracted with the terms of service. Uh, but I think it was more so just interesting that we discovered that these models pre-trained on web text had this sense of agency kind of internal in their weights.
- 5:29
Uh, we kind of take that for granted now, but 2021 was a very different time.
- 5:35
Here's a more recent demo from a friend of mine, Farza. Uh, he's using Gemini 2.5 Pro here today, so we're still discovering new capabilities. Here, he's making a basketball shot tracker where he's just putting in a video of him playing basketball, asking it to provide feedback as if Michael Jordan was watching his gameplay.
- 5:51
And I think this is a really great demo because it inspires people to realize that you can actually make video-first experiences with Gemini 2.5 Pro. Uh, before this, it was kind of like, oh, you can have it watch your screen, and it'll give you feedback on your code.
- 6:04
But there's so much more we can do, and I think one of the main reasons I find demos really, really interesting is that they inspire possibility. Um, so much so that There's probably so much low-hanging fruit today in these models that if you were to halt all capabilities, if you kept all the weights frozen, didn't do a
- 6:20
single backpropagation, I think you could build really amazing products for the next 10 years keeping everything constant. Um, and I think the way you do that is just by building these demos and following your curiosity.
- 6:33
So I have this really famous quote by Richard Hamming where he says, "In science, if you know what you're doing, you should not be doing it. Engineering, if you know what you're doing, you should not be doing it."
- 6:42
So traditional engineering is very teleological. It's very goal-oriented, very purpose-driven. Um, but I think AI engineering is a bit different. I think AI engineering is actually a bit more ki- It's, it's a bit closer to excavating.
- 6:56
You're looking for new capabilities hidden within these models, and your toolkit is a demo. Uh, it-- Your curiosity is kind of your flashlight guiding you to where the interesting bits of the models are.
- 7:06
And the way you kind of discover what's possible is just by making things. Um, and what's really, really interesting is that even the researchers today at labs like OpenAI and Anthropic actually don't have a full understanding of the capabilities of these models.
- 7:19
Um, I've had OpenAI researchers show me or tell me that they didn't even know GPT-3 could do this, uh, could browse the web or that it could generate fully functioning React components, which was pretty interesting.
- 7:32
Um, this is pretty funny. Uh, Charles Darwin was famous for coming, coming up with the theory of evolution, but little-known fact, he actually spent eight years studying barnacles, like the things on the sides of ships and piers and docks.
- 7:46
Uh, he spent, he spent eight years studying barnacles, so much so that people thought he was going crazy, uh, before he published Evolution. Um, in the moment, you wouldn't have known that it was important though, but the barnacle studying taught him that evolution was correct and it was kind of indisputable evidence for his theory.
- 8:03
Um, in the moment, you actually don't know what is actually work versus play. Uh, what you're doing might feel like it's useless. It might feel like it's leading nowhere.
- 8:12
Um, but sometimes you need to study barnacles for eight years before you can publish Evolution.
- 8:19
So I think we're in this really strange moment right now in 2025. Uh, these models can do amazing things. There's tons of them. Their context windows have now expanded from two thousand to maybe a million tokens or so.
- 8:30
Um, and I think demos are the way we explore what's possible. It's the way we s- w-we expand the search space and kind of see what we can do with these capabilities.
- 8:40
Um, and I think it's not something you can predict ahead of time. It's kind of like crossing a foggy pond. Uh, you kind of take one stone. You kind of step on a stone and then see where it leads.
- 8:50
If it leads somewhere interesting, you can keep going, but if it doesn't, you can always backtrack and go a different way. You'll never be able to plan your route across the pond, uh, ahead of time.
- 8:58
You just kind of have to take the first step.
- 9:02
I came across this really interesting tweet, um, and I really like it. It's because Anthropic really markets Claude as kind of a coding model or like a general reasoning model.
- 9:12
But it's like trying to sell an intergalactic spaceship as a toaster because one of its surfaces gets hot every once in a while. And I think this is a really, really good way of thinking of these models.
- 9:20
There are so much capabilities latent in them that, uh, we kind of only focus on the immediate and the obvious. But good demos reveal really interesting capabilities, uh, mainly through exploration and play.
- 9:32
And I think uncertainty is at the core of being an AI engineer. If you know what you're doing, you're kind of doing it wrong. And I think if you're uncertain and you're kind of just exploring, uh, you'll, you'll lead down interesting-- you'll find yourself being led down interesting paths.
- 9:46
Um, yeah, in subconsciously, you notice these patterns because you've worked with things that no one else has worked with before. Uh, your life is unique to you, your context window is unique to you, and no one else has that same shared context window.
- 9:59
So when you come across an idea in your head, oftentimes you're one of the only people to ever have that idea, and I think you'd be doing yourself a great injustice if you never actually tried to make that idea a reality.
- 10:14
So I'm gonna close with this slide. Um, one of the greatest computing papers ever written was "[REDACTED:gender]-Machine Symbiosis" by [REDACTED:username] in the 1960s. And the epitome of technology at the time were vacuum tube computers and punch cards.
- 10:29
Uh, if you wanted to write a program, it would probably take a few hours, maybe even days to run. Um, meanwhile, today we have Claude Opus 4 on our computers.
- 10:37
It's actually kind of insane, and I think [REDACTED:username] genuinely would have killed someone to have an hour with the tools we have today. Um, and like, I'm not even joking.
- 10:46
I think it's kind of important, so much so that I feel like today we have a moral obligation to do him justice and everyone else in the field that came before us.
- 10:56
Uh, not only to just follow your curiosity, but to share what you explore with the world. Um, because by sharing your demos, you kind of share what's possible with these models.
- 11:04
And I think that's how we move the field forward. And, um, yeah, that's, that's really it. Your unique perspective shouldn't be wasted, and I think you have a moral responsibility to share them with the world.
- 11:15
Thank you. [audience applauding]
- 11:19
All right. We've got some time for questions. Does anybody have any?
- 11:29
None?
- 11:30
Cool. Thank you, guys.
- 11:31
[REDACTED:gender].
- 11:31
Appreciate it.
- 11:32
We're doing really good for time. I was gonna ask you if you had like other demos that you wanted to show us because the... I, I liked seeing the 2020 versions of things.
- 11:40
I have a few more actually. Do you want me to pull them up?
- 11:41
We've got eight minutes.
- 11:42
All right. Let's do it.
- 11:43
You might as well. [chuckles] As long as they don't use Wi-Fi because apparently that's the running joke of this conference. [laughs]
- 11:50
I'm-- I think I might have a few downloaded. Let me check.
- 11:56
For the people in the room, I did try that basketball one, except I tried to apply it to running and it works really well. And it pretty much gave the same feedback that my $600 a month running coach would give me. [chuckles]
- 12:07
Oh, wow.
- 12:07
And I thought, "Wait, I think I can cancel this."
- 12:09
Did it give you like pretty good advice on your gait?
- 12:11
It, it gave me not, not just the gait per step And so that's something that my coach would never able, be able to do. What I couldn't get to figure out is, um, how it had the little arrow on top of the head, but if I had 20 more minutes, I probably could-
- 12:24
Yeah. Here's a pretty cool demo I think was also from 2020. Um,
- 12:28
let's see if it's playing. No, it's not. Let me... Oh, it is. Um, yeah, this was about a few weeks after the GPT-3 API came out, and I think, uh, the way I came across the API was really funny.
- 12:39
Uh, someone had said to me, "You have to try this out. OpenAI has created AGI and it's here available today, and no one's really talking about it." And I was like, "Okay, let's see what this is about."
- 12:48
Um, and I, I quickly realized it could actually write code, but writing code in the text interface was not really the best way to do it. Uh, so you actually hook it up to an API, put a compiler in the browser, and you get this, like, nice back-and-forth visual interface.
- 13:02
We kind of take this for granted for today with tools like Cursor, where you can kind of like chat with your code in the sidebar. But, uh, in 2020, this felt really, really different than what anything was possible.
- 13:10
Um, here I'm, like, working on a, like a really, really basic, like, banking app where you just ask it to add $3 or subtract another $5. Uh, and it was pretty funny because, like, the bugs were really bad.
- 13:21
You could actually-- There was a button where you could give away all your money, and if you were in debt, it would just, like, negate it and make you, make your balance go to zero again.
- 13:29
Um, but this really was, I think, the start of vibe coding, and it really a lot of people to take these models not only as, like, language models, but kind of reasoning engines.
- 13:38
Um, yeah, and I think, I think, um, the way to think about these models is really that. Like, they're these really, really intelligent, in a way, beings, which sounds kind of weird to say out loud, but that's like the mental model I have for them.
- 13:50
And you kind of hook them up to these different apparatuses, and they can kind of work them, and you kind of like instill these tools with a sense of, like, uh, purpose and agency.
- 13:59
Um, yeah, I, I really just hope a lot more people are inspired to work on demos because the capabilities we have today are really impressive, and you'd be really doing a disservice by not just, like, building something really fun and simple and sharing it with the world.
- 14:11
Yeah.
- 14:12
Sorry, we have a question over there.
- 14:13
Yeah. When you discovered GPT-3 in 2020, was it the base model or- Yes, it was the base model. Uh, we didn't have an in- we didn't have an instruct model until about a year later.
- 14:23
Um, so what w- it was essentially similar to base models we have today, where you give it a prefix and it just completes it. So you, you prompt engineer it with a few examples, and that's usually good enough.
- 14:35
Any other demos you got there? [laughs]
- 14:37
Uh, I can, I can go into the archive, but-
- 14:39
Do you want to go through your entire desktop while you're at it? [laughs]
- 14:43
Let me close this one. Uh, I, I have a few, but I, I don't know... Do we have time? We have five minutes.
- 14:51
You got five minutes.
- 14:51
Let's see if I can find anything. Um,
- 14:55
okay. Let, let me try something really quick.
- 15:11
I have a high-
- 15:13
Yeah, sure
- 15:15
... 'cause, um, this might be the first time I've come to a conference where the subconscious is mentioned in a talk, which I found just glorious. [laughs] Um, how did the concept or what was your thought process that led up to a presentation like this?
- 15:32
'Cause personally, I've never thought, uh, about doing demos as a moral obligation. I'm like, "Now do it. Okay." But maybe you could tell us, like, how this came to be.
- 15:44
Sure, yeah. I think, um, a lot of it was just kind of introspecting on why I made these demos in the first place. Um, a large part of it came from a sense of frustration that we have these really powerful models today and no one really knows what they're capable of doing.
- 15:57
Um, and I think I examined it a bit further, and it did feel like a sense of moral obligation. Uh, you have these pioneers of computing, uh, from the '60s and '70s and '80s, like [REDACTED:username] and Alan Kay and whatnot, and they came up with these grand ideas, uh, with the computers they had available to them.
- 16:16
They just couldn't make it possible. And I look at what we have today, and it's kind of like we're spoiled by so many amazing pieces of technology, and we're kind of just making the same things all over again.
- 16:27
Uh, but really, I think if you look back at, like, what people were writing about in the '60s and '70s, there is a whole goldmine of ideas there that we can revisit and actually make possible today.
- 16:37
Um, in the, uh, [REDACTED:gender]-Computer... [REDACTED:gender]-Machine Symbiosis paper, [REDACTED:username] talks about an assistant that knows everything you're working on and has, like, perfect context and can help you with anything immediately.
- 16:47
Um, and here we have, like, ChatGPT, where every time you want to talk to it, you press New Chat, and it has no memory of what you've talked about beforehand, minus like a few basic facts.
- 16:55
Um, and I think it's really just, it really boils down to wanting to, uh, kind of do the ideas that these pioneers came up with justice beforehand.
- 17:09
Well, you're inspiring.
- 17:10
Thank you.
- 17:12
You have three minutes, so did you have something loaded up?
- 17:14
I'm, I'm looking. [laughs] Um, um, yeah, this was an old GPT-3 demo where the idea was: How do you get these models to solve very large and ambitious problems? It was called Multivac, and the idea was you can't really fit everything into a 2,000 context window, uh, 2,000 token context window.
- 17:31
So what you do is you es- you essentially break down the problems into more digestible sub-problems, and you have this kind of visual interface to help you see where things are going.
- 17:40
So you can give it some really ambitious problem, like how do you solve climate change? And it might come up with things like convince more people to go vegetarian or build climate, or, uh, build wind turbines and, like, install more solar panels.
- 17:50
And then you can click on each of the sub-ideas, and it kind of breaks it down even further. Um, yeah, I, I think one of the core ideas behind this was like these models are a lot more than just text completion models, but I think they can be useful as, like, very helpful reasoning assistants, uh, specifically at
- 18:05
solving big problems. Uh, so much so that they could come up with ideas on their own one day and hopefully be really useful thought partners.
- 18:14
Yeah, I mean, looking at it now, it's pretty rudimentary, but I... Maybe someone should make a new version of this with, like, Opus Max. [laughs] That'd be sick. Yeah. I mean, someone here should do it.
- 18:25
I think that'd be pretty cool. [laughs] Um, yeah, that's about it, guys. Thanks. [upbeat music]