AI Engineer World's Fair 2026
The New Primitives: Building AI-Native Software
Read the talk
The New Primitives: Building AI-Native Software
From Bush’s Memex to a multiplayer game built around LLMs, the path beyond agents depends on new ways to combine computation, context, interfaces, and human intent.
From a talk by Kwindla Kramer and Kwindla Hultman Kramer
How do you imagine software before its infrastructure exists?
What should we build when a new computing capability arrives, but its useful forms are still unclear? Voice agents make that question immediate. Kwindla Hultman Kramer works at Daily on infrastructure for real-time audio, video, and AI, and on Pipecat, its open-source, vendor-neutral agent framework. Kramer describes Pipecat as the most widely used voice-agent framework, naming AWS, NVIDIA, Anthropic, and thousands of other organizations as users. That experience supplies the starting point: building agents is already practical; deciding what comes after them is harder.
An earlier version of the problem appears in Vannevar Bush’s 1945 essay As We May Think. Bush brought together experience with analog computers, photography, radio, and radar to imagine capabilities that the emerging computing industry could eventually deliver. His example matters because he was designing toward a future whose infrastructure barely existed.
Read through modern categories, Bush’s essay anticipates a remarkable collection of technologies. Kramer identifies documents displayed on screens, document scanning and OCR, speech-to-text and text-to-speech, programming languages, hypertext, search engines, and data networks. He also sees precursors to a GoPro-like camera, a Kindle Store-like service, voice interfaces, and brain-computer interfaces. These are retrospective analogies to Bush’s proposals, rather than the vocabulary of the original essay. The useful exercise is to connect technical possibilities into activities people might actually want to perform.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From agent harnesses to something beyond agents
Today’s immediate engineering task is to assemble a coherent software stack for agents. In the No Priors × Latent Space conversation with Satya Nadella, the organizing abstraction is the harness: it defines the models, data, and tools, then maintains a loop across all three. Nadella points to GitHub Copilot, the security system MDASH, and Discovery for Science as examples of products taking this direction. Their harnesses support multiple models, supply rich context, and progressively disclose tools so that every tool definition does not have to consume tokens at every step.
Loops, tool calls, and context engineering lead toward what Kramer calls Agents++: multi-model harnesses, copilots embedded throughout software, and harnesses operating at the organization level. The unit of design grows from an individual agent to the surrounding system that determines what it knows, what it can do, and how its work continues.
The early web offers a useful comparison. In 1995, Kramer wrote HTML by hand, built web servers and indexing and search software in C, and developed authoring and management infrastructure in Perl. Web pages were the object around which this work revolved, much as agents organize AI engineering today. His enthusiasm for HTML then was comparable to his enthusiasm for agents now.
Web pages remain useful, but the applications built on that foundation became much more expansive: web applications and native mobile applications coordinate services, data, and interaction in ways an early page did not. An agent can be a durable primitive without being the final form of AI-native software. The proposed progression is agents → Agents++ → applications whose behavior is designed around AI from the beginning.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turning calculation into a dialogue
To understand how a primitive becomes a platform, return to Bush’s longer timeline. His account treats the abacus as both an everyday calculating device and a conceptual tool associated with place value and zero. From there, he moves to the electromechanical keyboard calculators available in 1945, anticipates an arithmetical machine, and then imagines the Memex. The sequence moves beyond faster calculation toward a different relationship between a person and stored knowledge.
With hindsight, Kramer redraws that timeline around the stored-program computer, the personal computer roughly forty years later, and the agent era roughly another forty years after that. The intervening decades were spent building ways to express intent and interact with machines:
| Period | Engineering problem | New foundation |
|---|---|---|
| 1950s | Transmit human intent effectively | Programming languages and compilers |
| 1960s | Support two-way interaction | Interactive computing and graphical programming |
Languages combined mathematical formalisms with expressions closer to natural language. Interactivity then made computing a dialogue rather than only a sequence of submitted instructions. Ivan Sutherland’s Sketchpad extended that dialogue into graphical programming.
The popular imagination developed alongside those technical foundations, even when few people had access to a computer. In the Star Trek excerpt, a captain dictates a log, the computer records it, and then repeatedly addresses him with an unwanted term of endearment despite his correction. It is a compact vision of conversational computing with personality—and of a user struggling to control that personality. Kramer also draws attention to the punch-card-reader sounds in the background: the machine makes its work audible even while speaking about it.
HAL 9000 in Stanley Kubrick’s adaptation of Arthur C. Clarke’s 2001: A Space Odyssey supplies a much darker talking-computer vision. These fictional systems made interaction imaginable before it was broadly available. Meanwhile, growing computing power and data volumes created a practical problem for the next decade: software needed abstractions that could scale.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The spreadsheet made a capability broadly accessible
The 1970s supplied foundations for manipulating larger amounts of data and organizing more complex software. Relational databases introduced a theoretical basis for data manipulation; declarative languages gave programmers ways to use it. Smalltalk and object-oriented programming supplied another set of abstractions. These developments helped set the stage for personal computing, including the Macintosh in 1984 and Windows 1.0 in 1985.
Microsoft’s ambition was “A computer on every desk and in every home.” The reason to put one there was the tangible work it enabled. Kramer’s central example is VisiCalc, which he introduces as the first spreadsheet program: a two-dimensional, interactive abstraction for numerical computation. Its descendants, including Excel and Google Sheets, preserve that basic way of working because it remains useful.
The comparison between a spreadsheet in 1957 and one in 1985 is therefore about access to a capability. Work that once required a roomful of people with specialized knowledge could become something a person manipulated on a screen. The interface compressed a complex computational practice into a form that many more people could use.
Kramer offers VisiCalc as a possible counterargument to fears of mass unemployment from AI. In his account, spreadsheets did not eliminate accountants; they expanded the amount of accounting-like work people could perform and enabled categories of work that had been difficult even to imagine. This is an analogy for how access can expand demand, rather than a demonstrated forecast of AI’s employment effects.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Knowledge Navigator put the pieces together
Imagine looking ahead from Macintosh System 2 and Windows 1.0 in 1985. What would computing look like ten, twenty, or forty years later? Apple’s 1987 Knowledge Navigator concept video provides one answer. In the excerpt, an assistant reads messages from a research team in Guatemala, a student asking for another term-paper extension, and the user’s mother. The user interrupts the last message by supplying the birthday reminder himself. Personal information and conversational turn-taking are already part of the same interface.
The larger concept integrates several kinds of capability:
- Device and interaction: a foldable tablet, touch input, and a conversational voice assistant with a strong personality.
- Information and perception: access to global and personal information, real-time computer vision, and real-time video generation.
- Communication and agency: seamless video calls, delegation of complex tasks for autonomous execution, and what Kramer describes as continual learning.
The significance is their coexistence. The assistant does not inhabit an isolated voice interface; it participates in a workspace containing information, visual media, communication, and ongoing tasks.
Knowledge Navigator extends Bush’s vision with another forty years of technical experience. It anticipates the agent era more directly than the Memex could, because it can draw on personal computers, graphical interaction, and a much richer understanding of software. The vision changes as the available primitives change.
Suggest correction
A multimodal computer, connected and in your pocket
Knowledge Navigator falls roughly halfway through Kramer’s computing timeline. The next major layer was networking: local area networks and dial-up connections, followed by the internet and the web. Looking back, Kramer identifies the web’s multimodal character as its most important property. Text, audio, video, and data could belong together instead of being treated as separate things requiring separate programs. For many web builders, making something like Knowledge Navigator real was a conscious ambition.
The following decade made that networked, multimodal computer mobile and continually connected. Putting it in a pocket changed where interaction could happen. Portable computers, cameras everywhere, and increasingly inexpensive screens also gave designers and filmmakers a new set of near-feasible possibilities. The hardware was approaching capabilities that could be imagined coherently before they could quite be assembled into a working product.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Moving the interface into the surrounding world
John Underkoffler’s work on Minority Report and Iron Man supplied two influential cinematic interfaces from this period. In the Minority Report excerpt, gestural investigative work occurs alongside spoken requests to check neighbors and relations. Information handling is embedded in a larger activity involving people, instructions, and a shared environment.
The gestures shown in Minority Report were rendered with special effects, but Kramer explains that they were grounded in Underkoffler’s PhD research at the MIT Media Lab. The underlying move was to bring the interface out of a small screen and into the world around its users. Iron Man adopts a more playful future than Minority Report’s dystopia, while retaining recognizable elements of the same spatial-interface era.
The Iron Man excerpt adds the assistant’s social presence: a personalized welcome, references to recent public events, and sarcasm aimed at its user. As with Star Trek, personality is part of the imagined interface, not merely a voice wrapped around a command parser.
Kramer and Underkoffler then tried to move that vision into commercial software, co-founding a startup in 2006. The 2012 demo reel shown in the talk represents six years of that effort. It places the cinematic vision beside the much longer engineering process of turning an interaction concept into a product.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The infrastructure catches up with the vision
The continuing project is a computer that is multimodal, spans devices and screens, supports multiple people, and stays connected. The cloud buildout of the 2010s added another essential layer: the infrastructure, data centers, and data capacity needed to scale AI training and inference. That substrate brings the historical sequence back to agents and to the possibility of software beyond them.
Kramer condenses the progression to calculator → computer → personal computer → global cloud computer. His assessment is that this foundation now makes fully working versions of the Memex, JARVIS, and Knowledge Navigator possible. The broader opportunity is to build those systems and learn, through using them, which new forms of software become desirable.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Gradient Bang makes orchestration part of the game
Gradient Bang, a multiplayer game Kramer is developing with friends, provides a different testbed. LLMs sit at the core of every interaction rather than appearing as an optional assistant beside an otherwise conventional product. Kramer reports hundreds of inference calls happening at every moment in the game. He also says a comparable system could not have been built a year earlier. The inference count is his description of the system’s activity; no measurement window, player count, or model configuration accompanies it.
The demonstration begins with asynchronous, non-blocking context compression. A player asks the assistant to remember an intended future elimination of another in-game character, and the assistant acknowledges the note. The requested action is deferred: recording an intention is distinct from carrying it out. The pattern lets ongoing interaction coexist with work on the context needed for later turns.
Next come long-running subagents that share context. The assistant reports Eagle working through five trade loops, while Hawk and Raptor each work through five exploration loops. The player can ask about the fleet while those activities remain in progress; conversation is an interface to ongoing work rather than a requirement that every task finish within one exchange.
The remaining examples connect orchestration to the interface:
- Progressive skills loading: a question about average ship cost introduces the pattern. The assistant begins explaining that ship prices vary; the excerpt supplies no numeric price.
- Dynamic UI generation: the player requests task history, then asks to hide the map. The interaction can change what the interface presents as well as produce spoken responses.
- Conversational voice: the player corrects the intended transaction from exchanging a ship to selling it for cash. The assistant must follow the revised intent while still respecting the game’s rules.
The last request reaches a constraint: the assistant refuses a sale that would leave the player without a ship.
That final interaction separates understanding an action from permitting it. A small Python rule illustrates the boundary for the requested sale:
python
def validate_ship_sale(owned_ship_count: int) -> str | None:
if owned_ship_count <= 1:
return "Sale refused: you must retain a ship."
return None
requested_action = "sell ship for cash"
refusal = validate_ship_sale(owned_ship_count=1)
if refusal is not None:
print(refusal)
else:
print(f"Permitted for further processing: {requested_action}")
Here, passing validation would only permit further processing; it would not execute a sale. In the demonstration, the corrected request is understood but refused because the galaxy does not allow the player to become shipless and hitchhike. The conversational interface can be flexible while the world it operates on retains firm constraints.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Vannevar Bush's July 1945 essay describing the Memex and associative access to personal knowledge.
Open-source Python framework for real-time voice and multimodal agents, with setup instructions and examples.
Microsoft's companion page for the Build 2026 crossover interview about agent platforms and long-running assistants.
The Dom demonstration and its creator's explanation of perception, computer use, generated interfaces and memory.
Multiplayer universe for exploring, trading and collaborating with LLM agents, including local development instructions.
Read the complete timestamped transcript
- 0:00
[upbeat music] Good morning. I know a lot of you in this room.
- 0:16
It's great to see you. Welcome to the Voice Track at AI Engineer World's Fair. Uh, for those of you who don't know me, my name is Kwindla Hultman Kramer.
- 0:24
Uh, I work at a company called Daily. We make developer infrastructure for real-time audio, video, and AI, and we're the team behind Pipecat, which is the most widely used framework for building voice agents today.
- 0:37
Pipecat is open source and vendor neutral. It's used by companies like AWS and NVIDIA and Anthropic and thousands of startups and scale-ups and enterprises. And today, I'm gonna talk about what kind of agents we're building today, including voice agents, but not just voice agents, and what I'm interested in building next.
- 0:58
And I'm gonna try to put all this in the context of the roughly eighty-year history of digital computing so far. So we've got a lot to cover. We're gonna go fast.
- 1:09
But we're gonna start in 1945 with an essay called As We May Think, written by an engineer and academic, a civil servant named Vannevar Bush. Bush deeply understood technologies ranging from analog computers to photography to radio to radar, and As We May Think is a extraordinary piece of writing.
- 1:31
The essay predicts the development of, among other things, document display on a screen and document scanning and OCR and speech-to-text and text-to-speech and programming languages and hypertext and search engines and data networks.
- 1:45
Something like the GoPro camera, something weirdly like the Amazon Kindle Store, and voice interfaces and brain computer interfaces. And I've been thinking a lot about As We May Think lately because Bush wrote this essay right at the very beginning of the computing age.
- 2:03
And I think it feels to most of us like we're working right at the beginning of a new age, the intelligence age. So what will we build? Well, at the moment, we're building agents, and we're having a lot of fun doing it, and a lot of the AI engineering work we're all talking about this week is focused
- 2:22
on building a full coherent software stack for AI agents. Here is Satya Nadella talking a couple weeks ago on a crossover episode of the No Priors and Latent Space Pod about the challenges of building agents in 2026.
- 2:38
That's right. So, so in some sense, you kinda want the harness to define the models, the, the data, uh, and the tools, and so that you have a loop across those three.
- 2:51
And so what we are trying to, first of all, make sure is each of our products that we build, right? Whether it's GitHub Copilot or the Security Copi-- the, the stuff we showed with MDASH, or even the Discovery for Science, it doesn't matter.
- 3:04
All of them are multi-model harnesses, um, with tools access so that you can do this progressive, uh, disclosure of tools even so that they're token efficient. Uh, and then you're feeding it with very rich context.
- 3:20
So if you were here last year at AI Engineer World's Fair, you could draw a through line from the things we were talking about last year to loops and tool calls and context engineering and the stuff we're focused on this year to some emerging ideas.
- 3:33
Uh, you can hear that in Nadella's clip just there. I think of this as kind of Agents++ like multi-model harnesses and software copilots embedded in every single piece of software, organization-level harnesses.
- 3:47
So how do we go from agents to Agents++ to the next thing beyond agents? Well, the last time we had this kinda massive change in how we write software and what we write software for and to do was the early days of the World Wide Web.
- 4:04
And I was around for the early days of the World Wide Web. I was a baby programmer in 1995, and the thing we talked about all the time in 1995, the way we talk about agents today, is web pages.
- 4:15
I spent a lot of time writing HTML by hand and building web server software in C and indexing and search software in C and authoring tooling and management infrastructure for web pages in Perl.
- 4:27
I was as excited about HTML in 1995 as I am about agents today.
- 4:34
And the web page is still with us, and it's still important and useful. But today, we talk a lot more about web applications and native mobile applications than we talk about web pages.
- 4:44
So just like we went from web pages to full-blown web and native mobile, clearly we're gonna chart a path to a new fully AI native software that comes after agents and Agents++.
- 4:57
So let's keep going back in time to-- in order to think about this future. Uh, here's a timeline Vannevar Bush lays out in As We May Think. He talks about the abacus, which was both an immensely useful device for doing practical everyday mathematical calculations and also an incredibly important theoretical tool that led to ideas like numeric place
- 5:20
value and the concept of zero. And Bush talks about the massive jump from the abacus to the state-of-the-art electromechanical keyboard calculating machines that he had in 1945. And then he posits that we're about, or he is about to, uh,
- 5:40
witness and help create, uh, a, an, a, an equally large leap to what he calls the arithmetical machine. And then he goes a step even further than that, and he invents or designs in that essay a device he calls the Memex.
- 5:56
And we have a little bit of an advantage over, over Bush in 1945. We've seen eighty years of computing play out. So we can modify his timeline a little bit.
- 6:04
We can go from the abacus to the stored program computer To 40 years later, the personal computer, and 40 years after that, this AI agents era that we're all collectively helping to invent and create and bring into being.
- 6:19
So the question for me is, what did we build to go from those very first digital computers in the 1940s to the personal computer in the 1980s? Well, in the 1950s, the big job was to figure out more effective ways of transmitting human intent to these new computing machines.
- 6:36
We built the first programming languages. We wrote the first compilers. And the, the theoretical underpinnings here were figuring out how to combine the elegance of mathematical formalisms with something a little bit more like natural language.
- 6:51
And then building on that, in the 1960s, the challenge was to make these machines interactive, make these machines capable of a two-way dialogue with humans.
- 7:02
The '60s also saw the birth of graphical programming with systems like Ivan Sutherland's Sketchpad.
- 7:09
And the '60s were an amazing era for science fiction. Even though almost nobody had access to a computer, the computer became a big part of the popular imagination. The idea of a computer really resonated with people, and ideas matter.
- 7:24
For example, here is the idea of the computer in Star Trek.
- 7:29
Computer on. Record. Recording. Come. Captain's log supplemental.
- 7:39
Engineering Officer Scott informs warp engine's damaged, but can be made operational when re-energized. Computed and recorded, dear.
- 7:49
Computer, you will not address me in that manner. Compute.
- 7:53
Computed, dear. [laughs]
- 8:00
I love the background sound of punch cards going through a punch card reader, so, like, you know the computer is working even though it's talking to you about what it's actually computing.
- 8:11
There were, of course, a bunch of other talking computers in, in science fiction of the '60s, and the next year after this, the Kubrick movie, it was a interpretation of Arthur C.
- 8:19
Clarke's 2001: A Space Odyssey, had the HAL 9000 computer. This is a much, much more dystopian view of a talking computer than the Star Trek computers. And by the 1970s, computers had become powerful enough that the next big job was designing abstractions that could scale to much larger amounts of data and much more powerful computing substrates.
- 8:40
We got relational databases, which introduced new theoretical underpinnings for data manipulation. And we got declarative languages, which leveraged those new theoretical insights. And programming languages in general continued to evolve in what, to me at least, are really amazing ways.
- 8:56
We got Smalltalk and object-oriented programming in the '70s. And all of this set the stage for the personal computer in the 1980s. The Macintosh shipped in 1984. Windows 1.0 shipped in 1985.
- 9:08
And Microsoft's mission statement was, "A computer on every desk and in every home." And incredibly, Microsoft delivered on that mission statement. And we got a computer on every desk and in every home because these new personal computers delivered real, amazing, tangible benefits.
- 9:25
Take VisiCalc, for example, which was the first spreadsheet program, a truly new abstraction for doing computation, numerical computing, two-dimensional, interactive. So durable and so useful that probably most of us in this room use a direct descendant of VisiCalc regularly, Google, Google Sheets or Microsoft Excel or whatever.
- 9:49
Or put another way, this was a spreadsheet in 1957,
- 9:54
and this was a spreadsheet in 1985. And I think a lot about VisiCalc these days too because I think VisiCalc is an example of how transformative new technologies can be in the way of delivering a capability that used to require a lot of specialized people and specialized knowledge and making it generally accessible.
- 10:11
And I think VisiCalc is, uh, potentially a counterargument to the argument or the fear or the concern that AI is gonna lead to mass unemployment because VisiCalc didn't put accountants out of business.
- 10:23
Instead, it made much, much, much, much more accounting-like work possible, and it made new categories of work possible that we couldn't even really conceive of when a spreadsheet or doing a screen's worth of calculations as we think about it today took a room full of people.
- 10:42
So if we were here in the Moscone Center in 1985, and these two interfaces, the Macintosh System 2 and Windows 1.0, were state-of-the-art, what would we have said the world would look like in ten or 20 or 30 or 40 years?
- 10:57
Well, we actually have a really great example of a prediction from that time, like As We May Think, another famous document in the history of human-computer interaction, concept video from Apple made in 1987 called Knowledge Navigator.
- 11:10
This is very much worth tracking down online and watching all of if you haven't seen it. I'm just gonna play about 20 seconds from the middle. [instrumental music]
- 11:22
You have three messages: Your graduate research team in Guatemala just checking in, Robert Jordan, a second semester junior, requesting a second extension on his term paper,
- 11:34
and your mother reminding you about your fa-
- 11:36
Surprise birthday party next Sunday.
- 11:40
So the video shows a foldable tablet, a touchscreen interface, a conversational voice assistant with a really strong personality, access to both global and personal information, real-time video generation, real-time computer vision, seamless video call integration, delegation of complex tasks for autonomous execution, and what we might call today continual learning.
- 12:02
And it's really, really clearly influenced by As We May Think, but it's also quite different. It really is updated for 40 years of progress, and it really does sort of presage this AI agents era we're in now in a way that Vannevar Bush's Memex didn't and maybe couldn't.
- 12:17
The Knowledge Navigator video divides our timeline, I think, quite neatly in half. And hold that thought 'cause we're gonna come back to it. The 1990s were about the network.
- 12:26
First, local area networks and dial-up, and then the internet and the web. And with the benefit of hindsight, I now think that the single most important thing about the web was that it was multimodal from the very beginning.
- 12:42
More even than the GUIs of the 1980s, the web anticipated the text and audio and video and data were not different things to be used in different programs. They belonged together.
- 12:53
And in a real sense, the web was an attempt, and a conscious attempt on the part of a lot of people building the web, to make that Knowledge Navigator video real.
- 13:03
Then in the first decade of the new millennium, the big job was to make all of this computing stuff mobile and continually connected, to put this new multimodal networked computer in your pocket, literally to give a supercomputer to everybody in the world that they could carry around in their hand.
- 13:22
And as with the 1960s, there was an efflorescence of, like, futurism on screen in the first few years of the new millennium, and I think it was because computers you could carry around with you and cameras everywhere and a kinda Moore's law for pixels making screens super cheap really gave us a chance to think through what we
- 13:42
thought the future would look like in a new way. A lotta stuff we could almost but not quite build was cohering in the minds of people working on these machines.
- 13:52
And the best and most famous Hollywood computers from that era were created by John Underkoffler for the films Minority Report and Iron Man. Here's Minority Report from 2002.
- 14:05
It's no longer there. Shit. Timeframe? 13 minutes. Hey, Chief. Investigator from the fed's here. Yeah, I don't need some [REDACTED:gender] from the fed poking around right now. John, I wrote it down on your calendar.
- 14:15
I left you a message at your house. Check in with the papers they had at Ford and see if the neighbors knew where they went. Check all relations.
- 14:19
Checking neighbors and relations.
- 14:20
But John- Fletch, just get him some coffee. Tell him some stories how I save your ass every day and you can't live without me. I got coffee, thank you.
- 14:27
Danny Whitman, [REDACTED:gender] from the fed. Whoops, got one.
- 14:32
So the gestural interface in Minority Report was implemented on screen as special effects, but it was actually based on John's PhD work at the MIT Media Lab. In, in a real sense, this was real technology.
- 14:42
John had brought the UI out of the small screen and into the world with us in a bunch of really interesting and lovely ways. John also consulted on Iron Man, which is a very different view of the future than, you know, Minority Report, which was Spielberg working in, like, the American Kubrick dystopian tradition.
- 14:57
Iron Man is really squarely in that Star Trek goofy futurist tradition. But I think you can see the common elements in the UI depicted on screen. It's still from the same era. [clapping] [upbeat music]
- 15:12
Wake up, daddy's home.
- 15:13
Welcome home, sir. Congratulations on the opening ceremonies. They were such a success, as was your Senate hearing. And may I say how refreshing it is to finally see you in a video with your clothing on, sir. [laughs]
- 15:28
You. I swear to God, I'll dismantle you. I'll soak your motherboard. I'll turn you into a wine rack.
- 15:33
I co-founded a startup with John in 2006 to make the Minority Report interface into a commercial product. This is our demo reel from 2012, six years into that work. [upbeat music]
- 17:10
This long project to build the multimodal, multi-device, multi-screen, multiplayer, ubiquitously connected computer is still what I'm working on 15 years later. In 2010s, we built out the cloud, which laid the groundwork for the infrastructure and data centers and data capacity we would need to scale up AI training and inference, which brings us to now.
- 17:33
We're building agents, and we are starting to think about agents plus plus. But I think we can also start to think about the next thing, the AI native software that is to agents what today's internet is to the web pages of 1995.
- 17:47
And one way to think about the story is this. We went from the calculator to the computer to the personal computer to the global cloud computer, and now we actually have the ability to build the Memex and JARVIS from Iron Man and Knowledge Navigator from that 1987 video for real, completely working.
- 18:04
And by building those things, we'll figure out what we want to build next. A couple of weeks ago, the team at Tavis released a reimagined Knowledge Navigator video, this time entirely built on real and available technology.
- 18:17
I'll just play another 20 seconds of this, but like the original Knowledge Navigator video, this is worth tracking down and watching in full
- 18:31
Good evening, Hassan. I adore that houndstooth jacket you're wearing today. Anything I can help with or would you like to review tomorrow's schedule?
- 18:39
Thanks so much, Tom. Yeah, let's review tomorrow's schedule and see how busy it is.
- 18:44
Opening your calendar now. Here is the quick version since it is late. Tomorrow morning is slammed. Investor meeting at 9:00, then back-to-back one-on-ones and internal meetings until 1:00.
- 18:55
So the full four-minute video is one take, completely real. And when you watch it, it really does feel both like the Knowledge Navigator video from 1987, familiar but built on real technology, and like something brand new.
- 19:09
And I'll just close with a massively multiplayer game project I've been working on with some friends as a canvas to really think about what AI-native software can be. This game is built from the ground up with LLMs as the core of every interaction.
- 19:25
At every moment in the game, there are hundreds of inference calls happening, and we couldn't have built anything like this even a year ago.
- 19:32
Oh, sorry.
- 19:36
Welcome to Gradient Bang, a multiplayer game that showcases real-time agent orchestration. Gradient Bang demonstrates several patterns for AI subagents, such as asynchronous non-blocking context compression. Okay, make a note for later.
- 19:51
We are going to eliminate Hely Taitle from existence.
- 19:54
Noted.
- 19:55
Long-running subagents that share context.
- 19:58
Eagle is on five trade loops. Hawk and Raptor are on five exploration loops each. Your fleet is busy.
- 20:06
Progressive skills loading. How much does your average ship cost?
- 20:10
Ships range quite a bit, Captain.
- 20:12
Dynamic user interface generation. Show my task history.
- 20:16
Certainly.
- 20:17
Uh, hide the map.
- 20:19
S- okay.
- 20:20
And conversational voice.
- 20:21
No, I don't want to exchange it. I just wanna sell it for cold, hard cash, please.
- 20:26
I'm afraid the galaxy doesn't allow you to be shipless and hitchhike.
- 20:31
So I went long. I have to wrap up, but I will say that the first version of this new draft talk was an hour, so I have a lot more things I'm super excited to talk about with all of you.
- 20:41
So if you are interested in this stuff, come find me. Uh, we have a booth on the show floor. I'm online everywhere, and I'm excited to build agents 'cause agents are awesome, but also to build the next, next thing too.
- 20:53
Thank you. [clapping] [outro music]