AI Engineer World's Fair 2025
Invisible Users, Invisible Interfaces: Accelerating Design Iteration with AI Simulation
Read the talk
Invisible Users: Using AI Simulation to Accelerate Design
AI simulation can turn audience research into task-directed feedback, helping designers investigate friction before deciding what to build.
From a talk by Alex Liss
What happens when users cannot trust the interface?
What happens to a product when users cannot trust what its AI tells them? Alex Liss, introducing himself as VP of Data Science and AI at Huge, starts with that problem. He cites Edelman figures matching its 2025 Trust Barometer summary: 32% of US adults say they trust AI, while 44% of adults globally say they feel comfortable with businesses using AI. These measure different attitudes, but both make trust a central constraint for AI product design.
The failures are concrete: a web search recommends eating rocks every day, and a website offers a car for one dollar. Users can recognize that something has gone wrong without understanding the model behind the answer. Putting a chatbot into an interface and presenting it as magic creates expectations the product may immediately violate.
There is a more useful capability underneath those failures. Drawing on Cassie Kozyrkov’s framing of generative AI as a UX revolution, Liss points to natural-language interaction with machine-learning models. That capability invites a design question beyond where to put a chat window: how can AI help teams understand and remove the friction users already experience?
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make the interface disappear
Don Norman’s The Design of Everyday Things supplies the starting principles: simplicity and efficiency. The Invisible Computer extends the ambition to software so seamless and intuitive that users practically forget they are using it. In this framing, a successful interface lets the person concentrate on their goal rather than on operating the software.
AI can contribute upstream, through needfinding: understanding the needs that should guide design. Liss presents a data scientist’s proposal for accelerating that process, rather than a portfolio of designs he has produced. The intended output is better direction for designers—insight into which problems to solve and why they matter.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn research artifacts into participants
Modern interfaces can resemble a cockpit: many screens, abundant information, and considerable complexity. Pilots learn to work in that environment through simulation and practice. For design teams, the corresponding opportunity is to practice against a simulated audience while developing an experience. The existing process already collects qualitative observations, quantitative data, and ethnographic observations to guide prototypes.
The proposed change is to make those research artifacts active participants. An AI simulation represents invisible users and gives the designer an additional feedback cycle inside the broader needfinding process. The designer can investigate an interface through that simulation, then use the resulting observations to inform further work. The simulation loop sits within research and prototyping; it does not establish that simulated behavior is equivalent to human behavior.
The overall sequence remains recognizable:
- Define the audience whose experience matters.
- Map intentions to understand what that audience wants to accomplish.
- Identify tasks that serve those intentions.
- Analyze the process of attempting those tasks.
- Refine insights from the observations.
- Develop design alternatives that address the needs uncovered.
AI simulation changes some of the participants and the workflow inside this sequence, rather than removing the need to define an audience or interpret findings.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build an audience, then give it a task
Audience simulation begins with data representing the audience. Huge’s Live platform combines demographic, psychographic, and contextual datasets. Intent mapping then turns that foundation into an Intelligent Twin: a simulation representing user behaviors, desired outcomes, needs, and motivations. The distinction matters because an audience description alone does not specify what a simulated participant should try to do.
A task brief supplies that direction. Liss compares briefing an Intelligent Twin to asking a human designer to conduct a heuristic analysis: focus the evaluation on particular tasks within a particular interface. For the sample project, he chooses a global audit of sports websites. Imagine a business considering partnerships with sports leagues across countries and cultures; it needs to understand how those leagues engage their audiences. Simulation’s proposed speed and scale make that breadth attractive, although Liss supplies no measured speedup over a human audit.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A task matrix for the sports-site audit
The audit uses two contrasting personas:
- Casual fan: newer to the sport.
- Superfan: a knowledgeable, lifelong follower.
Each persona receives tasks across three websites. The tasks cover navigation, information architecture, and fan engagement, with four tasks in each category. Liss reports 72 AI-simulated actions in this sample audit: three websites × two personas × three categories × four tasks per category. This is the scope of the methodological test drive, not a count of human research participants.
The matrix can be made explicit before any simulation runs. This TypeScript expansion records the stated dimensions without inventing the task briefs; taskSlot identifies where each category’s four briefs would go.
typescript
const siteSlots = [1, 2, 3] as const;
const personas = ["casual-fan", "superfan"] as const;
const categories = [
"navigation",
"information-architecture",
"fan-engagement",
] as const;
const taskSlots = [1, 2, 3, 4] as const;
const auditPlan = siteSlots.flatMap((siteSlot) =>
personas.flatMap((persona) =>
categories.flatMap((category) =>
taskSlots.map((taskSlot) => ({
siteSlot,
persona,
category,
taskSlot,
status: "pending" as const,
})),
),
),
);
Every record is a planned task/persona/site combination. Keeping that distinction explicit prevents the size of a generated plan from being confused with completed observations or successful tasks.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From broad findings to a focused design brief
The audit can first be read at the category level, then examined more closely. Across the basketball, Olympics, and English Premier League sites, Liss reports relatively successful navigation task completion, followed by declining success deeper in content browsing, information architecture, and engagement pathways. He does not provide numerical completion rates in the spoken account.
That pattern separates reaching a website’s starting points from making useful progress inside it:
| Area | Reported pattern |
|---|---|
| Navigation | Relatively successful entry into the experience |
| Content and information architecture | Success drops deeper into browsing |
| Engagement pathways | Further friction after initial navigation |
A usable entrance does not guarantee a usable journey. Investigating the later friction gives designers a concrete place to direct their attention.
Solving those pain points is the proposed route back to trust: use AI to improve the experience people encounter. The same methodology can then move below the category-wide view to examine specific nuances of an experience. That range supports both broad briefs about recurring problems and focused briefs about particular sources of friction.
The handoff combines computer-use models, computer-vision models, and human-in-the-loop observation. The audit’s output is a design brief that helps the human team focus on distracting pain points within a category and develop responses to them. This preserves a clear division of work: simulation helps gather and organize observations; designers use those observations to decide how the experience should change.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Faster implementation makes the design question more important
Looking ahead from the first half of 2025, Liss points to Anthropic’s Model Context Protocol (MCP) and potential integrations connecting Figma prototype components to code using React, Node.js, and other frameworks. These are prospective tooling possibilities in the talk, not a demonstrated deployment. As creating an implementation becomes easier, the strategic work grows more consequential: identifying the user problem, understanding why it matters, and deciding which design deserves to be built.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make the method reproducible, then test its contribution
The methodology remains experimental. Liss’s next step is a code repository that standardizes the inputs and evaluation rules needed to reproduce an audit:
- Briefing instructions: what the simulated participant is asked to do.
- Audience dimensions: which characteristics define the simulated audience.
- Audit-run counts: how many runs the evaluation includes.
- Completion and failure parameters: what determines the outcome of a task.
These are proposed standardization targets, not a released implementation described in the talk.
Once those elements are standardized, Liss proposes a test-and-control methodology to isolate where Intelligent Twins contribute to needfinding. The question is how they can complement human teams across industries, geographies, and domains. The sports audit provides an example of the process; controlled evaluation would need to establish its strengths.
The final criterion is the experience users receive. Better websites, mobile apps, and other surfaces should provide clarity and simplicity. AI simulation earns its place if it helps design teams gather useful insights faster and turn them into those improvements. Restoring trust is the intended consequence of that better experience, not an outcome already demonstrated by the sample audit.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Huge's overview of audience personas and journey simulations powered by its LIVE data platform.
Updates since the talk
The June 2025 beta announcement explains how Figma supplies design context to AI coding tools.
Read the complete timestamped transcript
- 0:00
Hi, I'm Alex Liss, VP of Data Science and AI at Huge Design and Technology Company. And today I'm gonna talk about invisible users, invisible interfaces, which is how teams can use AI simulation to accelerate design.
- 0:18
We're gonna talk about three things today when it comes to the state of UX and AI. One, where we are now. Two, where we could be. And three, an idea for how to get there.
- 0:32
So in terms of where we are today, unfortunately, the reality is that AI has a trust gap. Some recent research by Edelman, which came out in December 2024, highlighted that only thirty-two percent of US adults say they trust AI, and only forty-four percent of adults globally say they feel comfortable with how businesses are using AI.
- 0:54
And the reason for that, unfortunately, is AI slop. And as these screen grabs show, uh, AI slop is when you go to use a website, a product, an interface, and there's a gen I- gen AI fail, uh, where it gives you something that you know is not correct, whether it's a search, a web search that tells you
- 1:15
you should eat rocks every day, or a website that tells you you can buy a car for one dollar. We know that neither of those things can be true.
- 1:25
Website creators are stuffing AI chatbots in everything and telling users it's magic. But the real magic of gen AI, uh, as, uh, Cassie Kozyrkov says, you know, it's, it is the fact that, that it's a UX revolution.
- 1:41
It's that users can actually talk to a machine learning model in natural language, which has never been possible before. So with that in mind, we w- w- really wanna think about the capability of that and go back to some UX first principles for us to figure out how we wanna use gen AI to help design.
- 2:03
So going back to how it started, you know, some writings by Don Norman of the Nielsen Norman Group talk about the design of everyday things, talking about simplicity and efficiency.
- 2:15
Uh, and he proffered another great principle in his book, The Invisible Computer, which talks about invisible interfaces, which is software that feels so seamless and intuitive, the users practically forget that they're using it.
- 2:30
And I think the opportunity here is to use AI to design interfaces that actually feel like magic, not by stuffing chatbots into the website, but by accelerating needfinding. And this part about needfinding is in asterisks because I just wanna make it clear I'm not a designer.
- 2:48
Uh, you're not gonna see designs I produced today. I'm a data scientist. But you are gonna see a proposal for a process for how we could deliver this AI accelerated needfinding.
- 3:01
So let's talk about what a new approach to the design life cycle looks like, because in the world today, all of us are bombarded with so many screens, so much information, so much complexity.
- 3:13
It's a little bit akin to a pilot operating in a cockpit. And the way that pilots learn to su- learn to operate in this environment is actually through simulation, through a lot of practice.
- 3:26
And I think we can take this idea of simulation and evolve it and reinterpret it a bit. Our current design process, which comes from the pre-ChatGPT area, uh, is data-driven as designers collect, you know, qualitative observations, uh, quantitative data, uh, ethno- ethnographic observations, and they use that to guide the prototyping process.
- 3:52
But going forward, what if we could empower designers to work with invisible users in the form of AI simulation to turn the data artifacts that they've always collected as part of needfinding to active participants in the design process, and give designers their own mini feedback cycle to work with AI simulation as part of the broader
- 4:16
needfinding process to help deliver better design? So let's take a, ta- let's take a look at what an example of an accelerated needfinding process could look like through an example project.
- 4:30
So the new process for needfinding in the AI-- era of AI simulation has a lot of simul- similarities with the existing process. We're gonna start by defining our audience, then mapping their intentions, identifying specific tasks they will perform to accomplish those intentions, conducting analysis of that process, refining insights, and ultimately developing design alternatives.
- 4:55
However, in this era of AI simulation, there's gonna be a few key differences in some of the components and some of the workflow.
- 5:04
So when we start thinking about audiences in the area-- era of AI simulation, we wanna start with data that represents these audiences. Uh, and at my company, Huge, we have a data platform, which we call Live, uh, that has a mixture of data sets, demographics, psychographic, contextual, that allow us to simulate audience behaviors in the real world.
- 5:28
And this is a critical foundation to build on. Once we have this foundation, the next step is to actually apply intent mapping. And here's where we turn the data into something active, a simulation which we call an Intelligent Twin, which represents a set of user behaviors and a set of desired outcomes, their needs and motivation.
- 5:50
Intelligent Twins become an active participant in the design simulation process.
- 5:56
And from there, we can brief Intelligent Twins to evaluate interfaces by focusing on specific tasks, much as we might brief a human designer to conduct a heuristic analysis. And in this way, we're able to evaluate a specific interface, or in the case of this sample project, a specific website.
- 6:17
Now for the sample project I looked at, uh, I wanted to think about the advantages of this, this simulated methodology, the scale and speed. Uh, intelligent twins can operate with different levels of velocity than a human team, so I wanted to apply that to do a global, uh, audit of sports websites.
- 6:39
And we can imagine this happening from the perspective of a business that wanted to partner with sports leagues globally and learn how to cut across different countries, different cultures, uh, to engage audiences there.
- 6:52
And this is where, uh, the intelligent twin methodology could provide some unique advantages.
- 6:59
So in terms of the actual data collection, so, uh, with intelligent twins here in this sample project, I actually broke them out into two audiences, two personas, a casual fan newer to sport versus super fan who's lifelong and very savvy.
- 7:17
And across these three different websites, uh, we briefed a series of n-- of tasks across different categories like navigation, information architecture, and fan engagement, uh, with four tasks per category.
- 7:31
And that allows us to simulate 72 AI simulated actions, uh, to give, uh, to give us a test drive for, for how this AI accelerated design audit will perform.
- 7:45
When we get into the actual findings, so, uh, w- with the speed and the scale of intelligent twins, we can roll these insights up at a very high level or dive more granular, as we'll see in the next couple of slides.
- 7:58
But one of the benefits of this kind of global audited speed is it gives us high level understandings across an entire category. And we can see from, from this audit that, uh, in the area of navigation, just kind of the starting place of a fan coming to the website, all the different leagues we looked at, um, you
- 8:18
know, basketball, the Olympics, and the English Premier League, all those sites performed pretty well off in terms of task completion when it comes to navigation. But as we get deeper into the process, as fans started to browse content and information architecture and engagement pathways, that's where those initial successes started to drop off.
- 8:39
And this goes back to the challenge I mentioned at the start of the presentation. Uh, there's an AI trust gap right now because AI is being shoved in people's face and websites, and it doesn't work very well.
- 8:52
But hopefully, this methodology could help us understand what user pain points are and actually solve those pain points, and in that way, start to repair that trust gap.
- 9:05
We can also go deeper with this methodology, so we can scale the simulation, uh, to surface friction across specific areas, specific nuances of the experience. Uh, the, the, the different levels of altitude this methodology can operate at allows us to create, uh, design briefs which are really focused, which are really broad and really deep as well.
- 9:29
And that's what we set up on this slide here, sort of closing out the audit process, uh, through AI acceleration, uh, computer use models, computer vision models, uh, human-in-the-loop observation.
- 9:41
We can use this process to generate a design brief that allows the human team to really focus in the pain points, uh, that are, that are distracting users in a specific category and try to solve them.
- 9:55
And in that way, uh, we can use AI simulation to, to hopefully accelerate and improve the design process.
- 10:04
As we look to the future, in the first half of 2025, there's been, uh, you know, tremendous progress. The, the MCP protocol from Anthropic is already showing potential integrations to turn prototype components in Figma through to actual code components in React or Node.js or other frameworks.
- 10:25
Uh, and this is gonna make it easier than ever to create new designs. And I think as the acceleration of these tools, um, just makes it easier to create design that really emphasizes the importance of focusing on the why, the strategy and the problem we wanna solve for users in the design process, because it'll be easier than
- 10:48
ever to solve for the how. In terms of this methodology, obviously this is experimental early stages, so there's some limitations and improvements to consider going forward. I think, uh, reproducibility is key, so as we evolve this methodology, my company and I will be standardizing things in a code repository that lays out parameters like the briefing
- 11:13
instructions, the simulated audience dimensions, the number of audit runs, and the parameters from task completion and failure. And also, uh, once we standardize this, applying it through a test and control methodology will also be useful to help us isolate what the strengths are of intelligent twins for design needfinding and how that can be used alongside or in
- 11:37
a complementary manner to human teams, especially applying to different industries, different de- geographies, and different domains.
- 11:46
In conclusion, just a few final thoughts here. Uh, when it comes to repairing the AI trust gap that we're struggling with at the moment, uh, I hope we can all re- recognize that users don't need more websites that have gen AI chatbots in them that don't work.
- 12:05
Uh, but users could always benefit from, from better websites, better mobile apps, better, better surfaces that provide clarity and simplicity. And hopefully, uh, I've been able to show in this talk today how AI simulation can be a tool to help us empower design teams to gather insights in a smarter, faster, and better process, and ultimately create websites
- 12:29
and interfaces that can restore and repair the trust gap. Thank you.