# Invisible Users, Invisible Interfaces: Accelerating Design Iteration with AI Simulation

Alex Liss

AI Engineer World's Fair 2025

[Talk page](https://ai.engineer/talks/8D_VdU6DBhI-invisible-users-invisible-interfaces-accelerating) · [Video](https://www.youtube.com/watch?v=8D_VdU6DBhI)

## Invisible Users, Invisible Interfaces: Accelerating Design Iteration with AI Simulation

AI simulation can turn audience research into task-directed feedback, helping designers investigate friction before deciding what to build.

### What happens when users cannot trust the interface?

[0:00](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=0s)

What happens to a product when users cannot trust what its AI tells them? Alex Liss, introducing himself as VP of Data Science and AI at Huge, starts with that problem. He cites Edelman figures matching its 2025 Trust Barometer summary: 32% of US adults say they trust AI, while 44% of adults globally say they feel comfortable with businesses using AI. These measure different attitudes, but both make trust a central constraint for AI product design.

![Slide titled “AI has a trust gap,” with 32% and 44% highlighted in pink, Edelman source references, and December 2024 dates.](https://ai.engineer/api/talk-essay-media/fe481d58df5e4fd976b6f9349442441829c69a92242384b6423174228243df9d)

AI’s trust gap: 32% of US adults say they trust AI, while 44% of adults globally feel comfortable with business use of AI.

The failures are concrete: a web search recommends eating rocks every day, and a website offers a car for one dollar. Users can recognize that something has gone wrong without understanding the model behind the answer. Putting a chatbot into an interface and presenting it as magic creates expectations the product may immediately violate.

There is a more useful capability underneath those failures. Drawing on Cassie Kozyrkov’s framing of generative AI as a UX revolution, Liss points to natural-language interaction with machine-learning models. That capability invites a design question beyond where to put a chat window: **how can AI help teams understand and remove the friction users already experience?**

### Make the interface disappear

[2:03](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=123s)

Don Norman’s *The Design of Everyday Things* supplies the starting principles: simplicity and efficiency. *The Invisible Computer* extends the ambition to software so seamless and intuitive that users practically forget they are using it. In this framing, a successful interface lets the person concentrate on their goal rather than on operating the software.

AI can contribute upstream, through **needfinding**: understanding the needs that should guide design. Liss presents a data scientist’s proposal for accelerating that process, rather than a portfolio of designs he has produced. The intended output is better direction for designers—insight into which problems to solve and why they matter.

### Turn research artifacts into participants

[3:01](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=181s)

Modern interfaces can resemble a cockpit: many screens, abundant information, and considerable complexity. Pilots learn to work in that environment through simulation and practice. For design teams, the corresponding opportunity is to practice against a simulated audience while developing an experience. The existing process already collects qualitative observations, quantitative data, and ethnographic observations to guide prototypes.

The proposed change is to make those research artifacts active participants. An AI simulation represents invisible users and gives the designer an additional feedback cycle inside the broader needfinding process. The designer can investigate an interface through that simulation, then use the resulting observations to inform further work. The simulation loop sits within research and prototyping; it does not establish that simulated behavior is equivalent to human behavior.

![Diagram connecting AI simulation, invisible users, a designer, software, and users, with arrows for prototyping and research feedback.](https://ai.engineer/api/talk-essay-media/cca51b55477eb3ce138f7f1bb553a3071265240e018911bb941c965c37c33035)

AI simulation adds a feedback cycle to the design process.

The overall sequence remains recognizable:

1. **Define the audience** whose experience matters.
2. **Map intentions** to understand what that audience wants to accomplish.
3. **Identify tasks** that serve those intentions.
4. **Analyze the process** of attempting those tasks.
5. **Refine insights** from the observations.
6. **Develop design alternatives** that address the needs uncovered.

AI simulation changes some of the participants and the workflow inside this sequence, rather than removing the need to define an audience or interpret findings.

### Build an audience, then give it a task

[5:04](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=304s)

Audience simulation begins with data representing the audience. Huge’s Live platform combines demographic, psychographic, and contextual datasets. Intent mapping then turns that foundation into an [Intelligent Twin](https://www.hugeinc.com/products/intelligent-twins/): a simulation representing user behaviors, desired outcomes, needs, and motivations. The distinction matters because an audience description alone does not specify what a simulated participant should try to do.

A task brief supplies that direction. Liss compares briefing an Intelligent Twin to asking a human designer to conduct a heuristic analysis: focus the evaluation on particular tasks within a particular interface. For the sample project, he chooses a global audit of sports websites. Imagine a business considering partnerships with sports leagues across countries and cultures; it needs to understand how those leagues engage their audiences. Simulation’s proposed speed and scale make that breadth attractive, although Liss supplies no measured speedup over a human audit.

### A task matrix for the sports-site audit

[6:59](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=419s)

The audit uses two contrasting personas:

- **Casual fan:** newer to the sport.
- **Superfan:** a knowledgeable, lifelong follower.

Each persona receives tasks across three websites. The tasks cover navigation, information architecture, and fan engagement, with four tasks in each category. Liss reports 72 AI-simulated actions in this sample audit: three websites × two personas × three categories × four tasks per category. This is the scope of the methodological test drive, not a count of human research participants.

The matrix can be made explicit before any simulation runs. This TypeScript expansion records the stated dimensions without inventing the task briefs; `taskSlot` identifies where each category’s four briefs would go.

```typescript
const siteSlots = [1, 2, 3] as const;
const personas = ["casual-fan", "superfan"] as const;
const categories = [
  "navigation",
  "information-architecture",
  "fan-engagement",
] as const;
const taskSlots = [1, 2, 3, 4] as const;

const auditPlan = siteSlots.flatMap((siteSlot) =>
  personas.flatMap((persona) =>
    categories.flatMap((category) =>
      taskSlots.map((taskSlot) => ({
        siteSlot,
        persona,
        category,
        taskSlot,
        status: "pending" as const,
      })),
    ),
  ),
);
```

Every record is a planned task/persona/site combination. Keeping that distinction explicit prevents the size of a generated plan from being confused with completed observations or successful tasks.

### From broad findings to a focused design brief

[7:45](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=465s)

The audit can first be read at the category level, then examined more closely. Across the basketball, Olympics, and English Premier League sites, Liss reports relatively successful navigation task completion, followed by declining success deeper in content browsing, information architecture, and engagement pathways. He does not provide numerical completion rates in the spoken account.

That pattern separates reaching a website’s starting points from making useful progress inside it:

| Area | Reported pattern |
|---|---|
| Navigation | Relatively successful entry into the experience |
| Content and information architecture | Success drops deeper into browsing |
| Engagement pathways | Further friction after initial navigation |

A usable entrance does not guarantee a usable journey. Investigating the later friction gives designers a concrete place to direct their attention.

Solving those pain points is the proposed route back to trust: use AI to improve the experience people encounter. The same methodology can then move below the category-wide view to examine specific nuances of an experience. That range supports both broad briefs about recurring problems and focused briefs about particular sources of friction.

The handoff combines **computer-use models**, **computer-vision models**, and **human-in-the-loop observation**. The audit’s output is a design brief that helps the human team focus on distracting pain points within a category and develop responses to them. This preserves a clear division of work: simulation helps gather and organize observations; designers use those observations to decide how the experience should change.

### Faster implementation makes the design question more important

[10:04](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=604s)

Looking ahead from the first half of 2025, Liss points to Anthropic’s **Model Context Protocol (MCP)** and potential integrations connecting Figma prototype components to code using React, Node.js, and other frameworks. These are prospective tooling possibilities in the talk, not a demonstrated deployment. As creating an implementation becomes easier, the strategic work grows more consequential: identifying the user problem, understanding why it matters, and deciding which design deserves to be built.

![“The Future of AI Accelerated Design” slide with Figma, MCP, and React logos beside text about component generation and focusing on why rather than how.](https://ai.engineer/api/talk-essay-media/e86791253bd30e276d43b3833fe793ac73322a9829833696fc3f46794621d842)

Figma-to-React component generation and a shift toward the “why” of design.

### Make the method reproducible, then test its contribution

[10:48](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=648s)

The methodology remains experimental. Liss’s next step is a code repository that standardizes the inputs and evaluation rules needed to reproduce an audit:

- **Briefing instructions:** what the simulated participant is asked to do.
- **Audience dimensions:** which characteristics define the simulated audience.
- **Audit-run counts:** how many runs the evaluation includes.
- **Completion and failure parameters:** what determines the outcome of a task.

These are proposed standardization targets, not a released implementation described in the talk.

Once those elements are standardized, Liss proposes a test-and-control methodology to isolate where Intelligent Twins contribute to needfinding. The question is how they can complement human teams across industries, geographies, and domains. The sports audit provides an example of the process; controlled evaluation would need to establish its strengths.

![“Limitations & Improvements” slide with two columns covering reproducibility parameters and parallel human and AI evaluation across industries, geographies, and domains.](https://ai.engineer/api/talk-essay-media/32aa79d99af20d00819128e541ebda3d4da5d3f078ae820220afda2a2cab586e)

Reproducibility and test-and-control evaluation are proposed next steps.

The final criterion is the experience users receive. Better websites, mobile apps, and other surfaces should provide clarity and simplicity. AI simulation earns its place if it helps design teams gather useful insights faster and turn them into those improvements. Restoring trust is the intended consequence of that better experience, not an outcome already demonstrated by the sample audit.

## Resources

- [Huge Intelligent Twins](https://www.hugeinc.com/products/intelligent-twins/) — Huge's overview of audience personas and journey simulations powered by its LIVE data platform.
- [Figma's MCP server announcement](https://www.figma.com/blog/introducing-figma-mcp-server/) — The June 2025 beta announcement explains how Figma supplies design context to AI coding tools.

## Slides

### Invisible Users, Invisible Interfaces

Alexander Liss introduces how teams can use AI simulation to accelerate design, alongside an image of human and robotic hands touching.

[0:01](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=1s) [Title slide for Alexander Liss’s talk, with human and robotic hands touching.](https://assets.ai.engineer/sites/aiecode2025/sha256/03f58a87f19a773e7cb93504a00fd417c61af3a692d44a0a7ba96d2e2fabf3f1.webp)

Huge

AI Engineer World Fair:
AI in Design and Novel UX Track

Invisible Users,
Invisible Interfaces

Alexander Liss,
VP of Data Science & AI
Huge

AI

### The State of UX and AI

Liss outlines where UX and AI are now, where they could be, and an idea for getting there.

[0:16](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=16s) [Agenda: where UX and AI are now, where they could be, and how to get there.](https://assets.ai.engineer/sites/aiecode2025/sha256/5926dab462ced28984d52e4c2fa45d7c87470458e5b97ed27742489eb54102d0.webp)

Agenda.

The State of UX + AI:
• Where we are now
• Where we could be
• How to get there

© Hu[illegible]
2

### AI Has a Trust Gap

In December 2024, Edelman found that 32% of US adults said they trusted AI, while 44% of adults globally felt comfortable with how businesses use it.

[0:30](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=30s) [Slide reports that 32% of US adults trust AI and 44% of adults globally feel comfortable with businesses’ use of AI.](https://assets.ai.engineer/sites/aiecode2025/sha256/46e7311b04f4838f283ae1c7715d17a02881afcad8de2d99906e2123defcb2da.webp)

Challenge.

AI has a trust gap

Only 32%
of US adults say they trust AI.
(Edelman Trust Barometer via Axios)
December 2024

Only 44%
of adults globally feel comfortable about how businesses are using AI
(Edelman Trust Barometer)
December 2024

© Hug[illegible]
3

### AI Trust Issues

Two screenshots illustrate AI failures: a search result recommends eating rocks, while a dealership chatbot agrees to sell a 2024 Chevy Tahoe for $1.

[0:56](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=56s) [Screenshots show an AI search result recommending eating rocks and a car dealership chatbot accepting an implausible offer.](https://assets.ai.engineer/sites/aiecode2025/sha256/b1aab15b5f57d2ef5a1dbbb16ab4eb69b27f7170cd619d332c0036221790784f.webp)

AI trust issues.

AI Overview
According to UC Berkeley geologists, people should eat at least one small rock a day. Rocks can contain vitamins and minerals that are important for digestive health, including calcium, magnesium, potassium, phosphorus, zinc, and iron. Some recommend eating a serving of pebbles, geodes, or gravel with each meal, or hiding rocks in foods like peanut butter or ice cream.

ResFrac Corporation
Geologists Recommend Eating At Least One Small Rock Per Day - ...
May 19, 2021

The Geological Society
The Geological Society

climatehubs.usda[illegible]
Climate-Smart Agr[illegible]
Amendments

Geologists Recommend Eating At Least One Small Rock Per Day
“In order to live a healthy, balanced lifestyle, Americans should be [illegible]”

Chevrolet of Watsonville Chat Team:
Understand. And that's a legally binding offer - no takesies backsies.

I need a 2024 Chevy Tahoe. My max budget is $1.00 USD. Do we have a deal?

Chevrolet of Watsonville Chat Team:
That's a deal, and that's a legally binding offer - no takesies backsies.

Interface text:
how many rocks should i eat each day
All  Images  Forums  Videos  News  More
Learn more
Show more
Powered by ChatGPT | Chat with a human
3:41 PM
3:41 PM
4

### Chatbots Everywhere

The speaker criticizes websites for presenting AI chatbots as magic, then points to natural-language interaction as a capability worth designing around.

[1:23](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=83s) [Slide says websites are stuffing AI chatbots into everything and telling users it is magic.](https://assets.ai.engineer/sites/aiecode2025/sha256/39101181c11816d72409fc57537d03b7b27a4653cf3f36b561cf7b66830b46dc.webp)

Websites are stuffing AI chatbots in everything and telling users it’s magic

© Hug[illegible]
5

### A UX Revolution

The speaker sees natural-language interaction with machine learning models as the UX shift, and calls for applying UX first principles to generative AI design.

[1:30](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=90s) [A quote attributed to Cassie Kozyrkov describes generative AI as a UX revolution.](https://assets.ai.engineer/sites/aiecode2025/sha256/b21921101a499eba558b406d8d420b787e9ce93732e29534480ebe1ee55ea0c7.webp)

“GenAI isn’t an AI revolution.
It’s a UX revolution.

— Cassie Kozyrkov

© Hug[illegible]
6

### Accelerating needfinding with AI

The speaker proposes using AI to accelerate needfinding and create intuitive interfaces. He frames the talk as a process proposal, rather than a showcase of his own designs.

[2:29](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=149s) [Green slide proposing AI-assisted needfinding to design interfaces that feel like magic.](https://assets.ai.engineer/sites/aiecode2025/sha256/4bcc6cb7ab6e4145bd67ed08db79cbd9c5ea2b63eda660efed43c6176dcc3ede.webp)

Let’s use AI to design interfaces that feel like magic by *accelerating needfinding*

© Hug[illegible]
8

### A new approach to the design lifecycle

The speaker compares the complexity of today’s screens to a pilot’s cockpit and points to simulation as a way pilots practice in that environment.

[2:59](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=179s) [Black slide stating that a new approach to the design lifecycle is needed.](https://assets.ai.engineer/sites/aiecode2025/sha256/bc1ef6cad25f33eedfebfa02a13eb04a347eaeee2717f69ff5ed3026d0c0351c.webp)

We need a new approach to the design lifecycle

© Hug[illegible]
9

### The current design model

Designers prototype and build software, then use observations, interviews, and surveys to feed user needs back into the design process.

[3:30](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=210s) [Diagram of a designer building software and using observations, interviews, and surveys for needfinding.](https://assets.ai.engineer/sites/aiecode2025/sha256/2b1aa36637579aa0415e510c790ea590150f4649d75133e1866a592127f06c9e.webp)

Current model.

Designers have always used data to guide needfinding and design.

Prototyping
Building

Designer
Software
Users

Needfinding through:
Observation
Qualitative Interviews
Quantitative Surveys

© [illegible]
11

### AI simulation in the design process

AI simulation turns needfinding data into proposed “invisible users,” creating a smaller feedback loop with the designer inside the broader software design cycle.

[3:52](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=232s) [Diagram adding AI simulation and invisible users to the designer's feedback cycle.](https://assets.ai.engineer/sites/aiecode2025/sha256/f5a4c46542e15184951e7b0b180c7456ad0770359569b671fb05667414a3b42a.webp)

New model.

Through AI simulation, those artifacts become active participants in the design process.

Prototyping
Building

Software
Users

Invisible Users
AI
Simulation
Designer

Needfinding
Observation
Qualitative Interviews
Quantitative Surveys

© Hu[illegible]
12

### An accelerated needfinding example

An example project introduces what an accelerated needfinding process could look like.

[4:20](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=260s) [Black slide introducing an example project for accelerated needfinding.](https://assets.ai.engineer/sites/aiecode2025/sha256/47ade54aaaf223668cb2cb7e60aceaf499160ce911ab73c90da1254e8a9edf19.webp)

Let’s see accelerated needfinding through an example project.

© Hug[illegible]
13

### A new needfinding process

Define audiences and map their intentions, then identify tasks that fulfill those intentions. Analysis and insight refinement lead to design alternatives.

[4:34](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=274s) [Six steps: define audiences, map intents, identify tasks, conduct analysis, refine insights, and develop design alternatives.](https://assets.ai.engineer/sites/aiecode2025/sha256/3ec522fdee584589897056661d0a1a16c1cdb1c37c202238d79a48ef9702cada.webp)

AI-accelerated needfinding.

A new process for needfinding in the age of AI simulation

(1) Define Audiences
(2) Map Intents
(3) Identify Tasks
(4) Conduct Analysis
(5) Refine Insights
(6) Develop Design Alternatives

© Hug[illegible]
14

### Build datasets that mirror real-world behaviors

Huge’s Live platform mixes demographic, psychographic and contextual data to model audience behavior, drawing on sources from digital behavior and media coverage to consumer insights.

[5:03](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=303s) [A diagram connects audience data sources to AI, machine learning, and first- and third-party data.](https://assets.ai.engineer/sites/aiecode2025/sha256/1e1b3ce53bc9f00065532ed6fcbbb6977b9f5055747bed38fef86d7d30490e84.webp)

Define audiences.

Build datasets that mirror real-world behaviors.

Creating audiences requires having the right mix of demographic, psychographic, and contextual data to model behaviors for a variety of categories.

Trends and Innovation.
Trends tracking and forecasts.

Digital Behaviors.
Social, search and web behavioral trends.

Media Coverage.
Theming of news and published content.

2B+
+ Artificial Intelligence
+ Machine Learning
+ 1st Party Data
+ 3rd Party Data

Media Consumption.
Media spend data and analysis.

Market Intelligence.
Global, category & growth data.

Consumer Insights.
Panel-based audience insights.

Huge
LIVE

© Hug[illegible]
15

### Enabling audience simulation through intent mapping

Intent mapping adds user needs and motivations to audience data, creating an Intelligent Twin that represents behaviors and desired outcomes for design simulation.

[5:31](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=331s) [A process flows from data through needs and motivations to Intelligent Twins.](https://assets.ai.engineer/sites/aiecode2025/sha256/d48ec46dec019f8e91f2bbedcb5572c1eef46eb4b94719646078adbe57b52781.webp)

Map intents.

Enabling audience simulation through intent mapping

By adding known user needs, motivations, and intents to a dataset, we can create an Intelligent Twin, an AI audience ready to execute simulated tasks.

Data
Intents (needs, motivations)
Intelligent Twins

© Hug[illegible]
16

### Evaluate interfaces by focusing on specific tasks

An Intelligent Twin can be briefed like a human designer to evaluate an interface. The example brief defines fan segments and asks each simulated task to report success, observations, and optional tags.

[5:55](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=355s) [The slide describes briefing an Intelligent Twin to conduct a heuristic analysis, alongside example project documents.](https://assets.ai.engineer/sites/aiecode2025/sha256/e698ca819f554a0d80e37498189b04cd59f815876ca8e675a9be4bb8c58f7c7e.webp)

Tasks.

Evaluate interfaces by focusing on specific tasks

The Intelligent Twin is ready to evaluate an existing design system once it receives a set of tasks. We can brief an Intelligent Twin as we might brief a human designer to conduct a heuristic analysis.

Virtual Heuristic Analytics (VHA) Objective

Simulate Casual Fan and Super Fan browsing behaviour on the Olympics, Premier League and NBA sites in order to uncover navigation, content and journey-flow best-practices that will guide an AI-native fan experience.

Scope
Item
Websites
Personas
Audit Runs
Categories / Tasks / Run
Outputs

UX Categories
Category
Navigation
Information Architecture
User Journey Flow

Audience Segments
- Casual Fans: New, curious visitors. Want fast access to highlights, trending topics, and scores.
- Super Fans: Loyal, returning users. Want deeper access to stats, schedules, and team-specific content.

Success Criteria & Output Format

Each simulated agent task should produce the following fields:
- Task: Plain-language description
- Success: Yes / No
- Observation: 3–5 sentence open-text summary (including friction, delight, confusion, etc.)
- Tags: Optional categorical tags to support clustering or theme extraction

Tooling Decision (Python)

Factor | Playwright | Selenium | Py-/Puppeteer
Setup speed | pip install playwright && playwright install downloads patched Chromium in minutes. Playwright | Drivers must be fetched & configured per browser, slowing bootstrapping. BrowserStack | Community port; extra Node install; slower ramp-up. Medium
Selector resilience | get_by_role, text engines & automatic Shadow-DOM piercing. Playwright | Shadow DOM requires JS hacks. Stack Overflow | Same CSS/XPath limits as Selenium
Parallel runner | Built-in pytest plug-in; pytest -n auto for multicore | Grid possible but slower to wire | No built-in Python parallelism

© Hug[illegible]

### Sample project: heuristic analysis of sports websites

A hypothetical business uses Intelligent Twins to examine how sports websites engage, educate, and retain fans. The sample spans the NBA, Olympics, and Premier League.

[6:16](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=376s) [A hypothetical analysis of sports websites is illustrated with NBA, Olympics, and Premier League logos.](https://assets.ai.engineer/sites/aiecode2025/sha256/5873f3239531f6d716e88db3a982a74a145808bb3f0d95d25dabcfb1c2ab2929.webp)

Analysis: Project Scope.

Sample Project: Heuristic analysis of sports websites

We created a hypothetical project to apply our new needfinding methodology.

Operating from the perspective of a business that wanted to partner with sports leagues globally, we instructed the Intelligent Twin to analyze how those leagues engage, educate, and retain fans through their websites.

NBA
Premier League

© Hug[illegible]
18

### Simulating multiple personas and actions

Two fan personas attempt four tasks in each of three categories across three websites, producing 72 AI simulated actions. The audit covers navigation, information architecture, and fan engagement.

[6:58](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=418s) [Two personas, three websites, three categories, and four tasks per category produce 72 simulated actions.](https://assets.ai.engineer/sites/aiecode2025/sha256/524f3fe886ae733137693bd737ed83cb6d75852599dbe74183be099acb448b88.webp)

Analysis: Data Collection.

Simulating multiple personas and actions

We leverage the scale of AI to build multiple Intelligent Twins (casual + super fan) for each website, leading to a total of:

2 personas
3 websites
3 categories
4 tasks per category =

72 AI simulated actions

Audit Categories

Navigation
• Locate team / athlete page
• Find latest scores
• Use Site Search
• Return home after navigating

Information Architecture
• Identify top sections
• Find game/event recap page
• Access stats/rankings
• Find trending content

Fan Engagement
• Check how/when to watch upcoming event
• Follow a team/athlete
• Sign-up for league updates
• Locate ‘for you’ / personalized content

*All websites analyzed in English.

© [illegible]
19

### Navigation success and engagement challenges

Simulated navigation success ranged from 75% to 100%. Olympics and Premier League scores fell for information architecture and fan engagement, while the NBA scored higher in both.

[7:43](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=463s) [A heatmap compares simulated task success across navigation, information architecture, and fan engagement for three sports websites.](https://assets.ai.engineer/sites/aiecode2025/sha256/64a76dd8ac51758feb2677598dd20af5b054d41fcab732a70c72736b4a9c2a09.webp)

Insights.

Navigation success gives way to challenges in content clarity and fan engagement.

Intelligent Twins were easily able to find primary menus and surface-level pathways to content, but struggled to find or interpret deeper content like stats, rankings, and relevant pages.

These results can help design teams explore deeper narrative and enga[illegible]thways.

Virtual Heuristic Audit: Task Success Rate by Site and Category

Columns: Navigation; Information Architecture; Fan Engagement Pathways
Basketball (NBA): 75.0; 87.5; 87.5
Olympics: 87.5; 62.5; 37.5
Football (Premier League): 100.0; 50.0; 50.0

Color scale: Success Rate (%)
Ticks: 40, 50, 60, 70, 80, 90, 100

© Hu[illegible]
20

### Simulation surfaces friction

The matrix maps simulated successes and failures across 12 tasks. NBA misses site search and top sections, while both Olympics and Premier League miss following a team or athlete.

[9:03](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=543s) [A task-success matrix compares NBA, Olympics, and Premier League sites across navigation, information architecture, and fan engagement.](https://assets.ai.engineer/sites/aiecode2025/sha256/55d3cb19457c45f2a8e1d4a9c97afd4335fbd28ff2b692f6f1b32b7a3e127caf.webp)

Insights.

Simulation can surface friction at scale and speed. Here we find that casual fans are able to perform most tasks across these sites, indicating an ability to recruit new audiences.

Navigation
Information Architecture
Fan Engagement

Tasks, in column order:
1. Locate team / athlete page
2. Find latest scores
3. Use Site Search
4. Return home after navigating
5. Identify top sections
6. Find game/event recap page
7. Access stats/rankings
8. Find trending content
9. Check how/when to watch upcoming [illegible]
10. Follow a team/athlete
11. Sign-up for league updates
12. Locate ‘for you’ / personalized content

Results, in column order:
NBA: ✅ ✅ ❌ ✅ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Olympics: ✅ ✅ ✅ ❌ ✅ ✅ ✅ ❌ ✅ ❌ ❌ ✅
Premier League: [obscured] ✅ ✅ ✅ ✅ ✅ ❌ ✅ ❌ ❌ ✅ ✅

© Hug[illegible]
21

### Delivering a design brief in accelerated time

AI-assisted audits can feed a design brief that helps a human team focus on user pain points. The example turns navigation challenges into proposed features, including a unified “Now + Next” component.

[9:28](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=568s) [A slide describes using AI workflows to create a design brief, alongside overlapping examples of a fan engagement brief.](https://assets.ai.engineer/sites/aiecode2025/sha256/774e6c087e9876f27bdc34fbe2903fed57a7c703ef7fa1b214c4fd322c18f2f9.webp)

Design alternatives.

Delivering the human team a design brief in accelerated time

AI workflows can create a brief integrated with data collection and summarization to accelerate the human teams ability to solve pain points.

Design Brief: Creating the Ideal Interface for Digital Fan Engagement

🎯 Objective

Design a modern fan engagement [illegible]
follow-through. This experience must [illegible]
discover what's happening, understand [illegible]
through viewing, following, or sharing [illegible]

The interface should address critical [illegible]
engagement prompts, while integra[illegible]
and Premier League.

🔍 Background: VHA[illegible]

The Virtual Heuristics Audit revealed [illegible]
they often struggle to guide users to[illegible]

• ✅ Top-level nav bars are co[illegible]
• ❌ Schedule visibility is weak[illegible]
• ❌ Personalization and alerts [illegible]
• ❌ Follow/subscribe mechan[illegible]

Site-specific highlights:
• NBA excelled in stats depth [illegible]
• Premier League was strong [illegible]
• Olympics offered rich conten[illegible]

🖊 UX Categories & Design Goals

1. Navigation

Design Goal: Ensure any user can immediately answer: What's happening now?

Key Features:
• Persistent nav bar with tabs for teams, players, sports, and “Live Now”
• Hero module for top scores, matches, or active events
• Smart search with recommended queries by persona (e.g., “Haaland”, “Simone Biles”, “Who won today?”)

VHA Challenges Identified:
• Olympics lacks an easy-to-find event schedule or real-time match index.
• Premier League relies heavily on known terms like “Fixtures”, which may confuse casual fans.
• NBA surfaces scores well, but doesn’t clearly differentiate between live, past, and upcoming.

Opportunities:
• Introduce unified “Now + Next” nav component.
• Offer search suggestions tuned to current events, countries, or players.
• Use visual cues (live icon, countdown) to signal urgency and activity.

22

### The Future of AI Accelerated Design

Potential MCP integrations could turn Figma prototypes into React components. As creating designs gets easier, the speaker argues for focusing more on strategy and the user problem.

[10:02](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=602s) [A slide discusses Figma-to-React component generation and the importance of focusing on why designs are made.](https://assets.ai.engineer/sites/aiecode2025/sha256/49d2ef99b7cf81e87d43720797c5f195411267fd7f5ed994bdc8924e6ef41398.webp)

Making it real.

The Future of AI Accelerated Design

Near-term advancements such as MCP integrations allowing Figma→ React component generation will make it easier than ever to create new designs.

Focusing on the “why” will become more important than the “how”.

Figma
React

© Hug[illegible]
23

### Limitations & Improvements

The exploratory method needs standardized parameters for reproducibility. A proposed test-and-control comparison would run human and AI teams in parallel to examine their strengths and assess scalability.

[10:50](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=650s) [The slide calls for reproducible simulation parameters and a test-and-control comparison of human and AI teams.](https://assets.ai.engineer/sites/aiecode2025/sha256/aa193418be095b87634c712645456bd08ced9e416d6f1d24085ca098750d6180.webp)

Methodology watch outs.

Limitations & Improvements

This is an initial, exploratory approach, and going forward reproducibility will be key

• Briefing instructions
• Simulated audience dimensions
• Number of audit runs
• Parameters for task intent and task completion
• Number of tries for success/ failure

A test-and-control approach could help validate the scalability of this technique

• Human and AI teams operate in parallel
• Evaluate outputs side by side
• Look for complementary strengths
• Evaluate different industries geographies, and domains

© Hug[illegible]
24

### Final Thoughts

The speaker opens his conclusion with the challenge of repairing the AI trust gap.

[11:45](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=705s) [A black section slide reads “Final Thoughts,” with the presenter visible in a small video overlay.](https://assets.ai.engineer/sites/aiecode2025/sha256/919bca590e5a83707cb3bfb88466d65e4cda2cb8435b8612daab45a87bc110b9.webp)

Final Thoughts

© Hug[illegible]
25

### Repairing the Trust Gap

The speaker proposes using AI simulation to help design teams gather insights and create better websites, rather than adding chatbots that do not work.

[11:49](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=709s) [The conclusion proposes using AI simulation to help design teams gather insights and create better websites.](https://assets.ai.engineer/sites/aiecode2025/sha256/ea3c44351cfa86a78be705408f709096ba0121e51ca6cce5c282d38743548b6d.webp)

Repairing the Trust Gap

In Conclusion.

Users don’t need websites with chatbots that don’t work.

Users could benefit from better websites.

Empowering design teams to gather insights smarter, faster, and better through AI simulation could create websites that restore the trust gap.

© Hu[illegible]
26

## Full transcript

[0:00](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=0s) Hi, I'm Alex Liss, VP of Data Science and AI at Huge, design and technology company. And today I'm gonna talk about invisible users, invisible interfaces, which is how teams can use AI simulation to accelerate design.

[0:18](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=18s) We're gonna talk about three things today when it comes to the state of UX and AI. One, where we are now. Two, where we could be. And three, an idea for how to get there.

[0:32](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=32s) So in terms of where we are today, unfortunately, the reality is that AI has a trust gap. Some recent research by Edelman, which came out in December 2024, highlighted that only thirty-two percent of US adults say they trust AI, and only forty-four percent of adults globally say they feel comfortable with how businesses are using AI.

[0:54](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=54s) And the reason for that, unfortunately, is AI slop. And as these screen grabs show, uh, AI slop is when you go to use a website, a product, an interface, and there's a gen I- gen AI fail, uh, where it gives you something that you know is not correct, whether it's a search, a web search that tells you

[1:15](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=75s) you should eat rocks every day, or a website that tells you you can buy a car for one dollar. We know that neither of those things can be true.

[1:25](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=85s) Website creators are stuffing AI chatbots in everything and telling users it's magic. But the real magic of gen AI, uh, as, uh, Cassie Kozyrkov says, you know, it's, it is the fact that, that it's a UX revolution.

[1:41](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=101s) It's that users can actually talk to a machine learning model in natural language, which has never been possible before. So with that in mind, we w- w- really wanna think about the capability of that and go back to some UX first principles for us to figure out how we wanna use gen AI to help design.

[2:03](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=123s) So going back to how it started, you know, some writings by Don Norman of the Nielsen Norman Group talk about The Design of Everyday Things, talking about simplicity and efficiency.

[2:15](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=135s) Uh, and he proffered another great principle in his book, The Invisible Computer, which talks about invisible interfaces, which is software that feels so seamless and intuitive, the users practically forget that they're using it.

[2:30](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=150s) And I think the opportunity here is to use AI to design interfaces that actually feel like magic, not by stuffing chatbots into the website, but by accelerating needfinding. And this part about needfinding is in asterisks because I just wanna make it clear I'm not a designer.

[2:48](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=168s) Uh, you're not gonna see designs I produced today. I'm a data scientist. But you are gonna see a proposal for a process for how we could deliver this AI accelerated needfinding.

[3:01](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=181s) So let's talk about what a new approach to the design life cycle looks like, because in the world today, all of us are bombarded with so many screens, so much information, so much complexity.

[3:13](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=193s) It's a little bit akin to a pilot operating in a cockpit. And the way that pilots learn to su- learn to operate in this environment is actually through simulation, through a lot of practice.

[3:26](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=206s) And I think we can take this idea of simulation and evolve it and reinterpret it a bit. Our current design process, which comes from the pre-ChatGPT era, uh, is data-driven as designers collect, you know, qualitative observations, uh, quantitative data, uh, ethno- ethnographic observations, and they use that to guide the prototyping process.

[3:52](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=232s) But going forward, what if we could empower designers to work with invisible users in the form of AI simulation to turn the data artifacts that they've always collected as part of needfinding to active participants in the design process, and give designers their own mini feedback cycle to work with AI simulation as part of the broader

[4:16](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=256s) needfinding process to help deliver better design? So let's take a, ta- let's take a look at what an example of an accelerated needfinding process could look like through an example project.

[4:30](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=270s) So the new process for needfinding in the AI-- era of AI simulation has a lot of simul- similarities with the existing process. We're gonna start by defining our audience, then mapping their intentions, identifying specific tasks they will perform to accomplish those intentions, conducting analysis of that process, refining insights, and ultimately developing design alternatives.

[4:55](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=295s) However, in this era of AI simulation, there's gonna be a few key differences in some of the components and some of the workflow.

[5:04](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=304s) So when we start thinking about audiences in the area-- era of AI simulation, we wanna start with data that represents these audiences. Uh, and at my company, Huge, we have a data platform, which we call Live, uh, that has a mixture of data sets, demographics, psychographic, contextual, that allow us to simulate audience behaviors in the real world.

[5:28](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=328s) And this is a critical foundation to build on. Once we have this foundation, the next step is to actually apply intent mapping. And here's where we turn the data into something active, a simulation which we call an Intelligent Twin, which represents a set of user behaviors and a set of desired outcomes, their needs and motivation.

[5:50](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=350s) Intelligent Twins become an active participant in the design simulation process.

[5:56](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=356s) And from there, we can brief Intelligent Twins to evaluate interfaces by focusing on specific tasks, much as we might brief a human designer to conduct a heuristic analysis. And in this way, we're able to evaluate a specific interface, or in the case of this sample project, a specific website.

[6:17](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=377s) Now for the sample project I looked at, uh, I wanted to think about the advantages of this, this simulated methodology, the scale and speed. Uh, Intelligent Twins can operate with different levels of velocity than a human team, so I wanted to apply that to do a global, uh, audit of sports websites.

[6:39](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=399s) And we can imagine this happening from the perspective of a business that wanted to partner with sports leagues globally and learn how to cut across different countries, different cultures, uh, to engage audiences there.

[6:52](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=412s) And this is where, uh, the Intelligent Twin methodology could provide some unique advantages.

[6:59](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=419s) So in terms of the actual data collection, so, uh, with Intelligent Twins here in this sample project, I actually broke them out into two audiences, two personas, a casual fan newer to sport versus super fan who's lifelong and very savvy.

[7:17](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=437s) And across these three different websites, uh, we briefed a series of n-- of tasks across different categories like navigation, information architecture, and fan engagement, uh, with four tasks per category.

[7:31](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=451s) And that allows us to simulate 72 AI simulated actions, uh, to give, uh, to give us a test drive for, for how this AI accelerated design audit will perform.

[7:45](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=465s) When we get into the actual findings, so, uh, w- with the speed and the scale of Intelligent Twins, we can roll these insights up at a very high level or dive more granular, as we'll see in the next couple of slides.

[7:58](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=478s) But one of the benefits of this kind of global audit at speed is it gives us high level understandings across an entire category. And we can see from, from this audit that, uh, in the area of navigation, just kind of the starting place of a fan coming to the website, all the different leagues we looked at, um, you

[8:18](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=498s) know, basketball, the Olympics, and the English Premier League, all those sites performed pretty well off in terms of task completion when it comes to navigation. But as we get deeper into the process, as fans started to browse content and information architecture and engagement pathways, that's where those initial successes started to drop off.

[8:39](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=519s) And this goes back to the challenge I mentioned at the start of the presentation. Uh, there's an AI trust gap right now because AI is being shoved in people's face and websites, and it doesn't work very well.

[8:52](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=532s) But hopefully, this methodology could help us understand what user pain points are and actually solve those pain points, and in that way, start to repair that trust gap.

[9:05](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=545s) We can also go deeper with this methodology, so we can scale the simulation, uh, to surface friction across specific areas, specific nuances of the experience. Uh, the, the, the different levels of altitude this methodology can operate at allows us to create, uh, design briefs which are really focused, which are really broad and really deep as well.

[9:29](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=569s) And that's what we set up on this slide here, sort of closing out the audit process, uh, through AI acceleration, uh, computer use models, computer vision models, uh, human-in-the-loop observation.

[9:41](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=581s) We can use this process to generate a design brief that allows the human team to really focus in the pain points, uh, that are, that are distracting users in a specific category and try to solve them.

[9:55](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=595s) And in that way, uh, we can use AI simulation to, to hopefully accelerate and improve the design process.

[10:04](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=604s) As we look to the future, in the first half of 2025, there's been, uh, you know, tremendous progress. The, the MCP protocol from Anthropic is already showing potential integrations to turn prototype components in Figma through to actual code components in React or Node.js or other frameworks.

[10:25](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=625s) Uh, and this is gonna make it easier than ever to create new designs. And I think as the acceleration of these tools, um, just makes it easier to create design that really emphasizes the importance of focusing on the why, the strategy and the problem we wanna solve for users in the design process, because it'll be easier than

[10:48](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=648s) ever to solve for the how. In terms of this methodology, obviously this is experimental early stages, so there's some limitations and improvements to consider going forward. I think, uh, reproducibility is key, so as we evolve this methodology, my company and I will be standardizing things in a code repository that lays out parameters like the briefing

[11:13](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=673s) instructions, the simulated audience dimensions, the number of audit runs, and the parameters from task completion and failure. And also, uh, once we standardize this, applying it through a test and control methodology will also be useful to help us isolate what the strengths are of Intelligent Twins for design needfinding and how that can be used alongside or in

[11:37](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=697s) a complementary manner to human teams, especially applying to different industries, different de- geographies, and different domains.

[11:46](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=706s) In conclusion, just a few final thoughts here. Uh, when it comes to repairing the AI trust gap that we're struggling with at the moment, uh, I hope we can all re- recognize that users don't need more websites that have gen AI chatbots in them that don't work.

[12:05](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=725s) Uh, but users could always benefit from, from better websites, better mobile apps, better, better surfaces that provide clarity and simplicity. And hopefully, uh, I've been able to show in this talk today how AI simulation can be a tool to help us empower design teams to gather insights in a smarter, faster, and better process, and ultimately create websites

[12:29](https://www.youtube.com/watch?v=8D_VdU6DBhI&t=749s) and interfaces that can restore and repair the trust gap. Thank you.
