The End of the Static Screen: Architecting Intent-Driven UX — Gus Iwanaga, commercetools

Read the talk

The End of the Static Screen: Architecting Intent-Driven UX

Gus Iwanaga traces commercetools’ move from unpredictable model-composed screens to a declarative architecture where intent selects tools and components, schemas preserve the design system, and encoded layout knowledge guides how native interfaces come together.

From a talk by Gus Iwanaga

At a glance

Ideas worth remembering

  • Repeatedly asking for the same Q1 sales report exposed the failure of loosely guided composition: layouts, copy, and presentation of the reporting period changed between turns.

  • Choose how much control to delegate: let the agent select finished components, deliver generated HTML through an MCP tool into a sandboxed iframe, or describe native components through a declarative UI specification.

  • In the declarative architecture, intent classification drives tool calls, tool entities map to eligible components, and a UI description using a catalog and Zod schema renders as native React UI.

  • Design-system compliance does not solve information architecture. The team guides placement through templates, slots, nested sub-slots, and eligible component categories, mapping upward from the selected components.

  • The catalog becomes the contract between agent and interface, shifting design work toward schemas, properties, mappings, synthetic inputs, and interaction patterns. That change requires attention to people and process as well as the product.

Static software makes users carry the complexity

Gus Iwanaga, a general manager leading product, UX, and engineering for zero-to-one products at commercetools, opens by rejecting his own catchy title. The useful subject is narrower: lessons from trying to build generative UX and UI, with particular attention to the interface problems that orchestration alone does not solve. The goal is a working mental model for deciding how much of an experience a model should control. 0:42

The starting complaint is familiar to anyone operating several business applications: software exposes its internal organization and asks people to learn it. Each SaaS product brings a separate navigation model, feature hierarchy, and method for completing work. Adding more applications therefore adds more mental models. The cognitive load that software was supposed to absorb remains with the user. 2:42

The examples are information-dense CRM views, elaborate tables, and interfaces shaped by many accumulated features. Even a polished interface can demand substantial onboarding because newcomers must learn where capabilities live and how that particular application expects work to proceed. The problem extends beyond visual clutter: people repeatedly translate their goals into each application’s private logic. 4:12

That burden is especially visible at an API-first company with more than 300 APIs. The question that started the project was whether AI could change the interaction model itself rather than merely help teams ship more static screens. An intent-driven interface would begin with what the user wants to accomplish, then assemble the relevant capabilities instead of making the user navigate the software’s structure. 5:42

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:12 · section reference included

One request produced four incompatible reports

The first implementation failed in a revealing way. The input stayed fixed—create a sales report for Q1—but four turns produced four different experiences. The model selected components from a catalog and decided their placement and information architecture. One result crowded the page with KPI cards. Another changed the period’s wording from Q1 to January and March. A third added still more cards, charts, and text. The fourth returned to Q1 but remained confusing. 6:12

Recording frame at 477 secondsOpen full source frame
Recording frame at 477 seconds

The observable defect was inconsistency, but the causal chain matters. The system gave the model a component catalog, asked it to choose and arrange the experience, and left too many UX decisions open. Repeated requests changed the layout, the copy, and how the reporting period was presented. Personalization became disorientation: the user had to interpret a new screen every time despite asking the same question. Iwanaga refused to ship that experience to production. 7:12

The revised campaign-planning demo uses an orchestrator before rendering. It extracts intent from the request, identifies applicable first- or third-party tools, gathers their outputs, and supplies the resulting context to a UX agent. Those capabilities can include agents on MCP servers. The interface is still selected with AI, but composition is more guided: retrieved data and applicable capabilities inform the UX agent’s decisions. 8:42

After the user approves the proposed campaign, the demonstration proceeds in a pre-production environment. The shown flow establishes feasibility; reliability at scale remained an open question for the team. The improved appearance is a UX judgment about this demonstration, rather than a measured consistency gain across repeated requests. 9:42

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:12 · section reference included

Choose how much interface control to delegate

Generative UI is a spectrum of control rather than one architecture. The practical decision is how much variability the product can tolerate and which decisions must remain predictable. Three approaches illustrate the range. 10:42

Recording frame at 683 secondsOpen full source frame
Recording frame at 683 seconds
  • Controlled component: The product ships a finished component, and the agent decides when to display it. A restaurant-search component in ChatGPT is the example: its structure remains opinionated and predictable. This works well for bounded flows such as booking, but can become too prescriptive for highly configurable B2B work.
  • Open-ended generation: The model creates the experience itself. A request for a three-level organizational chart in Claude demonstrates the appeal: a short request yields a useful diagram. In the architecture described here, an MCP tool delivers HTML that renders inside a sandboxed iframe in a host such as a chat application. Sandboxing supplies a rendering environment; it does not decide whether the resulting experience meets the company’s UX expectations.
  • Declarative generation: The model emits a constrained UI description that maps onto known native components. This middle position allows the interface to adapt while retaining the product’s component catalog and design rules. 11:11

The declarative option is supported by UI protocols and libraries. Iwanaga discusses several options for this middle approach. They have different characteristics; the talk groups them by the control they offer rather than comparing their individual APIs. The commercetools team chose the declarative approach and continued testing these options. 14:41

Where does control over the interface live? The comparison follows the shift from product-authored components toward model-authored markup. Open-ended generation gives the model more freedom over the experience, which makes its outcome harder for the business to predict. The declarative middle retains a defined rendering vocabulary while allowing the model to select and compose the interface.

Where does interface control live?

Controlled componenthigh product control — The product defines the component; the agent selects when to show it.
Declarative UI specshared control — The model describes an interface using eligible native components.
Generated markuphigh model control — An MCP tool delivers HTML for rendering inside a sandboxed iframe.
The declarative middle allows composition while keeping the application’s native component vocabulary.
Read the diagram as text
  • Controlled component. high product control — The product defines the component; the agent selects when to show it.
  • Declarative UI spec. shared control — The model describes an interface using eligible native components.
  • Generated markup. high model control — An MCP tool delivers HTML for rendering inside a sandboxed iframe.
  • Controlled component → Declarative UI spec: allow more composition.
  • Declarative UI spec → Generated markup: delegate the rendered experience.
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:42 · section reference included

The declarative pipeline turns intent into native UI

The declarative path begins with the user’s request. The orchestrator classifies its intent, invokes the relevant tools, and retrieves their data. It maps entities in those tool results to components that are eligible for the task. Instead of sending generated markup to the browser, it broadcasts a structured UI description—a specification of what the interface should contain. 15:11

Recording frame at 931 secondsOpen full source frame
Recording frame at 931 seconds

The component catalog defines what that specification may reference, with a Zod schema describing the supported structure and properties. The description must comply with the chosen protocol, and the renderer turns it into native React components. This separates selection from implementation: AI participates in choosing the interface, while the application supplies the components that actually render it. 15:41

How does the campaign-planning request reach the screen? The flow connects the demo’s request to intent classification, tool data, eligible components, a UI specification, and native rendering. The important relationship is between business entities and interface components: tools supply the context that makes component selection meaningful, while the catalog and schema define the interface language available to the agent.

This preserves design-system compliance, but it does not make the experience deterministic. The model can still choose among eligible components and decide where to place them. The Q1 report also showed why copy belongs in the UX problem: changing how a period is described can confuse the user even when the components themselves look correct. A defined rendering vocabulary does not, by itself, supply good information architecture. That remaining gap leads to the hardest layout question: who arranges what the agent picked? 16:11

How does a campaign request become native UI?

Plan a campaignThe user describes the intended task.
Intent classificationorchestration — The orchestrator identifies applicable tools.
Tool executionorchestration — First- or third-party capabilities return data and context.
Entity-to-component mappingUI planning — Tool entities map to eligible catalog components.
UI descriptionUI planning — The orchestrator broadcasts a spec using the catalog, Zod schema, and protocol requirements.
Native React UIapplication — The application renders its design-system components.
Tool entities map to eligible catalog components before a protocol-compliant UI description renders as React components.
Read the diagram as text
  • Plan a campaign. The user describes the intended task.
  • Intent classification. orchestration — The orchestrator identifies applicable tools.
  • Tool execution. orchestration — First- or third-party capabilities return data and context.
  • Entity-to-component mapping. UI planning — Tool entities map to eligible catalog components.
  • UI description. UI planning — The orchestrator broadcasts a spec using the catalog, Zod schema, and protocol requirements.
  • Native React UI. application — The application renders its design-system components.
  • Plan a campaign → Intent classification: classify request.
  • Intent classification → Tool execution: select and invoke.
  • Tool execution → Entity-to-component mapping: return entities and context.
  • Entity-to-component mapping → UI description: describe eligible UI.
  • UI description → Native React UI: render native components.
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:11 · section reference included

Layout knowledge becomes executable

To control arrangement, the team borrowed the hierarchy of atomic design and taught the UX agent what good layouts look like for different situations. A template catalog provides the larger structures. Each page layout contains slots such as a header or main region; slots may contain nested sub-slots; and those sub-slots accept particular categories of components. The hierarchy gives the system a way to guide placement instead of leaving it entirely open. 17:41

Recording frame at 1079 secondsOpen full source frame
Recording frame at 1079 seconds

The design hierarchy moves downward: layout → slot → sub-slot → component. The orchestrator starts with the components eligible for the current intent, so composition goes upward instead: component → sub-slot → slot → template. The system begins with the pieces needed to answer the request and maps them into the larger layout structure. This codifies the team’s UX knowledge in the agent’s composition process. Arrangement remained an ongoing challenge despite this mitigation. 18:41

How can selected components lead to a page layout? The upward mapping makes the dependency visible: component categories guide their placement in sub-slots, sub-slots connect to slots, and slots connect to templates. That relationship explains why layout metadata matters alongside the visual component implementation. The catalog must describe where a component fits as well as what it looks like. 19:11

How are agent-selected components fitted into a page?

Eligible componentsThe orchestrator selects components suitable for the user’s intent.
Sub-slotslayout hierarchy — Sub-slots accept eligible component categories and can be nested.
Slotslayout hierarchy — Larger regions include a header or main region.
Page templatelayout hierarchy — The template catalog provides overall layout structures.
Composition reverses the design hierarchy, mapping selected components upward into regions and templates.
Read the diagram as text
  • Eligible components. The orchestrator selects components suitable for the user’s intent.
  • Sub-slots. layout hierarchy — Sub-slots accept eligible component categories and can be nested.
  • Slots. layout hierarchy — Larger regions include a header or main region.
  • Page template. layout hierarchy — The template catalog provides overall layout structures.
  • Eligible components → Sub-slots: map eligible categories.
  • Sub-slots → Slots: map upward.
  • Slots → Page template: map to template.
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

17:41 · section reference included

Design work moves from pixels to rules

The second unresolved challenge is curation. Once agents compose interfaces through a catalog, that catalog becomes the contract between the agent and the UI. Every property matters—not only component attributes, but also the attributes of layouts, slots, and sub-slots. These definitions give the team a way to steer the generated experience. Testing UI protocols remains part of the work, but the protocol cannot substitute for a carefully described catalog. 20:11

Recording frame at 1259 secondsOpen full source frame
Recording frame at 1259 seconds

Curation is what lets the product offer useful flexibility while keeping UX guidance in the system. The agent needs descriptions of the available components and their properties, and the layout hierarchy needs its own attributes. Without that work, supplying a library of attractive components still leaves composition decisions underspecified—the problem exposed by the four Q1 reports. 20:40

The team consequently no longer designs every pixel and entire flow in advance. Product managers and UX designers now discuss a different set of materials:

  • Schemas and catalog rules: Define the properties and choices available during composition.
  • Synthetic data and representative queries: Develop inputs that map to particular components through the mapping logic.
  • Interaction patterns: Guide how users work with the generated experience.

Design judgment moves into the definitions that shape what the agent can generate. 21:11

That shift is organizational as well as technical. For nontechnical PMs and UX designers, discussing schemas, rules, and synthetic inputs was a substantial change in the nature of their work. Iwanaga closes with three concerns—people, product, and process—and calls for a lightweight process that takes the people element seriously. The interface can adapt to intent only if the team also learns to express its judgment in the catalog, mapping logic, and layout system. 22:11

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

20:11 · section reference included

Resources

  • A first-party explanation of the product’s orchestration and generative-UI pillars, with a campaign example that follows a user request through configuration, approval, autonomous operation, and monitoring.

Transcript & slides

Loading transcript and slides…

Transcript download