# Act, Confirm, or Stop? Smarter behavior for AI assistants, wearables & robots — Amit Desai, Roku

Amit Desai, Roku

[Talk page](https://ai.engineer/talks/Zd5b40Jbp_k-act-confirm-stop-smarter-behavior-ai-assistants) · [Video](https://www.youtube.com/watch?v=Zd5b40Jbp_k)

## Act, Confirm, or Stop? Smarter Behavior for AI Assistants, Wearables & Robots

Amit Desai shows how an assistant can reduce the pain of errors without improving recognition accuracy: assign a recovery cost to each possible outcome, then optimize when the system should act, ask for confirmation, or decline.

### Errors get more expensive when AI acts

[0:12](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=12s)

Amit Desai, credited with Roku for this recording, starts from a tension familiar to anyone building voice interfaces. Talking is natural, but speech systems remain error-prone. That becomes more consequential as assistants move from answering questions to taking digital or physical actions. Playing the wrong song is irritating; a robot throwing away your watch is materially worse.

This creates two independent controls over the experience. The familiar control is **accuracy**: improve wake-word detection, speech recognition, language understanding, intent classification, entity extraction, or another part of the interpretation stack. The second is the **system decision under uncertainty**: given a hypothesis that might be wrong, should the assistant act, confirm it, or stop? Better decisions can reduce user pain even when the underlying interpretation remains unchanged.

### Freeze accuracy at 79 percent

[3:43](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=223s)

The worked example is a simple music speaker. Collect 1,000 spoken requests, compare each requested song with what the system selected, and label the results. In the illustrative dataset, 790 hypotheses are correct and 210 are wrong: 79 percent accuracy. Conventional model work would try to shrink those 210 errors one percentage point at a time.

![The intuitive but arbitrary 65 percent threshold provides the comparison policy.](https://ai.engineer/api/talk-essay-media/2e20809039dadf954eeb44618ec83cccf5259a67a180f15f30fd06ad4f02f684)

The intuitive but arbitrary 65 percent threshold provides the comparison policy.

Instead, keep all 1,000 interpretations exactly as they are. The baseline assistant always acts, so every hypothesis immediately becomes playback. Desai adds one alternative: reject the hypothesis and ask the user to repeat. That sounds obviously safer, but it creates a harder quantitative question—exactly when should the system stop?

For the simplified exercise, every labeled hypothesis receives one reasonably calibrated confidence score between zero and one. A threshold `T` implements the policy: stop when confidence `C < T`; otherwise play. A cascaded production system may expose several confidence signals rather than one clean score, so this is a teaching model rather than the proposed final architecture.

A threshold such as 65 percent feels plausible: perhaps the system should act only when it is “confident enough.” But confidence alone does not identify the best decision. Moving the threshold prevents some wrong plays while also rejecting some requests the system understood correctly. The right boundary depends on what those outcomes cost the user.

### Count recovery effort, then minimize the OUCH

[7:41](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=461s)

The concrete failure is a request for “Kiss” by Prince that instead starts a Chris Brown song. The observable problem is not merely an incorrect label. Music begins playing; the user listens long enough to detect the error, talks over the speaker to stop it, and makes the request again. A refusal—“Sorry, I didn’t understand”—still delays success, but it avoids the work of undoing an incorrect action.

![The graph reaches roughly 1.27 OUCH per turn without any change to interpretation accuracy.](https://ai.engineer/api/talk-essay-media/a076c59a955c09e8a75ee17090c18aed12854e8bbf38d081c745bfed203cfb32)

The graph reaches roughly 1.27 OUCH per turn without any change to interpretation accuracy.

Desai turns that difference into a recovery-cost heuristic: estimate the additional time required to reach the intended result. He assigns 10 seconds to wrong playback and four seconds to stopping and repeating. These values are illustrative rather than measured universal costs; changing them, or changing the confidence distribution, changes the optimum.

For any candidate threshold, the cost is therefore `10 × wrong acts + 4 × stops`. Correct immediate playback contributes no recovery cost. Raising the threshold generally reduces wrong actions but creates more refusals, including refusals of correct hypotheses. The weighted sum makes that tradeoff explicit instead of hiding it inside an intuitive confidence cutoff.

Desai names the objective the **Outcome User Cost Heuristic**, or **OUCH**—a language nerd’s cost function for pain. With the original always-act policy, 210 wrong plays cost 2,100 points across 1,000 requests, or 2.1 per turn. A guessed 65 percent stop threshold lowers that to 1,904, or about 1.9 per turn. Searching the demonstrated curve finds a much better threshold at 43 percent, reducing the average to approximately 1.27 OUCH points per turn.

Nothing about the 790 correct and 210 incorrect interpretations has changed. The improvement comes entirely from routing uncertainty into a less costly behavior. Accuracy describes whether the hypothesis is right; the decision policy determines how much damage a possibly wrong hypothesis can cause.

### Add a confirmation region between stopping and acting

[12:04](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=724s)

The next behavior is confirmation. Instead of immediately playing or refusing, the speaker can state its interpretation—such as “Play ‘Kiss’ by Prince?”—and let the user affirm or correct it. This splits confidence into three regions governed by two thresholds: stop at low confidence, confirm in the middle, and act at high confidence.

![Two thresholds divide confidence into the three visible regions: stop, confirm, and act.](https://ai.engineer/api/talk-essay-media/ae81180def6499dbd601541bab0e1eed16a7f95d62072229ee6f333d484f2fff)

Two thresholds divide confidence into the three visible regions: stop, confirm, and act.

What does that three-way policy make visible? Low-confidence hypotheses are rejected before they cause damage, middling hypotheses pay the smaller cost of a question, and high-confidence hypotheses preserve the fast path. The two boundaries jointly determine how many correct and incorrect requests land in each behavior.

Routing one hypothesis by confidence
Two optimized boundaries divide the example’s confidence range into stop, confirm, and act regions. The policy changes behavior, not whether the original hypothesis was correct.
- Song hypothesis + confidence — The recognizer produces one proposed song and a simplified calibrated confidence score.
- Stop: C < T1 — Reject the hypothesis and ask the user to repeat; illustrative cost: 4.
- Confirm: T1 ≤ C < T2 — Read back the proposed interpretation; affirmation costs 2 and correction costs 6 in the example.
- Act: C ≥ T2 — Play immediately; an incorrect act carries the example’s largest cost, 10.
- Optimize T1 and T2 together — The demonstrated optimum is 41 percent and 49 percent for the selected data and costs.
- Song hypothesis + confidence → Stop: C < T1 (Low confidence)
- Song hypothesis + confidence → Confirm: T1 ≤ C < T2 (Middle confidence)
- Song hypothesis + confidence → Act: C ≥ T2 (High confidence)
- Stop: C < T1 → Optimize T1 and T2 together (Contributes stop cost)
- Confirm: T1 ≤ C < T2 → Optimize T1 and T2 together (Contributes yes/no costs)
- Act: C ≥ T2 → Optimize T1 and T2 together (Contributes wrong-action cost)

### Confirmation helps only when its own cost is modeled

[13:33](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=813s)

Confirmation is not free. Even a correct read-back delays playback, while an incorrect proposal forces the user to reject it and restate the request. The example assigns two seconds to affirmation and six seconds to correction, producing the extended objective `10 × wrong acts + 4 × stops + 2 × affirmations + 6 × corrections`. Because two thresholds must now be selected together, the cost landscape becomes a two-dimensional heat map.

![Closing instruction summarizes the talk: minimize the OUCH of the experience.](https://ai.engineer/api/talk-essay-media/5ef5aef1b6470e5fc75d3f34671771f9d71dc68a20e41284bb4bdcdade03fdaa)

Closing instruction summarizes the talk: minimize the OUCH of the experience.

For the demonstrated data and costs, the optimized boundaries are 41 and 49 percent. The recorded display distinguishes a current 30/60 configuration costing 1,464 in total from the optimized 41/49 configuration costing 1,260. The recap expresses the progression as 2.100 OUCH points per turn for always acting, 1.904 for the guessed stop threshold, 1.274 for optimized act-or-stop behavior, and 1.260 after optimized confirmation. Confirmation improves the result again, although much less dramatically than introducing and optimizing the stop behavior.

The thresholds are not reusable constants. If a wrong action costs 20 instead of 10, the optimum moves because preventing incorrect execution becomes more valuable. It also moves when the confidence distributions change. OUCH is therefore a way to derive a policy from a system’s observed outcomes and product-specific costs, not a recommendation that every assistant act above 49 percent confidence.

A production assistant would not necessarily use one or two offline scalar thresholds. Desai expects a learned real-time decision model that can consider richer signals. The simplified exercise supplies the objective: choose behavior under uncertainty according to expected user cost while keeping the interpretation model and the behavioral policy conceptually separate.

A television shows why the interface changes the equation. For “open a channel,” confirmation can present several visual choices instead of speaking one candidate. Selecting the right option with a remote may be cheaper than a spoken correction, so confirmation’s cost falls. But launching the wrong channel may eject the viewer from their current state, making an incorrect act more expensive.

The ending returns to higher-stakes actions: making a phone call, sending an email, or moving an object. As consequences rise, accuracy improvements alone become an increasingly narrow response to uncertainty. The assistant also needs a policy for when to proceed, seek permission, or decline. If that interface remains painful, it becomes the choke point even while the models behind it improve. Desai’s memorable instruction is the pun that carries the whole mechanism: minimize the OUCH of the experience.

## Key takeaways

- Interpretation accuracy and behavior under uncertainty are separate controls. The example keeps accuracy fixed at 79 percent while changing which hypotheses the assistant executes.
- Choose thresholds by minimizing weighted recovery cost, not by selecting a confidence percentage that merely feels safe.
- Wrong actions, refusals, affirmations, and corrections impose different costs. Treating them as equivalent hides the product decision that matters most to users.
- Adding confirmation creates a useful middle region, but it helps only when its delay and correction costs are included in the objective.
- Outcome costs must reflect the actual interface and action. Visual choices can make confirmation cheaper, while disruptive or irreversible actions make incorrect execution more expensive.

## Resources

- [Official AI Engineer talk page and corrected transcript](https://ai.engineer/talks/Zd5b40Jbp_k-act-confirm-stop-smarter-behavior-ai-assistants) — Watch the recording alongside its chapter markers, corrected transcript, numeric clarification, and discussion prompts.

## Related talks

- [Building Effective Voice Agents](https://ai.engineer/talks/-OXiljTJxQU-building-effective-voice-agents) — Extends the discussion from decision policies into production voice architecture, including latency, determinism, tool use, handoffs, and evaluation.
- [Designing Voice Agents for Real Conversations](https://ai.engineer/talks/hMlLw1LeIK8-designing-voice-agents-real-conversations) — Explains another behavior-under-uncertainty problem: deciding whether a pause means the user has finished speaking.
- [Bounded Autonomy: Between Free Will and Determinism](https://ai.engineer/talks/t4359sKBu4w-bounded-autonomy-between-free-will-determinism) — Develops the adjacent idea that agent autonomy should be constrained by context, curated knowledge, and short real-world feedback loops.

## Slides

### Act, Confirm, or Stop?

Amit Desai introduces a talk on smarter behavior for AI assistants, wearables and robots. A row of icons connects speech, a brain, a question and a person.

[0:20](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=20s) [Title slide introducing smarter behavior for AI assistants, wearables and robots, presented by Amit Desai.](https://assets.ai.engineer/sites/aiecode2025/sha256/2e8abef4d5907707a1852d8748e2cd5346d801036375bab58f3180746a312c2b.webp)

Act, Confirm, or Stop?

Smarter behavior for AI assistants, wearables & robots

Amit Desai

### My voice background

Amit Desai draws on voice AI work at Alexa, Roku and startups. He combines voice user interface expertise with new technical approaches to create compelling experiences.

[0:25](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=25s) [Speaker background in voice AI products and interfaces, with illustrations of voice-enabled devices.](https://assets.ai.engineer/sites/aiecode2025/sha256/bde0b910d6872f11e28f08f4f79f784ffcab90da10737bdd314b08de5c471598.webp)

My voice background

• SME in voice AI assistants, conversational UI, NLU, ASR, TTS, LLM, handsfree
• Led voice AI products and teams at Alexa, Roku, voice startups, App Store top 5
• Worked on voice-first speakers, displays, TV, mobile apps, customer service agents, ...

• Expert VUI (user interface) + new technical approaches = compelling experience!

### The power of voice

Voice offers a natural, hands-free interface across devices. A speaking person sits at the center of an ecosystem that includes phones, TVs, glasses, cars and robots.

[1:21](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=81s) [Voice is presented as a natural, hands-free interface across devices, illustrated by a person surrounded by device icons.](https://assets.ai.engineer/sites/aiecode2025/sha256/0085f92af8cd7a1533478b6ab0fd74efc6bb8c315aad6e89190fb646a4cead82.webp)

The power of voice

• Voice is our most natural interface – zero learning curve, hands-free, common UI
• AI Assistants: speaker, phone, TV, glasses, watch, call center, robot (and more)

### The pain of voice – error prone & growing

Voice errors can arise from recognition, intent, language models, wake words or mis-speech. Desai argues their consequences grow when assistants take physical action: a robot throwing away a watch is worse than playing the wrong song.

[1:40](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=100s) [Sources of voice errors and growing consequences are listed above an illustration of a distressed user surrounded by devices.](https://assets.ai.engineer/sites/aiecode2025/sha256/8b7b56f26b250407627e9dede10046fde29c6d3a744e4e8ff91267a5d206d9f5.webp)

The pain of voice – error prone & growing

• Accuracy is a problem even with today’s simple AI assistants (e.g. smart speakers)
• Many sources: ASR, NLU intent/entities, LLM, wakeword, user mis-speech
• Growing pains: worse consequences (embodied AI) and more demands (ambient)

### Smarter system decisions and user satisfaction

Desai proposes improving voice assistant satisfaction through system decisions under uncertainty, alongside accuracy. He suggests minimizing user costs can improve satisfaction even when accuracy stays unchanged.

[2:24](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=144s) [Summary proposes improving voice assistant satisfaction through accuracy and smarter system decisions under uncertainty.](https://assets.ai.engineer/sites/aiecode2025/sha256/063301462d84d3c901bd0299e8215f6f6019d599e8b17e0e8967e1e252a44aba.webp)

TLDR: Voice AI assistants can increase user satisfaction with smarter system decisions – not only accuracy

• Voice errors reduce user adoption of AI assistants, wearables, robots. Embodied AI increases the cost of errors: digital → physical.
• How to improve on voice errors?
  • Knob 1: Accuracy
  • Knob 2: System decisions - smarter conversational behavior under uncertainty; optimize system decisions to minimize user-cost function (aka O.U.C.H. :) Increase user satisfaction without changing accuracy
• Practical Example – smart speaker walkthrough
• Scalable across all voice AI assistant surfaces, devices, UIs

### Simple model: smart speaker

A music-only smart speaker hears a song request, thinks, and acts, playing either the right song or the wrong song.

[3:25](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=205s) [A music-only smart speaker processes a request through speak, hear, think and act stages, resulting in the right or wrong song.](https://assets.ai.engineer/sites/aiecode2025/sha256/256cb6892f8581833cd858d531650d0c57f28c4d26ec4121f92dfcd56639281d.webp)

Simple model: smart speaker (Alexo :)

• Alexo: smart speaker for music only
• User asks to play a song X

Speak → Hear → Think → Act
right song
wrong song

### Knob 2: Improve system decisions

At the same 79% accuracy, the system can use confidence and the user cost of outcomes to make decisions under uncertainty that aim for higher satisfaction.

[4:56](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=296s) [Change system behavior under uncertainty using confidence and user cost to improve satisfaction at the same accuracy.](https://assets.ai.engineer/sites/aiecode2025/sha256/2d8d9f4467c6f1787c38d64906dd7fe3bf4643dd7a7fb0a4754608a23eb95622.webp)

Knob 2: Improve system decisions

• Change system behavior under uncertainty
• Optimize system decisions to minimize user-cost function (aka O.U.C.H. :)
  1. Uncertainty (confidence)
  2. Outcome user cost
• Same accuracy → higher user satisfaction

### Original – system acts

Alexo hears a song request, processes it, and plays a song. The original system acts on every request, producing either the right song or the wrong song.

[5:15](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=315s) [A smart speaker processes a request through speak, hear, think, and act stages, producing either the right or wrong song.](https://assets.ai.engineer/sites/aiecode2025/sha256/28783ddf32c4f77bd85d7947eccf30ded704c38b55c7b3c52e4cce2793b78c6d.webp)

Original – system acts

• Alexo: smart speaker for music only
• User asks to play a song X

Speak → Hear → Think → Act

right song
wrong song

### Add system behavior: Stop

Desai proposes letting Alexo stop and apologize instead of playing a potentially wrong song. The challenge is deciding quantitatively when to stop.

[5:23](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=323s) [The system can act and play a song or stop and apologize when it cannot understand the request.](https://assets.ai.engineer/sites/aiecode2025/sha256/2704d75be87b27bfbc63dfdce96dc8fbf03d25029ead3a758fe578c66e213e39.webp)

Add system behavior: Stop

• System can decide to Stop (don’t play anything)
• Alexo: “Sorry, I didn’t get that”

Decide
Act
right song
wrong song
Stop
STOP
Sorry...

• Intuitively: stop is better than playing wrong song
• How do we decide to Stop? (quantitively)

### Stop or Act? — set threshold T

The system stops when confidence falls below T and plays otherwise. The right- and wrong-song distributions overlap, raising the question of how to choose T instead of guessing 65%.

[6:43](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=403s) [A graph shows correct and incorrect song distributions and a threshold separating stop from act.](https://assets.ai.engineer/sites/aiecode2025/sha256/5e56023729a8ba661bf4d9a1cbbc983a5aebcd71efe3ef2349d7381b860f4802.webp)

Stop or Act? – set threshold T

• Set a confidence threshold T
• System stops if Cᵢ if < T, otherwise acts
• How do we choose T?
• (Wing it... T = 65%? :)

threshold = T%
stop | act
STOP
Right song (790)
Wrong song (210)
X-axis ticks: 0%, 20%, 40%, 60%, 80%, 100%
C, confidence score [0-1] (%)

### Stop or Act? — choosing T

Any threshold creates two bad outcomes: stopping a request and playing the wrong song. The circled red tail shows wrong songs that pass the threshold.

[7:20](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=440s) [A threshold graph highlights bad acts and lists stops as another undesirable outcome.](https://assets.ai.engineer/sites/aiecode2025/sha256/ecb4b7651b19fc12c70e6d30622cd99100a124e5436be35f5ea76167e4a61b5b.webp)

Stop or Act? – choosing T

• Any T creates bad outcomes
1. Bad Acts
2. Stops (all)

T
stop | act
STOP
Right song (790)
Wrong song (210)

X-axis ticks: 0%, 20%, 40%, 60%, 80%, 100%
C, confidence score [0-1] (%)

### Introduce: outcome user cost

Playing the wrong song has the highest user cost: the listener must recognize the mistake, talk over the music, and request again. Stopping still requires another request; playing the right song succeeds.

[7:49](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=469s) [A decision tree compares the user costs of a correct song, a wrong song, and stopping.](https://assets.ai.engineer/sites/aiecode2025/sha256/cfa1188d773ebed30e059689e3df1530244c12f4946ba7529f02fe2d6783b3c4.webp)

Introduce: outcome user cost

• Bad outcomes vary in badness to users – some are worse!
• What is the user cost for each outcome?

Decide
Act
right song
Zero user cost: success!

wrong song
Bad Act = highest user cost: user must recognize song is wrong, talk over music, re-request

Stop
STOP
Sorry...
Stop = medium user cost: user must re-request

### Outcome User Cost Heuristic (OUCH)

OUCH assigns illustrative costs based on the extra time needed to get the right song: 10 seconds to recover from a wrong song, 4 seconds after a stop, and zero for success.

[8:42](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=522s) [A decision tree assigns zero extra seconds to success, 10 seconds to a wrong song, and 4 seconds to stopping.](https://assets.ai.engineer/sites/aiecode2025/sha256/c9338361efc648ad8237bfd9c36770f5c2e4a5080f71e312fe74b75966f3833c.webp)

Outcome User Cost Heuristic (OUCH)

• Define unit cost per outcome
• Heuristic: additional time (sec) required by user to play right song

Decide
Act
right song
0 extra (success)
wrong song
10 seconds extra for user: recognize song is wrong, talk over music, request again
Stop
STOP
Sorry...
4 seconds extra for user: re-request again

### Choose T to minimize total OUCH (user cost)

The threshold T separates stops from acts. Choose it to minimize estimated user time: 10 seconds per bad act plus 4 seconds per stop, while keeping the same accuracy.

[9:21](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=561s) [The slide defines total user cost as 10 times bad acts plus 4 times stops and recommends choosing the threshold that minimizes it.](https://assets.ai.engineer/sites/aiecode2025/sha256/f1a3bd45eed52d480e88e41e1f24cd09e524ba12eddecd157336ee16519d68c9.webp)

Choose T to minimize total OUCH (user cost)

• Total user cost = (# of Bad Acts)·(10) + (# of Stops)·(4)
• Choose T to minimize OUCH (user cost function)
• Improved user experience → same accuracy

stop | act
T
STOP
Wrong song (210)
Right song (790)
X-axis ticks: 0%, 20%, 40%, 60%, 80%, 100%
C, confidence score [0-1] (%)

### Add system behavior: Confirm

The assistant can confirm a song request, letting the user affirm or correct it. The diagram adds confirmation alongside acting and stopping.

[12:20](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=740s) [A decision diagram branches to act, confirm, or stop; confirmation allows the user to affirm or correct a song request.](https://assets.ai.engineer/sites/aiecode2025/sha256/8ba9386b3b0a2a2290f3938552ddca5c99cbea526ebfbf377854ac80f0bfc461.webp)

Add system behavior: Confirm

• System can Confirm: “Did you mean, Kiss by Prince?”
  • Yes
  • No + correction (“Kiss Me”)

Decide
Act
right song
wrong song
Confirm
X?
Stop
STOP
Sorry...

Interface text: CC

### Stop, act or confirm thresholds

Two confidence thresholds divide decisions into stop, confirm and act regions. The right- and wrong-song distributions overlap; Desai proposes using user outcome costs to optimize the thresholds.

[12:55](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=775s) [Confidence distributions divided into stop, confirm and act regions by two thresholds.](https://assets.ai.engineer/sites/aiecode2025/sha256/453778b49b55791270a18db1ca60151fd16934c82802af7fe5eb4e71ac654eb4.webp)

Stop, Act or Confirm? – optimize decision

Set 2 thresholds:
• Stop if Cᵢ < T-stop
• Act if Cᵢ > T-act
• Confirm otherwise

T-stop
T-act
stop
confirm
act
Wrong song (210)
Right song (790)
X-axis ticks: 0%, 20%, 40%, 60%, 80%, 100%
C, confidence score [0-1] (%)

Interface text: CC

### Confirmation outcomes

A confirmation asks the user to approve the proposed song or correct it. Both slow the user down; rejection also requires restating the request.

[13:57](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=837s) [A decision diagram shows act, confirm and stop paths, including confirmation and correction outcomes.](https://assets.ai.engineer/sites/aiecode2025/sha256/7530948976479ed2ef510e2a88e05ccba8ce111438ffc2f395e477d664b1aeca.webp)

Add system behavior: Confirm

• Systems confirms: “Did you mean, Kiss by Prince?”
  • Yes
  • No + correction (“Kiss Me”)

Decide
Act
right song
wrong song
Confirm
X?
(Yes)
(No, --- )
Stop
STOP
Sorry...

Confirm-Yes: user must process + Yes
Confirm-No: user must process + say request again

### Confirmation user cost

The heuristic assigns confirmation an extra 2 seconds for saying yes and 6 seconds for saying no and repeating the request.

[14:04](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=844s) [The confirmation diagram assigns two extra seconds to yes and six to no and correction.](https://assets.ai.engineer/sites/aiecode2025/sha256/474497706f8e3a1a02c67e9702024704e8a2bbb1477541fa75bcff6cab83b46b.webp)

Confirm: outcome user cost (OUCH)

• Define unit cost per outcome
• Heuristic: additional time (sec) required for success

Decide
Confirm
X?
Stop
STOP

right song
wrong song
(Yes)
(No, --- )
Sorry...

2 sec extra: user must process + Yes
6 sec extra: user must process + say request again

Interface text:
CC

### Minimize total user cost

Two confidence thresholds divide decisions into stop, confirm and act regions. The cost function weights bad acts at 10, stops at 4, and confirmation outcomes at 2 or 6.

[14:33](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=873s) [A weighted cost formula for bad acts, stops and confirmation outcomes accompanies the threshold graph.](https://assets.ai.engineer/sites/aiecode2025/sha256/d833dbe73f422984d12d2a51fb8043f7ee9f833a6fe1cf72edca3418ade41654.webp)

Choose T₁,T₂ to minimize total OUCH (user cost)

• Total user cost, OUCH =
(# of Bad Acts)·(10) + (# of Stops)·(4) + (# of Confirm-Yes)·(2) + (# of Confirm-No)·(6)
• Choose T-stop, T-act to minimize user cost function

T-stop    T-act
stop    confirm    act
Wrong song (210)
Right song (790)
X-axis ticks: 0%, 20%, 40%, 60%, 80%, 100%
C, confidence score [0-1] (%)

Interface text: CC

### Smarter decisions reduce user cost

In this simplified example, optimizing stop and confirm decisions lowers average user cost from 2.100 to 1.260 per turn without changing accuracy. Real systems would use learned decision models at runtime.

[16:10](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=970s) [A summary compares average user cost across four decision strategies, from 2.100 to 1.260 per turn.](https://assets.ai.engineer/sites/aiecode2025/sha256/816b899eb7078673975ef90ffd39544aff3b25d13cfb217277290317ced4ff86.webp)

Summary: smarter system decisions can increase user satisfaction (without changing accuracy)

Average user cost/turn drops as system behavior considers outcome user-cost (OUCH)

1. Naïve approach (Act only) → average 2.100 user cost/turn
2. Act + Stop (guesstimate) → average 1.904 user cost/turn
3. Act + Stop (OUCH-optimized) → average 1.274 user cost/turn
4. Act + Stop + Confirm (OUCH-optimized) → average 1.260 user cost/turn

• Note: Simplified example – real AI assistant system won’t use offline thresholds for decisions; learned decision models operating in runtime, etc.

style: bullets · layout: visual below

### Applying decisions across voice AI surfaces

Desai expects the decision principle to transfer from smart speakers to other voice AI devices, including phones, TVs, wearables and robots, with differences across devices.

[16:48](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1008s) [A person and speech bubbles are surrounded by illustrations of voice-enabled devices.](https://assets.ai.engineer/sites/aiecode2025/sha256/00a9f29f02f1102b908f3081042aa932943d9922319c090535d9a0a84b2c30aa.webp)

Scalable: apply across voice AI surfaces

• Voice is our most natural interface – zero learning curve, hands-free, common UI
• AI Assistants: speaker, phone, TV, glasses, watch, call center, robot (and more)

style: bullets · layout: visual below

### TV AI assistant example

For an “ABC” channel request, the assistant can act, confirm, or stop. The TV can show choices for confirmation; the example assigns a higher cost to a wrong action than to confirmation or stopping.

[16:58](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1018s) [A channel-opening request branches into act, confirm, and stop outcomes with example user costs.](https://assets.ai.engineer/sites/aiecode2025/sha256/6a8c827a25177094fba85f352a0a48eb92c922e9cb6a9cd26af8544cfc93d8b6.webp)

Example: TV AI assistant

• Action: Open a channel

“ABC”

Decide
Act
right  0
wrong  10
Confirm
X?
(Yes)  1
(No, ---)  5
Stop
STOP
Sorry...  4

OUCH

Embedded screenshot: Pr

Interface text: CC

## Full transcript

[0:01](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1s) [music]

[0:12](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=12s) Hi everyone. How's it going? Hey

[0:15](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=15s) Patricia, how are you?

[0:17](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=17s) >> Uh so last presentation of the day, so

[0:20](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=20s) let's make it count. Um,

[0:23](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=23s) all right. Let's, uh, let me start with

[0:25](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=25s) a little bit of background on myself.

[0:28](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=28s) And, um, my background, I'm a voice

[0:33](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=33s) subject matter expert. I've been working

[0:35](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=35s) in voice AI for a long time across

[0:37](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=37s) different surfaces, devices, and um,

[0:42](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=42s) both at Alexa, at at Roku, at my own

[0:46](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=46s) startups, you know, in the app store.

[0:49](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=49s) And my perspective is a little different

[0:52](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=52s) from a lot of other voice AI

[0:55](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=55s) practitioners. I think it's a

[0:58](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=58s) combination of um a deep um voice user

[1:02](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=62s) interface expertise and intu intuition

[1:05](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=65s) mixed in with new technical approaches

[1:09](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=69s) uh that I think can produce really

[1:11](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=71s) magical experiences. So I think it's

[1:12](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=72s) both sides and I think that's especially

[1:15](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=75s) true in this new area that we're in with

[1:17](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=77s) frontier tech where the human interface

[1:20](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=80s) is basically being redefined. So let me

[1:24](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=84s) start with uh I'll just blast through

[1:26](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=86s) the first couple of slides then get to

[1:28](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=88s) the premise. I think everybody knows

[1:30](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=90s) that voice has incredible potential.

[1:32](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=92s) There's the power of voice I think

[1:35](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=95s) across everywhere. It's the most natural

[1:37](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=97s) interface. Humans love talking. And uh

[1:40](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=100s) the problem is the other half is the

[1:43](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=103s) pain of voice. So it's the power and the

[1:44](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=104s) pain. Voice is errorprone. And I think

[1:49](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=109s) those errors are going to continue for a

[1:52](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=112s) while. And I think the cost or

[1:54](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=114s) consequence of those errors is going to

[1:56](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=116s) grow, especially as we go fromational

[1:59](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=119s) AI bots to embodied AI where rather than

[2:03](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=123s) just giving answers that might be

[2:05](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=125s) erroneous,

[2:07](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=127s) we're going to have AI systems take

[2:09](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=129s) physical actions or digital actions

[2:12](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=132s) where, you know, if the robot throws

[2:15](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=135s) your watch out with the trash, it's a

[2:17](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=137s) lot worse than playing the wrong song.

[2:19](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=139s) So I do think that a new approach is

[2:23](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=143s) definitely needed and here's the TLDDR

[2:26](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=146s) of the premise we're going to walk

[2:28](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=148s) through today. Um there are two ways to

[2:31](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=151s) improve customer or user satisfaction of

[2:35](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=155s) a voice AI assistant and that is by

[2:38](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=158s) increasing accuracy which people know

[2:39](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=159s) about I mean technically accuracy and

[2:43](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=163s) the other is a different knob that we

[2:46](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=166s) have that we are not using adequately

[2:48](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=168s) and I'll call that a system decision

[2:51](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=171s) which we will define which is orthogonal

[2:54](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=174s) which is different from accuracy and I

[2:57](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=177s) believe This approach which I have used

[3:01](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=181s) in several different environments and

[3:04](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=184s) seen some success I think is a promising

[3:07](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=187s) area that we should consider developing.

[3:10](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=190s) Um let me walk through this with a

[3:13](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=193s) simple smart speaker example and we'll

[3:16](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=196s) go step by step with this approach but

[3:19](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=199s) it is a scalable approach that I think

[3:22](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=202s) uh can apply across different surfaces

[3:24](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=204s) and devices. So let's get started. So

[3:27](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=207s) suppose we all you know are making a

[3:30](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=210s) smart speaker coincidentally called uh

[3:33](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=213s) Alexa and Alexa is very simple. It just

[3:37](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=217s) allows you to you know ask for music and

[3:40](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=220s) it'll play a song and of course it will

[3:43](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=223s) play either the song you wanted or a

[3:45](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=225s) different song. So it'll be right or

[3:47](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=227s) it'll be wrong. This isn't that

[3:49](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=229s) different from what you've seen out

[3:51](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=231s) there. Um now let's to first talk about

[3:54](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=234s) accuracy. Accuracy. Let's say we define

[3:57](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=237s) it as we you know take a thousand spoken

[3:59](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=239s) requests. We observe the input and the

[4:02](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=242s) output. We label it and we look at this.

[4:05](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=245s) This is the map of a thousand points and

[4:08](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=248s) 79% of the time 790 dots here were

[4:12](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=252s) actually the correct song. This is let's

[4:14](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=254s) say human annotated 20% 21% wrong song.

[4:19](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=259s) So that's the accuracy. Now, like I

[4:22](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=262s) said, knob one is to spend a lot of time

[4:25](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=265s) working on improving the accuracy, you

[4:28](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=268s) know, um, percentage point by percentage

[4:30](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=270s) point at any layer in the stack.

[4:33](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=273s) There's, if it's a cascaded system, you

[4:35](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=275s) know, there's a perhaps a wakeword layer

[4:38](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=278s) and a speech ASR layer and a NLU layer

[4:41](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=281s) which might have intent classification,

[4:44](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=284s) entity extraction, a lot of different

[4:45](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=285s) layers, VAD, etc. And any of those can

[4:47](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=287s) contribute to errors. So we spent time

[4:50](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=290s) we might be able to reduce that 210 to a

[4:52](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=292s) smaller number that is I think a known

[4:56](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=296s) area that we're tackling but I think

[4:59](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=299s) knob 2 which is what I was talking about

[5:01](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=301s) is what we'll go through here which is

[5:03](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=303s) keeping the accuracy exactly the same.

[5:06](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=306s) So 79% what could we do in conditions of

[5:10](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=310s) uncertainty to improve user satisfaction

[5:13](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=313s) apparent and I I think we can do a lot.

[5:15](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=315s) So let's start first with the original

[5:17](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=317s) system is just acting like I said user

[5:20](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=320s) says something system plays a song it's

[5:21](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=321s) either the right song or the wrong song

[5:24](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=324s) immediately I think just common sense

[5:26](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=326s) tells us that we could introduce at

[5:28](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=328s) least one system behavior to stop or

[5:30](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=330s) rather to reject the hypothesis and do

[5:33](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=333s) nothing. So uh there is now one more

[5:36](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=336s) option to decide the system may decide

[5:38](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=338s) and say sorry I didn't get that or sorry

[5:41](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=341s) could you repeat that? Uh the challenge

[5:43](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=343s) of course is how how when do we decide

[5:47](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=347s) to stop and I mean quantitatively. Um

[5:51](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=351s) here's one approach to kind of

[5:53](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=353s) visualizing this because if we don't

[5:55](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=355s) we'll just take probably some swag like

[5:57](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=357s) some guesstimate and I'll prove that if

[5:59](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=359s) we just took a guesstimate we would end

[6:02](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=362s) up with a worse situation than a more

[6:04](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=364s) rigorous approach. So let's just assume

[6:06](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=366s) I took those thousand data points and

[6:08](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=368s) like I said they've been annotated and

[6:10](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=370s) we assign a confidence score a single

[6:13](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=373s) confidence score to the hypothesis that

[6:16](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=376s) was generated by the system you know

[6:18](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=378s) between zero and one and let's say it's

[6:19](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=379s) reasonably calibrated. This is a

[6:21](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=381s) simplification of if it's a cascaded

[6:23](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=383s) system there are multiple layers and

[6:24](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=384s) multiple you know confidence scores but

[6:26](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=386s) let's just assume that for now. Whoops.

[6:28](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=388s) So we're going to have 790 points 200

[6:31](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=391s) that are correct 210 wrong. Each one has

[6:34](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=394s) a confidence score and we're going to

[6:36](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=396s) plot it, you know, plot the

[6:37](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=397s) distributions. Uh on the x-axis, I've

[6:40](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=400s) just converted from 0ero to one to

[6:42](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=402s) percentages. And the question is how do

[6:46](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=406s) we choose a threshold t such that

[6:49](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=409s) whatever that percentage is um to the

[6:52](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=412s) left of it meaning if when the system um

[6:55](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=415s) forms a hypothesis if the confidence

[6:58](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=418s) score c is less than that t stop and say

[7:01](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=421s) sorry otherwise play question is how do

[7:04](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=424s) we choose a t so far everything I'm

[7:06](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=426s) saying is fairly common sensical but

[7:08](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=428s) this is where um intuition will fail us

[7:12](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=432s) we might say something like okay I don't

[7:14](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=434s) know let's do 65%. It seems you know gut

[7:17](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=437s) feeling like okay it's kind of confident

[7:19](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=439s) that's probably when we should speak. Um

[7:22](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=442s) now here's where we start coming out

[7:24](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=444s) with some sophistication.

[7:26](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=446s) Any tea we choose is producing bad

[7:29](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=449s) outcomes. Bad in the in two fields. One

[7:33](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=453s) is obviously on the left side anytime

[7:35](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=455s) you stop it's bad. The user doesn't want

[7:38](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=458s) it to stop. He wants to they want to

[7:40](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=460s) hear their song. The other bad is if you

[7:43](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=463s) do play a wrong song, of course that's

[7:46](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=466s) bad as well. So these are two two kinds

[7:48](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=468s) of bad outcomes. But here's the

[7:51](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=471s) important part. Now I've like elaborated

[7:53](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=473s) on the um tree diagram on the right hand

[7:56](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=476s) side. The bad outcomes are not equally

[8:00](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=480s) bad. They're not the same thing from a

[8:02](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=482s) user perspective. And obviously let's

[8:05](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=485s) let's think about it. If the wrong song

[8:07](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=487s) plays, you said play kiss and it starts

[8:11](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=491s) playing kiss by Chris Brown instead of

[8:14](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=494s) the one by Prince. That's going to be um

[8:18](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=498s) the highest user cost. Now I'm defining

[8:21](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=501s) user cost from the user's perspective.

[8:23](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=503s) First I have to like hear music and

[8:25](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=505s) realize that is not Prince. Then I have

[8:27](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=507s) to shout over my Alexa and um you know

[8:31](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=511s) get it to stop and then I have to

[8:33](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=513s) re-request. All of that is a lot of

[8:35](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=515s) effort. that is definitely a worse

[8:37](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=517s) outcome than the system stopping and

[8:39](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=519s) saying sorry I didn't understand that

[8:42](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=522s) however we should go further and try to

[8:45](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=525s) quantify that relative badness and there

[8:47](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=527s) many ways to do it and I think this is

[8:49](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=529s) an area to be explored for now let's

[8:51](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=531s) just consider this a heristic of if that

[8:55](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=535s) outcome happens how many more seconds

[8:58](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=538s) additional seconds will it take for the

[8:59](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=539s) user to get back to success which is to

[9:01](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=541s) play the song they wanted kiss by Prince

[9:04](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=544s) and I'm I just put down some numbers

[9:06](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=546s) here. Let's say in the case of a bad

[9:08](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=548s) song, it's 10 seconds if you add up all

[9:10](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=550s) the things I got to do. And if it's a I

[9:14](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=554s) didn't understand you, it's 4 seconds

[9:15](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=555s) because that's how long it would take

[9:16](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=556s) you to respe and and the extra latency.

[9:19](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=559s) And now here's where we can start

[9:23](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=563s) utilizing that. If we go back to our

[9:25](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=565s) distribution curve on trying to find out

[9:27](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=567s) where is T. Now we've basically turned

[9:31](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=571s) this into a problem of minimizing a cost

[9:34](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=574s) function. It's a user cost function. It

[9:36](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=576s) is the number of bad acts wherever that

[9:38](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=578s) whatever the t causes times 10 because

[9:40](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=580s) that was a unit cost we gave plus the

[9:43](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=583s) number of stops times four because

[9:45](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=585s) that's the the unit cost we gave. By the

[9:48](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=588s) way, one thing I should have elaborated

[9:51](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=591s) because I work in voice and we like

[9:52](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=592s) language and we like puns. So this whole

[9:55](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=595s) thing is called an outcome user cost

[9:57](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=597s) heruristic. So that spells the word ouch

[10:00](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=600s) and that is some expression of pain.

[10:03](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=603s) Yes, we are you know language nerds. So

[10:06](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=606s) these kinds of things amuse us. Um so

[10:08](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=608s) now let's consider that is the cost

[10:10](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=610s) function is to minimize the ouch. And

[10:12](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=612s) now um that let's see if uh I'm going to

[10:16](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=616s) bring up a tool. Let's see if this

[10:18](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=618s) works.

[10:20](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=620s) Where I have actually gotten or with one

[10:23](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=623s) of my coding assistants gotten uh an

[10:26](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=626s) interactive

[10:29](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=629s) um graph where we have actually plotted

[10:32](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=632s) those thousand points and as we vary the

[10:36](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=636s) threshold t you can see that the total

[10:40](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=640s) user cost here which is that function of

[10:43](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=643s) you know x * y + a * b actually changes.

[10:46](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=646s) So let's in the very beginning when we

[10:49](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=649s) said the system was just playing

[10:52](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=652s) the the cost across those thousand

[10:54](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=654s) points was 2100 or divided by a,000 is

[10:57](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=657s) 2.1 ouch points per turn. Then we said

[11:01](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=661s) okay let's insert a stop behavior and

[11:04](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=664s) let's like wing it and say 65%. That's

[11:07](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=667s) when I want the threshold. If we brought

[11:10](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=670s) this up to 65 yeah that's better. Now

[11:13](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=673s) it's 1904 or 1.9 per turn, but it's not

[11:16](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=676s) optimal. As it turns out, if we do

[11:19](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=679s) actually um ask for the AI to solve the

[11:23](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=683s) uh the problem across this curve, it

[11:25](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=685s) turns out 43%. So I'll drag it now to 43

[11:30](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=690s) is in fact

[11:34](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=694s) the optimal

[11:36](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=696s) optimal point of t. This minimizes the

[11:39](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=699s) cost function. You can see it's the

[11:41](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=701s) lowest point on this graph down here to

[11:43](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=703s) 1

[11:44](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=704s) 27. So effectively we haven't changed

[11:48](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=708s) the accuracy at all. The system is not

[11:50](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=710s) any smarter in that sense. But with some

[11:52](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=712s) clever system behavior, conversational

[11:55](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=715s) behavior is what we'd call it and some

[11:56](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=716s) optimization and a cost function called

[11:59](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=719s) ouch. Um we have from the user's

[12:02](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=722s) perspective produced a more satisfactory

[12:06](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=726s) assistant. And this is not a trivial you

[12:08](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=728s) know accomplishment. Okay. Now, let me

[12:10](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=730s) go back to this. [clears throat] Let me

[12:12](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=732s) see if I can get this. Oh, great. Okay,

[12:16](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=736s) let's continue this. Let's continue this

[12:19](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=739s) with by now adding one more behavior.

[12:22](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=742s) Let's call it the confirm behavior. So,

[12:23](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=743s) there was play obviously, then stop,

[12:26](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=746s) confirm. Confirm is basically the system

[12:28](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=748s) after you said something saying uh kiss

[12:32](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=752s) play kiss by Prince or maybe play kiss

[12:35](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=755s) by Chris Brown. And uh you know the user

[12:38](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=758s) can either confirm like affirm it or

[12:40](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=760s) they can correct it. It is a different

[12:42](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=762s) kind of behavior and again this is kind

[12:44](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=764s) of how humans behave. Um that's

[12:46](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=766s) obviously the inspiration. Now if we go

[12:49](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=769s) back to our problem of optimization,

[12:52](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=772s) we have a third obviously um option

[12:55](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=775s) which is to confirm. And so this would

[12:58](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=778s) translate to two thresholds

[13:00](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=780s) um two thresholds which are separating

[13:03](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=783s) the distribution into three spaces of

[13:07](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=787s) stop, confirm and uh act. And the

[13:12](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=792s) question is now where are these T's? and

[13:15](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=795s) we have now given up on guesstimating

[13:16](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=796s) because we know it doesn't work. So

[13:18](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=798s) we're going to be a lot smarter and go

[13:21](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=801s) back to the concept of user outcome cost

[13:25](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=805s) and then you know use it go look for

[13:27](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=807s) some optimization in that graph. So

[13:29](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=809s) let's uh define what are the what are

[13:32](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=812s) all the possible bad outcomes that t1

[13:34](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=814s) and t2 um make for. So good you can see

[13:38](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=818s) my cursor. So uh of course any stops are

[13:42](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=822s) still bad. Then in the middle are

[13:45](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=825s) confirmations. Confirmations are bad

[13:47](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=827s) because they slow the user down. There

[13:49](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=829s) is a confirmation outcome called confirm

[13:52](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=832s) yes where they just affirmed it by

[13:54](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=834s) saying yeah or no where they had to

[13:56](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=836s) correct it. And going back to our

[13:59](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=839s) formula these outcomes are not equally

[14:02](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=842s) bad. And in fact, nobody will, I think,

[14:06](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=846s) argue here from a user's perspective.

[14:08](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=848s) Affirming, just saying yes is obviously

[14:11](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=851s) less painful than saying no and then

[14:13](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=853s) having to restate whatever it is that

[14:15](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=855s) you wanted in the first place. So now we

[14:17](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=857s) I've assigned values of two or six. And

[14:19](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=859s) again, I said it was a heristic. This

[14:21](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=861s) would be roughly the amount of time it

[14:23](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=863s) would take for the extra for the user to

[14:25](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=865s) get to the song they want. Saying

[14:27](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=867s) listening and then saying yes is like

[14:28](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=868s) two seconds. Um and then now

[14:34](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=874s) uh we restate the cost function for this

[14:37](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=877s) you know added behavior as this number

[14:40](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=880s) of you know bad type one times unit cost

[14:43](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=883s) bad type plus bad type two times unit

[14:45](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=885s) cost etc. And now we try to minimize

[14:49](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=889s) this user cost function and minimize the

[14:52](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=892s) ouch.

[14:53](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=893s) Yes, I'm going to keep doing that pun.

[14:56](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=896s) Um let's go back. So this is now the

[15:00](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=900s) interactive graph but

[15:03](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=903s) with

[15:06](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=906s) um the cost values the unit costs here

[15:08](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=908s) 10264

[15:10](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=910s) and uh you know we're just going to ask

[15:13](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=913s) the AI to tell us here's the heat map

[15:16](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=916s) because it's now two dimensions saying

[15:18](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=918s) that the optimal values are 41 for the

[15:22](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=922s) the T1 and 49 for the T2 and if we

[15:27](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=927s) employed that then we would go to 1464.

[15:31](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=931s) Uh, by the way, whatever numbers I put

[15:33](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=933s) in here, like let's say I thought wrong

[15:36](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=936s) act was 20. It's really irritating and

[15:39](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=939s) painful and takes way longer to actually

[15:42](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=942s) correct it when you hear a wrong song.

[15:44](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=944s) That would change you know all these

[15:45](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=945s) numbers uh and the optim optimal point.

[15:48](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=948s) So again it is about how what is the

[15:50](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=950s) relative badness of these outcomes also

[15:52](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=952s) of course the distribution curves

[15:54](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=954s) naturally. Uh let's go back here. Okay.

[15:58](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=958s) So, um I'm gonna

[16:02](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=962s) speed up a little bit. Uh let's go back

[16:06](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=966s) here.

[16:07](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=967s) Presentation mode. Okay. So, what have

[16:11](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=971s) we shown that if we did the super naive

[16:13](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=973s) approach, it's 2.1 act and stop 1.9 then

[16:19](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=979s) 1.27 then 1.26. We are able to bring

[16:22](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=982s) this with every added layer of

[16:25](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=985s) sophistication, adding more behaviors,

[16:27](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=987s) being smart about outcome, uh, user cost

[16:30](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=990s) and optimizing. Um, we have made a

[16:33](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=993s) tremendous difference without changing

[16:35](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=995s) the accuracy at all. Um, this was a

[16:38](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=998s) super simplified example. In real

[16:40](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1000s) systems, you're not going to have

[16:42](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1002s) obviously some offline decision

[16:44](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1004s) threshold or two. It's going to be a

[16:46](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1006s) real time, you know, learned decision

[16:48](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1008s) model. But the principle is the same.

[16:50](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1010s) And I believe this is uh scalable across

[16:54](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1014s) all voice AI surfaces. Obviously this is

[16:56](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1016s) a smart speaker but if we go across any

[17:01](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1021s) of these surfaces you will find the

[17:02](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1022s) equivalence. If we um we will find the

[17:07](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1027s) analogies with some differences but the

[17:09](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1029s) spirit and the I think the the gain will

[17:13](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1033s) be similar. So just for example in the

[17:16](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1036s) TV AI assistant space if you employ it

[17:20](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1040s) here it's going to you're going to have

[17:22](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1042s) the same thing when users express

[17:24](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1044s) intents like on TV it's you know open a

[17:27](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1047s) channel that's one of the most common

[17:29](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1049s) obviously um requests on a TV voice

[17:32](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1052s) assistant same thing you're going to

[17:34](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1054s) find you'll have exactly the same

[17:35](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1055s) approach but the difference will be

[17:38](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1058s) maybe in the the assignments of the user

[17:42](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1062s) outcomes because the UI and the

[17:43](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1063s) modalities are different when you have a

[17:46](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1066s) TV you have a multimodal interface where

[17:48](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1068s) choices can be shown. So instead of you

[17:51](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1071s) know asking did you mean ABC you know uh

[17:55](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1075s) news live by speech that you will the

[17:59](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1079s) system would display choices and not

[18:01](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1081s) just one it show ABC News live this that

[18:03](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1083s) would be the confirm step and if it's

[18:05](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1085s) visual and you can use your remote

[18:07](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1087s) control to select something it's less

[18:10](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1090s) pain so you would change some of these

[18:12](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1092s) values or if in fact launching the

[18:15](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1095s) channel would kick you out of your

[18:16](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1096s) current state then it would go in the

[18:18](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1098s) other direction than cost of you know a

[18:21](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1101s) bad act would go much higher. So it's

[18:24](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1104s) the same concept but in this new

[18:26](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1106s) modalities

[18:27](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1107s) um variables can change, values can

[18:30](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1110s) change, arguments can change but the

[18:32](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1112s) premise still holds and you can improve

[18:35](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1115s) from the user's perspective because

[18:37](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1117s) we're all about you know making humans

[18:39](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1119s) happy. Um you can make them happier and

[18:44](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1124s) this as I said in conclusion can be

[18:46](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1126s) applied across all surfaces. I did say

[18:49](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1129s) at the very beginning, just to recap for

[18:52](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1132s) us, that voice is great when it works,

[18:55](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1135s) bad when it doesn't. And as we get into

[18:58](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1138s) embodied AI, where these AI assistants

[19:01](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1141s) are taking actions, physical or even

[19:04](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1144s) digital, like making a phone call or

[19:06](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1146s) sending an email, it is getting more and

[19:09](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1149s) more difficult just to rely on accuracy

[19:12](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1152s) to improve user satisfaction. I believe

[19:15](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1155s) there's a whole knob the second knob

[19:17](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1157s) called smarter conversational behavior

[19:19](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1159s) under uncertainty

[19:21](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1161s) and um if we actually exploit that we

[19:25](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1165s) can uh very much help these AI systems

[19:30](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1170s) reach a acceptable user experience

[19:34](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1174s) otherwise I think this will continue to

[19:36](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1176s) be a bottleneck like a lot of things

[19:38](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1178s) will get better but if the voice

[19:40](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1180s) interface as experienced by user does

[19:43](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1183s) not improve it is going to be a a a

[19:46](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1186s) choke point. And um if you just remember

[19:50](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1190s) one word or two words from this whole um

[19:54](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1194s) presentation, it would be to minimize

[19:57](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1197s) the ouch of the experience. Um so thank

[20:00](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1200s) you. I'll stick around for questions if

[20:03](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1203s) you guys got any. Thanks a lot.

[20:06](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1206s) [applause]

[20:21](https://www.youtube.com/watch?v=Zd5b40Jbp_k&t=1221s) >> [music]
