# AI Consulting in Practice — Nathaniel Whittemore (NLW), Superintelligent

Nathaniel Whittemore (NLW)

AI Engineer Code 2025

[Talk page](https://ai.engineer/talks/ehQFj6VmuI8-ai-consulting-in-practice) · [Video](https://www.youtube.com/watch?v=ehQFj6VmuI8)

## AI Consulting in Practice

Early AI returns concentrate in productivity, but a preliminary survey of roughly 2,500 use cases points toward broader gains from risk reduction, automation and systematic adoption.

### What value are organizations finding now?

[0:24](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=24s)

Where are organizations actually finding value in AI? That question is more useful for planning an implementation than the competing narratives about an AI bubble. Nathaniel Whittemore opens with that tension, criticizing studies he considers dubious, including an unnamed MIT report, before turning to what organizations report from their own work.

His perspective combines daily AI news analysis through The AI Daily Brief with executive interviews through Superintelligent, which he introduces as an AI planning platform. The first gives a broad view of adoption; the second exposes what happens inside organizations. Alongside enterprise surveys, he presents an initial analysis of approximately 2,500 self-reported AI use cases collected from his audience.

### From general usage to production agents

[1:45](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=105s)

Enterprise AI usage is broadening across business functions. That establishes adoption, but not yet organization-wide scale: using a tool somewhere in a company is a different milestone from changing how the company operates.

![Slide titled “Increased General Usage” with a line chart of AI adoption and small charts showing usage across multiple business functions.](https://ai.engineer/api/talk-essay-media/757bf0d81ba19bc3a222884167f45eb2ec943ec214846b8e54296dc6a4930b1a)

AI usage is increasing across organizations and business functions.

Coding is a particularly visible inflection in 2025. Its reach extends beyond software engineering teams to other departments that can now communicate through code and build things themselves. The organizational change is therefore larger than faster software development: more people can express requirements in executable form.

Agents present a more mixed picture. The anticipated wholesale automation of enterprise work has not arrived, yet deployment has advanced. In KPMG’s [Q3 AI Quarterly Pulse release](https://kpmg.com/us/en/media/news/q3-ai-pulse.html), reported deployment of at least some agents rose from 11% in Q1 to 42% in Q3 2025. The Q3 survey covered 130 U.S. C-suite and business leaders at organizations with at least $1 billion in annual revenue. Whittemore describes these as production deployments, distinct from pilots; the measure does not establish automation across the whole enterprise.

As agents move into actual work, the implementation problem becomes partly human. Whittemore’s executive conversations suggest that some organizations moved through experimentation faster than expected. Their attention is shifting toward upskilling, enablement and how employees interact with agents. Sandboxes let people try those interactions, and he reports less resistance as employees gain experience.

### Deployment can grow while scaling remains difficult

[4:02](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=242s)

Moving from an exciting experiment to a scaled organizational capability remains difficult. Whittemore cites McKinsey figures in which only 7% describe themselves as fully at scale with AI and agents, while approximately 62% remain in experimentation or piloting. These describe a broader adoption stage than the presence of a deployed agent.

Progress also varies by company size and function. Larger organizations are generally further along in scaling, despite the expectation that smaller companies should move faster. Within companies, the earlier pattern of similar experimentation rates across departments is giving way to greater divergence, with IT operations pulling ahead.

**Leading organizations pursue a broader portfolio of changes.** They do more than run isolated experiments: they consider AI adoption systematically, undertake multiple initiatives and connect them to organizational strategy. Their objectives include time savings and productivity, but also revenue growth, new capabilities and new product lines.

![“High performers” slide with a workflow redesign comparison marked 2.8× and bar charts comparing efficiency, growth, and innovation objectives.](https://ai.engineer/api/talk-essay-media/f69828cfaf15e52d6f49b31765fbe9891b4a0dd3322e3de21a3456a0042d8627)

High performers report more workflow redesign and broader AI objectives.

### Larger budgets, shorter expected payback

[6:10](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=370s)

Spending intentions suggest continued commitment. The [KPMG Q3 2025 AI Quarterly Pulse Survey](https://kpmg.com/kpmg-us/content/dam/kpmg/pdf/2025/ai-quarterly-pulse-survey-q3-2025.pdf) reports average planned generative-AI investment over the following twelve months rising from $114 million in Q1 to $130 million in Q3. Whittemore also cites an approximately $88 million baseline in the preceding Q4. These are planned investments, not realized expenditure; the direction of change matters more to his argument than the absolute budget.

The [Deloitte investment findings](https://www.deloitte.com/nl/en/issues/generative-ai/ai-roi-the-paradox-of-rising-investment-and-elusive-returns.html) similarly report that 91% of surveyed organizations plan to increase AI investment over the twelve months through August 2026. Larger commitments bring greater pressure to show returns, while executives’ expected payback periods are shortening.

Whittemore compares the following distributions from KPMG’s annual CEO surveys. The [2024 global comparison](https://assets.kpmg.com/content/dam/kpmgsites/in/pdf/2024/10/kpmg-2024-India-ceo-outlook.pdf) asks about generative-AI payback; the 2025 chart refers to AI investment.

| Survey year | Anticipated payback | Share of CEOs |
|---|---|---:|
| 2024 | 3–5 years | 63% |
| 2024 | 1–3 years | 20% |
| 2024 | More than 5 years | 16% |
| 2025 | 1–3 years | 67% |
| 2025 | 6–12 months | 19% |
| 2025 | 3–5 years | 12% |

These selected categories show expectations moving toward earlier returns. They do not measure whether those returns have arrived.

### Measuring impact beyond conventional ROI

[7:46](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=466s)

Pressure to demonstrate value is increasing faster than confidence in measurement. Two distinct KPMG Pulse findings capture the gap: 78% say investor and board pressure for demonstrable returns will influence their generative-AI strategies over the next six months, and 78% say traditional metrics are insufficient to capture AI’s value. The first is more specific than the talk’s shorthand about ROI becoming a bigger consideration in the coming year. Whittemore hears the same measurement problem from CIOs: methods used for previous technologies and initiatives are falling short with AI.

That gap motivated a simpler research step: ask people what impact they are experiencing. The listener survey opened at the end of October; by the talk, roughly 1,000 organizations had submitted about 3,500 use cases. The presented analysis covers the first approximately 2,500.

The survey organizes impact into eight categories: **time savings**, **increased output**, **quality improvement**, **new capabilities**, **improved decision-making**, **cost savings**, **increased revenue** and **risk reduction**. This makes room for benefits that a narrow cost calculation might miss, while giving respondents a simple vocabulary for describing value.

### Reported returns begin with everyday productivity

[9:32](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=572s)

These results come from voluntary submissions by an unusually engaged daily AI podcast audience, not a representative sample of enterprises. Returns are respondents’ perceptions, and **high ROI** combines the significant and transformational response categories.

| Reported current ROI | Share |
|---|---:|
| Modest | 44.3% |
| High: significant or transformational | 37.6% |
| Negative | Approximately 5% |

Negative ROI means respondents believe they have spent more than they have gained so far. It does not necessarily mean they consider the program a failure.

Expectations are even more optimistic: 67% expect high growth in ROI over the next year. Among teams currently reporting negative ROI, 53% still expect high growth. These forecasts describe confidence about future returns, separately from the current results.

Time savings account for approximately 35% of submitted use cases. Increased output and quality improvement are also prominent, making everyday productivity the main entry point. Reported time savings cluster between one and ten hours, especially around five hours.

Recovering a few hours can matter substantially even when the eventual ambition is to create entirely new capabilities. Whittemore illustrates this with five to ten hours saved per week, describing the annual benefit as seven to ten workweeks. That is an illustration rather than a fixed conversion: the equivalent depends on the working-year and workweek assumptions. The practical point is that recurring productivity gains can be valuable before an organization achieves a more ambitious transformation.

### Different organizations and roles seek different benefits

[11:39](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=699s)

Time savings dominate the aggregate, but priorities differ within it. Organizations with 200–1,000 employees show a greater concentration of increased-output use cases. Whittemore tentatively connects this to companies that have reached some scale but are still pursuing expansion; the survey has not established that explanation. Leadership respondents likewise emphasize increased output and new capabilities more than time savings.

![Slide titled “Different roles are finding different benefits” with stacked horizontal bars for six role categories and a legend of benefits including time savings, increased output, and new capabilities.](https://ai.engineer/api/talk-essay-media/4099987b51f5f6fc98936ca4abe67bcaed1a8b5f4659615329e1b85f6988a28b)

Reported AI benefits differ across respondent roles.

Leaders report more optimism and transformational impact than junior respondents. One possible explanation is the work they choose: a senior leader may be evaluating a project whose scope is inherently more transformational if it succeeds. Among use cases submitted by leadership respondents, 17% are reported to already deliver transformational impact and ROI. Role-related project selection may therefore help explain the difference, rather than seniority itself producing better returns.

The smallest organizations also show a concentration of early transformational benefits. That observation concerns reported outcomes, whereas the earlier enterprise comparison concerned progress toward scale. Even the survey’s 1–50-person category may obscure important differences: a three-person company and a forty-person company can operate very differently. Whittemore proposes examining that range more closely. Coding and software use cases, meanwhile, show above-average ROI and below-average negative ROI in this sample, without a numerical uplift specified.

### Less common use cases can have greater impact

[14:15](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=855s)

Risk reduction is the least common primary benefit in the preliminary analysis. The [AI ROI Benchmarking Study](https://roisurvey.ai/) asks respondents to choose just one primary benefit, so category shares do not capture every benefit a use case may produce. Risk reduction is the primary benefit for 3.4% of use cases, but 25% of that subset reports transformational ROI—the highest transformational share among the categories described.

A possible explanation comes from conversations with practitioners in back-office, compliance and risk functions. Much of their difficulty involves the sheer volume of work, a burden AI can help address. That offers a plausible mechanism for large gains, although the survey does not demonstrate that volume reduction caused the reported returns.

Industry composition matters too. Technology and professional services are heavily represented in the listener sample. Whittemore describes some other industry samples as reasonably sized, without giving counts, and reports higher average impact for healthcare and manufacturing use cases than for the sample overall. Those differences are leads for further study, not a settled ranking of industries.

### Beyond isolated productivity experiments

[15:56](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=956s)

More content, better content and a few recovered hours characterize the first layer of enterprise adoption. Automation and agents point toward a further layer. In the preliminary analysis, use cases mentioning automation or agents substantially outperform on self-reported ROI. No numerical uplift or causal comparison is supplied, so the finding identifies an association with more advanced use cases rather than proving that adding an agent improves returns.

Breadth of adoption shows a related pattern: people or organizations submitting more use cases tend to report better ROI. Whittemore acknowledges that several explanations are possible, but connects the pattern to the earlier distinction between isolated experiments and systematic adoption. Exploring AI across disciplines and organizational functions creates more opportunities to find valuable applications than stopping after a single trial.

This remains a first look at roughly the first two-thirds of the use cases collected at that point. At the time of the talk, the survey was scheduled to remain open another week, with a full study planned for early December. The next step is to move beyond generic discussions of impact and intuitive impressions: collect more concrete reports, investigate the differences and use that evidence to decide where to explore next.

## Resources

- [KPMG Q3 2025 AI Quarterly Pulse Survey](https://kpmg.com/kpmg-us/content/dam/kpmg/pdf/2025/ai-quarterly-pulse-survey-q3-2025.pdf) — Survey findings on agent deployment, planned AI investment, workforce changes and measurement challenges.
- [KPMG 2024 India CEO Outlook: global comparison](https://assets.kpmg.com/content/dam/kpmgsites/in/pdf/2024/10/kpmg-2024-India-ceo-outlook.pdf) — Includes global and Indian CEO expectations for generative-AI investment payback in 2024.
- [AI ROI: The paradox of rising investment and elusive returns](https://www.deloitte.com/nl/en/issues/generative-ai/ai-roi-the-paradox-of-rising-investment-and-elusive-returns.html) — Deloitte examines rising AI investment intentions and the time needed to realize returns.

## Slides

### Two sets of experience

NLW brings two perspectives: macro-level AI news from The AI Daily Brief and a ground-level view from Superintelligent’s interviews with executives.

[0:58](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=58s) [Slide pairing The AI Daily Brief podcast with a Superintelligent interview interface.](https://assets.ai.engineer/sites/aiecode2025/sha256/9c37a46f85c5baa786c1292538c88e264005c72e7d99f92366f88c9da918e82c.webp)

ABOUT ME
Two sets of experience inform the convo

#1
Top Technology Podcasts

THE AI DAILY BRIEF
The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis
Jan 18, 2025
USA

Interface text:
Superintelligent
Interview Progress
1 of 11 questions completed

What is your role in the organization?
Which teams or departments do you usually work with?
What are your main responsibilities and daily tasks?
Which parts of your work are most repetitive or time-consuming?
What tasks require the most complex thinking or problem-solving?
How is your success measured, and how do you personally judge your own performance?
Do you use AI at work now, and if so, how?
Have you ever tried using AI for something and then stopped?
Has AI helped you try anything new or [illegible]

Super Agent
Hello! Let’s start by talking about your role. What is your position in the organization?

I run partnerships at Superintelligent.

Super Agent
Great, thank you for sharing that. Could you tell me a bit more about your responsibilities as the head of partnerships at Superintelligent?

Sure, I oversee the go-to-market planning with regards to clients or strategic partners and a little bit of fundraising sprinkled in there.

Super Agent
Thanks for clarifying your role. I’m curious, which teams or departments do you usually collaborate with at Superintelligent?

00:13

AIE / LEAD
PRESENTED BY
Google DeepMind
AI CONSULTING IN PRACTICE
NLW / Founder and CEO
Superintelligent

### The themes

NLW introduces enterprise adoption and the urgency of ROI in 2026, then shares early analysis of self-reported ROI across roughly 2,500 use cases.

[1:17](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=77s) [Three themes: enterprise adoption, urgency of ROI in 2026, and where companies are seeing ROI.](https://assets.ai.engineer/sites/aiecode2025/sha256/04d2fcfa9678b663d5c4f8a6bf8a0d364a8c8c31e5eb6026f40105593fffa307.webp)

THE THEMES

• Enterprise adoption status
• ROI becomes more urgent in 2026
• Where companies are seeing ROI so far

Interface text:
AIE / LEAD
PRESENTED BY
Google DeepMind
AI CONSULTING IN PRACTICE
NLW / Founder and CEO
Superintelligent

### Year of Agents?

In KPMG’s survey of enterprises with over $1 billion in revenue, the share deploying agents in full production rose from 11% in Q1 to 42% in Q3 2025.

[2:56](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=176s) [Chart showing 11%, 33%, and 42% for Q1, Q2, and Q3 2025, respectively.](https://assets.ai.engineer/sites/aiecode2025/sha256/79da76a0b57725032529385ac27330b06d7ab2c0957e4459d4c5b90bd687b08e.webp)

Year of Agents?

11% — Q1 2025
33% — Q2 2025
42% — Q3 2025

Interface text:
AIE / LEAD
PRESENTED BY
Google DeepMind
AI CONSULTING IN PRACTICE
NLW / Founder and CEO
Superintelligent

### Where the breakouts are

Among respondents, 39% at companies with $5 billion or more in revenue report scaling AI, versus 25% below $100 million. Agent deployment varies by function: IT leads at 95%, while HR is at 2% as of Q3 2025.

[4:34](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=274s) [Bar charts compare AI adoption by company revenue, with a list of agent deployment percentages by function.](https://assets.ai.engineer/sites/aiecode2025/sha256/bf60a49df24f1eff4a5f8b13a986b3ea9a4930ee393482f883ee6b784fae12bb.webp)

Where the breakouts are

Larger companies lead the way in scaling AI beyond pilots.

Phase of organization’s use of AI, by company revenues,¹ % of respondents
Legend: Not using at all; Experimenting; Piloting; Scaling; Fully scaled
<$100 million: 9; 39; 22; 25; 5
$100 million–$499 million: 8; 33; 32; 23; 4
$500 million–$999 million: 5; 31; 32; 29; 3
$1.0 billion–$4.9 billion: 6; 22; 31; 32; 9
$5 billion or more: 1; 17; 31; 39; 10

Functional deployment of AI agents into workflows as of Q3’ 25:
• Technology or IT (software engineers, IT specialists) – 95%
• Human Resources (recruiters, employee relations) – 2%
• Finance (analysts, accountants, controllers) – 45%
• Risk (risk analysts, compliance officers, fraud prevention specialists) – 66%
• Marketing and Sales (marketing managers, sales representatives, market research representatives) – 18%
• Operations (operation managers, quality assurance experts) – 89%

Interface text:
AI Engineer
Engineering the future of AI

### High performers

In McKinsey’s 2025 survey, among respondents whose organizations used AI in at least one function, 55% of AI high performers reported fundamental workflow redesign versus 20% of others. They also more often targeted growth and innovation.

[5:31](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=331s) [Charts compare workflow redesign and AI objectives among high performers and other respondents.](https://assets.ai.engineer/sites/aiecode2025/sha256/2be0d981b23a7a672877668f52040c5829938ab59e4e5a49e4f764f76fea0edb.webp)

High performers

High performers are nearly three times as likely as others are to fundamentally redesign their workflows in their deployment of AI.

Respondents who report fundamental redesign of organization’s workflows in their deployment of AI,¹
% of respondents
AI high performers²: 55
All other respondents: 20
2.8×

¹ Question was asked only of respondents whose organizations use AI in at least 1 function.
² AI high performers are respondents who reported that more than 5% of their organization’s EBIT and “significant value” are attributable to the organization’s use of AI. For AI high performers, n = 109; for all others, n = 1,644.
Source: McKinsey Global Survey on the state of AI, 1,993 participants at all levels of the organization, June 25–July 29, 2025

High performers set innovation or growth objectives for AI efforts, in addition to efficiency goals.

Objectives of AI efforts at respondents’ organizations,¹ % of respondents
Efficiency (eg, reducing costs by automating workflows): AI high performers² (n = 109): 84; All other respondents (n = 1,644): 80
Growth (eg, increasing revenues through improved customer targeting or new product features): AI high performers² (n = 109): 82; All other respondents (n = 1,644): 50
Innovation (eg, creating new businesses, transforming business models): AI high performers² (n = 109): 79; All other respondents (n = 1,644): 50

Interface text:
AI Engineer
Engineering the future of AI

### ROI expectations moving forward

In KPMG’s CEO survey, expected AI returns shifted earlier: 63% expected returns in 3–5 years in 2024, while 67% expected them in 1–3 years in 2025.

[6:49](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=409s) [Two bar charts compare expected times to realize returns on AI investment.](https://assets.ai.engineer/sites/aiecode2025/sha256/f1b5be43213c48416833450c51ab560ee3c24e342f484285ab952cb9b5296c58.webp)

ROI expectations moving forward

Left chart: Anticipated return on AI investment
2025 | 2024
More than 5 years: 16%
3 - 5 years: 63%
1 - 3 years: 20%
6 months - 1 year: 1%
Less than 6 months: 0%
X-axis ticks: 0, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%

Right chart: Anticipated return on AI investment
2025 | 2024
More than 5 years: 2%
3 - 5 years: 12%
1 - 3 years: 67%
6 months - 1 year: 19%
Less than 6 months: 0%
X-axis ticks: 0, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%

Interface text:
AI Engineer
Engineering the future of AI

[7:00](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=420s) [Charts compare anticipated time to AI investment returns in 2024 and 2025.](https://assets.ai.engineer/sites/aiecode2025/sha256/7433ef05571b4bf996d7e1c25442d420923b3c786ae750584a192081252bd8bf.webp)

ROI expectations moving forward

Left chart: Anticipated return on AI investment
2025 | 2024
More than 5 years: 16%
3 - 5 years: 63%
1 - 3 years: 20%
6 months - 1 year: 1%
Less than 6 months: 0%
X-axis ticks: 0, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%

Right chart: Anticipated return on AI investment
2025 | 2024
More than 5 years: 2%
3 - 5 years: 12%
1 - 3 years: 67%
6 months - 1 year: 19%
Less than 6 months: 0%
X-axis ticks: 0, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%

Interface text:
AI Engineer
Engineering the future of AI

### ROI is tough

Among leaders, 78% say traditional metrics fail to capture AI’s business impact, while 78% report significant pressure from investors and boards to demonstrate AI value.

[7:45](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=465s) [Two 78% figures describe difficulties measuring AI impact and pressure to demonstrate value.](https://assets.ai.engineer/sites/aiecode2025/sha256/9c4497ccffa8f5bbfea893e0f64edb1bd908405f7ba98c2f1841009d3d12ee4a.webp)

ROI is
TOUGH!

78%
AI is reshaping how organizations measure value, with 78% of leaders saying traditional metrics fail to capture its business impact.

78% of leaders report facing significant pressure from investors and boards to demonstrate AI value
78%

Interface text:
AI Engineer
Engineering the future of AI

### Impact categories

Among roughly 2,500 submitted use cases, time savings leads at 35.0%, followed by increased output and quality improvement at about 14% each. The submissions came from a self-selected AI podcast audience.

[9:08](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=548s) [Eight AI impact categories, led by time savings at 35% of use cases.](https://assets.ai.engineer/sites/aiecode2025/sha256/2269023b6bc20ce8e130cebd61f1e7bca0a036df3012d54788173e60090df0b4.webp)

Impact Categories

Time Savings
35.0% of use cases

Increased Output
14.5% of use cases

Quality Improvement
14.4% of use cases

New Capabilities
12.0% of use cases

Improved Decision Making
8.8% of use cases

Cost Savings
7.9% of use cases

Increased Revenue
4.2% of use cases

Risk Reduction
3.4% of use cases

Interface text:
AIE / LEAD
PRESENTED BY
Google DeepMind
AI CONSULTING IN PRACTICE
NLW / Founder and CEO
Superintelligent

### Expectations for increased ROI

67.2% anticipate high ROI growth over the next year: 51.6% expect a significant increase and 15.6% expect a transformational increase.

[10:11](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=611s) [A bar chart shows 67.2% anticipating significant or transformational ROI increases.](https://assets.ai.engineer/sites/aiecode2025/sha256/826c2f29da2fcbfb8ec4c1cb8ddac8a701a479bcaf8cf8055ccf9d81c556aa56.webp)

...and expected to grow
Big expectations for increased ROI

Significant Increase
Transformational Increase
Modest Increase
Same
Decreasing
X-axis ticks: 0%, 10%, 20%, 30%, 40%, 50%, 60%

↑ 67.2% High Growth

Key stats (anticipated ROI):
Significant Increase 51.6% (1,279)
Transformational Increase 15.6% (387)
Modest Increase 29.1% (721)
Same 3.5% (87)
Decreasing 0.1% (3)

AIE / LEAD
PRESENTED BY
Google DeepMind
AI CONSULTING IN PRACTICE
NLW / Founder and CEO
Superintelligent

### ROI growth expectations by current ROI

Among teams reporting negative ROI, 53% (69 of 131) expect significant or transformational ROI growth over the next year. The share peaks at 71% among teams with modest ROI.

[10:28](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=628s) [A chart shows that 53% of teams reporting negative ROI anticipate significant or transformational increases.](https://assets.ai.engineer/sites/aiecode2025/sha256/754abb1828355d509ef2ab87efbe2a9653de4471b9781a46f5371ebdc3af138e.webp)

Confidence extends to currently negative ROI teams
Even currently negative teams expect significant increases

Y-axis: % Expecting Significant/Transformational Increase
Y-axis ticks: 0%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, 100%
X-axis: Current ROI Level
X-axis categories: Negative, Breakeven, Modest, Significant, Transformational

53% of negative-ROI teams expect significant increases
Despite current challenges, most teams double down on AI investments

Anticipated High Growth by Current ROI:
Negative 53% (69 of 131)
Breakeven 64% (203 of 317)
Modest 71% (780 of 1,098)
Significant 68% (446 of 656)
Transformational 61% (168 of 275)

Note: "High Growth" = Significant or Transformational Increase

AIE / LEAD
PRESENTED BY
Google DeepMind
AI CONSULTING IN PRACTICE
NLW / Founder and CEO
Superintelligent

### Primary benefits of AI use cases

Time savings is the primary benefit for 35.0% of 2,477 use cases, more than twice the share for increased output or quality improvement.

[10:37](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=637s) [Bar chart of primary benefits for 2,477 use cases; time savings leads at 35%, followed by increased output at 14.5% and quality improvement at 14.4%.](https://assets.ai.engineer/sites/aiecode2025/sha256/b301503620a89e8837f47035bcdf6db02358ecd4e97c83cbbe1774a880dedad2.webp)

Buying back time is where most orgs are starting
Time-to-value dominates, with productivity and quality close behind.

Chart categories:
Time Savings
Increased Output
Quality Improvement
New Capabilities
Improved Decision Making
Cost Savings
Increased Revenue
Risk Reduction

X-axis: Percentage of Use Cases
X-axis ticks: 0%, 5%, 10%, 15%, 20%, 25%, 30%, 35%

1. Time Savings 35.0%
2. Increased Output 14.5%
3. Quality Improvement 14.4%

Primary benefit split (n=2,477):
Time Savings 35.0% (865)
Increased Output 14.5% (358)
Quality Improvement 14.4% (356)
New Capabilities 12.0% (296)
Improved Decision Making 8.8% (218)
Cost Savings 7.9% (195)
Increased Revenue 4.2% (105)
Risk Reduction 3.4% (84)

AIE / LEAD
PRESENTED BY
Google DeepMind

AI CONSULTING IN PRACTICE
NLW / Founder and CEO
Superintelligent

### Distribution of time savings

Time savings cluster between 1 and 10 hours a week, especially around five. NLW says saving five to ten hours weekly can buy back 7–10 work weeks a year.

[10:56](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=656s) [Histogram shows time savings concentrated near the low end of a 0–100 scale; the slide says most savings are between one and ten hours per week.](https://assets.ai.engineer/sites/aiecode2025/sha256/ee7231d4ce20a23712af68491e24231bf12af2edfa467e09de76de9d1eb755e5.webp)

Most time savings are between 1-10 hpw
Buying back 7-10 weeks worth of work a year ain’t bad

Distribution of Time Savings (0-100 Hours/Units Focus)
Y-axis: Count
Y-axis ticks: 0, 50, 100, 150, 200
X-axis: quantification_numeric
X-axis ticks: 0, 20, 40, 60, 80, 100

AIE / LEAD
PRESENTED BY
Google DeepMind
AI CONSULTING IN PRACTICE
NLW / Founder and CEO
Superintelligent

### Primary benefits by organization size

Time savings is the largest benefit category across all five organization sizes. Organizations with 201–1,000 people have the largest share of use cases focused on increased output.

[11:37](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=697s) [Stacked bars compare the proportions of eight primary benefit categories across five organization size groups.](https://assets.ai.engineer/sites/aiecode2025/sha256/0217e21edaa80f08cf37ff9360ad4c10364d8e97d8640545cbe53f36744d7280.webp)

The story goes far beyond just saving time

ROI Category Distribution by Organization Size

Y-axis: Organization Size
Y-axis ticks: 5,000+; 1,001-5,000; 201-1,000; 51-200; 1-50
X-axis: Proportion
X-axis ticks: 0.0; 0.2; 0.4; 0.6; 0.8; 1.0

Primary Benefit:
costSavings
improvedDecisionMaking
increasedOutput
increasedRevenue
newCapabilities
qualityImprovement
riskReduction
timeSavings

AIE / LEAD
PRESENTED BY Google DeepMind
AI CONSULTING IN PRACTICE
NLW / Founder and CEO
Superintelligent

### Current ROI by role

C-level and founder respondents report the largest share of transformational ROI. Whittemore suggests their choice of use cases may partly explain the difference.

[12:51](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=771s) [Stacked bars compare negative, breakeven, modest, significant and transformational ROI across role categories; C-level and founder respondents have the largest transformational share.](https://assets.ai.engineer/sites/aiecode2025/sha256/2e2d8248d2ff777efe5748456b99cc6d583d93e69e2901a06facae6adbe2d623.webp)

Senior leaders reporting more significant benefits
C-suite most likely to report transformational impact

Current ROI by Role Category

Y-axis: title_level
Categories: Other; C-Level/Founder; VP; Director; Manager; Individual Contributor
X-axis ticks: 0.0, 0.2, 0.4, 0.6, 0.8, 1.0
Legend: negative; breakeven; modest; significant; transformational

Interface text:
AI Engineer
Engineering the future of AI

### Transformational ROI by organization size

The transformational ROI rate falls from 17.6% for organizations with 1–50 employees to 0.9% for those with 1,001–5,000, then rises to 7.2% for those with 5,000+.

[13:19](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=799s) [Bar chart reports transformational ROI rates of 17.6% for 1–50 employees, 9.3% for 51–200, 4.6% for 201–1,000, 0.9% for 1,001–5,000 and 7.2% for 5,000 or more.](https://assets.ai.engineer/sites/aiecode2025/sha256/7736a912aa479195659b9f24ca8653fe772c71c035a3f3faa01bcbd2749a3c8c.webp)

Organization Size — Who Wins Big
Smaller organizations convert faster to transformational outcomes.

Transformational ROI Rate (%)
Y-axis ticks: 0%, 2%, 4%, 6%, 8%, 10%, 12%, 14%, 16%, 18%, 20%
Organization Size (Employees)
X-axis categories: 1-50, 51-200, 201-1,000, 1,001-5,000, 5,000+

Small orgs lead at 17.6%

Transformational ROI rate by org size:
1-50  17.6% (highest)
51-200  9.3%
201-1,000  4.6%
1,001-5,000  0.9% (lowest)
5,000+  7.2%

Large enterprises show higher breakeven/negative mix due to complexity and integration debt.

Interface text:
AI Engineer
Engineering the future of AI

### Coding and software ROI

Of 978 coding and software use cases, 38.5% show high ROI and 3.9% negative ROI. Coding cases have a higher share with high ROI and a lower share with negative ROI than use cases overall.

[14:04](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=844s) [Chart compares coding and overall use cases; the coding subset contains 978 cases, with 38.5% high ROI and 3.9% negative ROI.](https://assets.ai.engineer/sites/aiecode2025/sha256/97d4e2f69568210543b1eb2474e317cf0bbd2f98c43506c068012ea75705ad72.webp)

Coding/Software — Strong ROI

Software-centric initiatives are a major share with attractive ROI and lower downside.

Coding vs. Overall ROI Comparison
Legend: Coding Use Cases; Overall Use Cases
Categories: High ROI; Negative ROI
X-axis: Percentage of Use Cases
X-axis ticks: 0%, 5%, 10%, 15%, 20%, 25%, 30%, 35%, 40%, 45%, 50%

978 coding/software use cases
38.5% High ROI
Only 3.9% Negative

Current ROI (coding subset):
Modest 44.6% (436)
Significant 26.3% (257)
Transformational 12.2% (119)
Breakeven 13.1% (128)
Negative 3.9% (38)

AI Engineer
Engineering the future of AI

### Risk Reduction Champions

Risk reduction was the primary benefit in only 3.4% of use cases, yet those cases had the highest transformational ROI rate at 25%. Examples span compliance, security and safety.

[14:14](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=854s) [Risk reduction has a 25% transformational rate and represents 3.4% of use cases; examples include compliance, security, and safety.](https://assets.ai.engineer/sites/aiecode2025/sha256/2a2f0b01e54abff15dc410a843cbf4256a572d1ab9424183b4629011865456e3.webp)

Risk Reduction Champions
Quiet Wins with High Transformational Share

Y-axis: Transformational ROI Rate (%)
Y-axis ticks: 0%, 5%, 10%, 15%, 20%, 25%, 30%
X-axis: Primary Benefit Category
X-axis categories: Risk Reduction, Time Savings, Cost Savings, Quality Improvement, Increased Output, New Capabilities, Decision Making, Increased Revenue

25% Transformational Rate
Only 3.4% of Use Cases

Winning Industries:
Healthcare — Compliance
Financial — Security
Manufacturing — Safety

Winning Patterns:
Document validation & bond/contract checks
Compliance audit trails & verification
Exception flagging & early warning systems
Automated safety monitoring protocols

Interface text:
AI Engineer
Engineering the future of AI

### The Automation Dividend

The 168 automation use cases have a higher share of high ROI and a lower share of negative ROI than the overall average. The speaker describes the ROI as self-reported.

[16:09](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=969s) [Automation use cases show higher high-ROI and lower negative-ROI shares than the overall average, with four workflow examples.](https://assets.ai.engineer/sites/aiecode2025/sha256/6a73458d354030dda4be2126d58d16b3295cbe9b0d3d8e03b0ec9623b70e8873.webp)

The Automation Dividend
Repetitive workflow automation and agentic assists deliver above-average ROI

ROI Performance
Categories: High ROI %, Negative ROI %
X-axis: Percentage of Use Cases
X-axis ticks: 0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50
Legend: Automation Use Cases, Overall Average

168 Automation Use Cases
+5.8% Higher ROI Rate

Successful Automation Examples:
Payment Processing
Automated validation and status reporting for supplier/contractor payments

Content Operations
Streamlined content creation, review, and distribution workflows

Backlog Estimation
Intelligent analysis of work items, reducing planning cycles

Test Automation
Automated QA processes with agent-driven test case execution

### The Agent Revolution

Among 130 agent use cases, 44% have high self-reported ROI. The chart also shows a higher transformational rate than non-agent cases; examples include support triage and deployment validation.

[16:29](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=989s) [Agent use cases have higher high-ROI and transformational shares than non-agent cases; the slide lists success patterns and enabling conditions.](https://assets.ai.engineer/sites/aiecode2025/sha256/66cd49d25fbda76e5e02395d3b1f987d9405e2bc5d0a05f400ecfb1a8bcd2384.webp)

The Agent Revolution
Moving from assistive AI tools to autonomous agents with higher ROI potential.

Chart legend: Agent Use Cases; Non-Agent Use Cases
Y-axis: Percentage (%)
Y-axis ticks: 0%, 5%, 10%, 15%, 20%, 25%, 30%, 35%, 40%, 45%, 50%
X-axis categories: High ROI Share; Transformational Rate

130 Agent Use Cases
44% High ROI

Success Patterns:
Customer support triage & resolution
Backlog estimation & prioritization
Deployment operations & validation
Partner prospecting & qualification

Keys to Success:
Tool access (APIs & data sources)
Knowledge base grounding
Defined guardrails & error handling
Human review on exceptions

### Systems Change vs. Spot Change

High ROI rises from 35% of single-impact cases (n=428) to 38% with 2–3 impact areas (n=1,214) and 45% with 4+ areas (n=835). The slide describes this as cross-functional value capture.

[16:38](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=998s) [High-ROI shares rise from 35% for single-impact cases to 38% for two or three areas and 45% for four or more.](https://assets.ai.engineer/sites/aiecode2025/sha256/a1cf11ce0c8b2683729e3c709fd44c7485518298dffc96368bdee3b282ce36fc.webp)

Systems Change vs. Spot Change

Use cases citing 4+ impact areas achieve 45% significant/transformational ROI vs 35% for single-impact cases.

Y-axis: Significant + Transformational ROI (%)
Y-axis ticks: 0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50
X-axis: Number of Impact Areas
X-axis categories: Single Impact; 2-3 Impact Areas; 4+ Impact Areas

Cross-functional value capture
Multiple impact areas indicate solutions that work across functional boundaries, capturing value in multiple dimensions simultaneously.

High ROI by impact area count:
Single Impact  35%  (n=428)
2-3 Impact Areas  38%  (n=1,214)
4+ Impact Areas  45%  (n=835)

High ROI = Significant + Transformational outcomes

### Systems Change vs. Spot Change

In this first look, the high-ROI share rises from 35% for one impact area (n=428) to 38% for 2–3 (n=1,214) and 45% for 4+ (n=835).

[17:05](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=1025s) [A presenter stands beside a slide comparing high-ROI shares across one, two or three, and four or more impact areas.](https://assets.ai.engineer/sites/aiecode2025/sha256/7122e00e9e07c03894c53b198f44643b473f4863a89f0b375d70e7af14ccff56.webp)

Systems Change vs. Spot Change

Use cases citing 4+ impact areas achieve 45% significant/transformational ROI vs 35% for single-impact cases.

Y-axis: Significant + Transformational ROI (%)
Y-axis ticks: 0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50
X-axis: Number of Impact Areas
X-axis categories: Single Impact; 2-3 Impact Areas; 4+ Impact Areas

Cross-functional value capture
Multiple impact areas indicate solutions that work across functional boundaries, capturing value in multiple dimensions simultaneously.

High ROI by impact area count:
Single Impact 35% (n=428)
2-3 Impact Areas 38% (n=1,214)
4+ Impact Areas 45% (n=835)

High ROI = Significant + Transformational outcomes

AIE / LEAD
PRESENTED BY
Google DeepMind

AI CONSULTING IN PRACTICE
NLW / Founder and CEO
Superintelligent

### AI Engineer Code Summit

[18:12](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=1092s) [AI Engineer Code Summit branding and website address on a black background.](https://assets.ai.engineer/sites/aiecode2025/sha256/b30ca2841b9b29f685891124736a3ffc4ff250963de13628e4cf42817dcc6333.webp)

AI Engineer
Code Summit
https://ai.engineer

## Full transcript

[0:00](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=0s) [on-hold music] Today, I'm excited to talk, uh, about something a little bit different.

[0:24](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=24s) So right now, uh, there's been-- the last couple of months have been an interesting time in AI. There's been a sort of surge in the air, uh, the narrative of an AI bubble, a lot of it driven by dubious studies, uh, like the MIT report.

[0:38](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=38s) And so what I wanted to do today is get into not so much the practice of consulting and transforming, but what organizations are actually finding value in right now.

[0:49](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=49s) So for those of you who don't know me, uh, there's kind of two contexts I bring to this conversation. The first is as the host of The AI Daily Brief, which is a daily, uh, news analysis podcast about AI.

[0:57](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=57s) The second is as the CEO of Superintelligent, which is an AI planning platform. So the different perspectives are sort of very high-level macro thinking about the news that's happening, and then a much more kind of ground-level view where we're spending a ton of time interviewing executives about what's going on inside their organizations.

[1:13](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=73s) And what we're going to talk about is sort of, one, kind of briefly in the first part, just the status of enterprise adoption, uh, as it currently stands. And two, um, and the more interesting part is we've been live with a study in, in the market for about a month now, collecting self-reported information about ROI around different

[1:31](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=91s) use cases. And this will be the first time, uh, this week was the first time I did some analysis on it. And so I'm going to share what people have, uh, what people have told us around the first kind of twenty-five hundred or so use cases that they've shared.

[1:42](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=102s) Um, so it should be pretty, pretty interesting stuff.

[1:45](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=105s) Talking about kind of enterprise AI adoption first, I'll go through this pretty quickly because it's, um, pretty well-known stuff. Uh, the short of it is enterprises are adopting AI, uh, in a, in a growing fashion.

[1:56](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=116s) Um, pretty much everyone is using it a-at least a little bit, uh, and increasingly they're using it a lot.

[2:02](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=122s) Uh, this year, I will need to tell none of you that there is a major inflection around, um, specifically adoption in the, uh, coding and software engineering, right? You saw a huge, huge uptick in this.

[2:12](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=132s) Um, there's a lot that's interesting about that from an enterprise perspective because it wasn't just with the software engineering organizations. Other parts of the organization are also now thinking about how they can communicate with code, build things with code.

[2:23](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=143s) Uh, but that's a huge, huge theme of this year.

[2:26](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=146s) Coming into twenty twenty-five, one of the big sort of thoughts that many people had was that this would be the year of agents inside the enterprise, right? That big chunks of work would get automated away.

[2:36](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=156s) And on the one hand, I think it's pretty clear that we didn't see some sort of mass shift towards automation, uh, writ large across different functions of the organization.

[2:46](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=166s) But when you dig into the numbers, there has been actually pretty significant, uh, shifts in the patterns of, of agent adoption. So this is from KPMG's Quarterly Pulse Survey, and it's a measure of how many enterprises that are a part of their survey, which is all companies over a billion dollars in revenue, have, uh, actual sort of

[3:04](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=184s) full production agents in deployment. So this isn't pilots. This isn't experiments. This is where they consider, uh, some agent that's actually doing, doing kind of work in a, in a full way.

[3:13](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=193s) And it's jumped from eleven percent in Q1 of this year to forty-two percent in their most recent study for Q3. So you actually are seeing pretty meaningful uptake of, of agents inside the enterprise.

[3:23](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=203s) In fact, I would argue based on our conversations that people have-- that it's moved more quickly through the pilot or experimental phase than people might have thought. Um, so much so that you're actually seeing now a big shift in the emphasis around kind of the human side of agents and how humans are going to interact with agents.

[3:42](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=222s) And it's involving a shift in upskilling and, and, uh, and enablement work. Um, you're seeing a decrease in the sort of resistance to agents as people start to actually dig in with them.

[3:52](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=232s) You're seeing more experiments like these sandboxes where people can interact with agents. So this is a big theme, even if it wasn't necessarily the dominant theme that some thought it might be coming into this year.

[4:02](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=242s) At the same time, it is absolutely the case that many, many, if not most enterprises are, broadly speaking, stuck inside sort of pilot and experimental phases. There is a, a lot of challenge around moving from some of those first exciting experiments to something that's more scaled.

[4:19](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=259s) Um, so this is from, uh, McKinsey State of AI study, which came out, I think, a couple weeks ago now. And you can see only seven percent of the organizations that they talked to claim or sort of see themselves as, as fully at scale with, with AI and agents.

[4:30](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=270s) And it's something like sixty-two percent are either still experimenting or piloting.

[4:35](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=275s) Interestingly, big organizations are on-- in general, a little bit ahead in terms of, uh, the organizations that are scaling as compared to smaller organizations. This has been a, a thing that we've noticed kind of throughout the trajectory of, uh, uh, of AI, um, adoption over the last couple of years, that you would think that perhaps smaller, more

[4:54](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=294s) nimble companies, uh, would be more kind of quick to adopt these things. But in fact, it's often been the opposite, with the biggest organizations making the biggest efforts. You can also see from the chart on the bottom that there's very sort of jagged patterns of adoption, right?

[5:08](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=308s) You're starting to see, uh, from, you know, last year if you looked, there's very similar kind of rates of experimentation across lots of different departments. You're starting to see some pretty big breakouts now, uh, with, for example, you know, IT operations kind of jumping out ahead of other functions.

[5:24](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=324s) I won't spend too much time on this sort of high performer piece, but I think the thing to note, because it comes back in, in, in some of the stuff that we found with our ROI study, is that you are also starting to see a pretty significant bifurcation between leaders and laggards when it comes to AI adoption.

[5:39](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=339s) And one of the things that tends to distinguish the companies that are leading is that they are just doing more of it, and they are thinking more comprehensively and systematically about AI and agent adoption.

[5:50](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=350s) So they're not just sort of doing spot experiments. They're thinking about their strategy as a whole. They're doing multiple things at once. And importantly, they're not just thinking about sort of the very kind of first-tier time savings or productivity types of use cases.

[6:03](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=363s) They're also thinking about how do we grow revenue, how do we create new capabilities, how do we create new product lines.

[6:10](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=370s) Overall, it's very clear that despite what is sort of, you know, the, the, the concerns in the media, that spend is going to do nothing but increase on this.

[6:19](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=379s) Um, so the bottom is the KPMG pulse survey again, and this is a, a, an estimation of the amount of money that these organizations intend to spend on AI over the next twelve months.

[6:28](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=388s) At the beginning of the year, it was a hundred and fourteen, which by the way was up from like eighty-eight in Q4 of last year. It's now up to, in their last study, a hundred and thirty million is what they expect to spend, uh, in the, in the year ahead, which obviously the, the total magnitude doesn't matter

[6:40](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=400s) as much as the change. Um, you also see the green charts are from Deloitte, and you can see ninety percent plus of organizations intend to increase their spend o-on AI in the next twelve months.

[6:51](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=411s) And as part of that, I think that you're going to see a much more determined conversation around impact and ROI, uh, which is a particularly thorny topic. But interestingly, there has been an increase in optimism over the course of this year around the realization of ROI.

[7:09](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=429s) So this is from a different KPMG study, their annual CEO survey, which interviews tons and tons of CEOs. And if you look at the 2024 numbers, sixty-three percent of those polled thought that it would take between three and five years to realize ROI from their AI investments.

[7:25](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=445s) Twenty percent said one to three, and sixteen percent said more than five. This year, in that same survey, the number that said one to three years had gone up to sixty-seven percent.

[7:34](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=454s) There were now nineteen percent who said six months to one year, uh, and three to five years was down to just twelve percent. So huge, huge kind of pull forward of expectations of, of ROI realization.

[7:46](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=466s) The challenge is that ROI is really tough. So this is back to the pulse survey. Seventy-eight percent of those polled in that, in that survey said that they thought that ROI was going to basically become a bigger consideration in the year to come.

[8:00](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=480s) Uh, but also seventy-eight percent said that traditional impact metrics and measures were having a very hard time keeping up with the, with the new reality that we're living in.

[8:09](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=489s) And this is something that I've heard constantly over and over from CIOs and other people who are in charge of these investments, that the, the, the ways that we have measured impact of previous technologies and just previous initiatives are kind of falling flat with AI.

[8:22](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=502s) And so that got us thinking about the, the, the overall need that we have to just have more information. I'm not even talking about good, systematic information, just more information around what ROI looks like, what impact looks like.

[8:36](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=516s) And you know, I've got this great podcast audience. They're super engaged. And so we just decided, screw it, we're gonna ask them. We're just gonna ask them to report on what ROI they're finding from their use cases.

[8:47](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=527s) So this went up at the very end of October. Uh, like I said, as of this morning or when I looked-- last looked, we've had over a thousand submissions, uh, a thousand individual organizations rather submit something like thirty-five hundred use cases.

[8:58](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=538s) And, um, this is, uh, some, some of the first observations that we had around, um, kind of the first twenty-five hundred. So the impact categories, the way that we divided things was into sort of eight broad categories of impact, um, which will all, I think, be very intuitive to you guys.

[9:14](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=554s) Time savings, increased output, improvement in quality, new capabilities, improved decision-making, cost savings, increased revenue, and risk reduction. So basically, we're trying to think of like kind of a, a broad, simple heuristic for, uh, for, for kinda dividing or subdividing the different, the different ways that people are thinking about ROI.

[9:32](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=572s) And TLDR is that people are finding, uh, ROI right now. Um, now again, the caveats are that this is a highly enfranchised audience. They're listening to a daily AI podcast, and they are voluntarily sharing this.

[9:45](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=585s) So I think that, you know, there's, there's some caveating there. But you have forty-four point three percent saying that they're seeing modest ROI right now, and then you have another thirty-seven point six percent seeing high ROI.

[9:56](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=596s) For the purposes of a lot of these stats, high ROI will be significant plus transformational. Uh, only five percent or so are seeing negative ROI. And keep in mind, negative ROI doesn't mean that they think programs are failing.

[10:07](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=607s) That just means they haven't-- they've spent more than they've gained or, uh, uh, in terms of how their, their perception is. More than that, expectations are absolutely sky-high. Sixty-seven percent think over the next year they will see, uh, increase and high growth in their ROI.

[10:22](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=622s) So we've really optimistic sense from the ground view of where ROI is going to be in AI. Um, you even have the teams that are currently experiencing negative O-- uh, uh, ROI, fifty-three percent say that they're gonna see high growth.

[10:36](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=636s) So very, very optimistic. Um, as you might imagine, time savings is the default. It's the starting point for so many organizations. It represents about thirty-five percent of the use cases.

[10:47](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=647s) After that, increasing output, quality improvement, basically all those things that you would imagine around productivity are sort of like the dominant categories when it comes to these, uh, when it comes to these use cases.

[10:57](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=657s) When it comes to the specifics around time savings, you see a real cluster between one and ten hours, especially right around five hours. And I think this is interesting to call out because it's so obvious to all of us who are inside building these things, uh, whether you are a developer or an entrepreneur or just someone sort

[11:13](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=673s) of in and around it, how the, the vast breadth of opportunity that AI represents, new capabilities, things unimagined yet. It's hard to-- or it's easy to forget that if you save five hours a week or ten hours a week, you're talking about winning back seven to ten work weeks a year.

[11:29](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=689s) Uh, and that's very, very powerful. And when it comes to a lot of these enterprises, that is a very meaningful thing, even if it's not what they're ultimately in it for.

[11:39](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=699s) Interestingly, though, it's very clear that the story, although it might be, uh, have a concentration in time savings, is about much more than time savings. So this is the ROI distribution category, uh, uh, ROI distribution by organization size.

[11:52](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=712s) And this starts to get really interesting where you can see that there are some differences in where different size organizations are focused. So for example, the organization size between two hundred and a thousand people has a higher portion of their use cases concentrated in increasing output.

[12:09](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=729s) Now, we haven't taken the time yet to really figure out exactly what this means or even speculate on, on what this means. But I think it's interesting that this is a category of organization that has often reached a certain scale but is still very much striving for more and so seems to be focused more on use cases

[12:24](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=744s) that expand their capabilities. Same thing with, uh, when you start to divide things by role. You see a real kind of variance where, for example, C-suites and leaders, uh, are less focused on those time savings use cases and more focused on other things like increased output and, uh, and new capabilities.

[12:45](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=765s) In general, we're finding that C leaders, uh, and just sort of C-suite and, and leaders in general are even more optimistic and excited and, and seeing transformational impact than people who are in more junior positions.

[12:57](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=777s) Now, some of this might be sort of selection bias in terms of, um, what types of use cases you are focused on. If you are in that C-suite, you're thinking about things that inherently, if they work, are more transformational.

[13:08](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=788s) Uh, but it is notable that seventeen percent of, uh, of the use cases that, that people in those leadership positions have submitted, uh, they say have transformational impact and ROI already.

[13:19](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=799s) Uh, I'm gonna skip this 'cause there's-- we, we don't have time for too much. Um, you're seeing, uh, uh, interestingly, uh, a concentration, um, where the smallest organizations are getting more of that transformational benefit early.

[13:32](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=812s) Um, one of the things that I wanna do following this study is maybe do a sort of second round where we dig into what this one to fifty person, uh, size really looks like.

[13:44](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=824s) I actually think that whereas there might be a lot of similarity between a one thousand and a two thousand person organization, there could be a wild difference between a three-person, you know, small company and a forty-person company.

[13:56](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=836s) And so I'd really like to dig into that more. But you are definitely seeing a, a, a lot of impact in those sort of more small, nimble-moving organizations. Uh, as you might expect, coding and, uh, and software related or, uh, use cases have a higher ROI than average and a lower negative ROI than average.

[14:15](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=855s) Um, one really interesting kind of, you know, pulling on a specific category of use cases, risk reduction is our lowest category in terms of the percentage of use cases that, that that was their primary benefit.

[14:27](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=867s) So when you're filling out the survey, which is by the way at roisurvey.ai if you wanna check it out, uh, you basically only get to pick a primary ROI benefit.

[14:37](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=877s) We didn't want it to be super sort of, um, we wanted you to pick and, and hone in on the thing that was, uh, seemed most important or most significant.

[14:44](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=884s) And so only three point four percent have risk reduction as their primary benefit, uh, in terms of ROI categories. But it is by far, those use cases are by far the most likely to have transformational impact as, as the, a-as, as their outcome.

[15:01](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=901s) It's at twenty-five percent, so a full quarter of those, uh, have transformational ROI. And interestingly, I was having this conversation with a couple of my friends who work in sort of back office and compliance and risk functions, and this has been their experience as well, where there are a lot of, uh, a, a lot of the, the,

[15:17](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=917s) the challenges for those organizations involve sheer volume and quantity, uh, i-in ways that, that AI can be really helpful for. We also are finding some interesting patterns among organizations.

[15:29](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=929s) And again, this is where we get into some of the limits of this just being a whoever walks through the door of my listeners. We have a pretty heavy concentration among technology, as you might expect, industries and among professional services.

[15:40](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=940s) But we still have fairly decent sample sizes for some others. And in both healthcare and manufacturing, the use cases are meaningfully higher impact on average, uh, than the average across all organizations.

[15:51](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=951s) Um, which I think is, uh, it, it was kind of worthy of further study.

[15:56](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=956s) Last sort of part of this as I wrap up, you know, a lot of these use cases, as you saw, have to do with that sort of first tier that most enterprises are gonna be in.

[16:05](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=965s) Uh, increasing the amount of content that you output, increasing the quality of that content, just finding ways to win back, you know, your five hours a week. Um, but increasingly, there are automation and agentic use cases, and we are absolutely seeing that where those are the, the focus, where those use cases mention certain types of automation or

[16:24](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=984s) they mention agents, they wildly outperform in terms of the self-reported ROI from them. That's both on automation and it's on agents. And I think that that's sort of a, a trend towards where we're headed with sort of the next layer of more advanced use cases.

[16:40](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=1000s) The last thing that, uh, from this sort of first, first look of observations is there is clearly benefits, and this goes back to, to what we saw with that McKinsey study as well, of thinking about AI and agentic transformation in systematic cross-organizational, cross-disciplinary types of terms.

[16:59](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=1019s) Um, effectively, pretty much, uh, directly, the more use cases that a, a person or an organization submitted, the, the better they tended to see a ROI for. Now, there's lots of reasons for that, but I do think it speaks to that, that core idea that once you move beyond kind of your single spot experiments, there's a lot

[17:18](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=1038s) of opportunity, uh, to, to sort of grow, grow the impact of the organization. So like I said, that is the, the first look. Uh, it's kind of the first two-thirds of these, uh, of these use cases.

[17:28](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=1048s) We'll be open for another week, and then we'll have the full study out at the beginning of December. Um, I'm really excited, I think, heading into next year to see how we move from sort of generic conversations about impact, uh, and our gut senses about impact to a lot more random experiments like this to figure out where

[17:46](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=1066s) the impact really is and, uh, and where we go next. So look at that. I'm gonna end twenty-seven seconds early and really throw off the time. But appreciate you guys all being here.

[17:56](https://www.youtube.com/watch?v=ehQFj6VmuI8&t=1076s) Uh, and again, if you wanna check this out, it's at roisurvey.ai. [upbeat music]
