# Trends Across the AI Frontier

Micah Hill-Smith · George Cameron

AI Engineer World's Fair 2025

[Talk page](https://ai.engineer/talks/sRpqPgKeXNk-trends-across-ai-frontier) · [Video](https://www.youtube.com/watch?v=sRpqPgKeXNk)

## Trends Across the AI Frontier

Model intelligence is only one constraint: reasoning tokens, sequential latency, open weights, cost and inference speed determine what an application can actually do.

Familiarity with language-model APIs, tokens and basic inference terminology will help you follow the comparisons.

### What are you optimizing for?

[0:45](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=45s)

How should you compare AI systems when model intelligence, API performance and hardware performance measure different things? [Artificial Analysis](https://artificialanalysis.ai/) benchmarks all three: model capability, endpoint speed and cost, and the accelerators serving inference. Its coverage extends beyond language to vision, speech, image generation and video generation. At the time of George Cameron’s presentation, the company covered more than 150 models, published most results freely, and offered reports and enterprise subscriptions to support production adoption. Those separate measurements provide the basis for choosing a model that fits an application.

### Intelligence is only one frontier

[1:54](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=114s)

The launch of ChatGPT and **GPT-3.5** began the historical arc: more releases followed, from more labs, with increasingly crowded competition at the frontier. In the presentation’s ranking, **o3** led, followed closely by **o4-mini** with high reasoning. Other leading models included **[DeepSeek-R1](https://github.com/deepseek-ai/DeepSeek-R1)**, **Grok 3 Mini** with high reasoning, **Gemini 2.5 Pro**, and **Claude 4 Opus Thinking**.

![Step-line chart of the Artificial Analysis Intelligence Index by release date, with colored lines for six AI labs and labels for recent models.](https://ai.engineer/api/talk-essay-media/d6bc4b85253ba91fa9712e4b28cb7a0a9bf5d06167a494b761ffa2787871265f)

Frontier model intelligence over time shows increasingly close competition among AI labs.

The ranking used the **Artificial Analysis Intelligence Index**, a weighted composite of seven evaluations intended to provide a generalist view of capability. These are historical measurements using that seven-evaluation index and the configurations and prices reported in the talk; the [current methodology](https://artificialanalysis.ai/methodology/intelligence-benchmarking) and live rankings have since changed.

**The most intelligent model is not always the right model.** Accessing intelligence entails trade-offs, so a useful comparison must examine several frontiers. The progression here is reasoning versus non-reasoning models, open weights versus proprietary models, cost, and speed. Each changes the set of applications that is practical to build.

### Reasoning buys capability with more tokens

[4:06](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=246s)

Plot Intelligence Index scores against the output tokens required to complete the evaluations, and two groups emerge. Non-reasoning models use fewer output tokens but achieve lower measured intelligence in this comparison. Reasoning models achieve higher scores while generating substantially more tokens. The additional generation matters twice: it adds expense and extends the time required to finish a request.

The reported evaluation runs show the scale of that difference. **GPT-4.1** required millions of output tokens; the reasoning models required tens or hundreds of millions.

| Model | Output tokens for the full Intelligence Index run |
| --- | ---: |
| GPT-4.1 | 7 million |
| o4-mini, high reasoning | 72 million |
| Gemini 2.5 Pro | 130 million |

These are totals across the seven evaluations, not the length of an individual response. Cameron describes an order-of-magnitude difference between reasoning and non-reasoning models—large enough that token volume must be treated as an application constraint alongside capability.

### Measure the wait for a complete answer

[5:53](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=353s)

Across Artificial Analysis’s tested API requests, GPT-4.1 had a median full-response latency of 4.7 seconds; o4-mini with high reasoning took over 40 seconds in the reported comparison. The slide uses a 500-output-token workload, including reasoning time. That is roughly an order of magnitude more waiting. **Full-response latency** measures the wait for the complete answer, rather than time to first token or the rate at which tokens arrive. The slide separates input processing, thinking and output generation, making visible why a fast output stream alone does not describe the entire interaction.

![Stacked bar chart comparing model response times, with a shaded reasoning-model region and a purple 9X comparison arrow.](https://ai.engineer/api/talk-essay-media/d8e34cb9ecec4817b2984125fc3295e704ec459a8b1408ed80b1b70d0d6a640b)

End-to-end response times separate input processing, thinking and output generation.

That wait affects enterprise chatbots as well as consumer applications. A user does not necessarily need the strongest reasoning model for every interaction, particularly when responsiveness is part of the product’s value. Cameron invokes Facebook studies of latency-related user drop-off as supporting motivation, without giving a specific study or numerical result. The actionable question is whether the additional capability justifies the delay for the interaction being built.

### An agent multiplies per-call latency

[6:57](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=417s)

For an agent making successive requests, latency accumulates along the chain. Cameron uses 30 sequential queries as a plausible workload. Faster responses can change the workflow itself: instead of merely completing the same 30 calls sooner, an application might fit something closer to 100 calls into its waiting budget.

In the illustrative agent calculation, 30 sequential calls at 10 seconds each take 300 seconds, or five minutes; at one second each, they take 30 seconds. This is arithmetic for a sequential workload, not a measured agent benchmark. A small TypeScript calculation makes the dependency explicit:

```typescript
const calls = 30;

const scenarios = [
  { secondsPerCall: 10 },
  { secondsPerCall: 1 },
];

console.table(
  scenarios.map(({ secondsPerCall }) => {
    const totalSeconds = calls * secondsPerCall;
    return {
      calls,
      secondsPerCall,
      totalSeconds,
      totalMinutes: totalSeconds / 60,
    };
  }),
);
```

The calculation isolates model-call time. An actual workflow must also account for whatever happens between calls, but the sequential dependency alone already creates a substantial difference.

A contact-center application makes the product constraint concrete: a 30-second wait might be acceptable, while five minutes on the phone could make the interaction unusable. Likewise, imagine waiting five minutes every time you invoked a function in Google. The latency budget determines which workflows are viable, so measure the trade-off in the application rather than assuming a higher intelligence score will compensate for the wait.

### Open weights close the capability gap

[8:25](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=505s)

The open-weights frontier has moved unevenly. Around the release of **GPT-4**, **Llama 65B** and **Llama 2 70B** were substantially behind proprietary capability. Later releases such as **Mixtral 8x7B** and **Llama 405B** narrowed the gap. Then **o1** pulled ahead again in late 2024, reopening the separation.

**[DeepSeek-V3](https://arxiv.org/abs/2412.19437)** arrived in December, prompting Cameron to interrupt his Christmas holiday to read its technical report. R1 followed in January. By the presentation, Cameron reported that the recent R1 release was only a couple of Intelligence Index points behind the leading models. The progression matters: open weights were catching up not only to an older proprietary generation, but also to the newer reasoning frontier.

China-based labs led both the reasoning and non-reasoning open-weights rankings shown in the talk, with DeepSeek first in each. Alibaba’s **Qwen 3** series followed in reasoning. Meta and NVIDIA’s **Nemotron** fine-tunes of Llama were also competitive. The resulting choice was broader than a single leading proprietary API: open-weights models had become close contenders on the measured capability axis.

### Compare the cost of completing the work

[10:26](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=626s)

Cost changes what can be built just as latency does. At the prices used for the presentation’s evaluations, o3 cost almost $2,000 to complete the Intelligence Index. The cost of running evaluations had become substantial enough to attract TechCrunch coverage. Cameron then compared GPT-4.1 and **GPT-4.1 Nano** with more expensive reasoning models:

| Comparison on the Intelligence Index workload | Reported cost difference |
| --- | --- |
| GPT-4.1 versus o1 | Roughly 30 times cheaper |
| GPT-4.1 Nano versus o3 | More than 500 times cheaper |

The denominators differ: GPT-4.1 is compared with o1, while Nano is compared with o3. These ratios describe the evaluation-suite workload.

An application’s cost structure may therefore determine both which model it can use and how many times it can call that model. Cameron pushes the idea toward agents, suggesting that hundreds of inexpensive calls could cost less than one expensive call. That is a design possibility to investigate, not a universal per-query ratio established by the suite totals; actual prompts, responses and reasoning lengths determine the bill.

**Price per token is only half of the cost calculation.** The other half is how many tokens the model consumes and produces. In the billing behavior discussed here, reasoning tokens count as output tokens even when the provider does not expose their contents. A model can therefore have an attractive advertised price per million tokens and still be expensive to use because it generates much more internal reasoning or a longer answer.

Verbosity also varies among non-reasoning models. Compare the cost of completing the same application workload, including billed reasoning tokens, instead of assuming that every model will generate the same amount of text. The evaluation-suite comparison is useful precisely because it measures token quantity and token price together.

### Hold capability constant and watch cost fall

[12:50](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=770s)

To study cost over time, group models into **intelligence bands**. This asks what it costs to access a comparable level of capability, rather than what the newest frontier model costs. Cameron reports that the cost of accessing GPT-4-level intelligence fell by more than 100 times from mid-2023 to the presentation. Declining costs appeared across the plotted quality bands.

Even a newly reached band could become cheaper quickly. Cameron reports that the cost of accessing the intelligence band reached by **o1-mini** in late 2024 halved within a few months. That creates a useful planning exercise: what would you build if the current inference cost were no longer a barrier? An application whose economics fail today might become feasible six months later. That possibility is a reason to examine future cost scenarios, not a guarantee that a particular project’s economics will work on that schedule.

### The same capability arrives faster

[14:09](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=849s)

The speed frontier measures **output tokens per second**. Because capability and speed often trade off, intelligence bands again provide the useful comparison: how quickly can an application receive output at a given capability level? The plotted bands all show rising output speed.

Cameron recalls GPT-4 generating about 40 output tokens per second in 2023 and reports access to comparable intelligence at over 300 tokens per second by the presentation. The practical memory is familiar: waiting for ChatGPT to finish emitting code before copying it into an editor and trying it. Faster generation shortens that wait even when the model is not performing an extended reasoning step.

Several changes contribute to faster inference:

- **Model sparsity:** Mixture-of-experts models activate only a fraction of their parameters during inference. Less computation per token can mean faster generation, and increasingly sparse models extend that advantage.
- **Smaller, more capable models:** Distillation helps smaller models, including the cited 8B examples, deliver more intelligence without requiring the same model size.
- **Inference software:** [FlashAttention](https://arxiv.org/abs/2205.14135) and [speculative decoding](https://arxiv.org/abs/2211.17192) are two named optimization techniques.
- **Hardware:** The progression from A100 to H100 and then B200 systems provides another source of improvement.

Cameron reports that newly published Artificial Analysis B200 benchmarks exceeded 1,000 output tokens per second. The talk does not specify the model or workload for that result, so it is not a controlled hardware comparison with the earlier GPT-4 figure. Specialized accelerators from Cerebras, SambaNova and Groq add further options.

![Step-line chart of model output speed over release dates, grouped by intelligence bands, beside a panel listing sparsity, smaller models, inference software optimizations and hardware improvements.](https://ai.engineer/api/talk-essay-media/adacaddba1c96188b51ff1768aee66f161117ffb8c9b4c895427f791257a5496)

Output speed rises across intelligence bands, with four key inference drivers listed alongside.

### Efficiency can increase the appetite for compute

[16:34](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=994s)

Lower costs at a fixed intelligence level, better hardware and greater system output throughput all improve efficiency. Yet Artificial Analysis’s house view is that aggregate compute demand will continue to rise. Efficiency gains change what is affordable, while the desired scale of model use continues to expand.

Larger models are one driver. Cameron points to DeepSeek having more than 600 billion total parameters, explicitly correcting himself from active to total parameters. That distinction matters for sparse models: the full parameter count and the amount activated during inference are different quantities.

The other multipliers follow directly from the earlier comparisons. Demand for greater intelligence encourages more inference-time reasoning, which consumes additional compute. Agents turn one user task into sequences of 20, 30 or more than 100 model requests. More capable models, longer reasoning and more calls per task can absorb the savings from cheaper, faster inference. The closing expectation is therefore rising net compute demand—a house view about how these forces combine, rather than a quantified forecast.

## Resources

- [Artificial Analysis](https://artificialanalysis.ai/) — Interactive comparisons of model intelligence, token use, cost and API speed, with results updated over time.
- [Intelligence Index methodology and version history](https://artificialanalysis.ai/methodology/intelligence-benchmarking) — Current evaluation methods and a version history explaining how the Intelligence Index has changed.
- [DeepSeek-V3 Technical Report](https://arxiv.org/abs/2412.19437) — Describes DeepSeek-V3's mixture-of-experts architecture, with 671 billion total parameters and 37 billion activated per token.
- [DeepSeek-R1](https://github.com/deepseek-ai/DeepSeek-R1) — Official project repository for the DeepSeek-R1 reasoning-model family.
- [FlashAttention](https://arxiv.org/abs/2205.14135) — Introduces exact attention that reduces GPU memory transfers through tiling.
- [Fast Inference from Transformers via Speculative Decoding](https://arxiv.org/abs/2211.17192) — Explains how smaller-model proposals and parallel verification accelerate decoding while preserving the target output distribution.

## Slides

### AI Engineer World’s Fair

[0:01](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=1s) [White AI Engineer World’s Fair logo on a black background.](https://assets.ai.engineer/sites/aiecode2025/sha256/152e5540b17188c53681af561070a6f15662d862322df7046ca7cd2333d7a0aa.webp)

AI Engineer
World’s Fair

### Event partners and sponsors

[0:11](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=11s) [AWS innovation partner logo above six platinum sponsor logos.](https://assets.ai.engineer/sites/aiecode2025/sha256/d2ae3f1053a6492780e683137435e2fa9915ef9148871502286f5d9151daa057.webp)

INNOVATION PARTNER
aws

PLATINUM SPONSORS
Graphite
Windsurf
MongoDB®
daily™
augment code
WorkOS

### Trends Across the AI Frontier

George Cameron, co-founder of Artificial Analysis, presents Trends Across the AI Frontier.

[0:39](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=39s) [Artificial Analysis title slide for Trends Across the AI Frontier, presented by George Cameron.](https://assets.ai.engineer/sites/aiecode2025/sha256/9b2ab19ed555714126297e00d05b7ea3c9bcc68a0265c0c0791b8a8484fa7a8b.webp)

Artificial Analysis
Trends Across the AI Frontier

Presented by
George Cameron, Co-founder at Artificial Analysis

Interface text:
AI Engineer
World’s Fair
Microsoft
aws
smol ai

### Artificial Analysis benchmarking overview

Artificial Analysis benchmarks intelligence, speed, cost and AI modalities. Its website covers >150 models and >800 API endpoints; it also publishes reports and offers custom benchmarking.

[0:55](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=55s) [Overview of Artificial Analysis benchmarking coverage, website, reports, and custom benchmarking.](https://assets.ai.engineer/sites/aiecode2025/sha256/2dafb6bf01b6398b233f84e62b117ff71181bd30abef4515f15f7baecde6385c.webp)

Artificial Analysis is a leading independent AI benchmarking company

Benchmarking Coverage
Intelligence
Speed
Cost
Surveys & Market analysis
API Providers
Hardware
Speech
Video
Image

Website & Reports
Website hosting benchmarks of >150 models, >800 API endpoints & more
https://artificialanalysis.ai/
- In-depth reports, for enterprises and the wider community
- Custom benchmarking

Embedded website and report screenshots: [illegible]
Artificial Analysis

Interface text:
AI Engineer World’s Fair
Engineering the future of AI

[1:38](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=98s) [Overview of benchmarking categories, website coverage and reports.](https://assets.ai.engineer/sites/aiecode2025/sha256/86dbf7c7fd48e76d4dda9067e7a6b1b48bc666d9abf38f154bdedc02b4e28a82.webp)

Artificial Analysis is a leading independent AI benchmarking company

Benchmarking Coverage
Intelligence
Speed
Cost
Surveys & Market analysis
API Providers
Hardware
Speech
Video
Image

Website & Reports
Website hosting benchmarks of >150 models, >800 API endpoints & more
https://artificialanalysis.ai/
- In-depth reports, for enterprises and the wider community
- Custom benchmarking

Embedded website and report screenshots: [illegible]
Artificial Analysis

### Leading language models by intelligence score

o3 and o4-mini (high) both score 70, with DeepSeek R1 and Gemini 2.5 Pro at 68. The Intelligence Index combines seven weighted evaluations to give a generalist view of model intelligence.

[2:29](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=149s) [Bar chart comparing intelligence index scores for leading language models from several AI labs.](https://assets.ai.engineer/sites/aiecode2025/sha256/afbc47ff307dca4af8eb586ce4b73e4fc3f0ab379150ee3564ca4e47cbaf06ef.webp)

Frontier Intelligence: OpenAI, Google, DeepSeek and xAI lead frontier intelligence with their latest reasoning models, followed closely by other labs

Leading Large Language Model (LLMs), by AI lab
Artificial Analysis Intelligence Index (incorporates MMLU-Pro, GPQA, Humanity's Last Exam, LiveCodeBench, SciCode, AIME, MATH-500)

Models and scores:
o3: 70
o4-mini (high): 70
DeepSeek R1 0528 (May '25): 68
Gemini 2.5 Pro (May '25): 68
Grok 3 mini Reasoning (high): 67
Claude 4 Opus Thinking: 64
Qwen3 235B (Reasoning): 62
Llama Nemotron Ultra Reasoning: 61
Claude 3.7 Sonnet Thinking: 57
DeepSeek V3 0324 (Mar '25): 53
GPT-4.1: 53
Llama 4 Maverick: 51
Mistral Medium 3: 49
Nova Premier: 43
GPT-4o (Nov '24): 41

Source: Artificial Analysis independent benchmarking
Artificial Analysis

Interface text:
AI Engineer World's Fair
aws
Engineering the future of AI

[3:02](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=182s) [Bar chart comparing AI models on the Artificial Analysis Intelligence Index; o3 and o4-mini lead at 70.](https://assets.ai.engineer/sites/aiecode2025/sha256/e8398095d51d4a6c3f664b7f0600627eccbc9feb09dd9be3256046d4fe46eefc.webp)

Frontier Intelligence: OpenAI, Google, DeepSeek and xAI lead frontier intelligence with their latest reasoning models, followed closely by other labs

Leading Large Language Model (LLMs), by AI lab
Artificial Analysis Intelligence Index (incorporates MMLU-Pro, GPQA, Humanity's Last Exam, LiveCodeBench, SciCode, AIME, MATH-500)

Models and scores:
o3: 70
o4-mini (high): 70
DeepSeek R1 0528 (May '25): 68
Gemini 2.5 Pro (May '25): 68
Grok 3 mini Reasoning (high): 67
Claude 4 Opus Thinking: 64
Qwen3 235B (Reasoning): 62
Llama Nemotron Ultra Reasoning: 61
Claude 3.7 Sonnet Thinking: 57
DeepSeek V3 0324 (Mar '25): 53
GPT-4.1: 53
Llama 4 Maverick: 51
Mistral Medium 3: 49
Nova Premier: 43
GPT-4o (Nov '24): 41

Source: Artificial Analysis independent benchmarking
Artificial Analysis

### Multiple AI frontiers

The most intelligent model is not always the right choice: reasoning, open versus proprietary models, cost and speed are trade-offs, alongside size, license, context window and features.

[3:18](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=198s) [Slide identifies reasoning, open weights, cost and speed as trade-offs alongside intelligence, with other model characteristics also listed.](https://assets.ai.engineer/sites/aiecode2025/sha256/89c0a529f1310912915c8bafe77489a3902aa025debea77e67c8f22f3abd4fdc.webp)

What if I told you, there are multiple frontiers... all of which changing rapidly

WHAT IF I TOLD YOU

Trade offs to consider alongside intelligence

1 Reasoning vs. Non-reasoning
2 Open Weights vs. Proprietary
3 Cost
4 Speed

Others incl. model size, license, context window, features (function calling, etc.)

5
Artificial Analysis

Interface text:
AI Engineer World’s Fair
aws
tambo
neo4j
DISTRIBUTIONAL
Microsoft
smol ai

### Reasoning and non-reasoning models

In this Intelligence Index comparison, reasoning models cluster at higher scores and use more output tokens than non-reasoning models. The speaker notes that those extra tokens affect latency and cost.

[4:28](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=268s) [Scatterplot groups models by intelligence index score and output tokens, with reasoning models generally using more tokens and scoring higher in this comparison.](https://assets.ai.engineer/sites/aiecode2025/sha256/9d14633ebe4bf5fdef98ced48f02dbfdece954d5ae8baa7dbf03fddf0805d29d.webp)

1 Reasoning vs. Non-reasoning

Reasoning models: Treating reasoning & non-reasoning models as distinct categories is a helpful framework for understanding today’s model landscape

Intelligence vs. Output Tokens Used to Run Artificial Analysis Intelligence Index
Artificial Analysis Intelligence Index (Version 2, released Feb 25), Output Tokens Used (~5M input tokens)

Most attractive quadrant

Y-axis: Artificial Analysis Intelligence Index
Y-axis ticks: 35, 40, 45, 50, 55, 60, 65, 70, 75
X-axis: Output Tokens Used in Artificial Analysis Intelligence Index (Log Scale)
X-axis ticks: 4M, 8M, 20M, 30M, 70M, 100M, 300M

Non-Reasoning Models
GPT-4.1
GPT-4.1 mini
Grok 3
DeepSeek V3 0324
Llama 4 Maverick
Nova Premier
GPT-4o (Nov 24)
Llama 4 Scout
Gemma 3 27B
Mistral Large 2 (Nov 24)

Reasoning Models
o3
o4-mini (high)
Gemini 2.5 Pro
Grok 3 mini Reasoning (high)
Llama 3.1 Nemotron Ultra 253B Reasoning
DeepSeek R1
Gemini 2.5 Flash (Reasoning)
Claude 3.7 Sonnet Thinking

Artificial Analysis

### Reasoning model verbosity

Across the intelligence index’s ~5M input tokens, o4-mini (high) used 72M output tokens versus GPT-4.1’s 7M. Gemini 2.5 Pro used 130M. More output has implications for cost and latency.

[5:09](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=309s) [Bar chart compares output tokens used by reasoning and non-reasoning models, highlighting a roughly tenfold difference between o4-mini and GPT-4.1.](https://assets.ai.engineer/sites/aiecode2025/sha256/7f1152f308069368f8ede81ebeaa45bffe8b6b4b0363e4150ef23cdf7c3e85c6.webp)

1 Reasoning vs. Non-reasoning

Reasoning model verbosity: Reasoning models can use upwards of an order of magnitude more tokens than non-reasoning models

Intelligence vs. Output Tokens Used to Run Artificial Analysis Intelligence Index
Artificial Analysis Intelligence Index (incorporates 7 leading evaluations), Output Tokens Used in Artificial Analysis Intelligence Index (~5M input tokens)

Legend: Reasoning Tokens; Answer Tokens; Reasoning models
NON-EXHAUSTIVE

Chart labels, total and visible bar segments:
Gemini 2.5 Pro (May '25): 130M; 110M, 22M
DeepSeek R1 0528 (May '25): 99M; 91M
Claude 3.7 Sonnet Thinking: 98M; 90M
Grok 3 mini Reasoning (high): 95M; 92M
Gemini 2.5 Flash (Reasoning): 91M; 76M, 15M
Qwen3 235B (Reasoning): 74M; 71M
o4-mini (high): 72M; 69M
Llama Nemotron Ultra Reasoning: 57M; 56M
o3: 48M; 45M
DeepSeek V3 0324 (Mar '25): 11M
Llama 4 Maverick: 11M
Llama 4 Scout: 11M
Mistral Medium 3: 7.5M
Gemma 3 27B: 7.4M
GPT-4.1: 7.0M
GPT-4o (Nov '24): 5.5M
Nova Premier: 5.2M

10X
Artificial Analysis
Source: Artificial Analysis independent benchmarking
7
Artificial Analysis

Interface text:
AI Engineer World's Fair
Microsoft
AI Engineer World's Fair
Engineering the future of AI

[5:33](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=333s) [Bar chart comparing output tokens used by reasoning and non-reasoning models on an intelligence index.](https://assets.ai.engineer/sites/aiecode2025/sha256/570579bf4c8045d9b2b1d07ac958fd33f143d1df5bbf704dc15db8855ea9c623.webp)

1 Reasoning vs. Non-reasoning

Reasoning model verbosity: Reasoning models can use upwards of an order of magnitude more tokens than non-reasoning models

Intelligence vs. Output Tokens Used to Run Artificial Analysis Intelligence Index
Artificial Analysis Intelligence Index (incorporates 7 leading evaluations), Output Tokens Used in Artificial Analysis Intelligence Index (~5M input tokens)

Legend: Reasoning Tokens; Answer Tokens; Reasoning models
NON-EXHAUSTIVE

Chart labels, total and visible bar segments:
Gemini 2.5 Pro (May '25): 130M; 110M, 22M
DeepSeek R1 0528 (May '25): 99M; 91M
Claude 3.7 Sonnet Thinking: 98M; 90M
Grok 3 mini Reasoning (high): 95M; 92M
Gemini 2.5 Flash (Reasoning): 91M; 76M, 15M
Qwen3 235B (Reasoning): 74M; 71M
o4-mini (high): 72M; 69M
Llama Nemotron Ultra Reasoning: 57M; 56M
o3: 48M; 45M
DeepSeek V3 0324 (Mar '25): 11M
Llama 4 Maverick: 11M
Llama 4 Scout: 11M
Mistral Medium 3: 7.5M
Gemma 3 27B: 7.4M
GPT-4.1: 7.0M
GPT-4o (Nov '24): 5.5M
Nova Premier: 5.2M

10X
Artificial Analysis
Source: Artificial Analysis independent benchmarking
7
Artificial Analysis

### Reasoning model latency

In Artificial Analysis’s API benchmark for 500 output tokens, including thinking time, GPT-4.1 took 4.7 seconds versus 42.4 seconds for o4-mini (high)—a 9X gap that matters for responsive applications.

[5:51](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=351s) [Bar chart comparing response times for reasoning and non-reasoning models, with a ninefold comparison highlighted.](https://assets.ai.engineer/sites/aiecode2025/sha256/faf94ba1fbc95e0ad470adc62f40cecdad5f6668c581dbd74c7b5d3afe38822e.webp)

1 Reasoning vs. Non-reasoning

Reasoning model latency: Reasoning models are slower to provide their response, making them less suitable for latency sensitive tasks

End-to-End Response Time
Seconds to Output 500 Tokens, including reasoning model ‘thinking’ time; Lower is better

Reasoning models
NON-EXHAUSTIVE

Legend: Input processing time; 'Thinking' time (reasoning models); Outputting time

Reasoning models typically take longer to respond

Llama 4 Maverick: 3.3
Llama 4 Scout: 4.3
GPT-4o (Nov '24): 4.5
GPT-4.1: 4.7
Mistral Medium 3: 6.3
Nova Premier: 7.7
Gemma 3 27B: 11.5; 10.9
Grok 3 mini Reasoning (high): 15.5; 12.1
Gemini 2.5 Flash (Reasoning): 15.6; 14.2
o3: 19.5; 16.4
Claude 3.7 Sonnet Thinking: 19.8; 12.7
DeepSeek V3 0324 (Mar '25): 24; 20.5
Qwen3 235B (Reasoning): 36.9; 28.6
o4-mini (high): 42.4; 38.5
Gemini 2.5 Pro (May '25): 43.2; 39.8
Llama Nemotron Ultra Reasoning: 60.4; 12; 47.8
DeepSeek R1 0528 (May '25): 106.1; 20.5; 81.9

9X

Source: Artificial Analysis independent benchmarking
8
Artificial Analysis

### Reasoning model latency

For 500 output tokens, GPT-4.1 takes 4.7 seconds versus 42.4 seconds for o4-mini (high), about 9× longer. The speaker notes that successive agent calls compound latency.

[6:24](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=384s) [Bar chart comparing response times for reasoning and non-reasoning models, with a ninefold comparison highlighted.](https://assets.ai.engineer/sites/aiecode2025/sha256/21e70aa3f8565f554e4d15c731ac14b03a2c7bb48ff4080df6fd6175692b7b54.webp)

1 Reasoning vs. Non-reasoning

Reasoning model latency: Reasoning models are slower to provide their response, making them less suitable for latency sensitive tasks

End-to-End Response Time
Seconds to Output 500 Tokens, including reasoning model ‘thinking’ time; Lower is better

Reasoning models
NON-EXHAUSTIVE

Legend: Input processing time; 'Thinking' time (reasoning models); Outputting time

Reasoning models typically take longer to respond
9X

Chart labels and visible values:
Llama 4 Maverick: 3.3
Llama 4 Scout: 4.3
GPT-4o (Nov '24): 4.5
GPT-4.1: 4.7
Mistral Medium 3: 6.3
Nova Premier: 7.7
Gemma 3 27B: 11.5, 10.9
Grok 3 mini Reasoning (high): 15.5, 12.1
Gemini 2.5 Flash (Reasoning): 15.6, 14.2
o3: 19.5, 16.4
Claude 3.7 Sonnet Thinking: 19.8, 12.7
DeepSeek V3 0324 (Mar '25): 24, 20.5
Qwen3 235B (Reasoning): 36.9, 28.6
o4-mini (high): 42.4, 38.5
Gemini 2.5 Pro (May '25): 43.2, 39.8
Llama Nemotron Ultra Reasoning: 60.4, 47.8, 12
DeepSeek R1 0528 (May '25): 106.1, 81.9, 20.5

Source: Artificial Analysis independent benchmarking
8
Artificial Analysis

### Open weights versus proprietary models over time

After o1 opened a lead in late 2024, open-weight models caught up. DeepSeek R1 0528 sits a few Intelligence Index points behind the leading proprietary models by mid-2025.

[6:37](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=397s) [Step chart comparing intelligence-index scores of leading open-weights and proprietary models by release date.](https://assets.ai.engineer/sites/aiecode2025/sha256/7a3c1544dd9e93893ca7f104791fc58a8f163a5eae95697dc4f66ed76bd53c35.webp)

2 Open Weights vs. Proprietary

Open weights reasoning models closed the gap opened by OpenAI’s o1 in late 2024; proprietary models continue to lead by a small margin

Leading Language Models by License Type, Over Time
Artificial Analysis Intelligence Index (incorporates MMLU-Pro, GPQA, Humanity's Last Exam, LiveCodeBench, SciCode, AIME, MATH-500)

Legend: Proprietary; Open Weights
Y-axis: Artificial Analysis Intelligence Index
Y-axis ticks: 10, 20, 30, 40, 50, 60, 70
X-axis: Release Date
Timeline: Nov ’22, Jan ’23, Mar ’23, May ’23, Jul ’23, Sep ’23, Nov ’23, Jan ’24, Mar ’24, May ’24, Jul ’24, Sep ’24, Nov ’24, Jan ’25, Mar ’25, May ’25, Jul ’25

Chart labels:
o1-preview
Gemini 2.5 Pro Preview (Mar 25)
o4-mini (high)
DeepSeek R1
Qwen 235B A22B (Reasoning)
DeepSeek R1 0528
Artificial Analysis

9
Artificial Analysis

Interface text:
AI Engineer World's Fair
Microsoft
aws
smol ai

[6:42](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=402s) [Line chart comparing leading proprietary and open-weights models over time, showing a narrowing intelligence-index gap.](https://assets.ai.engineer/sites/aiecode2025/sha256/3be5f940f556fea5676f77bf9f7898b1da1acfc152c59e429e8623b50e1b960a.webp)

2 Open Weights vs. Proprietary

Open weights reasoning models closed the gap opened by OpenAI’s o1 in late 2024; proprietary models continue to lead by a small margin

Leading Language Models by License Type, Over Time
Artificial Analysis Intelligence Index (incorporates MMLU-Pro, GPQA, Humanity's Last Exam, LiveCodeBench, SciCode, AIME, MATH-500)

Legend: Proprietary; Open Weights
Y-axis: Artificial Analysis Intelligence Index
Y-axis ticks: 10, 20, 30, 40, 50, 60, 70
X-axis: Release Date
Timeline: Nov ’22, Jan ’23, Mar ’23, May ’23, Jul ’23, Sep ’23, Nov ’23, Jan ’24, Mar ’24, May ’24, Jul ’24, Sep ’24, Nov ’24, Jan ’25, Mar ’25, May ’25, Jul ’25

Chart labels:
o1-preview
Gemini 2.5 Pro Preview (Mar 25)
o4-mini (high)
DeepSeek R1
Qwen 235B A22B (Reasoning)
DeepSeek R1 0528
Artificial Analysis

9
Artificial Analysis

Interface text:
AI Engineer World's Fair
Microsoft
aws
smol ai

## Full transcript

[0:00](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=0s) [upbeat music] Hi, everyone.

[0:22](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=22s) I'm George, co-founder of Artificial Analysis. A quick background to who we are before we dive into things.

[0:32](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=32s) Do you see that? We see it here.

[0:36](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=36s) Sorry, I think my clicker's not working. Click.

[0:45](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=45s) Oh, there we go. Great. So a quick background to who we are. We're a leading independent AI benchmarking company. We benchmark a broad spectrum across AI. So we benchmark models for their intelligence.

[0:58](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=58s) We benchmark API endpoints for their speed, their cost. We also benchmark, uh, hardware and all the AI accelerators out there. Uh, and we also benchmark a range of modalities, not just language, but also vision, speech, image generation, video generation.

[1:17](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=77s) And we publish essentially nearly all of it, uh, for free on our website, artificialanalysis.ai, whereby we benchmark over a hundred and fifty different models, uh, across a range of metrics.

[1:32](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=92s) We also publish reports, many of which are publicly accessible, and we also have, uh, a subscription for enterprises looking to, uh, enter, uh, or bring AI to production in their environments, um, in an efficient, uh, and effective way.

[1:54](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=114s) Let's start off with AI progress. Let's set the scene. So it's been a crazy two years. I think that we've all felt it in this room, whereby OpenAI, uh, kicked off the race, uh, with the ChatGPT and GPT-3.5 launch, and since then, it's only gotten more hectic.

[2:16](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=136s) There's been f-- more and more, uh, model releases by more and more labs pushing the AI frontier.

[2:26](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=146s) So the current state now of frontier AI intelligence. I think this will be-- this order of models will be familiar to a lot in this room. o3 is the leader, but followed closely by o4-mini with reasoning mode high.

[2:40](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=160s) DeepSeek-R1, the release in the last week or two. Grok 3 Mini, reasoning high. Gemini 2.5 Pro. Claude 4 Opus Thinking.

[2:53](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=173s) This benchmark is our Artificial Analysis Intelligence Index. It's made up of a composite-- it's a composite index of seven evaluations, which we then weight to develop our Artificial Analysis Intelligence Index, which just provides a generalist perspective on the intelligence of these models.

[3:17](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=197s) We all have an understanding of what f-frontier AI intelligence is, but what I want to explore with you today is that there's more than one frontier in AI. There's trade-offs to accessing this intelligence.

[3:30](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=210s) You shouldn't always use the leading, most intelligent model. And so what we wanna do is we wanna explore the different frontiers out there, and as an AI benchmarking company, we're gonna bring some numbers to the fore to help you reason about this.

[3:46](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=226s) First, we'll be looking at reasoning models. Next, we'll be looking at the open weights frontier. Third, the cost frontier. And lastly, the speed frontier. There's other frontiers out there that we benchmark, but we'll focus on these key ones today.

[4:06](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=246s) Starting with reasoning models. What we've done here is we've taken our intelligence index and looked at that relative to the output tokens used to run the intelligence index. So we've measured all of how many tokens each model took to run our seven evaluations, and we've plotted it on this chart.

[4:27](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=267s) And you can see two distinct groups. It's helpful to think about these separately. So non-reasoning models, which offer less intelligence but, uh, require fewer output tokens.

[4:40](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=280s) And reasoning models, which use more output tokens but offer greater intelligence.

[4:45](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=285s) And the more out-- this is important to look at because more output tokens comes with trade-offs, both for request latency as well as cost. We're gonna bring some numbers to draw that out and look at the real differences here.

[5:01](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=301s) Starting with output tokens and the verbosity of these models, just how yappy these reasoning models are. We can see that there's an order of magnitude difference between reasoning and non-reasoning models.

[5:17](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=317s) It's not just that feeling, "Oh, this is taking a long time." It's real. It's an order of magnitude. So between GPT-4.1, it c-- uh, it required seven million tokens to run our intelligence index evaluations.

[5:31](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=331s) But then o4-mini-high took seventy-two million tokens. And the yappiest of them all, Gemini 2.5, uh, Pro, took a hundred and thirty million tokens to run our intelligence index.

[5:44](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=344s) And as mentioned, this has implications for cost as well as end-to-end latency responsiveness.

[5:53](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=353s) So looking at latency, we benchmark the API latency of how long it takes to receive a response when accessing these models via their APIs. Here we can see that GPT-4.1 on median across our request took four point seven seconds to return a full response.

[6:13](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=373s) o4-mini-high took over forty seconds, roughly another ten x or order of magnitude increase. This has implications for applications and uses which require responsiveness, even enterprise, uh, kind of chatbots.

[6:31](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=391s) You don't always reach for o3 in ChatGPT.

[6:36](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=396s) And it-- And, you know, Facebook's done a lot of studies on this where they've looked at the-- for consumer apps, where they've looked at, uh, user drop-off by latent-- uh, application latency, which clearly demonstrate this.

[6:49](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=409s) Sorry, do you mind if we jump back a slide?

[6:57](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=417s) And it also has up, uh-- implications for how we're building. So I think particularly with agents whereby thirty, uh, queries in succession is not uncommon. It has m- it's a multiplier effect on the latencies, uh, for your application and how you can build.

[7:17](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=437s) If you have faster responses, maybe you can make that thirty, uh, one hundred queries, for instance. And so putting numbers to that, in terms of agents, thirty is normal.

[7:27](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=447s) And so even less than the o4-mini, maybe you're at ten seconds for a reasoning model. If you're running thirty queries, that's three hundred seconds that a user might be waiting for a response or an application might be waiting for a response.

[7:40](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=460s) That's five minutes. If with the order of magnitudes that we're dealing with here, if that ten seconds was one second, then those thirty queries takes thirty seconds. Thirty seconds versus five minutes impacts what you can build.

[7:55](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=475s) Think of a contact center, uh, application. That might-- maybe thirty seconds is okay there, but five minutes, uh, definitely not. Who likes waiting on the phone, uh, that long?

[8:05](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=485s) Or imagine if you had to, uh, use Google, and each time that you wanted to use a function, it took five minutes. This impacts how we can build with these models.

[8:15](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=495s) And so I think bringing numbers to these trade-offs is really important. I'd encourage everybody, uh, to measure them.

[8:25](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=505s) Next, we're gonna move to the open weights frontier.

[8:29](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=509s) Around the time of GPT-4, there was a huge delta in terms of open weights intelligence versus proprietary intelligence.

[8:40](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=520s) Llama 65B or Llama 2 70B wasn't close to the intelligence of GPT-4. What I'd like to show here is where we plot our intelligence index by release date is that that gap had closed until with, with great models like Mixtral 8x7 and, uh, Llama 405B.

[9:02](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=542s) But o1 broke away in late twenty twenty-four.

[9:09](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=549s) But then, of course, I think we remember DeepSeek released V3, I think December twenty-sixth. Ruined some of my Christmas holiday plans. [laughs]

[9:20](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=560s) Had to tell my family, "I, I need to go read this paper. It's really exciting."

[9:26](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=566s) And then, of course, R1 in January. The gap between open-weights intelligence and proprietary model intelligence is less than it's ever been, particularly with the recent R1 release in the last couple of weeks, which is only a couple of points different in our intelligence index to the leading models.

[9:49](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=589s) You can't talk about open weights intelligence without talking about China. The leading open weights models across both reasoning models and non-reasoning models are from China-based AI labs. DeepSeek's leading in both.

[10:06](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=606s) Alibaba with their Qwen three series is leading-- is coming in second in reasoning.

[10:13](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=613s) But you also have other labs such as Meta, uh, and NVIDIA with their Nemotron fine-tunes of Llama coming in close as well.

[10:26](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=626s) Let's look at the cost frontier. This is really important, and I think similar to reas-- to, uh, end-to-end latency impacts what you can build. So bringing some numbers here, we can really see these order of magnitudes play out.

[10:40](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=640s) So o3 cost us almost two thousand dollars to run our intelligence index. TechCrunch actually wrote an article about how much money we were, we were spending on running evals.

[10:52](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=652s) We di- we didn't wanna read it. You can see four point one, a great model. It's thirty times roughly cheaper in terms of, uh, the cost to run our intelligence index compared to o1.

[11:08](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=668s) And four point one Nano, over five hundred times cheaper to run our intelligence index than o3.

[11:16](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=676s) You should think about these when building applications. The kinda co- uh, cost structure of your application might dictate what you can use here

[11:25](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=685s) and how you use them. Those thirty, uh, sequential, uh, API calls for your agentic application could be, uh, five hundred and still be cheaper than an o3 query.

[11:42](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=702s) A key point to note here with this cost to run intelligence index and why we don't just look at the per token price is that, and the labs maybe don't want you to think this way, [laughs] but you're paying for the cost per token, but then you're also paying for how verbose the models are.

[11:59](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=719s) All the reasoning tokens that are output when these models are in their thinking mode.

[12:06](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=726s) You pay for those as output tokens, even if some of the labs hide them. And so you need to think about this and measure it in your application, not-- and benchmark not just by the cost per million tokens, but also considering how many reasoning tokens there are and how verbose these models are.

[12:26](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=746s) You can see even amongst the non-reasoning models, there's big differences between how verbose these models are in responses.

[12:35](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=755s) So for instance-- Ah, we'll go to the next slide.

[12:42](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=762s) Do you mind if we go back one, please?

[12:50](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=770s) So what we've done here is we have-- now-- we're now gonna look at the trends in terms of cost. And so what you can see here is we've bucketed models by how intelligent they are, intelligence, uh, bands, if you will.

[13:07](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=787s) And what we can see here is that accessing GPT-4 level of intelligence has fallen over a hundred times since mid '23.

[13:18](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=798s) This is the case across all quality bands.

[13:22](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=802s) You can see that even when a new quality band, a new frontier is reached, o1-mini in late '24.

[13:30](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=810s) Quickly, within only a few months, the cost of accessing that level of intelligence halved. This is moving quickly. And so what I would say to you is when building applications, think about what if cost wasn't a barrier when you're building.

[13:47](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=827s) It's a, it's a very important kinda cost, uh, exercise because it might well be that if you build for a cost structure that doesn't work now, then maybe in six months' time that will be, uh, possible and it will be, uh, feasible.

[14:09](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=849s) Next, we're gonna look at the speed frontier. So this is how quickly you're receiving tokens, the output speed, output tokens per second that you're receiving after sending an A- API request.

[14:23](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=863s) This has been increasing and has increased dramatically since early '23 as well. So similarly, we've-- because there's a trade-off typically between intelligence and speed, we've grouped models into certain buckets.

[14:39](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=879s) And we can see here that they've all increased in terms of how quickly you can access a level of intelligence.

[14:47](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=887s) So 4, I believe, was around forty output tokens per second. Now you can access-- That was in 2023. Who remembers hitting-- It wasn't a reasoning model, hitting Enter in ChatGPT and just waiting for it to output, especially code, which you wanna just copy straight into your editor and, you know, hit Run [laughs] See if it works.

[15:11](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=911s) Now you can access that level of intelligence at over three hundred tokens per second. A few drivers here that I'll go through. It's not the focus of the talk, but important to, to reference.

[15:21](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=921s) Model sparsity, so we're seeing more MoEs, Mixture of Experts models, and they activate only a proportion of, uh, parameters at inference time. Less compute per token, which means it can go faster, essentially.

[15:39](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=939s) And MoEs were around back then, but they're getting more and more sparse. A smaller proportion of active parameters. Next, smaller models. Smaller models are getting more intelligence, uh, intelligent, particularly with distillations, you know, 8B distillations, et cetera.

[15:57](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=957s) Inference software optimizations like FlashAttention and speculative decoding.

[16:04](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=964s) And lastly, hardware improvements. So H100 was faster than A100. Now that we've recently launched benchmarks of the B200 on our Artificial Analysis website, and it's getting over a thousand output tokens a second.

[16:18](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=978s) Think about that relative to the forty output tokens per second of GPT, uh, four in '23.

[16:25](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=985s) There's also specialized, uh, accelerators like Cerebras, SambaNova, Groq.

[16:34](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=994s) I wanna share a house view here to frame things.

[16:38](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=998s) Yes, things are getting more efficient. Yes, the cost of accessing the same level of intelligence is decreasing, and hardware is getting better. We're getting more system output throughput

[16:49](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=1009s) on our-- on the chips. But our view is that demand for compute is gonna continue to increase.

[16:57](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=1017s) We're gonna see larger models. I mean, DeepSeek, it's over six hundred billion active, uh, sorry, not active, total parameters.

[17:08](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=1028s) And the demand for more intelligence is insatiable. Reasoning models, as we saw, the yappy models, they require more compute at inference time. And lastly, agents, whereby twenty, thirty, a hundred plus,

[17:26](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=1046s) uh, sequential requests to models is not uncommon. These act as multipliers on the demand for compute. And so the house view, playing with these numbers is net net. We're going to continue to see commute-- compute demand increase.

[17:44](https://www.youtube.com/watch?v=sRpqPgKeXNk&t=1064s) Thanks, everyone. [outro music]
