Scaling the Next Paradigm of Heterogeneous Intelligence

Why solve every part of a complex problem by scaling one model on a fleet of identical chips? That is the opening question behind Adrian Bertagnoli’s account of heterogeneous intelligence. Introducing himself as a founding engineer at Callosum, he starts with the prevailing alternative: homogeneous scaling, where more training data and more parameters produce a more capable model. Neural scaling laws made that approach productive; his question is what changes when the engineering emphasis moves from training models to running inference.

The shift is already visible at three layers:

These changes introduce specialization inside the model, between calls, and underneath inference. They do not yet require an entirely heterogeneous cluster.

At this mild stage, agents can use different prompts or different LLMs while still running on largely homogeneous infrastructure. Greater heterogeneity assigns different models to different GPUs, then combines architectures such as state-space models and diffusion models on hardware suited to their computation. The proposed endpoint goes further than choosing from existing chips: models, software systems, and hardware co-evolve into a vertically integrated system.

Real-world problems are multistep and open-ended. Their subproblems require different capabilities, so increasing one kind of intelligence can be an inefficient way to solve the whole task. Bertagnoli calls the alternative multi-agent heterogeneous intelligence: models of different sizes and architectures cooperating over long horizons. There is an infrastructure problem alongside the modeling problem. New silicon needs an interface through which it can contribute to the existing compute stack; simply making another accelerator available does not establish how agents should use it.

The principle of maximum heterogeneity formalizes this intuition through a distributed production model. In the diagram Bertagnoli describes, each agent has a distribution over a skill space, represented by its color. A ring connects the agents, representing communication. Together, their skills and interactions produce a collective production function: a profile of what the system can supply. That profile can fit one problem’s demand, labeled demand A, while serving demand B poorly.

Homogeneous scaling has two limitations in this picture. Replicating a specialist raises one peak without filling other gaps in capability. Replicating broad generalists spreads capacity across the skill space, producing the short cylinder in the illustration rather than enough capacity where demand is highest. Bertagnoli presents mathematical support and parallels across neuroscience, economics, and ecology. The corresponding distributed-production paper bounds the result by modeled skill distributions, communication, resource constraints, and environmental demand. The useful design principle is therefore to match diversity to the workload and its communication topology—not to maximize the number of different components indiscriminately.

This becomes an orchestration problem across three connected decisions:

• Hardware: Select where an agent runs according to its computational demands.
• Agents: Select the models and capabilities that perform the work.
• Workflows: Determine how agents interact and how their work is composed.

Bertagnoli develops two examples: heterogeneous recursion at the workflow layer, followed by a mixture of multimodal models for visual web navigation at the agent layer. In both, the opportunity comes from decomposing a task before deciding which resources should execute it.

A large context window does not guarantee that a model can reason reliably over everything in it. Recursive language models address that distinction. Bertagnoli refers to an MIT introduction from the previous October: the October 2025 author write-up preceded the paper submitted in December 2025. The motivating issue is context rot: performance can deteriorate even when the input occupies only part of the available window, depending on how much information the answer requires.

The distinction is easiest to see through two tasks:

A longer haystack does not necessarily require a longer answer or more facts to construct it. Aggregation does require more of the input to participate in the result. Bertagnoli says constant-demand retrieval can remain effective across the full context window, while linear or quadratic demands can degrade at roughly 30–60% occupancy. He does not assign separate thresholds to the two complexity classes or specify a universal cutoff.

The recursive approach changes how the agent accesses the input:

1. Keep the long context outside the model’s immediate prompt, in an environment the agent can inspect.
2. Let a coding agent use a Python REPL to search it with keywords, regular expressions, or other programmatic operations.
3. Extract relevant subcontexts and pass them to recursive calls.
4. Let each call answer its question or delegate again.

Bertagnoli describes a file-backed context and identical recursive agents. The original RLM design also supports context held in an environment variable and calls to other models; neither files nor identical child models are requirements. The central mechanism is programmatic access to context, followed by bounded reasoning over selected portions.

For example, a support log can stay in a file while Python extracts complete records mentioning refunds. The resulting strings are the subcontexts a recursive call would receive:

from pathlib import Path
import re

context = Path("support-log.txt").read_text(encoding="utf-8")
records = re.split(r"\n\s*\n", context.strip())
refund_records = [
    record
    for record in records
    if re.search(r"\brefund(?:s|ed)?\b", record, re.IGNORECASE)
]

batch_size = 20
subcontexts = [
    "\n\n".join(refund_records[start:start + batch_size])
    for start in range(0, len(refund_records), batch_size)
]

for index, subcontext in enumerate(subcontexts):
    print(f"--- Subcontext {index} ---")
    print(subcontext)

This illustrates the extraction step, leaving the recursive calls pending. The filter must match the question: a keyword search can select explicit refund mentions, but it would miss relevant records expressed with different vocabulary.

Once a workflow exposes smaller subcontexts, orchestration can choose how to process each one. Callosum’s extension maps generated subcontexts to different models and different chips, aiming to preserve the quality of frontier-model reasoning while reducing cost and latency. The distinguishing step is the joint selection of model and hardware, rather than recursion alone.

Bertagnoli presents results on OOLONG, the long-context reasoning and aggregation benchmark also used in the RLM work. He names GPT-5 and then GPT-5.2 as the contemporary baseline. For that baseline, Bertagnoli reports about 2,000 seconds to run through the benchmark and about $3.75 for one task. Those spoken units differ, so they should not be combined into a derived per-task throughput estimate.

The reported hardware choices produce different cost–latency tradeoffs:

Both rows compare with the baseline described in the talk. SambaNova reduces the price further than Cerebras while giving up some latency advantage. These are Callosum-reported OOLONG results; the exact evaluation split, timing aggregation, and accuracy comparison are not specified in the presentation. The architectural point is that model-and-chip assignment creates choices along both cost and time axes.

Visual web navigation introduces another kind of heterogeneity. A browser agent must interpret images, reason about text, and decide what action to take. Callosum combines open and closed video action language models for this work. Bertagnoli reports improvements of 18% over GPT-5.2 and 25% over Gemini 2.5 on the benchmark he calls Video Web Arena. The company’s companion account identifies the evaluation as VisualWebArena shopping and expresses the gains as relative pass-rate improvements over named agent systems, rather than bare models. That distinction matters when interpreting the claimed state of the art.

The decomposition is the mechanism: visual reasoning and textual reasoning become separate components that can use different models. Bertagnoli describes the resulting cost–latency comparison as a shift in the Pareto frontier: heterogeneous mixtures outperform the individual-model configurations plotted alongside them, including Kimi K2.5 and GPT-5.2.

The model pairings must be kept separate. Qwen3-VL-8B-Instruct supplies the smaller vision-language model in the configurations described here; the companion account resolves the talk’s abbreviated Qwen3-plus-GPT pairing as Qwen3-VL-8B-Instruct plus GPT-5.2.

The first row uses different baselines for speed and cost. Reading it as a single comparison against Kimi would change the claim.

One source of the savings is an operation as simple as zooming. A larger model may need a closer view before making a decision, but preparing that view does not require the same capability as interpreting it. Callosum maps zooming and other visual-view preparation to less capable models. Bertagnoli describes this construction as having no downside in the demonstrated comparison. For those visual subtasks alone, he reports 11× faster execution and 43× lower cost than using ChatGPT. These local savings contribute to the overall threefold speedup and 3.7-fold cost reduction of the GPT pairing; the much larger subtask ratios do not describe the whole workflow.

Bertagnoli places these examples in a three-era history of compute. The CPU era made computation faster. The GPU era made it massively parallel, with NVIDIA dominating that transition. His proposed third era maps multi-agent workloads onto different chips according to what each step needs. Scaling then depends not just on the capacity of a device, but on how effectively the system assigns work across devices.

He connects that direction to Callosum’s work with ARIA, reporting a rounded 3 million grant toward what he calls the UK’s first heterogeneous co-located cluster. Callosum’s funding announcement describes funding for cluster research and development with CommonAI, rather than establishing that a commissioned cluster was already operating. The infrastructure ambition is to bring different hardware into one coordinated system.

Homogeneous scaling still receives credit for the progress that made these systems possible. The closing proposal is to let models, workflows, and silicon co-evolve, so that new forms of specialization can improve the system’s intelligence, speed, and cost together. Bertagnoli ends the prepared talk with an infrastructure builder’s expression of optimism: “This is the worst our infrastructure will ever be.”

The audience’s final technical question returns to the zoom example: is the assignment hard-coded, or does another model decide which model should do the work? Bertagnoli says the first implementation used bespoke mappings from simple subtasks to simple models. Callosum subsequently added an automation layer that detects task complexity and predicts the best-suited model and hardware. He does not describe the predictor’s implementation or validation. This leaves the routing decision itself as a central part of the system: decomposition exposes the choices, and the orchestrator must turn them into assignments that preserve quality while reducing execution cost.