AI Engineer World's Fair 2026
The Desktop Frontier — Ahmad Osman, Osmantic
Read the talk
The Desktop Frontier: More Capability on the Hardware You Own
What happens when model efficiency improves faster than your hardware ages? Ahmad Osman traces the shrinking footprint of local AI and its implications for agents, ownership, and GPU purchases.
From a talk by Ahmad Osman
Before you start: Basic familiarity with language models, GPU memory, and coding agents will help; dense and mixture-of-experts parameter counts are explained in the article.
Frontier capability on one consumer GPU
Could a single consumer GPU run the equivalent of today’s largest open models? Ahmad Osman opens with a specific forecast: GLM 5.2-class capability on one RTX 5090 with 32 GB of VRAM in roughly 18 months, which he places in late 2027. He calls that estimate conservative. The prediction concerns equivalent capability in a future, more efficient model—not fitting today’s full GLM 5.2 weights onto that card.
The usual scaling story asks how much intelligence another increase in model size might buy. Osman turns the question around: for a given level of capability, how much hardware do we still need? He expects a gap between the frontier and local open models to persist, but also expects efficiency improvements to make that gap smaller.
Impact per parameter is his name for this comparison. Start with what a model can do, then compare the hardware required to obtain that capability across generations. His example is an RTX 3090 that once ran Llama 2 and now runs Qwen 3.5 or Qwen 3.6 at 27B parameters. Osman says the newer 27B models surpass Llama 3 405B, roughly a year and a half later, without specifying an evaluation. The useful question is whether the same physical machine has become capable of more valuable work.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From benchmark scores to useful local agents
A benchmark score is only one way to recognize progress. Another is whether a local model can function inside a coding harness such as Claude Code. In Osman’s account, this became practical with GLM 4.5 and GLM 4.5 Air. He describes a GLM 4.5 Air setup requiring at least four RTX 3090s or one RTX PRO 6000, followed by newer, more capable models running on a single RTX 3090 or RTX 5090. These are his deployment comparisons; precision, context length, and serving configuration are unspecified.
Context capacity has expanded too. Osman contrasts earlier models limited to about 4,000 tokens with local models supporting million-token contexts. A supported context window is not a promise that the whole window fits on the same consumer GPU: weights and the state retained while processing a long sequence both consume memory. Qwen3.6-27B’s current documentation, for example, distinguishes native context from an extended window and warns about memory limits.
These changes come from accumulated research, efficiency improvements, and architectural changes. Newer, more efficient models can outperform older, less efficient models; small size itself is not the cause of better capability. That distinction matters when deciding whether a smaller replacement can actually handle an existing workflow.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What the densing law measures
Osman connects impact per parameter to the densing law reported in Nature Machine Intelligence. He summarizes it as comparable capability requiring about 50% fewer parameters every three and a half months. The underlying study examined 51 open pretrained base models on MMLU, BBH, MATH, HumanEval, and MBPP, finding that maximum capability density approximately doubled over that interval. Equivalent-performance parameter halving follows from that empirical pattern; it does not establish a schedule for every future model or coding-agent workload.
The parameter count also needs a definition. A dense model uses its full parameter set for each token. A mixture-of-experts model, or MoE, routes tokens through selected experts, so its total stored parameters can greatly exceed the number activated per token. Total parameters help explain the weight-storage footprint; activated parameters help explain the computation involved. Neither number alone describes the complete serving system. Osman flags this distinction before turning to the largest local configurations.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A workstation frontier, then cheaper training
At the high end of local computing, Osman describes GLM 5.2 as having 744B total parameters, 40B activated parameters, and support for up to one million tokens of context. He says it can run in NVFP4 on a DGX Station or a server with eight RTX PRO 6000s. An under-desk workstation puts the model under its owner’s control, but it is a very different hardware class from the single consumer GPU in his opening forecast.
Osman reports that GLM 5.2 beats GPT 5.5 extra high on one unnamed benchmark. Without the task, score, or evaluation setup, that comparison establishes the scope of his claim—not general superiority over the cloud frontier. His broader interest is that locally owned systems can now participate in that comparison at all.
Nemotron 3 Ultra introduces a different efficiency mechanism: lower-precision training. NVIDIA documents NVFP4 pretraining for this model. Lower precision changes how numerical values are represented, potentially reducing storage and arithmetic costs. Osman extends that result into a prospect for cheaper training, fine-tuning, and specialized models. The documented pretraining result supports the mechanism; the savings for a particular specialization workflow still depend on that workflow.
The same efficiency trend can change how much parallel work existing hardware supports. Osman contrasts loading Llama 2 70B on eight RTX 3090s with running roughly 15 parallel Qwen 3.5-27B agents on those same eight GPUs. He does not supply precision, context, throughput, or agent-serving details, so this is a comparison of his described configurations, not a universal capacity figure. It makes impact per parameter tangible: a machine that once hosted one large model can potentially support many useful workers.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Ownership needs an economic base
The smaller-device end of this progression leads to sovereignty. Osman describes GPT-4o-quality capability running on an iPhone, though he names no local model, device configuration, or evaluation. His ownership argument does not require every cloud workload to fit on a phone: individuals and organizations benefit when they can retain access to a model and adapt it to their own needs.
Owning the model and infrastructure creates several related possibilities:
- Continued access: retain the model instead of depending on a provider keeping a particular service available.
- Specialization: optimize performance for the organization’s own tasks.
- Long-run economics: turn those optimizations into potential savings over repeated use.
Osman mentions ODS, the Osmantic Deployment System, as his route to supporting consumers and enterprises, without giving an implementation walkthrough.
For that ecosystem to last, he argues, enterprise spending must help sustain it. Companies currently financing cloud infrastructure through subscriptions could instead purchase hardware and own their stack end to end. That demand would create incentives for providers to keep developing open models. He also calls for licensing arrangements that allow the ecosystem to thrive, without proposing a specific license. Hardware ownership, model availability, and provider incentives form one economic problem.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
How smaller models became useful
The model history begins with Mistral 7B, an early example of a smaller model performing beyond expectations. Its significance at release does not make it suitable for today’s agent workflows: Osman says it cannot do the work he now expects inside Claude Code or OpenCode. He contrasts that experience with a contemporary, unnamed 9B model he can use through Telegram or Hermes. The target has moved from producing impressive text to participating in useful workflows.
Mixtral 8x7B marks the MoE milestone in this progression. Then came the Llama 3 family: an 8B model Osman valued for its distinctive identity, a 70B model he ran on his eight RTX 3090s at home, and the much larger 405B model. Looking back from Qwen’s newer models, he places this transition between summer 2024 and March 2026—roughly 21 months.
Gemma 2-27B and Qwen 2.5 extend the history beyond sparse expert routing. Better dense models also emerged from improvements to data, training recipes, tokenizers, post-training, and evaluation loops—the ingredients highlighted on the slide. Osman especially credits Qwen 2.5 with enabling fine-tuning, experiments, and research that benefited the community. Open models make a checkpoint available for further work; that work can then become part of the next capability improvement.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Local reasoning, then a smaller agent footprint
DeepSeek-R1 made reasoning another capability that owners could host themselves. Its nearly 700B-parameter MoE still required a substantial server in Osman’s account: local availability did not mean single-GPU accessibility. He then points to the subsequent May revision as evidence that more post-training could substantially improve an existing checkpoint. Capability could rise without requiring a corresponding increase in model size.
gpt-oss-120b is his next milestone. Osman remembers it as an early successful open-model tool-calling experience, allowing home hardware to do more agentic work. The significant shift is both behavioral and physical: tool use makes new workflows possible, while fewer total weights make the model easier to host.
Compared with DeepSeek-R1’s stated 671B parameters, gpt-oss-120b’s 120B count is roughly one-fifth to one-sixth the total size. Osman describes its agentic performance as comparable or possibly better, without a matched evaluation protocol. The comparison illustrates why total parameter count is a poor standalone ranking of usefulness: a smaller model that reliably invokes tools can be more valuable for an agent than a larger model with less suitable behavior.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A smaller model can activate more parameters
The Qwen comparison makes the distinction between stored and activated parameters concrete. Osman compares Qwen 3.5-397B-A17B, a large MoE, with the dense Qwen 3.6-27B. He reports that the newer dense model beats the larger model, despite a dramatic difference in total size.
He places this improvement within roughly two to three months, then returns to the practical appeal of owning the result: a capable model that remains available on your hardware, under your control, without depending on a provider’s continued access policies. He credits additional post-training with gains across the displayed benchmarks. The current Qwen model card supports selected coding gains, but the larger MoE still wins several knowledge and reasoning comparisons; the smaller model is not the stronger choice on every task.
| Model | Architecture | Total parameters | Activated parameters |
|---|---|---|---|
| Qwen 3.5-397B-A17B | MoE | 397B | 17B |
| Qwen 3.6-27B | Dense | 27B | 27B |
Using these counts, the dense model has about 14.7 times fewer total parameters while activating more parameters per token. That explains how a large reduction in weight storage can coexist with substantial active computation. It does not imply a proportional latency improvement: routing, memory bandwidth, precision, context, and serving implementation also affect performance. The explicit counts are more useful than the talk’s approximate percentage and processing-time comparisons.
This is the mechanism behind the opening forecast: future models might deliver a similar level of capability with far fewer stored weights. The question is how quickly those improvements can carry workstation-class capability into a consumer GPU’s memory budget.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What your hardware might run next
Returning to the single-5090 prediction, Osman gives a 17-month estimate and again calls it conservative. He recalls a December prediction that Opus 4.5-quality capability would run locally on one RTX PRO 6000, and says that happened by March. That is his retrospective assessment of quality, without a specified benchmark comparison. It leads to a more general question: can a hardware purchase become more useful as the software running on it improves?
His cloud-economics argument rests on expectations about future access. Osman predicts that token subsidies will disappear and that cloud models will come with increasing limitations. Owning hardware is his proposed way to reduce exposure to those changes. The trade is between paying for a service whose terms can change and owning capacity whose usefulness may grow with model efficiency.
That prospect explains why he keeps his RTX 3090s. He is more interested in what they might run in a year or two than in their immediate sale proceeds. For a DGX Station, he asks the same question over three, six, twelve, and eighteen months. He explicitly separates this interest in future workloads from financial advice.
The closing comparison is between the longevity of Ampere and the potential of Blackwell. Osman points to the continued usefulness of the RTX 3090’s 2020 architecture and says the cards are selling above their original MSRP—a time-sensitive market claim, not a demonstrated return on investment. If that older hardware still finds new uses, what might an actively developed Blackwell-based DGX Station run in a few months or a couple of years? The value he is exploring is future capability, not just resale price.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Should you buy a GPU?
If a 32 GB RTX 5090 can deliver GLM 5.2-class capability in 18 months, the purchasing decision depends on more than what the card can run today. It also depends on the future workloads that more efficient models could make possible. Osman leaves that condition unresolved and ends with the question: “Should you buy a GPU?”
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Model architecture, benchmark results, evaluation conditions and instructions for serving Qwen3.6-27B.
NVIDIA's research overview of its hybrid model, including NVFP4 pretraining and links to model artifacts.
Release announcement covering open weights, reasoning, tool use and deployment memory requirements.
A local AI server stack with inference, chat, agents and workflows, plus installation and hardware guidance.
Further reading
Osman's earlier argument for technical depth, local AI and ownership of the software stack.
Read the complete timestamped transcript
- 0:00
[upbeat music] Hey everyone. We are about to start this presentation.
- 0:17
Uh, it's called the Desktop Frontier, and, um,
- 0:23
it's basically about where we started and how far we've come with local and open source models. Um,
- 0:35
how many like... J- just a quick question. How many of you here follow me on X? [audience cheering]
- 0:42
I'm, I'm amazing. Love you all. Love you all. Uh, so you know, I sometimes every now and then I would say a prediction. Uh, here is a new one.
- 0:52
Within roughly 18 months, we are gonna have the equivalent of GLM 5.2 class intelligence running on a single RTX 5090 with 32 gigabytes of VRAM. Um, that's basically late 2027.
- 1:10
Um, this, this is conservative. We might actually get there faster.
- 1:17
So, um, you know, for a long time, uh, the story has been bigger models, bigger models, bigger models. How can we get to the next five trillion? How can we get to the 20 trillion?
- 1:29
And I'm not saying that there won't ever be like a gap between frontier intelligence, um, and, um, you know, open source models. There will always be a gap. But that gap, um, will shrink and the efficiency of the models will get exponentially better.
- 1:51
So the, the term that I like to think about is impact per parameter. Um, you know, what capability are we talking about? What could the model do? Um, what footprint, like hardware footprint did it have last year in comparison to now?
- 2:10
And, uh, what hardware does that use, and what hardware did it need to use a year ago? And, uh, you know, are we moving down for the same kind of quality on the, on that hardware?
- 2:23
Again, as I was saying earlier, I used to run Llama 2 on an RTX 3090. It's now running Qwen 3.5, 3.6, 27 billion parameter. That's better than Llama 3 405.
- 2:35
That's a 400 billion plu- 400 billion-plus parameters model that you beat with a 27 billion parameter model a year and a half after.
- 2:48
So, um, yeah, as I was saying, similar capabilities are moving into smaller hardware footprint. Um,
- 2:56
benchmark scores are one thing, but also, you know, a year ago, this time a year ago, we didn't have any local models that were able to successfully run within Claude Code, right?
- 3:10
It wasn't until GLM 4.5 that came out in late July, and GLM 4.5 Air required at least four RTX, uh, 3090s or an RTX Pro 6000. Now that footprint for hardware is not needed anymore.
- 3:27
All that you need is a single RTX 3090, 5090, and you have something much more capable, much more intelligent. So is this trend just random or is there more to it?
- 3:41
That's a question that everyone should ask. Um, um, is it just by, by random chance that we've gotten this far from models that were- weren't able to sustain more than four thousand, um, tokens in terms of context lengths, and now we have things that are million param- million, uh, tokens locally on your hardware that you own.
- 4:06
It's not by chance, you know, it's not just a coincidence that we got here. Um, there is research being done. There is efficiency gains to be made. There are architecture hacks that compound, and they will continue to compound.
- 4:21
And, uh, I think I like this line. It's not that small models are beating big models, it's that newer, more efficient models are beating older, less efficient ones.
- 4:33
Uh, so yeah, capability density, uh, is, you know, the literature I back this up with. Uh, uh, Nature Machine Intelligence, uh, calls this pattern dancing law. And, uh, basically, um, you know, every three and a half months, we are having 50% fewer parameters.
- 4:55
Whether that's in dense or activated, that's a different story. But we are getting way more intelligence out of the models that we're running.
- 5:09
So, you know, right now we're at, at, uh, GLM 5.2. That's, uh, our, you know, biggest player. And, um, it's 744 billion parameters total with only 40 billion parameter activated, and that supports up to one million context lengths.
- 5:26
You can run this in NVME four on a machine, uh, on a DGX station or on a server with eight RTX Pro 6000s. That's something that you-- like a DGX station is something that you can sit under your desk, and it's running this kind of frontier intelligence.
- 5:43
Whether, you know, it, it's a on one benchmark it actually beats GBT 5.5 extra high. Doesn't that mean that we're getting somewhere with local and open source models that we can compete with the frontier?
- 5:56
That we're not that far off from the best that you can get from the cloud?
- 6:02
We also have Nemotron 3 Ultra, which proved that NVFP4 training, more efficient training can be done on, uh, on, on hardware, right? That's, that's very important. That means that the footprint even for training these models, for fine-tuning them, for making small and specialized models as I was talking earlier, could be more efficient, could be done cheaper, and
- 6:22
could be, you know, could deliver you value in terms of economics way sooner or, you know, for much less money than you used to.
- 6:33
Yeah. So, you know, again, Llama 2, uh, that was a 70 billion parameter model. If you try to run that right now, you're, you're-- you'd laugh at it, right?
- 6:41
That used to take eight RTX 3090s to load up, and it-- those same eight RTX 3090s could run something like 15 parallel agents right now with Qwen 3.5-27B. That's, that's a massive jump in terms of performance gains.
- 6:58
Um, so the fencing wall basically means that we have similar or better, um, capabilities with significantly fewer parameters. That's the impact per parameter, as I was saying. I want everybody to leave here thinking about this term and, you know, thinking where are we gonna get a year from today?
- 7:19
As I was saying earlier, everyone here has a phone I'm assuming. Raise your hand if you have a phone.
- 7:25
If, if you didn't raise your hand, we know you lie about other things as well. So come on, guys.
- 7:31
So, you know, you can ri- you can now run GPT-4o quality on your iPhone. That, that's massive. That thing required data centers to serve. So why wouldn't you invest, you know, in Sovereign AI?
- 7:48
Why wouldn't you as a consumer, as an individual, as a small-sized business, middle-sized business enterprise, why wouldn't you want to be in control of the models that you're on?
- 7:59
Why wouldn't you want to make sure that nothing gets taken away from you? That every little thing can be optimized for you later on. That the performance gains can be made specially and specifically for your use cases, and that you can save more money that way in the long run.
- 8:19
And, you know, ODS for consumers, it's basically the way that we support individuals, but enterprises also. And I think that there is something that we like as a community we need to think about deeply.
- 8:33
We need enterprises for open-source AI to win. We need these people that are using the cloud right now, that are basically supporting data centers being built for cloud providers to come on this side, to own their own hardware, to own the stack fully end-to-end so that we can keep delivering open-source models.
- 8:54
So that there is an incentive for open-source providers to actually come up with models so that we can come up with new licenses that allows open source to thrive and...
- 9:08
So again, um, open weight and the frontier I think I-- Yeah, sorry, that was a misclick. Um, you know, so smaller models started bouncing above the weight after Llama 2 with Mistral 7B, one of my favorite models.
- 9:23
If you try to build that model right now in, um, Claude Code or OpenCode, it's not gonna work. But it used to take so much in terms of hardware, right?
- 9:34
That you would now get from a 9B milli-- model that I can run with Telegram, with-- or with Hermes, for example, and, uh, do a lot of stuff with.
- 9:44
So, so we've come a long way. Uh, we had that. We had Mistral 8x7B, which, you know, everybody knows is an MoE. Then the progression went, uh, from that to Llama 3.
- 9:56
You know, Llama 38B was one of my favorites. Still is. Um, it had unique identity in my opinion. Uh, then we had like the 70 billion, which was like the, the thing that I would run basically on my eight RTX 3090s at home.
- 10:10
Then there was like the 405, the 400 billion-plus parameter Llama 3 which again required a lot of hardware and if you bought it now against Qwen 3.5, the 27 billion parameter would lose against it.
- 10:24
That's in the span of what? Two years? Two years and some?
- 10:28
No, I think, I think less than two years. That's summer 2024 to March 2026. That's, uh, that's about 21 months. And the next big thing in my opinion, Gemma 2-27V and then we had the Qwen 2.5 and that, that was the moment that I was like, "Okay, we actually are making progress," and the gap was shrinking between
- 10:49
open-source models and, uh, the frontier. Uh, really Llama 3 saved like, you know, it really helped us a lot and then, um, Qwen 2.5 delivered a massive improvement and there was a lot of fine-tuning and experiments that could be done on that one.
- 11:05
There was amazing papers and, um, they helped the community im-immensely in my opinion.
- 11:12
Then the next big thing was DeepSeek-R1 in my opinion, and the reasoning becoming something that you can run at home. That was a massive MoE, almost 700 billion parameters.
- 11:22
Um, you know, you had to have like a very beefy server to actually get it up and running. Um, and then, you know, the improvements that came from just
- 11:33
more training on that one and DeepSeek-R1 that was released in May last year made massive jump again. So it showed that both training could deliver more improvements on the same, on the same checkpoints.
- 11:53
Then GPT open source like GPT-OS 120B. Anyone remembers that one from last summer? Yeah.
- 12:01
Nobody here used it Come on, guys. I, I need some help here. [laughs]
- 12:07
Um, it was, it was, it was one of the first open source models that were able to successfully do tool calling. Um, and, uh, it was a step forward.
- 12:17
It showed us that we can do more with, uh, with the hardware that we have at running at home. Uh, that was a footprint shift, right? From like, you know, that massive 700 billion parameters DeepSeek, uh, R1 that was, yeah, 671 billion parameters to something that was one-fifth, one-sixth of its size.
- 12:36
And GBT-oSS was comparable, maybe better, more agentic performance. Um,
- 12:48
then the moment of, uh, Qwen 3.5, that's 39.7... That's three-- That's 397 billion parameters. Uh, that's, uh, that's a beefy MoE. And, uh, you know, what's funny is that about, uh, it's about 15 times the size of, uh, that Qwen 3.6.
- 13:08
And I'm-- here I'm comparing 3.5 to 3.6 of the dense 27 billion parameter model, and that dense model beats it. And that dense model has 40% higher number of activated parameters, so it's not that far off.
- 13:24
That's massive amount of performance gains in a very small amount of time with massively different footprint in terms of hardware, uh, requirements. And that trend happened in like what?
- 13:36
Two, three months. So, you know, how far could we go from here? Um, how far before we get to, you know, uh, a recent model that there were some, uh, news about, you know, that is, uh, finally relaunched again?
- 13:51
How far before open source delivers something of that quality that you could run on your own hardware and you can control and will not be taken away from you and will not refuse a request from you?
- 14:08
So again, these are just the-- some benchmarks where you can see that an iteration on the 27 billion parameter model, a little bit more post-training improved it across all benchmarks and made it win against a model that is almost 15 size-- for 15 times its size.
- 14:32
And again, remember, this is 27 billion parameters activated versus 17 billion parameter activated. It's still massively the same amount. Like, you know, it's, it's only 40% less in terms of the amount of time it would take to process things, but it's 15 times smaller.
- 14:50
That's a lot. So again, um, how long until the prediction I made earlier becomes plausible when I said that we're gonna have the equivalent of GLM 5.2 running on an RTX, uh, 5090?
- 15:06
This is the math. Seventeen months. And this is a conservative math. Uh, earlier this year in December, I had a, um, a very viral post that I predicted that we're gonna have the quality of, uh, Opus 4.5 running locally at home on a single RTX, uh, Pro 6000.
- 15:24
That happened by March. [laughs] So a question, um, hardware purchases today, does it get more valuable as models become more efficient and smaller in size? [laughs]
- 15:43
That's a good question. So why are you funding other people to build data centers so that you can subscribe to them and pay subsidized tokens and then later on get- [clapping] Th- those subsidies are gonna go away, and you're not gonna be able to run those models, and they will have so many limi-limitations.
- 16:02
So might as well ask yourself, why not own the hardware yourself and be in control?
- 16:07
Um, so yeah, the forward-looking question is basically, what will a DGX station be able to run in three, six, 12, 18 months from now? That's something that there is a reason that I'm not selling any of my RTX 3090s if you follow me, and I have a lot of hardware, guys.
- 16:22
Um, but I'm interested in seeing what I could do with them in a year or two from now, more than in the amount of money I would get for them today.
- 16:32
This is not a financial advice, by the way. Like, let me make that very clear. [laughs]
- 16:36
Um, so yeah. Um, the disk size frontier potential, um, you know. An NVIDIA DGX station could run a lot of, uh... Today it could run GLM 5.2. What will it be able to run tomorrow, six months, 18 months, two years from today?
- 16:53
We know that, you know, RTX 3090s, Amber, uh, architecture from 2020 sells at a higher value than MSRP today and is still being utilized for a lot of use cases.
- 17:06
So what will a DGX station, the actively developed Blackwell architecture, will be able to run in a few months, a couple of years?
- 17:16
That's a good question. So the question you have to ask yourself, um, if an RTX 3050 90 with 32 gigabytes of VRAM runs in the equivalent of a GLM 5.2 in 18 months.
- 17:30
And this is the question that everybody should be asking themselves, and I want you all to be looking at the screen, taking this very seriously. Okay?
- 17:40
Should you buy a GPU? [clapping] Thank you. [outro jingle]