The Future of Qwen: A Generalist Agent Model

How can one model family move beyond answering text questions to working with images, video, speech and tools? Qwen starts with both language and multimodal models, with the longer-term goal of building a generalist agent. In Junyang Lin’s 2025 presentation, Qwen Chat is the entry point: users can upload images and videos, interact through voice or video chat, and try applications such as WebDev and deep research.

For developers, the Qwen release blog provides technical explanations, while GitHub code and Hugging Face checkpoints provide a route from trying a hosted model to running and adapting one. Those access points matter because the family spans different tasks and deployment sizes; there is no single checkpoint that represents every Qwen capability.

Qwen2.5-Max, released before Spring Festival 2025, establishes the instruction-following baseline. It is a large mixture-of-experts, or MoE, model. Lin describes its benchmark performance as competitive with Claude 3.5, GPT-4o and DeepSeek-V3. The accompanying chart compares models across Arena-Hard, MMLU-Pro, GPQA-Diamond, LiveCodeBench and LiveBench, separating several kinds of capability rather than reducing them to one score.

The next question is whether reinforcement learning can make a model better at solving problems, rather than merely following instructions. Lin reports consistent gains in mathematics and coding while developing QwQ-32B. Lin reports that a 32-billion-parameter model’s AIME 2024 score rose from roughly 65 to 80 during reinforcement learning. The presentation does not specify the checkpoint sequence or scoring protocol behind that trajectory. He also reports sustained top-15 placement in an arena evaluation and competitiveness with larger models, without establishing a usable leaderboard identity for that comparison.

Qwen3 combines the instruction-following and reasoning work in a family of dense and MoE models. The flagship, Qwen3-235B-A22B, has 235 billion total parameters but activates 22 billion per token. This distinction is central to MoE efficiency: total capacity and the parameters used for a particular token are different quantities. Lin places the flagship near o3-mini and slightly behind Gemini 2.5 Pro in the comparisons he presents. He also describes the largest dense model as competitive.

The smaller models make two different efficiency arguments:

• Sparse activation: Qwen3-30B-A3B has 30 billion total parameters and activates 3 billion. Lin says it can outperform QwQ-32B on some tasks despite activating far fewer parameters.
• Distillation: Qwen3-4B transfers knowledge from larger models into a smaller dense model. With thinking enabled, Lin describes it as competitive with the previous Qwen2.5-72B flagship and suitable for mobile deployment.

These are different routes to reducing inference requirements: MoE selects a subset of a larger model, while distillation improves a smaller model’s capabilities. The mobile claim comes without a device, quantization setting or latency measurement.

Hybrid thinking puts two response behaviors in one model. Thinking mode explores possibilities and reflects before producing the answer, as in o1 and DeepSeek-R1. Non-thinking mode behaves like a conventional instruction-tuned chatbot, moving directly to the response with less delay. The useful control is whether a request needs deliberation, rather than whether the application should switch to an entirely different model.

Prompts or configuration parameters select the behavior. Lin tentatively presents combining both modes as a possible first for the open-source community; the practical feature is the shared model, regardless of priority. The original Qwen3 release exposes enable_thinking through its Python chat template. For example, an application can prepare the same request in either mode:

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B")
messages = [
    {"role": "user", "content": "What is 17 × 24?"}
]

prompts = {
    mode: tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
        enable_thinking=enabled,
    )
    for mode, enabled in {
        "thinking": True,
        "non_thinking": False,
    }.items()
}

print(prompts["non_thinking"])

This selects the prompt format; it does not itself run generation or set a thinking-token budget.

Once a model can think before answering, the application needs a way to limit that work. A thinking budget is the maximum number of tokens available for reasoning. Consider Lin’s example of a task whose reasoning would finish after 8,000 tokens:

A budget is a ceiling, not a requirement to consume every token. It also differs from a total output limit, which can constrain the answer as well as reasoning. Current implementation guidance separates capped reasoning from final-answer generation; the presentation does not specify its exact implementation.

Lin reports AIME 2024 performance just above 40 with a very small thinking budget and above 80 with a 32,000-token budget. The smallest budget, model variant and sampling protocol are not specified in the passage. The slide places AIME24 alongside AIME25, LiveCodeBench and GPQA Diamond, showing thinking-mode curves against non-thinking baselines. The engineering question is how much additional reasoning improves the task enough to justify its token cost.

Lin illustrates that decision with a hypothetical application requiring 95% accuracy: if an 8,000-token budget already exceeds the requirement, there is no need to allocate more thinking tokens merely because the model permits it. The example is a budget-selection rule, not a measured 95% result. He then turns from reasoning controls to another part of the release: broader language coverage.

Qwen3’s release specifies support for 119 languages and dialects, compared with 29 languages for Qwen2.5. The release provides the detailed language list. The motivation is practical: developers working in languages underserved by earlier open-weight models need models they can use in their own domains, not just models that perform well in a few widely represented languages.

The next capability changes the structure of a response. Qwen3 strengthens agent and coding support, including Model Context Protocol (MCP) integration. A function call can occur during thinking; the tool’s result then becomes new evidence for continued reasoning. Instead of generating an entire plan from a fixed prompt, the model can revise its next step after observing what happened.

Desktop organization makes that loop concrete. Given filesystem access, the model must select tools, use them, inspect their feedback and continue until the work is complete:

1. Determine which filesystem tool is needed for the next step.
2. Invoke the tool and receive its result.
3. Continue reasoning with that environmental feedback.
4. Repeat as needed, then report the completed organization to the user.

The desktop example is one of two tool-use examples Lin introduces. Its distinctive feature is thinking that continues through interaction: tool use is part of the reasoning process, and the environment helps determine what happens next. That is the bridge from a chatbot to an agent doing useful work.

The initial open-weight Qwen3 lineup contains two MoE models—the 30B/3B-active and 235B/22B-active variants—and six dense models. The sizes serve different deployment and experimentation roles:

Lin highlights Qwen3-32B as a strong model developers can run locally and use for RL. Dense models remain part of the release, but he expects future development to favor MoE, with more releases and better support from third-party frameworks.

The same progression extends to vision-language models. Qwen2-VL was followed by Qwen2.5-VL in January 2025. Lin cites competitive performance on MMMU, MathVista and general visual question-answering benchmarks. These evaluations cover understanding and reasoning about visual inputs, rather than only generating text from text.

QVQ explores thinking in this setting. Increasing the maximum thinking length gives the model more room to reason about visual problems, especially mathematics. The displayed MathVision chart reports accuracy rising from 43.5% at 4,000 thinking tokens to 48.1% at 24,000, with intermediate points of 45.6% at 8,000 and 46.7% at 16,000. The exact QVQ variant and evaluation protocol are not identified here, but the slide makes the proposed scaling axis explicit: more inference-time reasoning for a visual task.

An omni model broadens both what the model can perceive and what it can produce. Qwen2.5-Omni is built on a 7B language model and accepts text, vision—including images and video—and audio. Its outputs in this release are text and audio, enabling text, voice and video chat. High-quality image and video generation remain future ambitions, not capabilities of this demonstrated release.

Lin reports leading audio-task performance among similarly sized 7B models and is encouraged by the visual results. The comparison with Qwen2.5-VL-7B is task-dependent: the original technical report shows Omni ahead on MMMU validation but slightly behind on MathVista testmini. Combining modalities can therefore preserve strong visual understanding without uniformly outperforming the specialist vision model.

There is also an acknowledged regression. Language intelligence and agent-task performance have dropped, and Lin proposes better data quality and training methods as ways to recover them. Broader modality coverage does not automatically preserve every existing capability. The goal is a model that can work across domains without sacrificing the reasoning and agent behavior that make those inputs useful.

Open releases create a feedback loop with developers: people use the models in different settings, report what fails and help guide improvements. Qwen2.5-Coder is already used for local development in Lin’s account, while Qwen3-Coder is still under development at the time of the talk. The breadth of the lineup reflects demand at every scale, from 0.6B models to the earlier 72B dense models and the 235B MoE.

Distribution also means supplying formats people can deploy. Lin lists GGUF, GPTQ, AWQ and MLX for Apple hardware. He says most Qwen models use Apache 2.0, allowing business use and modification without separately requesting permission; that family-level statement should not be read as a license declaration for every checkpoint. The initial eight Qwen3 models are explicitly Apache 2.0. Broad third-party framework and API-platform support is another part of making the family accessible beyond its own chat interface.

Qwen Chat packages model capabilities into workflows. In WebDev, a simple request to create a Twitter website produces code and an artifact-like preview. The next step is deployment: the user can obtain a URL and share the resulting site. This connects generation to an inspectable output rather than leaving the user with code alone.

A sunscreen product introduction site provides another example, including clickable buttons. The same workflow can produce something smaller than a website: a card based on a supplied link and additional information. Lin says the team uses these cards on Twitter. Across the examples, the input stays simple while the output becomes a visual object the user can inspect, interact with or share.

Deep research turns a broad topic into a longer sequence of work. A user might request a report about healthcare or artificial intelligence. The system first asks what to focus on; the user can narrow the scope or leave that choice to the system. The workflow then proceeds through several stages:

1. Make a research plan.
2. Search step by step.
3. Write parts of the report incrementally.
4. Continue searching as the report develops.
5. Assemble a comprehensive report that can be downloaded as a PDF.

Search and writing are interleaved rather than treated as one retrieval followed by one final response.

Lin describes reinforcement-learning fine-tuning for a specialized deep-research model as ongoing and difficult. A usable research product requires sustained progress across a sequence of decisions, not just a good answer to one prompt. There remains substantial room to improve the quality of that process, but its attraction is concrete: reducing the work needed to produce a useful report.

The future agenda begins with pretraining, where Lin still sees considerable headroom. Some useful data has not been included; other data has not been cleaned well enough. Multimodal and synthetic data could expand the tasks and domains a model handles. He also raises the possibility of changing the training objective beyond next-token prediction, including reinforcement learning during pretraining. That last proposal is an avenue to explore, not a completed training recipe.

The second direction is scaling reinforcement-learning compute. Earlier scaling emphasized model size and pretraining data. Agent training adds long-horizon reasoning with environmental feedback: act, observe the result, continue thinking and decide what to do next. This returns to the desktop-organizing mechanism, but now as a training objective. Inference-time scaling can include repeated interaction with the world, not just a longer uninterrupted stream of reasoning.

That interaction creates the third pressure: context. Long reasoning trajectories generate more history, while memory increases the input an agent needs to use. Lin’s proposed sequence is to handle one million tokens well, then pursue ten million and eventually an aspirational infinite context. Lin sets a 2025 target of at least one million context tokens for most Qwen models. This is a roadmap commitment; the original Qwen3 release lists 32K or 128K context depending on the model.

The fourth direction is input and output modalities. Lin distinguishes increased productivity from increased intelligence: another modality can enable a new kind of work even if it does not make the model a better reasoner. Vision-language understanding, for example, enables a GUI agent to inspect an interface and perform computer-use tasks; he describes such work as almost impossible without vision. The output side matters too. Unifying image understanding with image generation would let one system both interpret and create visual material, with GPT-4o’s image generation serving as the example.

The development target is shifting from training models to training agents. Pretraining supplies the foundation, while reinforcement learning—especially interaction with environments—trains behavior across extended tasks. Larger contexts and more modalities give those behaviors room to operate. For Lin, the agenda for 2025 and the following year is to bring these directions together so that the model can keep working, observing and reasoning until a task is done.