The 2025 AI Engineering Report — Barr Yaron, Amplify Partners

Does AI engineering describe a job title or the work people actually do? The early findings of the 2025 State of AI Engineering Survey give that question a concrete setting. Barr Yaron, an investment partner at Amplify backing technical founders, introduces the results with a promise of less biography and more charts. The survey has 500 respondents, including conference attendees and livestream viewers. Software engineers and AI engineers make up the largest group.

Then comes a quick test in the room. Yaron asks who actually holds the title AI engineer; few hands rise. She asks people with other titles to raise their hands, then keep them up if they do the same work as AI engineers. The response makes the distinction visible: the work extends beyond the title. Yaron expects the label to gain ground as this broad technical community grows.

Her Google Trends observation places the emergence of the term around the launch of ChatGPT: AI engineering barely registered before late 2022, then attracted sustained interest. A new discipline does not necessarily mean inexperienced practitioners. Among surveyed software engineers with at least ten years of software experience, nearly half have worked with AI for three years or less, and one in ten started in the past year. Even veteran developers are learning a new practice.

More than half of respondents use LLMs for both internal and external applications. For customer-facing products, OpenAI supplies three of the five most-used models and five of the top ten. These rankings capture a historical snapshot preceding Claude 4 and GPT-4.1.

Code generation and code intelligence lead the applications, alongside writing assistance and content generation. But the more consequential pattern is breadth: among respondents using LLMs, 94% use them for at least two use cases and 82% for at least three. Adoption here means using models across several kinds of work, often both inside the company and in products customers use.

Beyond few-shot learning, retrieval-augmented generation, or RAG, is the most popular system-customization approach: 70% of respondents report using it. Fine-tuning is also more common than Yaron expected, with researchers and research engineers doing it most often.

An open-ended question asks fine-tuners which techniques they use. Forty percent of those responses mention LoRA or QLoRA, which Yaron reads as a strong preference for parameter-efficient methods. The responses also name Direct Preference Optimization, or DPO, and reinforcement fine-tuning. Supervised fine-tuning remains the most popular core training approach, and respondents describe hybrid approaches as well. The resulting picture is a mix of customization methods rather than a single standard recipe.

Just as one model integration settles, another release arrives with better benchmarks and a breaking change. Respondents report frequent changes to both models and prompts:

Prompt revision is especially frequent; Yaron jokes that some engineers have not stopped typing since GPT-4 arrived.

A post from Simon Willison can be enough to make yesterday’s trusted prompt feel inadequate. Yet 31% of respondents report no structured prompt-management tool. Frequent revision and explicit management have not advanced together. The survey does not ask how engineers feel about that gap; Yaron leaves that question for a jokingly proposed 2026 follow-up.

Text is well ahead of image, video, and audio in workplace usage. Yaron calls this the multimodal production gap. The gap persists when the chart expands beyond deployments working well to include models already in production but with less traction. Broadening the definition of production adoption does not erase text’s lead.

The next reveal separates nonusers into two groups: those planning eventual adoption and those with no current plan. Audio has the highest adoption intent: 37% of respondents not currently using audio plan to adopt it eventually. That percentage describes intentions among audio nonusers, not current usage across the whole sample. Yaron anticipates an audio wave and expects better, more accessible models to encourage further adoption.

Before asking about agents, the survey fixes a definition: an AI agent is a system in which an LLM controls the core decision-making or workflow. That avoids collecting adoption figures against hundreds of different interpretations. Under this definition, 80% of respondents say LLMs work well at work, while fewer than 20% say the same about agents. This is respondent-reported workplace success, not benchmark accuracy or a success rate restricted to agent users.

Most respondents are not yet using agents, but most at least plan to; fewer than one in ten say they will never use them. The gap between current success and future interest is substantial.

For agents already in production, permissions are more consequential than the adoption label alone. Most have write access, typically with a human in the loop, and some can act independently. An agent allowed to change a system presents a different operational responsibility from one that only reads it. Yaron identifies tool permissioning as an area she wants to understand better as deployments grow.

Production systems need monitoring, but respondents do not rely on one method. The monitoring question allows multiple selections, and most select multiple approaches. Standard observability is used by 60% of respondents, while more than 50% rely on offline evaluation.

Accuracy and quality evaluation likewise combine several methods, including collecting user data and using benchmarks. Human review remains the most popular. For tracking their own model usage, most respondents rely on internal metrics. These are distinct concerns: observing system behavior, judging output quality, and understanding model usage do not collapse into a single measurement.

Context storage adds another infrastructure choice. Dedicated vector databases are used by 65% of respondents. Yaron interprets that adoption as evidence that specialized databases provide enough value over general-purpose databases with vector extensions for many use cases. She then reports 35% primarily self-hosting and 30% primarily using a third-party provider, introducing those figures as being among the dedicated-vector-database group. The denominator is not explicit enough to convert those hosting figures into respondent counts.

The survey then turns from deployed systems to opinions and forecasts:

• AI disclosure: Most respondents think agents should be required to disclose that they are AI rather than human.
• Inference-time compute: Willingness to pay more wins, but only by a narrow margin.
• Model architecture: Respondents generally expect transformer-based models to remain dominant in 2030, prompting Yaron’s callback that attention may be all we will need.
• Open and closed models: A majority expect open-source and closed-source models to converge.

On a more speculative question, the mean respondent estimate for the share of the US Gen Z population that will have AI girlfriends or boyfriends is 26%. This is a forecast, not measured prevalence; the spoken question gives no forecast year.

Asked for the single most painful part of AI engineering, respondents put evaluation first. Yaron connects that result to the conference’s emphasis on evals and the preceding talk: the attention is responding to a problem practitioners already feel.

The closing survey question asks which podcasts and newsletters respondents actively learn from at least once a month, allowing multiple selections. The slide presents the top ten in each category. Latent Space heads both lists, and Yaron singles out Swyx for appearing in both. The lists offer a view of this community’s learning habits, rather than a population-wide ranking of publications.

At the time of the presentation, Yaron says the full report will appear the following week, adding detail on respondents’ favorite models and tools across the stack. She closes by inviting further discussion of AI trends online and in the conference hallways. The early findings leave evaluation as the leading practical pain point, alongside a community still exchanging the knowledge needed to address it.