Field guide · Updated August 21, 2026
What is an AI Engineer?
An AI engineer is a software engineer who designs, builds, evaluates, and operates products powered by foundation models. AI engineering is the discipline of turning model capabilities into reliable user-facing systems by combining software architecture, data and context, model selection, evaluation, safety, and production operations.
The role is broader than prompt writing and does not require training every model from scratch. Its defining responsibility is making an AI system useful and dependable in the real world.
- Primary output
- A reliable AI product, feature, or workflow
- Typical collaborators
- Product, design, data, ML, platform, and domain experts
- Success measure
- Useful outcomes within quality, safety, cost, and latency limits
The work
What does an AI engineer do?
Exact ownership changes with the team and product. These duties form the common center of the role.
Choose and test models
Compare hosted and open models against quality, latency, cost, privacy, and deployment needs.
Design the AI system
Combine prompts, retrieval, tools, memory, workflows, and application code into a useful system.
Build evaluations
Turn requirements and real failure modes into repeatable tests, datasets, review, and release criteria.
Ship production software
Integrate models with interfaces, APIs, data systems, authentication, and the rest of the product stack.
Operate for reliability
Monitor quality, safety, security, latency, and cost as models, data, and user behavior change.
Close the feedback loop
Study outcomes and traces, diagnose failures, and decide whether to change code, context, data, or models.
The operating loop
AI engineering continues after launch.
Production evidence flows back into every earlier decision.
- 01
User problem
Define the outcome and constraints.
- 02
Model + context
Choose capabilities, data, and tools.
- 03
Evaluation
Measure quality and failure modes.
- 04
Product
Design the interface and workflow.
- 05
Production
Ship with security and reliability.
- 06
Telemetry
Learn from real outcomes and iterate.
The artifacts
What AI engineers ship
The product is more than a model call. It includes the surrounding software and the evidence that it works.
- OUTPUT 01A support copilot grounded in company knowledge
- OUTPUT 02An agent that completes a bounded workflow with tools
- OUTPUT 03A search or recommendation experience with measurable quality
- OUTPUT 04An evaluation suite that catches regressions before release
- OUTPUT 05A multimodal interface that understands text, audio, or images
- OUTPUT 06An inference service with cost, latency, and safety controls
The toolkit
Skills for AI engineering
Tools change quickly; capability areas last longer. Strong AI engineers combine software depth with enough model, data, product, and communication fluency to own the whole outcome.
Software and product
APIs, backend and frontend systems, databases, authentication, UX, testing, product judgment, and communication
Models and context
Model APIs, prompting, structured outputs, retrieval, embeddings, tool use, agents, and fine-tuning
Evaluation and operations
Datasets, graders, human review, observability, latency, cost, incident response, and experimentation
Data, safety, and security
Data quality, privacy, access control, prompt injection defenses, abuse testing, and responsible deployment
Related roles
AI engineer vs. machine learning engineer
The distinction is about emphasis, not rank. AI engineers usually own the product system around a model; machine learning engineers usually own more of the data, training, and model-performance loop. In practice, the roles overlap substantially.
AI engineering emphasis
User workflow · application architecture · context and tools · product evaluation · UX · production operations
Shared zone
Serving
fine-tuning
evaluation
observability
ML engineering emphasis
Data pipelines · feature and dataset design · training · experiments · model performance · scalable serving
A living discipline
The definition keeps being tested by the people shipping the work.
AI Engineer events bring together practitioners working across models, infrastructure, evaluation, agents, product, and research. The talks below add first-hand technical perspectives to this field guide.
Watch and learn
Four perspectives from the field
These talks span the role's definition, production patterns, evaluation, and infrastructure. Players load only when requested.
Greg Brockman
Cofounder · OpenAI
#define AI Engineer
A direct discussion of the emerging role and the opportunities available to engineers building with increasingly capable models.
“The most important determiner is a specific flavor of technical humility.”Greg Brockman, 2022 essay
“In numbers, there's probably going to be significantly more AI Engineers than there are ML engineers / LLM engineers. One can be quite successful in this role without ever training anything.”Andrej Karpathy, 2023Open on YouTube
Dex Horthy
Founder · HumanLayer
12-Factor Agents
Production patterns for moving agentic software beyond a demo and toward a system that can be understood, tested, and operated.
“Not every problem needs an agent.”Dex Horthy, from the talkOpen on YouTube
Sayash Kapoor
Author · AI Snake Oil
Oil
Building and evaluating AI Agents
A practical look at what evaluation can and cannot tell us about agent performance, grounded in evidence rather than vibes.
“AI agents don't (yet) work well.”Sayash Kapoor, on the keynoteOpen on YouTube
Charles Frye
Developer Advocate · Modal
What every AI engineer needs to know about GPUs
A systems-level explanation of the hardware constraints behind inference, serving, latency, and cost decisions.
“What I think every person working with language models needs to know about GPUs.”Charles Frye, on the talkOpen on YouTube
Career path
How to become an AI engineer
There is no single credential or sequence. The most legible path is to pair software fundamentals with model fluency, then prove both in a complete production-minded project.
- STEP 01
Build strong software foundations
Learn to design APIs, work with data, test code, use version control, and deploy a normal application before adding model uncertainty.
- STEP 02
Learn the model interface
Practice prompting, structured outputs, retrieval, tool calling, and model selection. Understand tokens, context limits, latency, and cost.
- STEP 03
Make evaluation part of development
Collect representative examples, define success, test failure cases, and compare changes instead of relying on a compelling demo.
- STEP 04
Ship a complete, narrow system
Build something real people can use. Add authentication, feedback, observability, safety controls, and a written account of the tradeoffs.
Degree
Useful, not universal. Research-heavy roles may require advanced ML depth; applied roles can be reached through software experience and strong shipped work.
Certification
Optional. A vendor credential can structure learning, but it is evidence of platform familiarity—not proof that you can make an AI product reliable.
Interview
Expect architecture, debugging, evaluation design, tradeoff reasoning, communication, and coding. Be ready to explain failures and what production evidence changed.
US compensation and outlook
Use title-specific and official benchmarks together.
Indeed reported average US base salary of $153,445 for “AI/ML engineer,” based on 1,800 salaries in postings from the prior 36 months and updated August 16, 2026. This is a changing job-posting estimate, not an official occupational statistic.
The US Bureau of Labor Statistics does not publish an occupation called “AI engineer.” Two adjacent benchmarks are software developers, with median annual pay of $133,080 in May 2024 and 16% projected employment growth from 2024–2034, and computer and information research scientists, with median pay of $140,910 and 20% projected growth. Neither figure is an AI-engineer salary.
Indeed's career guide also emphasizes collaboration, communication, analytical judgment, and business knowledge alongside technical skills. Those durable capabilities are included above; its older model-training-centric definition is not adopted here because this guide focuses on the foundation-model-era role. Read Indeed's career guide.
Editorial provenance
The framing that stuck
Original essay · June 30, 2023
AI engineer was already a job title. In 2023, swyx's Latent Space essay The Rise of the AI Engineer popularized the foundation-model-era framing of the role: software engineers turning increasingly capable models into useful products.
It placed product-specific data, evaluations, and the practical overlap between AI and ML within the role. This guide carries that framing forward with the operating practices, career paths, and role boundaries that have emerged since.
The term predates the essay; its contribution was the framing, not the title.Read the original essay Common questions
AI engineering FAQ
Does an AI engineer train models from scratch?
Usually not. Many AI engineers begin with hosted or open foundation models and focus on the system around them. They may fine-tune, distill, or host models when product requirements justify it, while research and ML teams often own more of the training loop.
Is prompt engineering the same as AI engineering?
No. Prompt design is one technique inside a broader discipline. AI engineering also includes software architecture, data and retrieval, tool integration, evaluation, security, user experience, deployment, monitoring, and iteration.
Do you need a computer science or machine learning degree?
There is no universal degree requirement. A relevant degree can help, especially for research-heavy or model-training roles, but strong software fundamentals and evidence that you can ship and evaluate complete AI systems are often more important for applied roles.
Which programming language should an AI engineer learn?
Python is common across model, data, and evaluation tooling. TypeScript is common for product interfaces and full-stack AI applications. The better choice is the language that lets you build a complete system in your target environment; many teams use both.
Are AI engineering certifications required?
No. Cloud and platform certifications can provide a structured curriculum or demonstrate familiarity with a particular stack, but they do not replace a portfolio of deployed systems, clear evaluation work, and sound engineering judgment.
What should an AI engineering portfolio show?
Show a narrow product working end to end, the evaluation set and release criteria, important failure cases, architecture decisions, and what changed after observing users. A polished demo without evidence of reliability reveals less.
Why do AI engineer job descriptions vary so much?
The title is young and company needs differ. At one team it may mean product-focused foundation-model work; at another it may include data pipelines, fine-tuning, model serving, or research. Read the outcomes and responsibilities, not only the title.
Where did the term AI engineer come from?
The title existed before the current foundation-model wave. Shawn “swyx” Wang's 2023 Latent Space essay popularized a foundation-model-era framing centered on software engineers who apply powerful models to products; he did not coin the term.
Put the definition to work