AI Engineer Summit 2025 Schedule

Talk title and abstract TBA

Want to guarantee your AI initiatives are great boardroom presentations but don't actually work? This talk explores the most reliable ways to waste time, money, and resources while building AI strategies. One proven method? Cultivate the perfect disconnect between those building AI solutions and the executives charting the strategic direction.

For business leaders, navigating the AI space is overwhelming. There's a sea of advice on how to allocate resources, hire and execute on AI, but much of it comes from sources that aren't hands-on with the technology. Without practical experience, it's challenging to separate fact from fiction and make informed decisions. On the flip side, AI practitioners who are in the trenches often struggle to communicate their knowledge in a way that resonates with executives. They tend to think and speak in technical terms, which can be off-putting or confusing for those focused on business outcomes.
After showing you common failure modes, Hamel and Greg will discuss how to overcome and avoid them by drawing on their experience helping 30+ companies develop and execute successful AI projects.

Multi-agent systems are revolutionizing industries, driving innovation with autonomous agents that collaborate in dynamic environments. But with great complexity comes great vulnerability.
If you’ve ever worried about adversarial behaviors, privacy breaches, or data leakages in these systems, you’re not alone. These threats can erode trust and jeopardize the integrity of even the most cutting-edge applications.
What if there were ways to safeguard these systems without stifling innovation? Enter a balanced approach that combines robust security frameworks with adaptive, innovation-friendly strategies.
In this talk, we’ll explore how to identify and mitigate the unique risks inherent in multi-agent systems while fostering an environment where creativity thrives. Learn practical techniques for maintaining security, ensuring safety, and enabling innovation in these transformative systems.

What if you built an AI coding agent that…built itself? It sounds like science fiction, but AI coding startup Augment Code built an AI agent that not only writes code, but continuously improves its own codebase - including performance optimizations and system architecture. This level of AI autonomy raises fascinating questions about the future of software engineering, where human engineers may shift from writing code to orchestrating multiple self-improving AI agents working in parallel.

In this talk, Augment co-founder and Chief Scientist Guy Gur-Ari will explore:

• What does it look like to supervise AI coding agents at scale?
• How far can we push AI coding agents - what tasks are out of scope?
• What might unlock when AI handles complexity and humans drive innovation?

Talk title and abstract TBA

In this talk, we will offer insights into building LinkedIn’s GenAI Platform team from the perspective of a first-line manager. We will explore the journey of developing a scalable and adaptable GenAI technology stack in a rapidly evolving tech landscape, seamlessly integrating AI-first principles into LinkedIn’s predominantly Java-based ecosystem, and achieving significant advancements in multi-agent system architectures. The presentation will highlight LinkedIn's GenAI platform innovations, including a centralized registry for dynamic skill-sharing, contextual memory for agent interactions, seamless integration with existing infrastructure, and a balanced approach to leveraging both in-house solutions and popular open-source libraries. Attendees will gain practical insights into fostering collaboration, cultivating a developer-friendly environment, aligning agile execution with long-term platform goals, and identifying the critical talent and skills essential for success. This talk is designed to inspire and guide leaders as they build foundational GenAI teams in dynamic, fast-changing environments.

Talk title and abstract TBA

The latest generation of LLMs is demonstrating impressive test time reasoning capabilities. However, to be truly valuable in an enterprise setting requires those agentic capabilities to be applied to the right enterprise data. In all the excitement around AI agents, many of us have somehow forgotten the timeless adage “garbage in; garbage out” – language models can only do their job if they are contextualized properly. In this talk, Douwe Kiela will share lessons learned from deploying enterprise RAG systems at scale and how to design a system robust enough for the Fortune 500.

Talk title and abstract TBA

Gen AI is cool, but what does that mean for your function? This is a talk on when AI hit the road, and where it missed and why. 2024 has been such a confusing year! Tech giants vs startups, established service providers vs internal redefinition of tool stack, trials to licenses, and more. As a functional leader for any org, how do you know the state of AI for your business processes? 2024 has been the year we experimented to find that answer. We tried out lots of vendors, open source, even in-house solutions for AI driven workflow automation and business processes across different functions in our enterprise. Join me as we describe the potential for Agentic workflows, learning from our narrative of what worked in 2024, and what we want changing in 2025.

Turning AI agents into reliable, production-ready tools that deliver tangible business results requires more than just great models. It demands robust evaluation frameworks that ensure agents perform at scale, align with organizational objectives, and continuously improve in dynamic environments.

This session provides an executive-level perspective on evaluating AI agents at scale. We’ll explore practical strategies for designing evaluation processes that drive measurable impact, identifying and mitigating performance bottlenecks, and implementing observability practices to maintain reliability over time. Through insights from real-world deployments, we’ll highlight common pitfalls, share best practices for iterative improvement, and demonstrate how effective evaluation frameworks can transform experimental agents into enterprise-grade solutions.

Whether you're shaping your organization’s GenAI strategy or looking to unlock the full potential of AI agents, this talk offers actionable insights to ensure your agents work—and scale—successfully.

This presentation talks to AI Executives who are tasked with building self managed AI Networks. It will talk about some new terminology that is used, list the key considerations around power to the racks, new hardware (GPU servers) and software (workload managers, GPU management and programming) that's required, there are specific storage requirements, multi-tenancy options, uptime, telemetry and visibility. It will then drill down into the details of the differences between the Data Center networks we build today versus what's required for the next generation of AI Networking.

Operate services without getting paged, and not because you phone ran out of batter. Ever get woken up in the middle of the night because a service has gone down, stumbling around looking for your laptop. What if there was an oncall engineer who didn’t need to sleep, always ready to handle issues instead, letting you sleep. In this talk we’ll share how we’re using AI Agents at Datadog to handle alerts and incidents for you, and what we’ve learned building agents to use Datadog and handle DevOps like a human would.

As enterprises have embraced GenAI architectures the biggest gap is connecting all of the structured and unstructured datasets across your organization. Knowledge graphs are a natural representation of connected data that allows you to enable higher accuracy and more explainable retrieval using GraphRAG. In this session we will talk about how to build organizational support for a connected data architecture that is the fuel for the current and future GenAI applications of your company.

In today's rapidly evolving financial landscape, AI applications and agents are transforming high-value workflows, like risk assessment, fraud detection, and customer service. As financial institutions increasingly integrate AI into their operations, the reliability and trustworthiness of these systems – and the underlying LLMs – are paramount. The consequences of inaccurate outputs in finance can be significant, ranging from financial losses to reputational damage, underscoring the need for models that are tailored for industry-specific challenges like nuanced terminology and complex regulatory requirements. In this session, Waseem AlShikh, CTO and Co-founder of Writer, will challenge the scaling narrative around general-purpose models and demonstrate how domain-specific LLMs, particularly within the high-stakes finance industry, are delivering state-of-the-art performance, without the need for endless pre-training budgets and resources.

Is 2025 the year of AI agents? Will reasoning models allow agents to solve challenging open problems? From software engineering to web task automation, it has been claimed that agents will solve challenging open problems. Unfortunately, current agents suffer from many shortcomings that reduce their utility in real-world tasks — look no further than Rabbit R1 and the Humane Pin. In this talk, we will explore how current agents fall far short of their claimed performance in the real world and understand best practices for improving agent evaluation. Learn how to avoid known pitfalls and build AI agents that actually matter.

You’re doing research all the time, we just don't always call it that. From figuring out the right summer camp options for your kids to running due-diligence on an investment opportunity, there are plenty of tasks that take up hours of your day and dozens of chrome tabs. Gemini Deep Research is our initiative to turbocharge this process. In this talk we’ll discuss how we built it, learnings along the way and our vision for the future of personal research assistants.

What if AI agents could build and launch successful products on their own?

I built a system where AI agents handle almost everything - from software development to customer support to marketing - to build, iterate, and maintain a live product. After 5 iterations, my most successful AI-built product was able to hit $3k MRR in under 2 weeks.

In this talk, I'll share practical lessons learned from building these autonomous product systems. You'll learn how each agent works, what succeeded, what failed, and how to build your own autonomous product development system (APDS).

tbc

tba

tbd

AI Engineering: Agents

The Ideal AI Engineering Platform

tbc

tbc

tbc

"If you've built a advanced LLM app, you know how finicky it is to break down complex goals into sub-tasks and organize them into a workflow. For instance, what are the best practices when decomposing tasks? How to manage the state within a workflow? Are generic flows better than complicated ones?

Flow engineering is an emerging skill among AI engineers that covers these questions.

In this workshop, we'll explore the four fundamentals of flow engineering: managing state, curating context, improving conditional edges, and best architectural practices. Through these fundamentals, you'll learn to transform complex LLM interactions into well-structured, maintainable systems that deliver reliable results."

In this workshop, Nick (LangChain) will explain agent evaluation concepts. Jeff Barg will then show how Clay performs E2E agent tests for Claygent for a few different prompt categories against a ground truth dataset, and discuss how Clay is running prompt-level evals.

tbc

tbc

In fast-paced technical roles like Solutions Architects and Cloud Engineers, repetitive, time-consuming tasks can pull us away from high-impact work. In this session, you’ll learn how to leverage Multimodal AI agents to perform multi-step tasks by integrating tools that streamline software development—from ideation and design to estimation, deployment, and testing. We’ll discuss key considerations and challenges when building multi-agent collaboration.

Building LLM-based applications is exciting, but keeping them reliably performant in production is a challenge—even for the best teams. How do you track user interactions? Diagnose issues? Prevent regressions when prompts or models change? At Weights & Biases, we’ve worked with hundreds of companies to tackle questions like these.

Join us for this exclusive workshop, where we’ll guide you through building and evaluating an AI application using a proven, future-proof workflow, learning how to:

• Trace and Log everything
• Collect and leverage user feedback
• Build evaluation datasets
• Design evaluations (Programmatic, HITL, LLM as a judge)
• Confidently iterate and optimize performance of your applications

No prior experience with evaluations? No problem. Just bring your laptop, and leave with practical skills to future-proof your AI applications.

Learn how to leverage the latest LLMs, tools and patterns to build AI-powered experiences.

You can think of this workshop as the hands-on version of this article by Anthropic https://anthropic.com/research/building-effective-agents

The workshop will feature a mix of technical deep dives and hands-on exercises. Coding will happen in Stackblitz, a cloud-based IDE, to spend the least amount of time on environment/ project setup. The entire workshop material and exercises will be available on GitHub so that students who want to operate at their own pace are able to do so. This is a great chance for you to be hands on with Vercel AI SDK.

When it comes to assumptions & prerequisites, workshop attendees should have:

1. Knowledge of building APIs and server-side code with JavaScript/TypeScript
2. Familiarity with command-line tools and installing Node.js packages

The exercises will add backend functionality to a fullstack app. No front-end knowledge is needed, the workshop will focus on the backend.

This workshop is aimed at the generalist TypeScript developer onboarding to AI, they will walk away with the necessary knowledge to build AI features.

tbc

Discover the transformative potential of Multimodal Agentic Retrieval-Augmented Generation (RAG) in this hands-on workshop. Participants will explore the integration of advanced AI engineering techniques with Amazon Nova to build systems that leverage text and image embeddings, LangChain, and Amazon Bedrock for creating context-aware multimodal AI agents. The workshop focuses on the practical application of Multimodal RAG to automate complex workflows, such as insurance claim optimization, through enhanced decision-making and operational efficiency.

This workshop will provide attendees with step-by-step guidance to:

• Build multimodal AI agents that process text, images, and video.
• Implement sophisticated retrieval systems using Amazon Nova and LangChain.
• Create real-world applications leveraging Bedrock-powered agents and knowledge bases.
• Attendees will leave with a solid understanding of building and deploying Multimodal RAG systems that integrate seamlessly into enterprise frameworks, paving the way for innovation in AI-powered solutions.

see https://build-ai-app-dec-24.vercel.app/docs for prereading

Agents aren't really agents unless they can discover the world outside of the chatbox, and act on it -- and the MCP (Model Context Protocol) is an emerging standard for connecting agents to physical and digital worlds. This workshop will teach you how to build your own MCP client (not just the one built into Claude) and connect it to a multi-agent system (we'll be using LangGraph but this approach works with Autogen, Crew AI and many more). This allows you to build powerful workflows that go way beyond writing code, and rethink how to express business logic as a set of roles, tools, and desired outcomes. We'll walk through examples together and then put them into practice -- come with ideas of workflows you want to build! [List of existing MCP servers here]

tbd