# Emergence Launch: AI Agents and the future enterprise

Satya Nitta, Emergence

AI Engineer World's Fair 2024

[Talk page](https://ai.engineer/talks/xhQfRSueFZI-emergence-launch-ai-agents-future-enterprise) · [Video](https://www.youtube.com/watch?v=xhQfRSueFZI)

## Emergence Launch: AI Agents and the future enterprise

Emergence’s launch connects a planning-and-verification Orchestrator with Agent-E, a web agent, to turn delegated requests into coordinated software actions.

### Distributed systems meets AI research

[0:17](https://www.youtube.com/watch?v=xhQfRSueFZI&t=17s)

What does it take to build AI agents that can carry out work across software systems? Emergence approaches that problem as both AI research and distributed systems engineering. Satya Nitta introduces an R&D-led company founded by a team from IBM Research, with colleagues drawn from Google Brain, Alexa, the Allen Institute of AI, Meta, and Microsoft.

Nitta cites the team’s experience with IBM Watson, Alexa, and recommendation engines behind Amazon Prime, Amazon Video, Twitter, and Eventbrite. That deployment background explains the company’s emphasis: develop the science of agents while building infrastructure that lets other developers connect them into working systems.

### From answers to actions

[1:09](https://www.youtube.com/watch?v=xhQfRSueFZI&t=69s)

The ambition is older than today’s language models. Nitta traces it to science fiction of the 1940s and 1950s: AI would perform actions for people. Here, that means operating browsers, enterprise systems, and other software to automate complex human workflows. **The desired output is work performed through software**, with productivity benefits for both consumers and enterprises.

![Slide stating that AI agents will operate most of the world’s software, automate complex human workflows, and unlock the full promise of AI.](https://ai.engineer/api/talk-essay-media/d7244a50201e6c3e1cd98bc1e010d66ebe1ecc985f2167e05560e2170ee3d486)

AI agents operating software to automate complex human workflows.

Enterprise is Emergence’s focus because its workflows are demanding. Coordinating work across business software is expected to stress the limits of autonomous AI, making enterprise applications both a commercial destination and a proving ground for the research.

### An agent that coordinates other agents

[1:57](https://www.youtube.com/watch?v=xhQfRSueFZI&t=117s)

Emergence introduces two infrastructure platforms, starting with the **Orchestrator**. At the time of the recording, Nitta says early access is live and forecasts general availability in August. Those are the launch’s contemporaneous availability claims.

The Orchestrator is itself an agent. Its operating loop has three parts:

1. **Plan** the work.
2. **Act** to carry it out.
3. **Verify** the result.

Nitta also describes memory and improvement over time. The purpose of this loop is to coordinate multiple agents and stitch their contributions into a larger workflow. Verification is therefore part of the agent’s job, rather than treating an issued action as the end of the task.

The scope ranges from choosing among language models to coordinating business processes:

| Scope | Orchestrated components | Intended work |
|---|---|---|
| Simple | Generalist and open-source LLMs | Coordinate model use |
| Complex | Multiple agents | Enterprise workflows such as claims processing |

The difference is the work being composed: model responses at the simpler end, contributions from several agents toward a business outcome at the more complex end.

### Giving the Orchestrator a web agent

[2:51](https://www.youtube.com/watch?v=xhQfRSueFZI&t=171s)

**An orchestrator’s capabilities depend on the agents it can call.** Emergence is integrating the Orchestrator with [Agent-E](https://github.com/EmergenceAI/Agent-E), an open-source project intended to understand and operate web interfaces as a human would. Nitta points to an earlier session by his colleague Tamer for more on that component. The division of responsibility is straightforward: the Orchestrator coordinates the workflow; Agent-E supplies the ability to act through the web.

![Agent-E slide describing understanding and operating web interfaces like a human, with a GitHub logo and QR code.](https://ai.engineer/api/talk-essay-media/a9627516d1e86f5c3fb4d5cfc84c68d73e7d7dbec19cbf7ad7121adb15d212f3)

Agent-E: an open-source web agent for enterprise workflows.

Nitta reports that Agent-E tops the [WebVoyager benchmark](https://arxiv.org/abs/2401.13919). The talk supplies neither a score nor evaluation conditions, so its description of Agent-E as the best web agent should be read as a benchmark-specific company claim. The intended product role is broader: support multiple enterprise workflows while working with the Orchestrator.

### Developer controls and a delegated reservation

[3:47](https://www.youtube.com/watch?v=xhQfRSueFZI&t=227s)

The product video adds a developer-facing layer around this coordination. Generalist and task-specific LLMs and agents are described as operating with appropriate guardrails, though the video does not explain their implementation. Its dashboard capabilities fall into three groups:

- **Prompt analysis:** inspect prompts used by the system.
- **Cost and latency optimization:** tune the economics and responsiveness of model use.
- **Model development:** create models with your data and enhance existing models.

The video also promises on-demand migration to newer LLMs and describes the Orchestrator as improving with use. Agent-E, still described as being in R&D development, appears as one connected component in the LLM and agent garden.

![Diagram labeled LLM & Agent Garden lists Emergence-1, Agent-E, Custom-Agent, ChatGPT, Meta, and Gemini, with Agent-E highlighted.](https://ai.engineer/api/talk-essay-media/be4e2d27ed17957ed713cc62c068493ddb44c65badcdbca5bf15c91a30311d5b)

The Orchestrator connects to Agent-E in an LLM and agent garden.

The concrete example is a delegated restaurant booking: find a reservation for 15 people near the office around 6 PM, considering Asian, Mediterranean, or Mexican food, then send the details to Levi. As a structured task record, the request contains both booking constraints and a communication requirement:

```json
{
  "reservation": {
    "partySize": 15,
    "location": "near the office",
    "time": "around 6 PM",
    "cuisines": ["Asian", "Mediterranean", "Mexican"]
  },
  "followUp": {
    "recipient": "Levi",
    "content": "reservation details"
  }
}
```

This is a requested outcome, not a shown completed transaction. Its significance is the composition of work: finding an option that satisfies several constraints, securing a reservation, and communicating the result are all part of the delegation. The video closes by identifying planning and self-improvement as core research problems behind that ambition.

### Self-improvement and agent-oriented programming

[4:52](https://www.youtube.com/watch?v=xhQfRSueFZI&t=292s)

Nitta closes by making **self-improving agents** the labs’ central research focus. Planning and reasoning are supporting directions, alongside **agent-oriented programming**: ways to stitch agents together so their combined behavior can accomplish more complex work. He names the direction without presenting a programming language or composition API.

That research remains anchored in enterprise applications, including robotic process automation (RPA) and document processing. These are the contexts in which better planning, reasoning, and agent composition are meant to become useful systems that perform work.

## Resources

- [Agent-E](https://github.com/EmergenceAI/Agent-E) — Open-source browser automation agent with installation instructions, natural-language commands, and configuration examples.
- [Agent-E architecture and evaluation](https://arxiv.org/html/2407.13032v1) — Explains hierarchical planning, DOM processing, change observation, and the original WebVoyager evaluation.
- [WebVoyager](https://arxiv.org/abs/2401.13919) — Introduces a web agent, tasks across 15 real websites, and an automated evaluation protocol.
- [Introducing the Emergence Multi-Agent Orchestrator](https://www.emergence.ai/blog/q7yz1gypvzqcdalwwh2mxocdai8eth) — A later product account describing coordination between web automation and API agents.

## Slides

### Presenting sponsor: Microsoft

Microsoft is the presenting sponsor.

[0:05](https://www.youtube.com/watch?v=xhQfRSueFZI&t=5s) [Microsoft logo beneath the words presenting sponsor.](https://assets.ai.engineer/sites/aiecode2025/sha256/5d3d51b7f4395ac7638521872389073b482489fa8b12ec0e10957e6f0474a758.webp)

PRESENTING SPONSOR

Microsoft

### Innovation and platinum sponsors

AWS is the innovation sponsor; MongoDB, Google Cloud, and Neo4j are platinum sponsors.

[0:10](https://www.youtube.com/watch?v=xhQfRSueFZI&t=10s) [AWS is the innovation sponsor; MongoDB, Google Cloud and Neo4j are platinum sponsors.](https://assets.ai.engineer/sites/aiecode2025/sha256/89df67840b7375b18048f17cebc1f421962fa066f1ba8ac3ab753cbb7d1e710e.webp)

INNOVATION SPONSOR
aws

PLATINUM SPONSORS
MongoDB®
Google Cloud
neo4j

### About Emergence

Emergence describes itself as an R&D-led AI agent company; its founding team came from IBM Research.

[0:21](https://www.youtube.com/watch?v=xhQfRSueFZI&t=21s) [Emergence describes itself as an R&D-led AI agent company advancing the science of AI agents.](https://assets.ai.engineer/sites/aiecode2025/sha256/b5ea5e6a736264c3a43a26448a738fbd20476231a5d9f04c244a5d2153cb92e0.webp)

Emergence is an R&D led AI Agent company,
advancing the science of AI agents

emergence

### Emergence team experience

Emergence says its team has built and deployed AI at scale, from IBM Watson and Alexa to recommendation engines behind Amazon Prime, Amazon Video, Twitter and Eventbrite.

[0:25](https://www.youtube.com/watch?v=xhQfRSueFZI&t=25s) [Emergence says its technologists have worked on large-scale AI deployments, with a row of company and product logos.](https://assets.ai.engineer/sites/aiecode2025/sha256/c31b43aba08a4e3eaacf9674eb3b3cfa23546edfb902a87206bb37283448aef9.webp)

Technologists at Emergence have worked on some of the most scaled deployments of AI on the planet.

amazon
alexa
prime video
eventbrite
watsonx
airtel
LaMDA
emergence

### AI agents and software

Emergence predicts that AI agents will operate most of the world’s software, automating complex human workflows. The speaker points to web browsers and enterprise systems as examples.

[1:06](https://www.youtube.com/watch?v=xhQfRSueFZI&t=66s) [The slide predicts that AI agents will operate software and automate complex human workflows.](https://assets.ai.engineer/sites/aiecode2025/sha256/20a50968e38b9dfca73643de4dd8db92576cdaaecf3caa8804679d82fd5a57a4.webp)

AI agents will operate most of the world’s software, automating complex human workflows and unlocking the full promise of AI.

emergence

### Agent productivity benefits

Emergence expects agents to improve productivity for consumers and enterprises. It focuses on enterprise workflows because their complexity will push the limits of autonomous AI.

[1:38](https://www.youtube.com/watch?v=xhQfRSueFZI&t=98s) [Agents will drive productivity benefits for consumers and enterprises, with enterprise as the focus.](https://assets.ai.engineer/sites/aiecode2025/sha256/4541b2d271ad26c7340a8566edb03786d3adf878a796b7d17839550bea5af80e.webp)

Agents will drive great productivity benefits for consumer and enterprise (our focus area)

emergence

### Emergence’s Orchestrator

Emergence’s Orchestrator routes tasks to an LLM or agent. The speaker says it plans, acts, and verifies, and can stitch multiple agents into complex workflows.

[2:05](https://www.youtube.com/watch?v=xhQfRSueFZI&t=125s) [Emergence describes its Orchestrator as a meta-agent that routes tasks to an LLM or agent.](https://assets.ai.engineer/sites/aiecode2025/sha256/a4ddb8ba003e6343775885b81657a944b6ec355fa293462d3a11f38887da04ac.webp)

Emergence’s Orchestrator

is a meta-agent that intelligently routes tasks to the most optimal language LLM or agent.

emergence

### Agent-E

Agent-E is an open-source web agent designed to operate web interfaces for enterprise workflows. The team is integrating it with the Orchestrator.

[3:02](https://www.youtube.com/watch?v=xhQfRSueFZI&t=182s) [Agent-E is presented as an open-source web agent for automating enterprise workflows; the slide includes a QR code.](https://assets.ai.engineer/sites/aiecode2025/sha256/beb8f5152311b8dbab521b1f78ba6f8ec2298552382fda7b3ad89318f69dd6e1.webp)

Agent-E

is our open-source web agent, designed to seamlessly understand and operate web interfaces, just as a human can, to automate complex enterprise workflows.

emergence

[3:26](https://www.youtube.com/watch?v=xhQfRSueFZI&t=206s) [Agent-E is introduced as an open-source web agent for automating enterprise workflows, alongside a QR code.](https://assets.ai.engineer/sites/aiecode2025/sha256/380508ba74f7d3573d94383238d1989e84bbf377487f7d07b48a6150b8d7f6ee.webp)

Agent-E

is our open-source web agent, designed to seamlessly understand and operate web interfaces, just as a human can, to automate complex enterprise workflows.

emergence

Interface text:
AIE
Microsoft
[illegible]oogle Cloud
neo4j
AI Engineer World’s Fair
Lambda
Covalent
neptu[illegible]

### Routing to LLMs and agents

The Orchestrator routes prompts to models and agents, with guardrails, access to enterprise tools and data, and feedback on the output.

[3:47](https://www.youtube.com/watch?v=xhQfRSueFZI&t=227s) [An orchestrator routes prompts to models and agents, with guardrails, enterprise tools, and feedback.](https://assets.ai.engineer/sites/aiecode2025/sha256/961bb7e6877aa49e09325e4912c34c436a57039b53b1bf4b1ae30b33e4e014d9.webp)

Routes to generalist and task specific LLMs as well as agents

Input Prompt
Guardrail Agent
Orchestrator Agent
Guardrail Agent
Output

LLM & Agent Garden
Emergence-1
Agent-E
Custom-Agent
ChatGPT
Meta
Gemini

Enterprise Tools & Data

User Feedback
Critique feedback / Golden Samples

Developers tools: 1. LLM and Agent Registry  2. Prompt analytics  3. Prompt compression  4. LLM tuning on custom

emergence

### Dashboard usage overview

The usage overview displays year-to-date requests, GFM calls and estimated cost overall and for Orchestrator A. The figures shown are placeholders.

[3:55](https://www.youtube.com/watch?v=xhQfRSueFZI&t=235s) [A dashboard shows usage summaries for requests, model calls, and estimated costs.](https://assets.ai.engineer/sites/aiecode2025/sha256/4b9f9a45dca51a6691870d55c0e75824f96fff79e8f2ab30694648da51edb804.webp)

Dashboard: enhance models

Usage Overview
Monitor your orchestrator's performance with detailed usage statistics. See the difference in resource consumption between a GFM (generative foundation model) and our proprietary agents.

Usage Summary
YTD Requests
TOTAL REQUESTS  #
YTD GFM Calls
TOTAL CALLS  #
YTD Estimated Cost
ESTIMATED COST  $$$

ORCHESTRATOR A
HIDE DETAILED REPORT
YTD Requests
TOTAL REQUESTS  #
YTD GFM Calls
TOTAL CALLS  #
YTD Estimated Cost
ESTIMATED COST  $$$

Query ID  Date  Time  Agent  Est. Cost

Interface text:
emergence
Home
Orchestrators
API Keys
Usage Overview
Playground
emergence

### Migrate to the latest LLMs on demand

The configuration screen shows Claude and three Emergence AI agents selected alongside unselected Gemini, GPT-4o and LLaMA options. It also shows cost optimization and a domain guardrails toggle.

[4:08](https://www.youtube.com/watch?v=xhQfRSueFZI&t=248s) [An orchestrator configuration screen offers model and agent selections, a cost setting, and domain guardrails.](https://assets.ai.engineer/sites/aiecode2025/sha256/50e17b36d0155eb67cfe480b3d1e7b3e6f01c467b3eb196e02d4dd9c63abd414.webp)

Migrate to the latest LLMs on demand

Orchestrators

Select agents
■ Anthropic Claude 3 Sonnet
■ Emergence AI: Online Search + Q&A
■ Emergence AI: RAG Tools + Q&A
■ Emergence AI: Aligned Summarization
□ Gemini Pro 1.5
□ GPT-4o
□ LLaMA-3-Instruct-Efficient
□ LLaMA-3-Instruct-Powerful

Optimize for
Cost

Enable domain guardrails

BACK
SAVE

Interface text:
emergence
Home
Orchestrators
API Keys
Usage Overview
Playground

## Full transcript

[0:00](https://www.youtube.com/watch?v=xhQfRSueFZI&t=0s) [on-hold music] I'm here to tell you a little bit about what we're doing at Emergence in the field of AI agents.

[0:17](https://www.youtube.com/watch?v=xhQfRSueFZI&t=17s) But first, I wanna tell, tell you guys a few things about who we are. So we are an R&D-led AI company advancing the science and development of agents. And, uh, we come from some of the world's, uh, top AI labs, uh, founding team out of IBM Research, uh, and then we attracted, uh, talent from places like Google

[0:35](https://www.youtube.com/watch?v=xhQfRSueFZI&t=35s) Brain, Alexa, Allen Institute of AI, Meta, Microsoft, et cetera. But even more importantly, we have built and deployed some of the most scaled AI deployments on the planet, from the IBM Watson platform, to Alexa, to the backend recommendation engines behind Amazon Prime, Amazon Video, Twitter, Eventbrite, et cetera.

[0:55](https://www.youtube.com/watch?v=xhQfRSueFZI&t=55s) So we really think of ourselves as a distributed systems meets AI R&D team. Uh, and our goal is to work on AI agents and to enable all of you to build agents to transform the world.

[1:09](https://www.youtube.com/watch?v=xhQfRSueFZI&t=69s) So we're all here today because we are excited about what's actually coming out in AI. So the long promise of AI over this last several decades has always been that AI will perform actions for us.

[1:20](https://www.youtube.com/watch?v=xhQfRSueFZI&t=80s) Uh, this is what science fiction authors have told us since the nineteen forties and fifties, and I think the time is finally here. And in particular, what we're very excited about is AI that will operate things like, uh, you know, web browsers and, uh, various other enterprise systems and software, and in the process, uh, drive great productivity

[1:39](https://www.youtube.com/watch?v=xhQfRSueFZI&t=99s) benefits for everybody, both in consumer and especially in enterprise, which is our focus. So we're interested in enterprise because the most interesting workflows are actually in enterprise, and this will really stress and push the limits of what AI can do, in particular, what autonomous AI can do.

[1:57](https://www.youtube.com/watch?v=xhQfRSueFZI&t=117s) And, uh, so what we're building to enable that future to come to pass, uh, with, uh, with the help of all of you, uh, are two very infrastructural platforms, the first of which will be GA'd in August.

[2:08](https://www.youtube.com/watch?v=xhQfRSueFZI&t=128s) Uh, early access al- is already, uh, live right now, so you can go sign up for it. This is, uh, called an orchestrator agent. So first of all, it's an agent and, uh, in the sense that it acts, it plans, it basically also verifies.

[2:23](https://www.youtube.com/watch?v=xhQfRSueFZI&t=143s) So it, it finishes that agentic loop between planning, acting, and verifying. It remembers and improves over time. But what does it do? It actually allows you to orchestrate across multiple agents and stitch them together in complex workflows.

[2:37](https://www.youtube.com/watch?v=xhQfRSueFZI&t=157s) Uh, a simple version of it is you could simply orchestrate across multiple LLMs, generalist and open source LLMs, but more complex versions of it will basically enable you to solve very complex enterprise workflows like claims processing, et cetera.

[2:51](https://www.youtube.com/watch?v=xhQfRSueFZI&t=171s) Now, an orchestrator is only as good as the agents it will orchestrate to. So one of the exciting things that we're doing here is integrating the orchestrator with a project that is currently being developed in the open source called Agent-E.

[3:06](https://www.youtube.com/watch?v=xhQfRSueFZI&t=186s) Um, Agent-E is a project that, uh, uh, my, my colleague Tamer had a session on earlier this afternoon. It's a web agent. It's basically meant to control the web like a human would.

[3:18](https://www.youtube.com/watch?v=xhQfRSueFZI&t=198s) And currently, it's the best web agent on the planet. It's topping the WebVoyager benchmark, and, uh, it's designed to basically, uh, be used to build multiple enterprise workflows and work seamlessly with agent-- with the, with the orchestrator agent.

[3:33](https://www.youtube.com/watch?v=xhQfRSueFZI&t=213s) So I'm gonna play a short video that will show you a little bit more about both these agents, and then I'll, uh, wrap, wrap the talk up.

[3:44](https://www.youtube.com/watch?v=xhQfRSueFZI&t=224s) Okay, we don't have sound, so maybe I'll talk over it. So our first product is the Orchestrator-

[3:47](https://www.youtube.com/watch?v=xhQfRSueFZI&t=227s) With generalist and task-specific LLMs and agents, all using appropriate guardrails. The developer dashboard helps analyze prompts, optimize cost and latency, create new models with your data, and enhance existing ones.

[4:05](https://www.youtube.com/watch?v=xhQfRSueFZI&t=245s) Build with confidence, knowing your solution is future-proof and can migrate to the latest LLMs on demand. The Orchestrator is an intelligent agent that improves with use, connecting to agents like Agent-E, our web automation agent in development in our R&D labs.

[4:22](https://www.youtube.com/watch?v=xhQfRSueFZI&t=262s) Agent-E is an open source agent designed to learn how to autonomously operate the web and automate complex workflows. For example, get us a reservation for fifteen people near the office around six PM.

[4:34](https://www.youtube.com/watch?v=xhQfRSueFZI&t=274s) We like Asian, Mediterranean, and Mexican. Send the details to Levi, please. At Emergence, our mission is to advance the science of AI agents by tackling core AI problems like planning and self-improvement to enable the full transformation of AI and benefit the world.

[4:52](https://www.youtube.com/watch?v=xhQfRSueFZI&t=292s) Okay, so just in conclusion, uh, in our R&D labs, our area of focus is around self-improvement agents. This is our core focus. And in the process, we will advance things like AI planning and reasoning, and also solve things like, uh, you know, how agents should be stitched together in really interesting ways through something called agent-oriented programming.

[5:12](https://www.youtube.com/watch?v=xhQfRSueFZI&t=312s) And we're doing all of this in the context of enterprise workflows like RPA and, uh, document processing access, et cetera. So that's my time. Thank you so much. See us at our booth. [outro jingle]
