AI Engineer World's Fair 2026
Tribal Dungeons of Global Shipping: AI Agents at Global Scale — Dmitry Buykin, Maersk
Read the talk
Turning Shipping Knowledge into Executable Agent Procedures
Dmitry Buykin’s practitioner account of how explicit procedures, bounded execution, shared traces, and repeated corrections support agents in global shipping.
From a talk by Dmitry Buykin
At a glance
Ideas worth remembering
Agent SOPs must specify preconditions, decisions, identifiers, backend calls, validation, recovery, and evidence of success. Screenshot sequences leave that execution contract implicit.
Reliability depends on the refinement process: use shared traces to diagnose failures, turn corrections into executable changes, and replay real cases with production-affecting rights disabled.
Buykin reports over 100,000 corrections over nine months. Heat maps group tracked scenarios to prioritize expert and engineering effort; the correction count alone does not establish an accuracy rate.
Production harnesses should eliminate unsafe paths, with review and approval retained for critical work.
Successful repeated sequences become composite tools. The team uses distilled responses and tailored function calls to control execution quality, choosing not to use MCP for this system.
The expensive work begins when systems disagree
Dmitry Buykin opens with a production problem: turning messy operational knowledge into something an agent can execute safely. His setting is global shipping, where a workflow that looks singular on paper actually coordinates many parallel state machines. As long as those systems agree, the happy path works. When one drifts, the shipment becomes exception work.
The easy majority, he says, is already automated in many companies. The remaining long tail contains more exceptions than the existing systems were built to handle, making it disproportionately expensive. A process depends on multiple systems remaining coherent at once; if one step cannot complete, an expert must orchestrate work across incomplete systems. That is the operational challenge his agents have to address.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A screenshot sequence is not an executable procedure
Standard operating procedures, or SOPs, are supposed to capture the variations and pathways experts handle. Making them useful to agents involves stable intent detection, safe tool calls, integration with legacy backends, and evaluation with experts. Buykin calls the gap “tribal dungeons”: the organization possesses the knowledge, but has not represented it in a form an agent can safely execute.
His concrete example is a legacy SOP consisting of screenshots arranged in sequence. It records what a person sees and clicks. An agent procedure needs preconditions, decisions, identifiers, backend calls, validation, recovery, and evidence of successful execution. These elements make explicit when work may start, which branch to follow, what records and operations are involved, how to check the result, and what to do when execution fails. A record of clicks alone does not supply that execution contract.
Buykin assigns experts ownership of the what and agents ownership of the how. Most of the effort lies in translating and negotiating between them until the procedure reflects a shared understanding. An exception can then become a guardrail: knowledge that once required an expert’s intervention becomes an explicit constraint on subsequent execution.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The procedure corpus outweighs the runtime
The architecture has three parts: SOP memory organized as a corpus, an execution runtime, and feedback capture. Buykin places the greatest complexity in the refinement process surrounding the agent. Running a loop is only one part of the system; the organization also needs a way to retain process knowledge and revise it as execution exposes gaps.
A slide correction becomes an illustration of country variation: what appears to be the same thing can mean something different, and be described differently, in different countries. The corpus therefore represents company process memory adapted to local conditions. Buykin gives its proportion to the runtime as 20 to 1. He does not specify a measurement unit, so the comparison establishes the corpus’s relative importance without establishing a ratio of code, storage, or engineering hours.
At the reported production scale, the system runs over 200 instances, with latency ranging from a few minutes to as much as 10 minutes. Buykin attributes the delay mainly to dependencies on legacy systems. Concurrency and completion time are separate concerns here: many instances can operate at once while individual workflows still wait on the backends they must coordinate.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn expert feedback into executable changes
Expert time becomes the bottleneck, so the team uses triage tooling to cluster failures into actionable groups. The trace supplies shared evidence: an expert and an engineer can inspect the same case and agree on what happened. Clustering helps decide where to spend attention, while the individual trace supports a concrete diagnosis.
A correction counts only when it becomes an executable change. Agreement about what should have happened is not yet a production fix; the system’s behavior must change. The team checks those changes by replaying real examples with rights disabled to protect production systems, then examining whether behavior improved. This connects diagnosis, implementation, and verification in one refinement cycle.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Accuracy accumulates through correction
Buykin argues that vibe coding and spec-driven development reach limits in this project: further accuracy at this scale requires sustained engineering work. He frames that work as established software engineering practice rather than an exotic new discipline, and says AI coding agents need that engineering knowledge to help build reliable production systems.
Accuracy, he says, was earned one small correction at a time. He reports over 100,000 corrections during nine months of development. That number describes the scale of refinement; the supplied account does not define the counting unit or give an accuracy percentage, so it cannot establish a success rate by itself.
Heat maps turn thousands of traces into priorities. Each cell represents a group of tracked scenarios, allowing experts and engineers to focus on the same problems and identify where work would be most beneficial. Buykin associates changing one block with around one or two months of effort from the whole team, including engineers and AI agents. The effort estimate makes clear that a compact visual summary can represent substantial work.
“The agent failed” is the beginning of investigation. Each failure needs to map to a specific fix. This requirement keeps a broad failure label from becoming the final explanation..
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Production constraints must eliminate unsafe paths
Buykin draws a boundary between discovery and production. Discovery benefits from agent freedom; production needs a constrained harness that makes avoidable mistakes impossible. Asking an agent to be careful does not create that constraint. A preventive measure must remove the unsafe execution path.
One clearly stated example connects selecting the wrong workflow to classifier evaluation: the remedy should address the mechanism that chose the workflow. Specific failures require specific preventive measures. On critical paths, review and approval remain in the loop, with engineering focused on safe handoffs and a trustworthy trail of what happened.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make improvement reusable
Buykin presents the methodology as the central outcome of the project. Its five moves are to make work representable, make execution bounded, make every agent’s behavior observable, make correction cheap, and make improvement compound. Together they connect process definition to controlled execution and then to a systematic way of improving quality.
The compounding mechanism is concrete: learn which sequences work and fold them back into code as composite tools. Repeated successful sequences are aggregated into larger tools and reusable snippets that other agents can use. Operational learning therefore changes the software available to later executions, while the architecture adapts to the applications and people around it.
Buykin identifies this adaptive architecture as the lasting asset. He says reusable tools make it possible to roll out successful sequences across hundreds of countries in one go. This is a claim about the potential reach of reuse, not a documented rollout count.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Why the team chooses tailored function calls
In his closing technical point, Buykin says the team does not use MCP for this system. He describes the connected systems as bloated and says the team needs to distill their responses and tune tools to the agents through function calling. The decision centers on controlling what the agent receives and how its tools support assigned tasks.
This is a system-specific integration choice. Buykin’s stated benefit is control over software quality and correct task processing; he provides no comparative benchmark establishing a general limitation of MCP.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Read the complete timestamped transcript
- 0:01
[music]
- 0:13
Hello everyone. Um, this is a
- 0:15
practitioner report uh from real
- 0:17
production work. So, let's get into it.
- 0:21
Um, I'll skip the generic uh yet another
- 0:24
loop agent intro. This is about the hard
- 0:26
part most agent demos skip.
- 0:30
and about turning messy operational
- 0:32
knowledge into something an agent can
- 0:35
execute safely.
- 0:37
This comes from real work uh in my
- 0:39
company I'm working for supporting
- 0:42
global shipping operations and grounded
- 0:44
in production.
- 0:48
On paper it's uh one workflow usually
- 0:50
but uh in reality every shipment is an
- 0:53
orchestration of many parallel state
- 0:55
machines. While they agree the happy
- 0:59
paths work the moment one drifts you get
- 1:02
exception work.
- 1:06
The easy majority is already automated
- 1:09
in many companies. What's left is the
- 1:12
long tail and more exceptions than
- 1:15
system built uh to handle them. That
- 1:18
tail is uh the expensive part.
- 1:23
And then there's my favorite category.
- 1:25
And it comes with a special uh plate
- 1:29
here. See for EI builder dreams and
- 1:33
their laptops. This what you can find
- 1:36
outside of AI bubble in San Francisco.
- 1:42
The signal process uh depends on many
- 1:44
systems being coherent at once. If any
- 1:47
step uh can't complete the happy path
- 1:50
breaks and then it takes expert uh
- 1:53
archist expert orchestration across uh
- 1:56
multiple incomplete systems.
- 2:00
All these uh variations um path pathways
- 2:04
should be captured in SOPs. SOPs is a
- 2:07
standard operating procedure common and
- 2:09
regulated industries. So an expert and
- 2:11
the model read them uh the same way.
- 2:16
That gap is the hard part. Stable intent
- 2:20
detection tool calls you can guarantee
- 2:23
are safe integrating with legacy back
- 2:26
ends and results evaluated with experts.
- 2:32
Uh I call this uh tribal dungeons. Uh
- 2:35
the knowledge exists but not in a form
- 2:38
uh agent can execute and you can safely
- 2:41
run a process. You can't safely run a
- 2:44
process. The organization cannot
- 2:45
represent
- 2:49
standard legacy SOPs [clears throat]
- 2:51
bunch of bunch of screenshots organized
- 2:55
in sequence and but screenshots not uh a
- 2:58
process. A legacy SOPs explain what a
- 3:01
person sees and clicks. And an agent SOP
- 3:05
needs a more complex uh setup,
- 3:08
preconditions, uh decisions,
- 3:10
identifiers, back end calls, validation,
- 3:13
recovery, and evidence of uh successful
- 3:16
execution.
- 3:21
Experts own the what, agents own the
- 3:24
how. And exception becomes a guardrail.
- 3:28
Most of the effort is the translation
- 3:30
and negotiation between them to align on
- 3:34
common sense.
- 3:38
Three parts here um in this architecture
- 3:41
it's SOP memory uh organized as SOP
- 3:44
corpus
- 3:46
execution runtime and theme feedback
- 3:48
capture. The agent loop is not the
- 3:51
system. The refining loop around the
- 3:53
agent is the system
- 3:56
and it's the most complex part. Oh,
- 3:58
sorry SAP is okay. It's this slide for
- 4:02
UK. This is correct one. So and it's
- 4:06
good illustration why the the same thing
- 4:09
is means different and uh describing
- 4:12
differently in different countries and
- 4:15
it's creating a lot of variations
- 4:16
between each country
- 4:19
and that corpus is a asset the company
- 4:22
company's process memory uh modified and
- 4:25
aligned with every country um conditions
- 4:30
and far bigger than than than runtime
- 4:32
you could see the proportion 20 to1
- 4:36
So and this is concurrently operating
- 4:38
system
- 4:41
and this is the scale we run in
- 4:43
production today
- 4:46
over 200 instances and spikes and
- 4:50
latencies deviates from few minutes to
- 4:53
up to 10 minutes.
- 4:56
Um and mainly yeah the mainly main
- 5:00
reason for it that u we depending on
- 5:02
many legacy system which is uh so cannot
- 5:05
be faster than agent loop itself.
- 5:11
Expert time is the bottleneck. So the
- 5:15
theme bench uh does the triage for us.
- 5:19
It clusters the failures and hands back
- 5:22
something you can act on. Not just look
- 5:24
at look at it.
- 5:28
The trace is the shared evidence that
- 5:31
lets an expert and an engineer review
- 5:34
the same case and agree on what
- 5:36
happened.
- 5:39
A correction only counts when it becomes
- 5:41
an executable change. And that's the
- 5:45
line between an opinion and a production
- 5:47
fix.
- 5:52
And and this is where quality comes
- 5:55
from. not from vibes uh not from a
- 5:59
bigger model from replaying real
- 6:01
examples with u disabled rights to uh
- 6:06
protect the production systems and
- 6:08
checking whether behavior improved.
- 6:13
You can see here on the
- 6:16
uh cognitive proportion u or this effort
- 6:19
ratio uh between each activity in our
- 6:21
project. So usually uh pipe coding ends
- 6:25
here.
- 6:27
Here there ends um
- 6:30
specdriven development because it cannot
- 6:33
uh grow improve accuracy more than this
- 6:37
stage on this scale. And this is uh
- 6:40
where the real work starts. Nothing
- 6:44
exotic. It's engineering common
- 6:46
engineering sense applied at scale.
- 6:50
So if uh you don't know all this uh
- 6:53
terminology which developed over lastuh
- 6:57
30 years in software development
- 6:58
argument to check because this is what
- 7:01
every AI agent uh AI coding agent should
- 7:03
know uh to help you develop reliable
- 7:07
production systems
- 7:12
and accuracy it's uh wasn't designed uh
- 7:15
in one diagram up front it was earned
- 7:19
one small correction at the time at the
- 7:21
scale you see here. So we have over
- 7:24
100,000
- 7:26
corrections over last 9 months in the
- 7:29
system when we developing it
- 7:33
[clears throat]
- 7:35
and this um heat maps uh turned
- 7:39
thousands of traces into priorities. is
- 7:41
how we keep experts and engineers uh
- 7:44
looking at the same problems and
- 7:46
prioritize where the the most beneficial
- 7:49
work for them. Every cell is a group of
- 7:53
tracked scenarios we have and uh usually
- 7:57
to turn one block in red it's around one
- 8:02
two months of force for the whole team
- 8:06
whole team of engineers and also AI
- 8:09
agents
- 8:11
um the agent failed is uh where the
- 8:13
investigation starts not where it ends
- 8:16
each failure maps to a specific uh fix
- 8:22
discovery needs agent freedom and
- 8:25
production needs a cage. Uh the harness
- 8:28
isn't there to give the agent more room.
- 8:30
It's there to make the dumb mistakes
- 8:32
impossible.
- 8:36
So on this scale please be careful is
- 8:38
not a guard guard. Uh if we have wrong
- 8:41
workflow then classifier eval. If it's
- 8:44
wrong right then right gate. If it's
- 8:47
wrong assumption then it's a mere view.
- 8:49
A preventive measure eliminates the
- 8:51
unsafe path
- 8:55
on critical paths. U review and approval
- 8:58
stay in the loop. The engine engineering
- 9:00
focus is uh to build safe hands offs and
- 9:03
a trail you can trust.
- 9:08
The real outcome
- 9:11
uh wasn't the agent in the system. It
- 9:13
was the [clears throat] methodology we
- 9:15
built around it. If you want the
- 9:18
blueprint, then it's uh these five
- 9:20
moves. Make work representable. Make exe
- 9:24
execution bounded. Make behavior
- 9:26
observable for every agent and make
- 9:29
correction cheap. And last thing is make
- 9:32
improvement compound. So gradually
- 9:35
systematically improve the quality of
- 9:37
the system.
- 9:41
AI native um operation is more than
- 9:45
agents in workflow. It's a system that
- 9:48
learns from what works and fold folds it
- 9:51
back into code as new composite tools
- 9:54
adapting to the applications and the
- 9:56
people around it. The best AI models um
- 10:00
oriented intelligence for us. The
- 10:03
adaptive architecture we built is the
- 10:05
asset, the final asset
- 10:10
and
- 10:12
we aggregating all um repeatable
- 10:16
sequences of steps successful scenarios
- 10:18
and uh merging them into bigger tools
- 10:22
which uh combine the disproven scenarios
- 10:25
into the reusable snippets by other
- 10:27
agents. So and then um it's possible to
- 10:31
roll out them not only for one country
- 10:33
but for hundreds country in one go.
- 10:38
So this is um um all for the talk and
- 10:43
little time for questions and I'll be
- 10:44
around afterwards. And the final
- 10:47
reminder you know if you you know if you
- 10:50
are AI builder if you emotionally
- 10:54
attached to tools not MCPS
- 10:57
we're not using MCPS because uh for us
- 10:59
it's uh always not the best choice. So
- 11:02
because all all systems usually really
- 11:05
bloated and we have to distill responses
- 11:08
and uh tune the tools through function
- 11:12
calling uh to our agents then we can
- 11:15
control quality of um our software and
- 11:19
ensure that uh it's correctly
- 11:22
processing assigned tasks.
- 11:27
Thank you. Any questions?
- 11:33
Okay, then um thanks for your attent u
- 11:37
attention. Then I will be around so you
- 11:39
can ask me questions if you want.
- 11:44
[applause]
- 12:00
>> [music]