Agents in Orbit: Workflows, Not Autopilot — Eduardo Simon & Javier Casal, Flywire

Hello, everyone, and welcome. My name is Edu, and alongside Javi, we represent the AI Platform Experience team at Flywire. Our core mission is to build practical agentic infrastructure that empowers our software engineers while following our established engineering standards and practices. Today, we'll share how structured workflow automation delivers real enterprise value, ensuring control, safety, and reliability across our development process. Before we dive into the deep

technical details of our AI platform, let us give you some essential context about Flywire. Flywire is built to process the world's most complex payments across global industries like education, healthcare, travel, and business services. Today, we serve over five thousand clients across more than two hundred and forty countries around the world. Because we process high-value international transactions, regulatory compliance, privacy, and security are not

optional items. They are core requirements of our business. We operate under strict compliance frameworks like PCI DSS and SOC 2. This means that customer data, personal identification, and payment details must remain completely safe. We cannot simply take sensitive context and pass it to third-party services without strict governance and architectural boundaries. To give you an idea of our engineering organization, we have around four hundred

software engineers distributed globally. Across our teams, we maintain diverse technology stacks, including Java, Ruby, Go, Python, .NET, and PHP. Almost everything we build and run is deployed on Victoria, our internal platform as a service that operates on top of GitLab and AWS infrastructure. Victoria provides standardized build pipelines and consistent deployment practices. So how did our AI journey begin?

About six months ago, back in April, we formally established a dedicated AI platform experience team. Even before that, passionate engineers across Flywire were already experimenting with AI tools, creating skills, building MCP servers, and assembling an early code review bot. That organic activity proved that our engineers wanted AI automation. It also showed that we needed a dedicated team to provide infrastructure

for safe and reliable agentic solutions. Since then, we have moved fast with close to a thousand commits to our internal tool. But the industry does not stop. In the same six months, Claude Code alone published more than a hundred and sixty releases, almost one per day, and the major AI labs released more than twenty new models. Every week brings a new model and a new tool or a new way of working. Our

challenge is not only to build fast but to keep up.

This brings us to Orbit, our own toil killer. Orbit is an internal platform that eliminates daily engineering repetitive tasks by including LLMs directly into structured workflows. Instead of letting AI agents run freely in open-ended loops, Orbit pairs deterministic code with small LLM steps. We crystallize repetitive procedural logic into code, and we call a model only where reasoning is actually needed.

When we evaluated whether to build or buy, building in-house made complete sense for our environment. Frontier models made feature development really fast, which drastically lowers the cost of a custom platform. More importantly, as a financial enterprise handling PCI, PII, and PHI data, keeping execution inside our existing Victoria infrastructure ensures that sensitive data never leaves our controlled environment.

We also skip long vendor security reviews and extra compliance work. So what does a workflow actually look like? Every agent in Orbit is defined in a single YAML file that lives in a repository and goes through the same merge request review as any other code. Take our code review agent as an example. It starts with a trigger, a GitHub webhook that fires when an engineer mentions the review bot on a

merge request. From there, a sequence of deterministic activities fetches the merge request metadata, classifies its risk tier, loads prior findings, and clones the repository. Only then we call the model in an LLM step that can read the code but has no permission to modify it, and that must answer in a strict output schema. Finally, another deterministic step posts the findings back to the merge request. This is what we mean by workflows,

not autopilot. Code handles the predictable parts, and the model is called only where reasoning is actually needed. We choose to start with YAML files and a merge request review process on purpose. YAML is a format our engineers already know, and the review works the way they already work every day. It also keeps the code base AI-friendly. Coding agents can read, write, and review workflow definitions as easily as people can. However, not every use case

comes from an engineering team. For corporate use cases, we are considering an authoring tool in the Orbit UI, where people can build a workflow by dragging and dropping from a set of predefined activities without writing any YAML. A workflow can start in three ways. Engineers and other systems can call Orbit's internal API directly or using our internal CLI. Webhooks from GitLab, Jira, Sentry, and AWS start

a workflow from the moment something happens, such as a new merge request or a new production error. Scheduled jobs run workflows on a fixed cadence, like our daily translation syncs, feature flag cleanups, and knowledge base refreshes. Beyond these entry points, we are currently building an MCP server for Orbit so that the other AI services and tools we use across Flywire can start workflows directly. For example, an engineer will be able to ask Claude in Slack

through Claude Tag to run an Orbit workflow without ever leaving the conversation. We started small with clear high impact use cases. First, the daily engineering work. Automated code reviews on merge requests and our Victoria upgrades agent, which fixes Victorial interfin- findings automatically. Second, translations. One workflow now handles the full cycle. It fetches the new strings, translates them, updates the files, and notifies the teams with the

updated docs. Third, production errors. A Sentry webhook starts the Sentry investigator. It assesses the issue, posts the assessment in Slack, and shares the merge request with the fix in the same place. On the support side, ticket triage classifies new tickets and routes them to the right engineering area. AI readiness scorecards measures how ready a repository is for AI-assisted development, and a fixer acts on the

findings. Finally, the Jira implementer takes a ticket and turns it into a merge request. It works like a small software factory. And the biggest benefits that we found is that we can make changes in bulk across many repositories. That lets us do software engineering at scale. It is very convenient for mi- migrations, library updates, and deprecations. And we also use it to run Claude Doctor and to

launch our own AI readiness scorecard. Even with all this automation, the merge request is still our unit of work, and it is where a human stays in the loop. We operate in a regulated environment, so people still review and approve those merge requests. Let's take a deeper look on the technical architecture of Orbit. Originally, we began with a simple container setup on AWS ECS Fargate,

where the orchestration layer harness and the runtime environment lived inside the same container. As our needs grew, we evolved towards separating the orchestration layer, the agent runtime environment, and the session state into distinct components. This improves isolation, maintainability, and resource allocation. It also makes the system more resilient because an error in the front does not kill the sandbox. And it lets us scale each piece independently and adapt to changing requirements.

Fargate lets us scale tasks elastically so we can run a very large number of agents in parallel. It was also really easy to deploy Orbit because we already had Victoria, our PaaS that scales applications without any effort. But that scale needs guardrails so the platform is not misused. First, timeouts. Every agent stops after a configurable amount of time. Second, rate limits, which are aggressive by default. A broken API integration

could start spawning thousands of agents and make our bill much bigger. The limits keep that under control. Third, concurrency control. Our control plane uses locks to prevent duplicate executions of the same task. For example, if a code review is already running, we drop the duplicated request. Running agents inside Victoria comes with container limits. Victoria uses standard Fargate task definitions, so the containers are lightweight and tightly restricted. Our

baseline agent container includes the common language runtimes, but it intentionally leaves out package managers like Bundler, Maven, and Composer. So heavy agents, like our Jira implementer, cannot easily run native builds or unit tests inside the container. When we started Orbit in April, AWS Lambda microVMs did not exist yet. AWS released them in June, so our first version ran only on Fargate. These are managed microVMs that

work like Lambda functions, resumable processes, so they can pause and continue where they stopped. That fits the nature of agentic execution really well, so we now use them with a runtime chosen by project type. Each tech stack gets its own toolchain quickly in an isolated environment. This is not full sandboxing yet, and we are actively moving there. The goal is to support setups like Docker in Docker or multiple microVMs, so agents can run tests or more complex verifications on

any repository. Inside each LLM step, we did not build our own harness. Instead, we reused the Claude Agent SDK, the same harness that powers Claude Code. This gives us a mature, well-tested harness with tool usage, subagents, and skills out of the box, so we can focus on the workflow rather than on the plumbing. What we add on top is control. We restrict four different things for every step. The network the agent can reach, the

tools it can call, the skills it can load, and the MCP servers it can use, down to the specific tools allowed from each server. All of this is declared in the YAML file itself. A code review step, for example, can read and search the code but cannot edit files or push changes. And our Sentry investigator can query traces through our MCP server, but only with a short list of read-only tools. In practice, each LLM step runs with only the access it needs, and the access is visible and reviewable like any other

code.

Skills themselves come from a single place. At Flywire, we maintain a central repository of skills that is published internally as a Claude Code plugin marketplace. And Orbit connects to that marketplace at build time. This means the same skills works in two places. An engineer can run it in Claude Code on their laptop, and Orbit can run it inside an automated workflow. That shared foundation also shapes our recommendation to teams. Start with the

skill. Skill is cheap to write, easy to iterate on, and useful from day one. Once a team is sure that a task is worth automating, we move it into Orbit and turn it into a workflow. The predictable parts become deterministic code. The model is called only where it adds value, and each run becomes more reliable and less expensive than an open-ended session. Workflows also run in the background, so engineers do not have to sit with an open session and guide it step

by step. That means less context switching and more time for the work that needs them. Beyond runtime isolation, we also rely heavily on structured output to guarantee systemic reliability. Every LLM step in an Orbit workflow enforces strict JSON schemas for model responses. By constraining output formats, we ensure that subsequent execution branches, error handling, and state transitions are driven by deterministic control

flow rather than unpredictable string parsing. If an LLM response fails validation against the expected JSON schema, Orbit automatically triggers a targeted retry or fallback path before proceeding. Another crucial pattern we rely on is isolated component evaluation. Evaluating complex end-to-end agentic workflows in synthetic benchmark environments is often noisy, expensive, and difficult to reproduce consistently.

Instead of trying to evaluate entire workflows from start to finish, our evaluation harness isolates individual LLM steps. By injecting exact historical state and context directly into specific workflow nodes, we can evaluate model output quality, measure prompt adjustments, and catch performance regressions quickly and cost-effectively. So how do we know that an agent is good enough? Today, a new agent usually reaches production after a vibe check. The team tries it on a few real cases and

confirms that it behaves as expected. Once it runs in production, we trace every execution end to end with OpenTelemetry, including each activity, model call, and tool call. We then extract traces from real executions and add them to a golden dataset. For now, each golden dataset is versioned in the repository next to the code, and we run it on demand. Based on those results, we improve the prompt or change the model, and we run the dataset again to

compare. We do not have online evaluations yet, but we are working on them internally. Our goal is to run the whole evaluation loop inside Orbit from the UI or through the MCP server. First, you deploy your agent, then you monitor and evaluate its execution in production. Finally, you improve the agent based on those evaluations without the need to deploy it again. That is our roadmap for evaluations: one place to ship, measure, and improve every

workflow. The operational impact of this approach has been significant across our engineering organization. In the past thirty days alone, our automated code review agent completed over nine thousand seven hundred runs, averaging under three minutes per merge request. Beyond code review, our audit fixer opened a hundred and fifty-nine merge requests to fix vulnerable dependencies, and the AI readiness fixer improved around eighty repositories. The Victoria

linter fixer fixed four hundred and thirteen projects, and the merge conflict resolver ran nine hundred and twenty-one times. Across these workflows, we ran over eleven thousand executions with a ninety-nine percent success rate.

Reflecting on our journey, adopting a self-service model was one of the best decisions we've made. Standardized guardrails allow engineers across SRE, security, IT, and support to build custom workflow steps easily and safely. We also learned that cost control starts with model selection. Not every step needs the most capable model, so each LLM step in a workflow declares its own model. For example, a planning step and an implementation step can use different models.

Matching the model to the task keeps costs under control, and it is why a cost-aware model router is in our roadmap. On the other hand, managing data separation remains an ongoing responsibility. We maintain a strict distillation between engineering SDLC data, corporate operational data, and sensitive payment production data. Additionally, because demand for automation is high, maintaining focus and prioritizing high-impact work is essential.

Looking ahead, our roadmap focuses on expanding our platform features and increasing overall architecture flexibility. We are actively moving towards a complete sandboxing, so agents can handle complex builds, local environments, and test suites seamlessly across all project types. Moving forward, we are also introducing resumable sessions to handle complex multi-step tasks. We are also building more evaluation tools directly into Orbit, so teams can measure the quality of their workflows on the platform itself. In parallel,

we are refactoring our orchestration layer to be model-agnostic, enabling flexible routing across different model vendors, including an automated model router that selects the most cost-effective model for each task. That will help us control costs as we keep scaling the platform. We also, uh, want to make the platform harness-agnostic so that we are not tied to a single harness. To wrap up, here are three things we learned. First, start small. Pick a repetitive,

well-defined task and prove it works before you scale it. Second, use the model only where it adds value. Everything predictable stays as code, and that is what keeps the workflows reliable. And third, keep guardrails and humans in the loop. Owners, limits, and merge request reviews. That is how we move from AI experiments to workflows our engineers use every day.

Thank you all so much for your time and attention today. Feel free to reach out to us in case you have further questions or comments.