# No, That's Not a Software Factory — Ryan Cooke, WorkOS Ryan Cooke, WorkOS AI Engineer World's Fair 2026 [Talk page](https://ai.engineer/talks/HvboD89DyQ8-no-thats-not-software-factory) · [Video](https://www.youtube.com/watch?v=HvboD89DyQ8) ## No, That's Not a Software Factory WorkOS’s first sandbox could produce pull requests, but Ryan Cooke found little difference from engineers using coding agents locally. TARS and Horizon extend automation into planning, review, project coordination and context gathering—the work that turns code into a product. ### A pull request is an output. What did it change? [0:12](https://www.youtube.com/watch?v=HvboD89DyQ8&t=12s) The familiar software-factory recipe is straightforward: put a repository in a sandbox, add a coding agent, give it a prompt, and merge the resulting pull request. Ryan Cooke, an engineer at WorkOS, opens with that recipe because it leaves an important question unanswered: does the organization actually deliver more useful software? ![Source frame: A pull request is an output. What did it change?](https://ai.engineer/api/talk-essay-media/a8e107432559d19cc1f430169f1216ba0498324f176de14515c409d6bfbaf62f) Source frame: A pull request is an output. What did it change? Counts of pull requests, percentages attributed to AI, and quantities of generated code describe production activity. They do not establish that the activity advances a customer outcome. More pull requests can make an automation system look successful while leaving the harder question—whether features reach customers sooner—unresolved. That distinction matters when building the factory itself consumes engineering time and potentially a dedicated team. WorkOS’s ambition is to give each engineer something like a small engineering team. The consequence should be observable in delivery: more features shipped, with complex work completed faster. Cooke’s account offers qualitative experience and proposed success measures rather than numerical delivery or reliability results; it explains the design and its intended benefits without establishing their magnitude. ### The sandbox worked; the surrounding process needed work [3:11](https://www.youtube.com/watch?v=HvboD89DyQ8&t=191s) WorkOS began with a sandbox built on Cloudflare, a model router, and prompts. The disappointing result was how ordinary it felt. Cooke found the outcomes pretty indistinguishable from engineers driving coding agents on their laptops. Moving execution into a sandbox had not, by itself, removed enough of the surrounding work to change product delivery. ![Source frame: The sandbox worked; the surrounding process needed work](https://ai.engineer/api/talk-essay-media/4a7c3908d3e21bb50ae6bd085a7f1ec8da27e101b9d85f30863612a611e55c52) Source frame: The sandbox worked; the surrounding process needed work The next design question was how to put WorkOS’s engineering processes inside the factory. Engineers do more than write code: they organize projects and carry work through the steps needed to produce a product. Automating those steps required the system to participate in the tools where that work already happened. That led to two parts with different responsibilities: - **TARS:** The interface to the coding agent lives in Slack, Linear and GitHub. It also subscribes to webhooks so it can track project progress alongside its code work. - **Horizon:** The infrastructure orchestration layer sits in front of an internal MCP gateway. It provides the execution infrastructure, while the gateway connects the agent to organizational tools and context. ### Webhooks turn ticket completion into the next action [5:07](https://www.youtube.com/watch?v=HvboD89DyQ8&t=307s) A dependency between Linear tickets gives the factory a plan it can follow. One ticket blocks another; completing the first changes which work can proceed. Because TARS receives ticket-completion webhooks, it can pick up the next ticket automatically. Planning therefore supplies both the units of work and their execution order, reducing the need for a person to issue a fresh prompt at every handoff. ![Source frame: Webhooks turn ticket completion into the next action](https://ai.engineer/api/talk-essay-media/f1895364fe8e160203cd80984a53b1a263da074952f7be1e6219520cf5d2de02) Source frame: Webhooks turn ticket completion into the next action The plan can also change as implementation reveals gaps. Between completed tickets, an engineer can ask TARS to reevaluate the Linear project and identify missing work. This is a separate operation from advancing to the next ticket: advancement follows the existing plan, while reevaluation checks whether the plan still covers the project’s goal. What connects a finished ticket to continued execution, and where can the plan be reconsidered? The diagram separates those paths. The completion event drives the next handoff; an intervening request can send TARS back to the project plan to look for gaps. The useful autonomy comes from reacting to project state, rather than treating each coding session as an isolated task. Execution follows project events; reevaluation revisits the plan Ticket completion can trigger the next unit of work. A requested reevaluation checks for missing tickets as implementation exposes gaps. - Linear project plan — Tickets describe smaller units of work and their dependencies. - Ticket completed — Linear emits a completion event received by TARS. - TARS picks up the next ticket — Execution continues along the planned work. - Requested project reevaluation — TARS checks whether the project is missing work discovered during implementation. - Linear project plan → TARS picks up the next ticket (Dependencies guide execution) - Ticket completed → TARS picks up the next ticket (Completion webhook) - Ticket completed → Requested project reevaluation (Engineer can request a check between steps) - Requested project reevaluation → Linear project plan (Identify gaps and keep the plan fresh) ### From a Slack brief to a reviewed Hilltop [6:31](https://www.youtube.com/watch?v=HvboD89DyQ8&t=391s) At WorkOS, engineers carry many product responsibilities; Cooke says teams do not have product managers. Their planning ritual is a **Hilltop** document, a product requirements document that gathers the project’s purpose, customer needs, competitive analysis, early designs and major milestones. Consolidating those inputs gives an agent a coherent resource from which to break the project into units of work. ![Source frame: From a Slack brief to a reviewed Hilltop](https://ai.engineer/api/talk-essay-media/cabafa4c472a99e84a965b5bbe0082e1266b846fed5c8dd0e17bf3b029747a5e) Source frame: From a Slack brief to a reviewed Hilltop The PM agent automates the first pass through that ritual. A brief specification becomes a draft Hilltop; the agent adds context, reads human reviews, and breaks implementation into tickets. Humans can comment on generated tickets and refine the work throughout the process. Approval is also an event the factory understands: once the Hilltop has been reviewed and approved through Linear, the system can begin the project automatically. The concrete example is a new API for Vaults, a product Cooke leads. He starts the project with a Slack command and a few sentences describing the goal. TARS creates the Linear project, a draft Notion document, and first passes at decision logs and open questions. The observable change is from a short request to an organized set of project resources that the lead engineer can immediately edit. Cooke then adds specifications and fills in poorly defined areas using his knowledge of the project. The example is still awaiting its Hilltop review: completing that review ticket will let TARS detect the change and move to the next implementation stage. The review is therefore doing two jobs. It improves the product definition, and its recorded completion tells the automation when work may advance. The benefit is partly the familiar “blank page problem.” Cooke reports that teams are doing more project-brief and Hilltop work because the agent seeds the information. The drafts can grossly overestimate a project’s scope, requiring engineers to cut substantial portions. The tradeoff is deliberate: reviewing and trimming an existing draft can be easier than creating every project primitive from scratch. Work then continues through Slack and Linear toward GitHub pull requests. ### The MCP gateway teaches agents where to look [11:03](https://www.youtube.com/watch?v=HvboD89DyQ8&t=663s) The resulting project documents are useful beyond WorkOS’s own coding agent. Engineers can point Devin at the same tickets and documentation, or let a local coding harness retrieve the material through MCP. Security colleagues can also join the project channels and weigh in. The planning work remains useful even when a different agent performs implementation. ![Source frame: The MCP gateway teaches agents where to look](https://ai.engineer/api/talk-essay-media/1e87cbb915e5fcf4a560ece4f21699143b25ccdad834fbde2c49601f778fde9f) Source frame: The MCP gateway teaches agents where to look WorkOS calls its internal MCP gateway a “context engine.” Connecting tools is only part of its job. It also builds system prompts and context that explain when to use a tool, how to navigate it, and how WorkOS organizes information inside it. Those descriptions give an agent a starting map rather than merely a list of available operations. Snowflake makes the mechanism concrete. It is WorkOS’s main data lake, with semantic tables describing product utilization and customer conversations. The MCP tool description can list those tables, explain their contents, and direct the agent to query the appropriate tables for a particular type of question. The query still has to be written; the gateway supplies the organizational knowledge needed to choose where to write it. How does one gateway serve both software implementation and internal analysis? The diagram shows the shared connection: multiple agent-facing workflows use the same tool access and navigation guidance. WorkOS opened the MCP server to Slack queries for data and customer analysis, then began building other internal tools on it. That reuse is why Cooke recommends investing early in a gateway that explains both the tools and the company’s information organization. One context engine, several uses The gateway combines access to internal systems with instructions about their contents and use. Coding and analysis workflows can reuse that knowledge. - Coding workflows — Horizon and local coding harnesses use organizational tools and documentation. - Slack analysis — People query data and customer information through Slack. - Internal MCP gateway — Tool connections, system prompts, and guidance on where information lives. - Snowflake — Semantic tables for product utilization and customer conversations. - Project documentation — Tickets and supporting documents provide implementation context. - Coding workflows → Internal MCP gateway (Retrieve context and use tools) - Slack analysis → Internal MCP gateway (Query through Slack) - Internal MCP gateway → Snowflake (Describe tables and guide queries) - Internal MCP gateway → Project documentation (Expose documentation through MCP) ### Extend the process to triage, then improve its infrastructure [14:06](https://www.youtube.com/watch?v=HvboD89DyQ8&t=846s) The same event-driven approach extends beyond planned features: - **Bug requests:** TARS listens through webhooks to requests arriving in Slack, takes a first pass at triage, and can implement fixes and open pull requests. - **Customer support:** Shared customer Slack channels provide support requests. TARS can inspect the code to help identify where a customer may be encountering a product problem. ![Source frame: Extend the process to triage, then improve its infrastructure](https://ai.engineer/api/talk-essay-media/93e277958a419825a41c5d57260552cb25ec83ebe5be50306d13fa3cc8ef42c3) Source frame: Extend the process to triage, then improve its infrastructure WorkOS is also using TARS to build its own sandbox infrastructure. The motivation for moving away from some sandbox services is deeper control over session information and the ability to move workloads among parts of its infrastructure. Owning that layer becomes relevant when the factory needs to inspect how work happens, rather than simply launch an agent and collect its output. The next planned component is a company-wide memory layer: continually useful context about what people are working on, their teams, the products they own, and how WorkOS does its work. The intended consumers include both the factory and other AI tools. This is a proposed expansion of shared organizational context, with no completed memory implementation described in the talk. ### Measure delivery, reliability and the choice to use it [15:57](https://www.youtube.com/watch?v=HvboD89DyQ8&t=957s) The closing measurement discussion returns to customer impact. Shipping faster is valuable only if the products remain dependable, and a tool that engineers voluntarily choose offers a different signal from one that merely generates activity. Cooke identifies three areas to watch: - **Delivery and customer impact:** Does the factory help useful work reach customers? - **Reliability:** Track defect rate and time to recovery to watch for instability introduced by the new workflow. - **Voluntary adoption:** Look for engineers choosing TARS and cloud sandboxes over their local harnesses. Cooke treats this as an anecdotal indication that the system helps them work. ![Source frame: Measure delivery, reliability and the choice to use it](https://ai.engineer/api/talk-essay-media/76b52b7db9f793755ec3c60bad3e584f9c86211e055c8c9a9a5c716b53b98ab9) Source frame: Measure delivery, reliability and the choice to use it Infrastructure visibility is meant to feed a further improvement loop. Agents could examine sessions to find gaps in how engineers use the factory, identify mistakes that call for a new skill, and recognize skills that have become obsolete as the code changes. A skill written six months earlier may no longer fit the current code. Better code generation and better engineering practice are goals of this loop, rather than demonstrated self-improvement results. ### The process can advance; authorization remains unresolved [18:02](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1082s) “Sandboxes are great.” Opening a pull request is useful. WorkOS’s larger aim is to encode the engineering practices that carry a project from intention through planning and execution. The Vaults example makes that ambition concrete: a Slack brief creates resources, an engineer refines them, a review ticket records approval, and TARS can react to that event to continue the project. ![Source frame: The process can advance; authorization remains unresolved](https://ai.engineer/api/talk-essay-media/1db676efbacfe5163c03204244be84264c9e0f239f69926da008a0249f7b52e6) Source frame: The process can advance; authorization remains unresolved Cooke ends with an unresolved question: authorization. WorkOS has not yet figured out how it should handle authorization for this factory, and he asks other builders how they approach it. That limits what the workflow demonstrates. Approval events can coordinate when work advances, but the talk does not supply a settled model for who may authorize the agent’s actions across its connected systems. ## Key takeaways - A sandbox can generate code without improving product delivery. WorkOS’s initial experience pushed it toward automating planning and coordination alongside implementation. - Ticket dependencies and completion webhooks let TARS continue a project without a fresh prompt at every handoff; requested reevaluation helps expose missing work. - Draft Hilltop documents reduce setup work and give engineers something to refine, but scope control remains an important human contribution. - An internal MCP gateway becomes more useful when it explains where information lives and when to query it. WorkOS reuses that guidance for coding, data analysis and customer analysis. - Evaluate customer delivery alongside defect rate, recovery time and voluntary adoption. Company-wide memory and session-driven improvement are future directions; authorization remains unresolved. ## Slides ### No, That's Not a Software Factory Ryan Cooke of WorkOS introduces his talk on software factories at AI Engineer World's Fair 2026. [0:21](https://www.youtube.com/watch?v=HvboD89DyQ8&t=21s) [WorkOS title slide for Ryan Cooke's talk, No, That's Not a Software Factory.](https://assets.ai.engineer/sites/aiecode2025/sha256/2a12534ccf804423d836055042bd3f4f28cdbdbfd67ced54155cd4494e16cfc3.webp) No, That's Not a Software Factory WorkOS SOFTWARE FACTORIES · AIEW 2026 Ryan Cooke · Product Engineering AI Engineer World's Fair 2026 Interface text: 1 / 16 ### The Standard Playbook Cooke describes a setup where code and an AI agent share a sandbox: a prompt generates a PR, which is then merged. Ramp, Spotify, Stripe and WorkOS have each built a version. [0:25](https://www.youtube.com/watch?v=HvboD89DyQ8&t=25s) [A workflow from sandbox, model and repo to pull request and merge, with examples from several companies.](https://assets.ai.engineer/sites/aiecode2025/sha256/eb11d8549ed9378419cc22a427d2661386bc3fdd0b7f7b478b990b36e042dde2.webp) The Standard Playbook Sandbox → Model → Repo → PR → Merge Ramp built Inspect · Spotify built Honk · Stripe built Minions · We built Horizon You've seen this. You may have built this. So have we. WorkOS 2 / 16 ### The Industry Is Measuring Output Company PR counts and AI shares measure production. Cooke argues these output metrics can obscure whether the systems actually drive outcomes. [1:24](https://www.youtube.com/watch?v=HvboD89DyQ8&t=84s) [Reported AI pull request metrics for Ramp, Spotify, Stripe, Abnormal and Coinbase, labeled as production rates rather than value delivered.](https://assets.ai.engineer/sites/aiecode2025/sha256/6c5eaea42e607509d290dcfd02355bddb8ac64a0019998d1807abae2ea2e2bc8.webp) The Industry Is Measuring Output >50% — Ramp of merged PRs from AI 1,500+ — Spotify AI-generated PRs merged 1K+ — Stripe merged PRs per week 13% — Abnormal of merged PRs from AI 5% — Coinbase of merged PRs from AI These are production rates. Not value-delivered rates. WorkOS Interface text: 3 / 16 ### Output vs. Outcomes Cooke argues that PR counts and generated code can obscure whether a software factory creates value. He instead tracks outcomes such as feature delivery time, defect rate at ship, and time to resolve issues. [2:10](https://www.youtube.com/watch?v=HvboD89DyQ8&t=130s) [Output metrics such as pull requests and lines of code contrasted with outcome metrics such as delivery time and defect rate.](https://assets.ai.engineer/sites/aiecode2025/sha256/a8499616a6894ad9cad9d2bd82478debc3cf55ffd418e1961e581187b16e970b.webp) Output vs. Outcomes The industry is optimizing for the wrong variable Output metrics — PRs merged / week — % of code written by AI — Lines of code generated — PR velocity vs. Outcome metrics + Feature delivery time + Defect rate at ship + Engineer leverage + Time to resolve issues “More code, faster, is not the same as more value, faster.” WorkOS Interface text: 5 / 16 ### Sandbox + Model + Repo WorkOS found its initial sandbox and model setup produced outcomes indistinguishable from engineers using coding tools on their laptops. The team began considering how to embed engineering processes in the factory. [3:11](https://www.youtube.com/watch?v=HvboD89DyQ8&t=191s) [Sandbox, model and repo are described as necessary but incomplete.](https://assets.ai.engineer/sites/aiecode2025/sha256/e04a7edf079f88f4a67f365ced9f22c47389c953fb708a54eb8ac910501dee2d.webp) Sandbox + Model + Repo = Necessary. Incomplete. Every serious team has checked this box. So what's left? WorkOS Interface text: 5 / 16 ### What We Built: Horizon / TARS TARS embeds the coding agent in tools such as Slack, Linear and GitHub. Horizon orchestrates the infrastructure, including warm-pooled sandboxes and an event loop from Linear to the repo and back. [3:46](https://www.youtube.com/watch?v=HvboD89DyQ8&t=226s) [TARS is presented as the user-facing product and Horizon as the infrastructure supporting it.](https://assets.ai.engineer/sites/aiecode2025/sha256/669bc5359bd0342ad2f3fab34992fc470d6c0cdaf73865ff8be568c06f4a4acd.webp) What We Built Horizon / TARS TARS — The Product What users interact with. Slack, Linear, GitHub, web UI. Receives instructions, opens PRs, reports status. Horizon — The Infrastructure Cloudflare Containers, webhook-driven orchestration, OpenCode runtime. Warm-pooled sandboxes. Event loop from Linear to repo and back. WorkOS 6 / 16 ### Infrastructure is only half the story TARS uses ticket-completion webhooks to pick up the next dependent Linear ticket. It can also reevaluate the project for missing tickets as work progresses. [5:04](https://www.youtube.com/watch?v=HvboD89DyQ8&t=304s) [Slide explaining how webhooks, shared MCP surfaces, and sandboxes connect agents to existing processes.](https://assets.ai.engineer/sites/aiecode2025/sha256/383c107fe3443f9db1b6e57864d960849f001e8ce2ec032d26b53e55cd4733f0.webp) Infrastructure is only half the story. Webhooks, shared MCP surfaces, and reproducible sandboxes plug agents into the systems and processes we already run. We built an agent that participates in our engineering process WorkOS Interface text: 7 / 16 ### The Blueprint WorkOS gives agents its Hilltop documents to break projects into units of work. Humans weigh trade-offs, while dependency chains highlight blockers and branches. [6:29](https://www.youtube.com/watch?v=HvboD89DyQ8&t=389s) [Slide listing Hilltop documents, small teams, dependency chains, and a review-first loop.](https://assets.ai.engineer/sites/aiecode2025/sha256/41bb934ddc70846c14a98e8ed68b32d50b440490575b4a7e3d37432888323f86.webp) The Blueprint WorkOS engineering culture, made executable Hilltop Documents Write the narrative before writing code. Small Teams Humans weigh trade-offs Dependency Chains Plans highlight blockers and branches Review-First Loop Confident AI reviews are easily merged WorkOS 8 / 16 ### The PM Agent The PM agent turns a Hilltop document into dependent issues. Humans add context and review pull requests; after a merge, Horizon unblocks the next issue. [8:08](https://www.youtube.com/watch?v=HvboD89DyQ8&t=488s) [Five steps from planning and human review through implementation, pull request review, and automatic unblocking.](https://assets.ai.engineer/sites/aiecode2025/sha256/87da7782422eec9e5c89d9857c6f1b2227f00036b5edb6ef9b387a5a00e8dbd6.webp) The PM Agent Encoding the planning process 01 PM agent reads the Hilltop doc Divides project into issues with explicit dependency chains 02 Human reviews + adds context Issues reflect real decisions; context injected for implementation 03 Implementation agent picks up issues Works in order as issues unblock 04 PR opened, human reviews Quality gate: correct diff, tests pass, lint clean 05 After merge: Horizon auto-unblocks Dependency graph evaluated; next issue starts immediately WorkOS Interface text: 9 / 16 ### Project issue list The project groups work by status, with Hilltop Review still in the backlog. Cooke says Tars will move to the next project stage after the review ticket is completed. [9:54](https://www.youtube.com/watch?v=HvboD89DyQ8&t=594s) [A project issue list grouped by progress status.](https://assets.ai.engineer/sites/aiecode2025/sha256/12697d952a61e87863b1f0974d2fc7d9fb5721300402fcf05f229a5c308f5303.webp) Vault Internal KEK API In Progress 1 PVIT-364 Project Channel — May 8 Backlog 13 PVIT-352 Close the loop with customers — May 8 PVIT-353 Product Demo — May 8 PVIT-354 Changelog & Blog — May 8 PVIT-358 Update SDKs — May 8 PVIT-361 Product Tier — May 8 PVIT-363 Write the hilltop doc — May 8 PVIT-355 Enablement — May 8 PVIT-359 Infra self assessment — May 8 PVIT-356 Testing — May 8 PVIT-357 Observability — May 8 PVIT-362 Security self assessment — May 8 PVIT-351 Announce in #all-shipped — May 8 PVIT-360 Hilltop Review — May 8 Done 2 PVIT-366 Write the Brief — May 8 PVIT-365 Brief review — May 8 Interface text: WorkOS Pulse 99+ Inbox 77 My issues Reviews 5 Drafts 3 Workspace Initiatives Projects Views More Favorites Vault All issues Radar Your teams Security PVIT Issues Projects Views Radar Triage 1 Issues Projects Views Infrastructure Foundation Triage Issues Overview Activity Issues Search PVIT for KMS keys ticket Ask Linear https://linear.app/workos/issue/PVIT-360/hilltop-review ### Review issue The backlog ticket calls for stakeholder review, a recording, and a security review when the self-assessment is medium or higher. Cooke says Tars will continue implementation once the ticket is complete. [9:59](https://www.youtube.com/watch?v=HvboD89DyQ8&t=599s) [An issue detail page with review instructions and related work.](https://assets.ai.engineer/sites/aiecode2025/sha256/86883719af54e67afb0f3d905df1846ede4861e5397e37a49c0ade8b96f96638.webp) Hilltop Review • Present with relevant stakeholders. • Be sure to record and share. • If security self-assessment was medium or higher then ensure there is a Security Review • Post to Hilltop Reviews Log - this will auto update the #epd-hilltop-updates Add sub-issues Activity service created the issue · 8w ago ryancooke changed team from Security to PVIT · 8w ago ryancooke moved from Triage to Backlog · 8w ago Properties Backlog Set priority Assign Labels Add label Project Vault Internal KEK API Shaping & Hilltop Suggestions SEC Security Relations Related Write the hilltop doc Security self assessment Interface text: Vault Internal KEK API › PVIT-360 Hilltop Review 14 / 16 Subscribe Leave a comment... WorkOS Pulse 99+ Inbox 77 My issues Reviews 5 Drafts 3 Workspace Initiatives Projects Views More Favorites Vault All issues Radar Your teams Security PVIT Issues Projects Views Radar Triage 1 Issues Projects Views Infrastructure Foundation Triage Issues Projects Search PVIT for KMS keys ticket Ask Linear ### The Context Engine WorkOS’s MCP gateway connects internal systems and guides agents through their conventions. Its tool descriptions, for example, explain which Snowflake tables to query. [11:57](https://www.youtube.com/watch?v=HvboD89DyQ8&t=717s) [A slide describing standard tool shapes, built-in paths, and conventions for an internal MCP server.](https://assets.ai.engineer/sites/aiecode2025/sha256/26f49e0c6455b0fd0e836abbdb0ebdfc4d951f7bbafb365852afa74d7a81e838.webp) The Context Engine The MCP server isn't data plumbing. It's culture encoding. Standard tool shapes Consistent naming, opinionated schemas — not just data sources but how WorkOS engineers navigate this codebase Paved paths built in Datadog logs, Sentry errors, Slack context — plus the patterns for how to use them. AGENTS.md and CLAUDE.md as first-class artifacts Convention over exploration "Agents can figure it out with minimal context, but it's slow. They spend tokens rediscovering conventions. We keep tightening that loop." Interface text: WorkOS 10 / 16 ### Infrastructure Alone Doesn't Produce This TARS opens PRs for visual bugs reported in Slack and triages customer messages before a human looks. WorkOS also uses a PM agent to build features and is rebuilding its sandbox runtime with both agents. [14:03](https://www.youtube.com/watch?v=HvboD89DyQ8&t=843s) [Four examples: Slack bug fixes, customer triage, shipped features, and rebuilding the factory's infrastructure.](https://assets.ai.engineer/sites/aiecode2025/sha256/63d1db80f19103a0452db9ca28190035868efe8f422989a2aa894eae62e4520a.webp) Infrastructure alone doesn't produce this. Encoding how we build software did. Bug fixes from Slack TARS monitors #ui-nits — when someone reports a visual bug, it opens a PR automatically. No ticket, no assignment. Customer triage at scale Incoming messages in customer Slack channels get a first-pass triage through TARS before a human looks. Major features shipped Released significant product features using the PM agent to bootstrap engineering — Hilltop in, implementation running. The factory rebuilds itself We are rebuilding the sandbox runtime with TARS and the PM agent. The factory is now building its own infrastructure. WorkOS Interface text: 11 / 16 ### Memory: The Next Layer WorkOS plans a memory layer with user, team and organization context, scoped by identity. Cooke wants that context available both in the software factory and in other AI tools. [15:12](https://www.youtube.com/watch?v=HvboD89DyQ8&t=912s) [Memory organized per user, per team, and across the organization.](https://assets.ai.engineer/sites/aiecode2025/sha256/45a00f1f43b6368b6488274463d3dbfd8317f41a8ac80318bd1d4cdc5ea3659c.webp) Memory: The Next Layer Learnings compound across the org — scoped by WorkOS identity, not by harness. Per-User What this engineer works on, their preferences, context from past sessions Per-Team How this team scopes work, their conventions, team-level patterns and decisions Org-Wide WorkOS-wide standards, golden principles, company-level judgment calls Interface text: WorkOS 12 / 16 ### Engineering Process as the Blueprint Cooke describes using agents to examine sessions for mistakes and obsolete skills, feeding those findings into a factory designed to improve with each run. [16:55](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1015s) [A quotation describes each run producing improvements; the takeaway calls the engineering process the blueprint for agent workflows.](https://assets.ai.engineer/sites/aiecode2025/sha256/2812363ef090ba96349964027ff0bd6fe45fb5d3ba421f32b00c495b94137b99.webp) "Every run ships work and produces the next set of fixes: better verification, clearer guidelines, tighter paved paths, so the platform learns faster than the failures repeat." Your product engineering process is the blueprint for every agent workflow you build. Interface text: WorkOS 14 / 16 ### A Software Factory Is A sandbox, model, and repo are necessary foundations. Cooke argues that teams must also encode their planning, feature scoping, verification, and engineering conventions in automation. [17:59](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1079s) [A software factory combines infrastructure with planning, feature scoping, verification, and engineering conventions.](https://assets.ai.engineer/sites/aiecode2025/sha256/e22478a12b4fd58a7f36bf7e1a54b14a7cd8d0b86c8e9f414bf106271b6d455b.webp) A Software Factory Is: Sandbox + Model + Repo necessary condition How you plan work Hilltop docs · issues · dependency chains How you scope features Small teams · explicit acceptance criteria How you verify Enforced gates, not prose instructions Your conventions Judgment calls that define your engineering culture The unlock isn't infrastructure. It's institutional knowledge, made executable. WorkOS Interface text: 15 / 16 ### Ryan Cooke — WorkOS [18:19](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1099s) [WorkOS closing slide naming Ryan Cooke, Product Engineering, and AI Engineer World's Fair 2026.](https://assets.ai.engineer/sites/aiecode2025/sha256/07fd8c26ec323524afa551474d825e360dd4d16900305c584fc0e84c2e92e2f6.webp) WorkOS Ryan Cooke Product Engineering · WorkOS AI Engineer World's Fair · 2026 Interface text: 16 / 16 ## Full transcript [0:12](https://www.youtube.com/watch?v=HvboD89DyQ8&t=12s) Well, thank you for joining [0:14](https://www.youtube.com/watch?v=HvboD89DyQ8&t=14s) the final day of the conference here. I [0:15](https://www.youtube.com/watch?v=HvboD89DyQ8&t=15s) know it's been a long week. [0:18](https://www.youtube.com/watch?v=HvboD89DyQ8&t=18s) But I'm excited to talk about a topic [0:21](https://www.youtube.com/watch?v=HvboD89DyQ8&t=21s) here that is very near and dear to me. [0:24](https://www.youtube.com/watch?v=HvboD89DyQ8&t=24s) Software factories. [0:26](https://www.youtube.com/watch?v=HvboD89DyQ8&t=26s) Um [0:27](https://www.youtube.com/watch?v=HvboD89DyQ8&t=27s) You this is probably not a new concept [0:30](https://www.youtube.com/watch?v=HvboD89DyQ8&t=30s) to any of you. I'm guessing if you're [0:31](https://www.youtube.com/watch?v=HvboD89DyQ8&t=31s) all here, you're very familiar with [0:33](https://www.youtube.com/watch?v=HvboD89DyQ8&t=33s) software factories. You know, Ramp [0:35](https://www.youtube.com/watch?v=HvboD89DyQ8&t=35s) blogged about their inspects [0:38](https://www.youtube.com/watch?v=HvboD89DyQ8&t=38s) several months ago. I think maybe [0:40](https://www.youtube.com/watch?v=HvboD89DyQ8&t=40s) December last year. I forget exactly [0:41](https://www.youtube.com/watch?v=HvboD89DyQ8&t=41s) when. It's been a little while. And that [0:43](https://www.youtube.com/watch?v=HvboD89DyQ8&t=43s) really kind of like excited the entire [0:45](https://www.youtube.com/watch?v=HvboD89DyQ8&t=45s) industry of thinking like, oh, we can [0:47](https://www.youtube.com/watch?v=HvboD89DyQ8&t=47s) actually build software very differently [0:49](https://www.youtube.com/watch?v=HvboD89DyQ8&t=49s) now that AI is extremely capable of [0:52](https://www.youtube.com/watch?v=HvboD89DyQ8&t=52s) writing code. Um a lot of other [0:55](https://www.youtube.com/watch?v=HvboD89DyQ8&t=55s) companies have followed suit. Um and [0:58](https://www.youtube.com/watch?v=HvboD89DyQ8&t=58s) there's kind of like a standard set up [1:00](https://www.youtube.com/watch?v=HvboD89DyQ8&t=60s) here. Uh you have a sandbox that you put [1:02](https://www.youtube.com/watch?v=HvboD89DyQ8&t=62s) your code into. Um you put a an AI agent [1:06](https://www.youtube.com/watch?v=HvboD89DyQ8&t=66s) in there. Maybe a Claude code section or [1:09](https://www.youtube.com/watch?v=HvboD89DyQ8&t=69s) an open code. [1:11](https://www.youtube.com/watch?v=HvboD89DyQ8&t=71s) Um and with a prompt, it generates a PR [1:13](https://www.youtube.com/watch?v=HvboD89DyQ8&t=73s) and you merge it. [1:15](https://www.youtube.com/watch?v=HvboD89DyQ8&t=75s) Um [1:15](https://www.youtube.com/watch?v=HvboD89DyQ8&t=75s) You might be building this already [1:17](https://www.youtube.com/watch?v=HvboD89DyQ8&t=77s) [clears throat] yourself. [1:18](https://www.youtube.com/watch?v=HvboD89DyQ8&t=78s) We have taken a slightly different [1:21](https://www.youtube.com/watch?v=HvboD89DyQ8&t=81s) approach at WorkOS. [1:23](https://www.youtube.com/watch?v=HvboD89DyQ8&t=83s) Um and the motivation for what we are [1:26](https://www.youtube.com/watch?v=HvboD89DyQ8&t=86s) building is that we see a lot of the [1:30](https://www.youtube.com/watch?v=HvboD89DyQ8&t=90s) kind of success metrics um really [1:33](https://www.youtube.com/watch?v=HvboD89DyQ8&t=93s) focusing on output, right? Like you'll [1:35](https://www.youtube.com/watch?v=HvboD89DyQ8&t=95s) see posts on X and blogs [1:38](https://www.youtube.com/watch?v=HvboD89DyQ8&t=98s) talking about the percentage of PRs or [1:41](https://www.youtube.com/watch?v=HvboD89DyQ8&t=101s) the number of pull requests or how much [1:43](https://www.youtube.com/watch?v=HvboD89DyQ8&t=103s) code uh is being generated by AI [1:47](https://www.youtube.com/watch?v=HvboD89DyQ8&t=107s) moving into production. Um and this is [1:49](https://www.youtube.com/watch?v=HvboD89DyQ8&t=109s) just really output metrics that [1:53](https://www.youtube.com/watch?v=HvboD89DyQ8&t=113s) um [1:53](https://www.youtube.com/watch?v=HvboD89DyQ8&t=113s) you know, this can actually disguise how [1:56](https://www.youtube.com/watch?v=HvboD89DyQ8&t=116s) well these systems are working. [1:59](https://www.youtube.com/watch?v=HvboD89DyQ8&t=119s) Percentage of PRs or number of PRs [2:02](https://www.youtube.com/watch?v=HvboD89DyQ8&t=122s) may be lying in overall increase in PRs. [2:04](https://www.youtube.com/watch?v=HvboD89DyQ8&t=124s) It's very hard to distinguish [2:06](https://www.youtube.com/watch?v=HvboD89DyQ8&t=126s) whether the output is actually driving [2:09](https://www.youtube.com/watch?v=HvboD89DyQ8&t=129s) outcomes. [2:11](https://www.youtube.com/watch?v=HvboD89DyQ8&t=131s) And so that's kind of where we focus our [2:14](https://www.youtube.com/watch?v=HvboD89DyQ8&t=134s) efforts when we're thinking about a [2:15](https://www.youtube.com/watch?v=HvboD89DyQ8&t=135s) software factory. It's like if we're [2:17](https://www.youtube.com/watch?v=HvboD89DyQ8&t=137s) going to go put engineering time into [2:20](https://www.youtube.com/watch?v=HvboD89DyQ8&t=140s) building this level of automations and [2:22](https://www.youtube.com/watch?v=HvboD89DyQ8&t=142s) there's a [2:23](https://www.youtube.com/watch?v=HvboD89DyQ8&t=143s) bit of work that needs to go into this. [2:25](https://www.youtube.com/watch?v=HvboD89DyQ8&t=145s) Full-time teams dedicated to this. [2:27](https://www.youtube.com/watch?v=HvboD89DyQ8&t=147s) How do we want to think about the [2:30](https://www.youtube.com/watch?v=HvboD89DyQ8&t=150s) success of this and whether this is [2:32](https://www.youtube.com/watch?v=HvboD89DyQ8&t=152s) actually driving value for our [2:33](https://www.youtube.com/watch?v=HvboD89DyQ8&t=153s) organization, our engineering [2:35](https://www.youtube.com/watch?v=HvboD89DyQ8&t=155s) organization? [2:36](https://www.youtube.com/watch?v=HvboD89DyQ8&t=156s) And so we measure instead of just code [2:40](https://www.youtube.com/watch?v=HvboD89DyQ8&t=160s) output, we're looking at outcome [2:42](https://www.youtube.com/watch?v=HvboD89DyQ8&t=162s) metrics. [2:44](https://www.youtube.com/watch?v=HvboD89DyQ8&t=164s) And that is really about like are we [2:46](https://www.youtube.com/watch?v=HvboD89DyQ8&t=166s) accelerating our ability to deliver [2:48](https://www.youtube.com/watch?v=HvboD89DyQ8&t=168s) features? [2:50](https://www.youtube.com/watch?v=HvboD89DyQ8&t=170s) We think that the dream of the software [2:51](https://www.youtube.com/watch?v=HvboD89DyQ8&t=171s) factory is that it gives each of our [2:54](https://www.youtube.com/watch?v=HvboD89DyQ8&t=174s) engineers a small engineering team for [2:56](https://www.youtube.com/watch?v=HvboD89DyQ8&t=176s) themselves. And so the consequence of [2:58](https://www.youtube.com/watch?v=HvboD89DyQ8&t=178s) that really needs to be that we're [3:00](https://www.youtube.com/watch?v=HvboD89DyQ8&t=180s) building more and we're shipping more [3:02](https://www.youtube.com/watch?v=HvboD89DyQ8&t=182s) and that as we embark on complex [3:04](https://www.youtube.com/watch?v=HvboD89DyQ8&t=184s) features, those can get built a lot more [3:07](https://www.youtube.com/watch?v=HvboD89DyQ8&t=187s) a lot more quickly. [3:11](https://www.youtube.com/watch?v=HvboD89DyQ8&t=191s) So what does this actually look like? [3:12](https://www.youtube.com/watch?v=HvboD89DyQ8&t=192s) Well, [3:14](https://www.youtube.com/watch?v=HvboD89DyQ8&t=194s) we kind of started with the same thing [3:16](https://www.youtube.com/watch?v=HvboD89DyQ8&t=196s) you've seen in other other [3:19](https://www.youtube.com/watch?v=HvboD89DyQ8&t=199s) factories. [3:21](https://www.youtube.com/watch?v=HvboD89DyQ8&t=201s) We start with a sandbox we built on top [3:22](https://www.youtube.com/watch?v=HvboD89DyQ8&t=202s) of Cloudflare. We put an open code model [3:26](https://www.youtube.com/watch?v=HvboD89DyQ8&t=206s) router in there. We're feeding it [3:27](https://www.youtube.com/watch?v=HvboD89DyQ8&t=207s) prompts. [3:29](https://www.youtube.com/watch?v=HvboD89DyQ8&t=209s) Um, [3:29](https://www.youtube.com/watch?v=HvboD89DyQ8&t=209s) and we kind of very quickly kind of ran [3:32](https://www.youtube.com/watch?v=HvboD89DyQ8&t=212s) into this situation where this wasn't [3:34](https://www.youtube.com/watch?v=HvboD89DyQ8&t=214s) driving more outcomes. This was not an [3:37](https://www.youtube.com/watch?v=HvboD89DyQ8&t=217s) incremental increase or an exponential [3:39](https://www.youtube.com/watch?v=HvboD89DyQ8&t=219s) increase over engineers just driving [3:41](https://www.youtube.com/watch?v=HvboD89DyQ8&t=221s) cloud code on their laptops. It was [3:43](https://www.youtube.com/watch?v=HvboD89DyQ8&t=223s) actually pretty indistinguishable for [3:45](https://www.youtube.com/watch?v=HvboD89DyQ8&t=225s) us. [3:47](https://www.youtube.com/watch?v=HvboD89DyQ8&t=227s) And so we really kind of started to [3:48](https://www.youtube.com/watch?v=HvboD89DyQ8&t=228s) think about, well, [3:49](https://www.youtube.com/watch?v=HvboD89DyQ8&t=229s) how can we embed our engineering [3:52](https://www.youtube.com/watch?v=HvboD89DyQ8&t=232s) processes into the factory itself? It's [3:56](https://www.youtube.com/watch?v=HvboD89DyQ8&t=236s) not sufficient that the factory is [3:58](https://www.youtube.com/watch?v=HvboD89DyQ8&t=238s) producing code. We actually want it to [4:00](https://www.youtube.com/watch?v=HvboD89DyQ8&t=240s) take a lot of the other work that our [4:02](https://www.youtube.com/watch?v=HvboD89DyQ8&t=242s) engineers do to produce products and [4:05](https://www.youtube.com/watch?v=HvboD89DyQ8&t=245s) automate that as well. [4:08](https://www.youtube.com/watch?v=HvboD89DyQ8&t=248s) So, we kind of split this into two [4:09](https://www.youtube.com/watch?v=HvboD89DyQ8&t=249s) parts. [4:10](https://www.youtube.com/watch?v=HvboD89DyQ8&t=250s) One is we have the system we call TARS. [4:13](https://www.youtube.com/watch?v=HvboD89DyQ8&t=253s) This is a way for users to interact with [4:18](https://www.youtube.com/watch?v=HvboD89DyQ8&t=258s) the coding agent. And it is embedded [4:21](https://www.youtube.com/watch?v=HvboD89DyQ8&t=261s) into the tools that we use. So, not just [4:24](https://www.youtube.com/watch?v=HvboD89DyQ8&t=264s) Slack, but also Linear and GitHub. [4:27](https://www.youtube.com/watch?v=HvboD89DyQ8&t=267s) We subscribe to webhooks through TARS so [4:29](https://www.youtube.com/watch?v=HvboD89DyQ8&t=269s) that TARS can actually track the [4:31](https://www.youtube.com/watch?v=HvboD89DyQ8&t=271s) progress of projects [4:34](https://www.youtube.com/watch?v=HvboD89DyQ8&t=274s) in addition to generating the code [4:36](https://www.youtube.com/watch?v=HvboD89DyQ8&t=276s) outputs. [4:38](https://www.youtube.com/watch?v=HvboD89DyQ8&t=278s) And then we built a separate system [4:39](https://www.youtube.com/watch?v=HvboD89DyQ8&t=279s) called Horizon. [4:42](https://www.youtube.com/watch?v=HvboD89DyQ8&t=282s) And this is the infrastructure [4:43](https://www.youtube.com/watch?v=HvboD89DyQ8&t=283s) orchestration layer. This is a lot more [4:45](https://www.youtube.com/watch?v=HvboD89DyQ8&t=285s) similar to [4:47](https://www.youtube.com/watch?v=HvboD89DyQ8&t=287s) Inspects and Minions and and some of the [4:49](https://www.youtube.com/watch?v=HvboD89DyQ8&t=289s) other systems that other companies have [4:51](https://www.youtube.com/watch?v=HvboD89DyQ8&t=291s) built. [4:52](https://www.youtube.com/watch?v=HvboD89DyQ8&t=292s) But, it sits in front of an MCP gateway. [4:56](https://www.youtube.com/watch?v=HvboD89DyQ8&t=296s) And I'm going to talk a little bit more [4:57](https://www.youtube.com/watch?v=HvboD89DyQ8&t=297s) about why that MCP gateway has been [5:01](https://www.youtube.com/watch?v=HvboD89DyQ8&t=301s) really transformational for us. [5:07](https://www.youtube.com/watch?v=HvboD89DyQ8&t=307s) So, by [5:08](https://www.youtube.com/watch?v=HvboD89DyQ8&t=308s) feeding our webhooks, activities that [5:12](https://www.youtube.com/watch?v=HvboD89DyQ8&t=312s) are happening in other uh source code [5:14](https://www.youtube.com/watch?v=HvboD89DyQ8&t=314s) systems and project tracking systems, [5:17](https://www.youtube.com/watch?v=HvboD89DyQ8&t=317s) we're starting to get to that level of [5:18](https://www.youtube.com/watch?v=HvboD89DyQ8&t=318s) autonomy where [5:20](https://www.youtube.com/watch?v=HvboD89DyQ8&t=320s) our [5:21](https://www.youtube.com/watch?v=HvboD89DyQ8&t=321s) factory is performing product [5:23](https://www.youtube.com/watch?v=HvboD89DyQ8&t=323s) engineering work, not just code work. [5:25](https://www.youtube.com/watch?v=HvboD89DyQ8&t=325s) So, what's an example of this? [5:27](https://www.youtube.com/watch?v=HvboD89DyQ8&t=327s) In Linear, we can define tickets that [5:30](https://www.youtube.com/watch?v=HvboD89DyQ8&t=330s) have dependencies. So, one ticket blocks [5:32](https://www.youtube.com/watch?v=HvboD89DyQ8&t=332s) another ticket. Pretty common for, you [5:34](https://www.youtube.com/watch?v=HvboD89DyQ8&t=334s) know, breaking down uh large units of [5:36](https://www.youtube.com/watch?v=HvboD89DyQ8&t=336s) work into smaller ones. [5:38](https://www.youtube.com/watch?v=HvboD89DyQ8&t=338s) Because [5:40](https://www.youtube.com/watch?v=HvboD89DyQ8&t=340s) uh TARS is getting webhooks on ticket [5:42](https://www.youtube.com/watch?v=HvboD89DyQ8&t=342s) completion, it can automatically pick up [5:44](https://www.youtube.com/watch?v=HvboD89DyQ8&t=344s) the next ticket in a cycle. And so, we [5:47](https://www.youtube.com/watch?v=HvboD89DyQ8&t=347s) can do through our planning process [5:50](https://www.youtube.com/watch?v=HvboD89DyQ8&t=350s) construct like a map of how this plan we [5:53](https://www.youtube.com/watch?v=HvboD89DyQ8&t=353s) think this plan is going to get executed [5:55](https://www.youtube.com/watch?v=HvboD89DyQ8&t=355s) and TARS can start to execute on that [5:57](https://www.youtube.com/watch?v=HvboD89DyQ8&t=357s) autonomously. The other thing that we [5:59](https://www.youtube.com/watch?v=HvboD89DyQ8&t=359s) have is in between those steps when a [6:01](https://www.youtube.com/watch?v=HvboD89DyQ8&t=361s) ticket is completed, we can ask TARS [6:04](https://www.youtube.com/watch?v=HvboD89DyQ8&t=364s) "Can you reevaluate the linear project [6:06](https://www.youtube.com/watch?v=HvboD89DyQ8&t=366s) and let me know if it's missing now [6:07](https://www.youtube.com/watch?v=HvboD89DyQ8&t=367s) tickets?" Because as you do work, you're [6:10](https://www.youtube.com/watch?v=HvboD89DyQ8&t=370s) learning about where the gaps in your [6:12](https://www.youtube.com/watch?v=HvboD89DyQ8&t=372s) plan. And we want to continuously keep [6:16](https://www.youtube.com/watch?v=HvboD89DyQ8&t=376s) our our plan fresh by using the agent [6:19](https://www.youtube.com/watch?v=HvboD89DyQ8&t=379s) itself. Because the agent is determining [6:22](https://www.youtube.com/watch?v=HvboD89DyQ8&t=382s) that there may be missing pieces to what [6:25](https://www.youtube.com/watch?v=HvboD89DyQ8&t=385s) we initially planned for the project and [6:27](https://www.youtube.com/watch?v=HvboD89DyQ8&t=387s) the goal that we have in mind. [6:31](https://www.youtube.com/watch?v=HvboD89DyQ8&t=391s) So, we run this product engineering [6:33](https://www.youtube.com/watch?v=HvboD89DyQ8&t=393s) culture at WorkOS. This is where [6:35](https://www.youtube.com/watch?v=HvboD89DyQ8&t=395s) engineers are responsible for a lot of [6:37](https://www.youtube.com/watch?v=HvboD89DyQ8&t=397s) the product functions. We don't have [6:39](https://www.youtube.com/watch?v=HvboD89DyQ8&t=399s) product managers on teams today. [6:41](https://www.youtube.com/watch?v=HvboD89DyQ8&t=401s) Um and one of the rituals as part of [6:45](https://www.youtube.com/watch?v=HvboD89DyQ8&t=405s) this product engineering process is we [6:46](https://www.youtube.com/watch?v=HvboD89DyQ8&t=406s) create a hilltop document. It's a PRD [6:49](https://www.youtube.com/watch?v=HvboD89DyQ8&t=409s) and it is intended to [6:52](https://www.youtube.com/watch?v=HvboD89DyQ8&t=412s) both define what is the purpose of this [6:54](https://www.youtube.com/watch?v=HvboD89DyQ8&t=414s) project, but also incorporate [6:58](https://www.youtube.com/watch?v=HvboD89DyQ8&t=418s) um what are customers talking about? [7:00](https://www.youtube.com/watch?v=HvboD89DyQ8&t=420s) Like where are we seeing the need for [7:01](https://www.youtube.com/watch?v=HvboD89DyQ8&t=421s) this unit of work? Uh it looks at [7:04](https://www.youtube.com/watch?v=HvboD89DyQ8&t=424s) competitive analysis. Like are there [7:06](https://www.youtube.com/watch?v=HvboD89DyQ8&t=426s) similar products out in the market today [7:08](https://www.youtube.com/watch?v=HvboD89DyQ8&t=428s) that we can draw inspiration from? We [7:10](https://www.youtube.com/watch?v=HvboD89DyQ8&t=430s) start to bring design early design [7:12](https://www.youtube.com/watch?v=HvboD89DyQ8&t=432s) screens into this. [7:13](https://www.youtube.com/watch?v=HvboD89DyQ8&t=433s) Um and we outline like what are the [7:15](https://www.youtube.com/watch?v=HvboD89DyQ8&t=435s) major milestones. And so, by kind of [7:18](https://www.youtube.com/watch?v=HvboD89DyQ8&t=438s) consolidating and canonicalizing this [7:21](https://www.youtube.com/watch?v=HvboD89DyQ8&t=441s) information into a document, this is [7:23](https://www.youtube.com/watch?v=HvboD89DyQ8&t=443s) something our product engineers have [7:24](https://www.youtube.com/watch?v=HvboD89DyQ8&t=444s) been doing uh throughout the entire uh [7:27](https://www.youtube.com/watch?v=HvboD89DyQ8&t=447s) company. [7:28](https://www.youtube.com/watch?v=HvboD89DyQ8&t=448s) Um [7:29](https://www.youtube.com/watch?v=HvboD89DyQ8&t=449s) now we can ask give this resource to an [7:31](https://www.youtube.com/watch?v=HvboD89DyQ8&t=451s) agent and an agent can break this into [7:33](https://www.youtube.com/watch?v=HvboD89DyQ8&t=453s) units of work. [7:34](https://www.youtube.com/watch?v=HvboD89DyQ8&t=454s) Uh and so, this has been a really uh [7:37](https://www.youtube.com/watch?v=HvboD89DyQ8&t=457s) powerful way for us to take existing [7:39](https://www.youtube.com/watch?v=HvboD89DyQ8&t=459s) processes [7:40](https://www.youtube.com/watch?v=HvboD89DyQ8&t=460s) and encode them into the factory [7:42](https://www.youtube.com/watch?v=HvboD89DyQ8&t=462s) themselves. Like I mentioned that it's [7:45](https://www.youtube.com/watch?v=HvboD89DyQ8&t=465s) listening for web hooks. I'll show an [7:46](https://www.youtube.com/watch?v=HvboD89DyQ8&t=466s) example of what this looks like, but it [7:48](https://www.youtube.com/watch?v=HvboD89DyQ8&t=468s) it can see that the hilltop document [7:51](https://www.youtube.com/watch?v=HvboD89DyQ8&t=471s) through linear tickets has been reviewed [7:54](https://www.youtube.com/watch?v=HvboD89DyQ8&t=474s) and approved and by virtue of it being [7:57](https://www.youtube.com/watch?v=HvboD89DyQ8&t=477s) approved can start work on this project [7:59](https://www.youtube.com/watch?v=HvboD89DyQ8&t=479s) automatically. So we don't need a human [8:01](https://www.youtube.com/watch?v=HvboD89DyQ8&t=481s) to be shepherding this agent through [8:03](https://www.youtube.com/watch?v=HvboD89DyQ8&t=483s) every single step of the life cycle. [8:09](https://www.youtube.com/watch?v=HvboD89DyQ8&t=489s) So we created an agent specifically for [8:12](https://www.youtube.com/watch?v=HvboD89DyQ8&t=492s) this part of the process. We call it the [8:13](https://www.youtube.com/watch?v=HvboD89DyQ8&t=493s) PM. [8:14](https://www.youtube.com/watch?v=HvboD89DyQ8&t=494s) And it is doing the first draft of the [8:17](https://www.youtube.com/watch?v=HvboD89DyQ8&t=497s) hilltop based on the brief specification [8:20](https://www.youtube.com/watch?v=HvboD89DyQ8&t=500s) that we give it. [8:21](https://www.youtube.com/watch?v=HvboD89DyQ8&t=501s) It's adding context to that in addition [8:25](https://www.youtube.com/watch?v=HvboD89DyQ8&t=505s) to reading the human reviews and then it [8:27](https://www.youtube.com/watch?v=HvboD89DyQ8&t=507s) picks up the implementation and breaking [8:29](https://www.youtube.com/watch?v=HvboD89DyQ8&t=509s) that into tickets. [8:31](https://www.youtube.com/watch?v=HvboD89DyQ8&t=511s) And there's an opportunity for humans to [8:33](https://www.youtube.com/watch?v=HvboD89DyQ8&t=513s) stay in the loop for each part of this [8:35](https://www.youtube.com/watch?v=HvboD89DyQ8&t=515s) process. I mean it's very common for a [8:38](https://www.youtube.com/watch?v=HvboD89DyQ8&t=518s) human to intervene or comment on a [8:39](https://www.youtube.com/watch?v=HvboD89DyQ8&t=519s) ticket that's generated by AI, give it [8:41](https://www.youtube.com/watch?v=HvboD89DyQ8&t=521s) further guidance or refinements. [8:44](https://www.youtube.com/watch?v=HvboD89DyQ8&t=524s) Um but it's really about that like cold [8:46](https://www.youtube.com/watch?v=HvboD89DyQ8&t=526s) start problem where we can kind of get [8:49](https://www.youtube.com/watch?v=HvboD89DyQ8&t=529s) over the hump of creating all these [8:50](https://www.youtube.com/watch?v=HvboD89DyQ8&t=530s) different resources. Okay, so what does [8:52](https://www.youtube.com/watch?v=HvboD89DyQ8&t=532s) this actually look like? [8:55](https://www.youtube.com/watch?v=HvboD89DyQ8&t=535s) So here's a project I kicked off last [8:57](https://www.youtube.com/watch?v=HvboD89DyQ8&t=537s) month two months ago. [8:59](https://www.youtube.com/watch?v=HvboD89DyQ8&t=539s) Um we're adding a new API to products I [9:01](https://www.youtube.com/watch?v=HvboD89DyQ8&t=541s) lead called Vaults. [9:03](https://www.youtube.com/watch?v=HvboD89DyQ8&t=543s) And with a command here in [9:07](https://www.youtube.com/watch?v=HvboD89DyQ8&t=547s) um [9:08](https://www.youtube.com/watch?v=HvboD89DyQ8&t=548s) in Slack [9:09](https://www.youtube.com/watch?v=HvboD89DyQ8&t=549s) I can kick off this project with a [9:11](https://www.youtube.com/watch?v=HvboD89DyQ8&t=551s) simple description, usually a few [9:12](https://www.youtube.com/watch?v=HvboD89DyQ8&t=552s) sentences of what I'm trying to [9:14](https://www.youtube.com/watch?v=HvboD89DyQ8&t=554s) accomplish. [9:16](https://www.youtube.com/watch?v=HvboD89DyQ8&t=556s) Tarsin goes in and creates all these [9:18](https://www.youtube.com/watch?v=HvboD89DyQ8&t=558s) resources for me. I don't need to go and [9:20](https://www.youtube.com/watch?v=HvboD89DyQ8&t=560s) create the project in linear. I don't [9:21](https://www.youtube.com/watch?v=HvboD89DyQ8&t=561s) need to create a draft of the notion [9:23](https://www.youtube.com/watch?v=HvboD89DyQ8&t=563s) document. [9:25](https://www.youtube.com/watch?v=HvboD89DyQ8&t=565s) Um [9:25](https://www.youtube.com/watch?v=HvboD89DyQ8&t=565s) all of the decision logs and the open [9:28](https://www.youtube.com/watch?v=HvboD89DyQ8&t=568s) questions. [9:29](https://www.youtube.com/watch?v=HvboD89DyQ8&t=569s) Um Tars is creating a first pass at [9:31](https://www.youtube.com/watch?v=HvboD89DyQ8&t=571s) that. And so this then gives me an easy [9:34](https://www.youtube.com/watch?v=HvboD89DyQ8&t=574s) framework for me to step into as the [9:37](https://www.youtube.com/watch?v=HvboD89DyQ8&t=577s) lead product engineer and start giving [9:39](https://www.youtube.com/watch?v=HvboD89DyQ8&t=579s) more specification, rounding out areas [9:42](https://www.youtube.com/watch?v=HvboD89DyQ8&t=582s) that aren't well defined with knowledge [9:45](https://www.youtube.com/watch?v=HvboD89DyQ8&t=585s) that I had know about what we're trying [9:46](https://www.youtube.com/watch?v=HvboD89DyQ8&t=586s) to accomplish with the project, and then [9:49](https://www.youtube.com/watch?v=HvboD89DyQ8&t=589s) hand it back to Tars uh for execution. [9:53](https://www.youtube.com/watch?v=HvboD89DyQ8&t=593s) So, here's an example of uh the project. [9:57](https://www.youtube.com/watch?v=HvboD89DyQ8&t=597s) Um and it's like I said, it sets up [9:58](https://www.youtube.com/watch?v=HvboD89DyQ8&t=598s) these first [10:00](https://www.youtube.com/watch?v=HvboD89DyQ8&t=600s) milestones around our product [10:02](https://www.youtube.com/watch?v=HvboD89DyQ8&t=602s) engineering process. So, we haven't done [10:05](https://www.youtube.com/watch?v=HvboD89DyQ8&t=605s) the hilltop review yet here. When that's [10:07](https://www.youtube.com/watch?v=HvboD89DyQ8&t=607s) done, we'll uh [10:09](https://www.youtube.com/watch?v=HvboD89DyQ8&t=609s) we'll we'll mark this ticket as complete [10:12](https://www.youtube.com/watch?v=HvboD89DyQ8&t=612s) and Tars will pick that up and move on [10:14](https://www.youtube.com/watch?v=HvboD89DyQ8&t=614s) to the next stage of project [10:15](https://www.youtube.com/watch?v=HvboD89DyQ8&t=615s) implementation. [10:17](https://www.youtube.com/watch?v=HvboD89DyQ8&t=617s) So, [10:18](https://www.youtube.com/watch?v=HvboD89DyQ8&t=618s) what we found is like a lot of [10:21](https://www.youtube.com/watch?v=HvboD89DyQ8&t=621s) teams are now doing more project brief [10:24](https://www.youtube.com/watch?v=HvboD89DyQ8&t=624s) and hilltop work because our agent can [10:27](https://www.youtube.com/watch?v=HvboD89DyQ8&t=627s) seed all that information. It's that [10:29](https://www.youtube.com/watch?v=HvboD89DyQ8&t=629s) blank page problem with writing. If you [10:32](https://www.youtube.com/watch?v=HvboD89DyQ8&t=632s) can come into a document that already [10:33](https://www.youtube.com/watch?v=HvboD89DyQ8&t=633s) has information, and you know, these [10:35](https://www.youtube.com/watch?v=HvboD89DyQ8&t=635s) agents aren't perfect. There are times [10:37](https://www.youtube.com/watch?v=HvboD89DyQ8&t=637s) where it grossly overestimates what [10:40](https://www.youtube.com/watch?v=HvboD89DyQ8&t=640s) we're trying to accomplish with this [10:42](https://www.youtube.com/watch?v=HvboD89DyQ8&t=642s) project, and we have to cut out a lot of [10:43](https://www.youtube.com/watch?v=HvboD89DyQ8&t=643s) the scope that it comes up with. But, [10:45](https://www.youtube.com/watch?v=HvboD89DyQ8&t=645s) that's fine. That's a lot simpler for an [10:47](https://www.youtube.com/watch?v=HvboD89DyQ8&t=647s) engineer to add input into into rather [10:51](https://www.youtube.com/watch?v=HvboD89DyQ8&t=651s) than them spending time to set up all [10:52](https://www.youtube.com/watch?v=HvboD89DyQ8&t=652s) these primitives themselves. [10:54](https://www.youtube.com/watch?v=HvboD89DyQ8&t=654s) And then we can continue to drive this [10:56](https://www.youtube.com/watch?v=HvboD89DyQ8&t=656s) work, you know, through Slack, through [10:58](https://www.youtube.com/watch?v=HvboD89DyQ8&t=658s) Linear, [10:59](https://www.youtube.com/watch?v=HvboD89DyQ8&t=659s) um and ultimately uh results in PRs in [11:02](https://www.youtube.com/watch?v=HvboD89DyQ8&t=662s) GitHub. [11:03](https://www.youtube.com/watch?v=HvboD89DyQ8&t=663s) The other thing that this lets us do is [11:05](https://www.youtube.com/watch?v=HvboD89DyQ8&t=665s) we don't necessarily always need to use [11:07](https://www.youtube.com/watch?v=HvboD89DyQ8&t=667s) our coding agent to implement uh pieces [11:11](https://www.youtube.com/watch?v=HvboD89DyQ8&t=671s) of work. Uh Devin is very popular. We [11:13](https://www.youtube.com/watch?v=HvboD89DyQ8&t=673s) have a lot of folks that really like [11:14](https://www.youtube.com/watch?v=HvboD89DyQ8&t=674s) using Devin. They can point Devin to [11:17](https://www.youtube.com/watch?v=HvboD89DyQ8&t=677s) these tickets and the same documentation [11:20](https://www.youtube.com/watch?v=HvboD89DyQ8&t=680s) to give Devin the context it needs to go [11:22](https://www.youtube.com/watch?v=HvboD89DyQ8&t=682s) and implement different parts of the [11:23](https://www.youtube.com/watch?v=HvboD89DyQ8&t=683s) work. Uh same with the local cloud code [11:26](https://www.youtube.com/watch?v=HvboD89DyQ8&t=686s) if you're just using Opus in a local [11:28](https://www.youtube.com/watch?v=HvboD89DyQ8&t=688s) harness. [11:29](https://www.youtube.com/watch?v=HvboD89DyQ8&t=689s) Um it can grab through MCP all of these [11:33](https://www.youtube.com/watch?v=HvboD89DyQ8&t=693s) uh [11:34](https://www.youtube.com/watch?v=HvboD89DyQ8&t=694s) all of these pieces of documentation and [11:38](https://www.youtube.com/watch?v=HvboD89DyQ8&t=698s) use that as context for its work. [11:41](https://www.youtube.com/watch?v=HvboD89DyQ8&t=701s) And then, of course, we have people that [11:43](https://www.youtube.com/watch?v=HvboD89DyQ8&t=703s) collaborate in these project channels as [11:44](https://www.youtube.com/watch?v=HvboD89DyQ8&t=704s) well. You'll notice our security team [11:46](https://www.youtube.com/watch?v=HvboD89DyQ8&t=706s) will come in and look take a look at new [11:48](https://www.youtube.com/watch?v=HvboD89DyQ8&t=708s) projects, weigh in on the security [11:50](https://www.youtube.com/watch?v=HvboD89DyQ8&t=710s) implications, and so forth. [11:59](https://www.youtube.com/watch?v=HvboD89DyQ8&t=719s) So, I alluded to we we kind of early on [12:03](https://www.youtube.com/watch?v=HvboD89DyQ8&t=723s) in the the development of our factory is [12:05](https://www.youtube.com/watch?v=HvboD89DyQ8&t=725s) created our own MCP gateway. [12:07](https://www.youtube.com/watch?v=HvboD89DyQ8&t=727s) Um, we call it our context engine. [12:10](https://www.youtube.com/watch?v=HvboD89DyQ8&t=730s) Um, this connects into all of our [12:11](https://www.youtube.com/watch?v=HvboD89DyQ8&t=731s) internal systems, but it also builds [12:15](https://www.youtube.com/watch?v=HvboD89DyQ8&t=735s) system prompts and context around what [12:18](https://www.youtube.com/watch?v=HvboD89DyQ8&t=738s) how to navigate these tools and when to [12:20](https://www.youtube.com/watch?v=HvboD89DyQ8&t=740s) use these tools. [12:21](https://www.youtube.com/watch?v=HvboD89DyQ8&t=741s) So, it is connected into Snowflake, [12:25](https://www.youtube.com/watch?v=HvboD89DyQ8&t=745s) which is our main data lake. We have [12:28](https://www.youtube.com/watch?v=HvboD89DyQ8&t=748s) semantic tables that we've built in [12:30](https://www.youtube.com/watch?v=HvboD89DyQ8&t=750s) Snowflake that describe product [12:32](https://www.youtube.com/watch?v=HvboD89DyQ8&t=752s) utilization or customer conversations. [12:35](https://www.youtube.com/watch?v=HvboD89DyQ8&t=755s) And then, we can provide an agent [12:38](https://www.youtube.com/watch?v=HvboD89DyQ8&t=758s) through our MCP server in the tool [12:40](https://www.youtube.com/watch?v=HvboD89DyQ8&t=760s) description [12:42](https://www.youtube.com/watch?v=HvboD89DyQ8&t=762s) list of here are the tables, this is the [12:44](https://www.youtube.com/watch?v=HvboD89DyQ8&t=764s) content they [12:45](https://www.youtube.com/watch?v=HvboD89DyQ8&t=765s) contain. If you're trying to answer [12:47](https://www.youtube.com/watch?v=HvboD89DyQ8&t=767s) questions about this type of content, [12:50](https://www.youtube.com/watch?v=HvboD89DyQ8&t=770s) write a query for these tables. And so, [12:52](https://www.youtube.com/watch?v=HvboD89DyQ8&t=772s) it gives us a little bit of orientation [12:54](https://www.youtube.com/watch?v=HvboD89DyQ8&t=774s) to an agent to understand how to use how [12:57](https://www.youtube.com/watch?v=HvboD89DyQ8&t=777s) we use our tools, how we've organized [12:59](https://www.youtube.com/watch?v=HvboD89DyQ8&t=779s) linear in Snowflake, and give some [13:03](https://www.youtube.com/watch?v=HvboD89DyQ8&t=783s) guidance on where to find information. [13:06](https://www.youtube.com/watch?v=HvboD89DyQ8&t=786s) What has been really [13:08](https://www.youtube.com/watch?v=HvboD89DyQ8&t=788s) surprising about this is we actually now [13:10](https://www.youtube.com/watch?v=HvboD89DyQ8&t=790s) use this MCP server across a lot of [13:12](https://www.youtube.com/watch?v=HvboD89DyQ8&t=792s) other different internal tools. [13:15](https://www.youtube.com/watch?v=HvboD89DyQ8&t=795s) We open it up for people to query [13:17](https://www.youtube.com/watch?v=HvboD89DyQ8&t=797s) directly from Slack [13:19](https://www.youtube.com/watch?v=HvboD89DyQ8&t=799s) to do data or customer analysis. And so, [13:21](https://www.youtube.com/watch?v=HvboD89DyQ8&t=801s) while we initially kind of envisioned [13:23](https://www.youtube.com/watch?v=HvboD89DyQ8&t=803s) that this would just be a way to connect [13:25](https://www.youtube.com/watch?v=HvboD89DyQ8&t=805s) our agent or coding agent to all of our [13:27](https://www.youtube.com/watch?v=HvboD89DyQ8&t=807s) systems. Uh this has been an incredible [13:30](https://www.youtube.com/watch?v=HvboD89DyQ8&t=810s) piece of leverage for a lot of our [13:32](https://www.youtube.com/watch?v=HvboD89DyQ8&t=812s) internal teams, and we're building a lot [13:34](https://www.youtube.com/watch?v=HvboD89DyQ8&t=814s) of other tools on top of this MCP [13:37](https://www.youtube.com/watch?v=HvboD89DyQ8&t=817s) gateway. So, if you're just getting [13:38](https://www.youtube.com/watch?v=HvboD89DyQ8&t=818s) started with thinking about a software [13:40](https://www.youtube.com/watch?v=HvboD89DyQ8&t=820s) factory, it's well worth your time to [13:43](https://www.youtube.com/watch?v=HvboD89DyQ8&t=823s) invest in an internal MCP gateway server [13:46](https://www.youtube.com/watch?v=HvboD89DyQ8&t=826s) that both connects all your tools, [13:49](https://www.youtube.com/watch?v=HvboD89DyQ8&t=829s) has descriptions that can tell the agent [13:51](https://www.youtube.com/watch?v=HvboD89DyQ8&t=831s) how to use those tools and how you [13:53](https://www.youtube.com/watch?v=HvboD89DyQ8&t=833s) specifically organize the information in [13:55](https://www.youtube.com/watch?v=HvboD89DyQ8&t=835s) them, and I think you'll find that that [13:58](https://www.youtube.com/watch?v=HvboD89DyQ8&t=838s) ends up being useful in a lot of other [14:00](https://www.youtube.com/watch?v=HvboD89DyQ8&t=840s) use cases. [14:06](https://www.youtube.com/watch?v=HvboD89DyQ8&t=846s) Like I mentioned, we try to automate [14:09](https://www.youtube.com/watch?v=HvboD89DyQ8&t=849s) other parts of our software development. [14:11](https://www.youtube.com/watch?v=HvboD89DyQ8&t=851s) Um we have like bug [14:14](https://www.youtube.com/watch?v=HvboD89DyQ8&t=854s) bug requests that come in through Slack. [14:17](https://www.youtube.com/watch?v=HvboD89DyQ8&t=857s) Uh so, TARS through webhooks can listen [14:18](https://www.youtube.com/watch?v=HvboD89DyQ8&t=858s) to those and take a first pass at [14:21](https://www.youtube.com/watch?v=HvboD89DyQ8&t=861s) triaging and implementing opening PRs to [14:24](https://www.youtube.com/watch?v=HvboD89DyQ8&t=864s) fix bugs. [14:25](https://www.youtube.com/watch?v=HvboD89DyQ8&t=865s) Uh we have a lot of our customers uh in [14:28](https://www.youtube.com/watch?v=HvboD89DyQ8&t=868s) shared channels in Slack, as well. [14:30](https://www.youtube.com/watch?v=HvboD89DyQ8&t=870s) Uh we found TARS to be pretty useful to [14:33](https://www.youtube.com/watch?v=HvboD89DyQ8&t=873s) uh triage [14:34](https://www.youtube.com/watch?v=HvboD89DyQ8&t=874s) support requests that we're getting [14:36](https://www.youtube.com/watch?v=HvboD89DyQ8&t=876s) through them, because it can look at the [14:38](https://www.youtube.com/watch?v=HvboD89DyQ8&t=878s) code, it can often understand like where [14:40](https://www.youtube.com/watch?v=HvboD89DyQ8&t=880s) the customer might be running into [14:42](https://www.youtube.com/watch?v=HvboD89DyQ8&t=882s) problems with our product uh from the [14:44](https://www.youtube.com/watch?v=HvboD89DyQ8&t=884s) code perspective. [14:46](https://www.youtube.com/watch?v=HvboD89DyQ8&t=886s) Um I showed you an example of how we [14:48](https://www.youtube.com/watch?v=HvboD89DyQ8&t=888s) kind of organize major features around [14:50](https://www.youtube.com/watch?v=HvboD89DyQ8&t=890s) this. [14:51](https://www.youtube.com/watch?v=HvboD89DyQ8&t=891s) Uh and then the last place I think is [14:53](https://www.youtube.com/watch?v=HvboD89DyQ8&t=893s) really exciting, we're all trying to get [14:55](https://www.youtube.com/watch?v=HvboD89DyQ8&t=895s) to the self-improving software, or the [14:57](https://www.youtube.com/watch?v=HvboD89DyQ8&t=897s) self-driving software. So, we're using [14:59](https://www.youtube.com/watch?v=HvboD89DyQ8&t=899s) uh TARS to actually build out our own uh [15:02](https://www.youtube.com/watch?v=HvboD89DyQ8&t=902s) sandbox infrastructure. [15:05](https://www.youtube.com/watch?v=HvboD89DyQ8&t=905s) So, we want to move off of some of the [15:07](https://www.youtube.com/watch?v=HvboD89DyQ8&t=907s) sandboxes as a service and own that [15:09](https://www.youtube.com/watch?v=HvboD89DyQ8&t=909s) infrastructure layer ourselves. Um [15:13](https://www.youtube.com/watch?v=HvboD89DyQ8&t=913s) Some of the reasons we have for that is [15:15](https://www.youtube.com/watch?v=HvboD89DyQ8&t=915s) we want really deep control over the [15:18](https://www.youtube.com/watch?v=HvboD89DyQ8&t=918s) session information, and [15:20](https://www.youtube.com/watch?v=HvboD89DyQ8&t=920s) uh be able to move workloads around [15:23](https://www.youtube.com/watch?v=HvboD89DyQ8&t=923s) different parts of our infrastructure. [15:25](https://www.youtube.com/watch?v=HvboD89DyQ8&t=925s) Uh so what we're building out next is [15:27](https://www.youtube.com/watch?v=HvboD89DyQ8&t=927s) our memory layer and we want this to be [15:29](https://www.youtube.com/watch?v=HvboD89DyQ8&t=929s) an evergreen context uh about what every [15:34](https://www.youtube.com/watch?v=HvboD89DyQ8&t=934s) person in the company is working on, [15:36](https://www.youtube.com/watch?v=HvboD89DyQ8&t=936s) what team do they sit in, what products [15:38](https://www.youtube.com/watch?v=HvboD89DyQ8&t=938s) are they responsible for, and then at an [15:40](https://www.youtube.com/watch?v=HvboD89DyQ8&t=940s) organization level, what is the [15:42](https://www.youtube.com/watch?v=HvboD89DyQ8&t=942s) semantics about Work OS and how we do [15:44](https://www.youtube.com/watch?v=HvboD89DyQ8&t=944s) our work. And this we want to be able to [15:46](https://www.youtube.com/watch?v=HvboD89DyQ8&t=946s) plug into both our software factory, but [15:49](https://www.youtube.com/watch?v=HvboD89DyQ8&t=949s) lift that context out of our factory and [15:51](https://www.youtube.com/watch?v=HvboD89DyQ8&t=951s) into our other AI tools as well. [15:57](https://www.youtube.com/watch?v=HvboD89DyQ8&t=957s) So, I think this is kind of our our [16:00](https://www.youtube.com/watch?v=HvboD89DyQ8&t=960s) major takeaway of where we've seen a lot [16:03](https://www.youtube.com/watch?v=HvboD89DyQ8&t=963s) of folks invest effort just around [16:05](https://www.youtube.com/watch?v=HvboD89DyQ8&t=965s) outputs and where we think we can [16:06](https://www.youtube.com/watch?v=HvboD89DyQ8&t=966s) actually get that human exponential [16:09](https://www.youtube.com/watch?v=HvboD89DyQ8&t=969s) value out of our factory. [16:11](https://www.youtube.com/watch?v=HvboD89DyQ8&t=971s) Can we be actually delivering values to [16:14](https://www.youtube.com/watch?v=HvboD89DyQ8&t=974s) our customers? We think a lot about [16:16](https://www.youtube.com/watch?v=HvboD89DyQ8&t=976s) shipping and customer impact and so we [16:18](https://www.youtube.com/watch?v=HvboD89DyQ8&t=978s) want to encode that into their our [16:20](https://www.youtube.com/watch?v=HvboD89DyQ8&t=980s) software factory itself. Um certainly [16:24](https://www.youtube.com/watch?v=HvboD89DyQ8&t=984s) worried about, you know, introducing [16:26](https://www.youtube.com/watch?v=HvboD89DyQ8&t=986s) instability into our products. So, all [16:28](https://www.youtube.com/watch?v=HvboD89DyQ8&t=988s) right, can we measure [16:30](https://www.youtube.com/watch?v=HvboD89DyQ8&t=990s) uh defect rate and other [16:32](https://www.youtube.com/watch?v=HvboD89DyQ8&t=992s) um [16:33](https://www.youtube.com/watch?v=HvboD89DyQ8&t=993s) you know, time to recovery uh metrics? [16:36](https://www.youtube.com/watch?v=HvboD89DyQ8&t=996s) Um and then a lot of anecdotal, like we [16:38](https://www.youtube.com/watch?v=HvboD89DyQ8&t=998s) want to see our engineers using TARS as [16:42](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1002s) a sign that this is giving them value [16:44](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1004s) and accelerating their work. We want to [16:47](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1007s) see that they're moving off electing to [16:48](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1008s) move out of their local harness and into [16:51](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1011s) a cloud sandbox. [16:58](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1018s) And then the real goal is we can use [17:00](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1020s) this information because we own the [17:01](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1021s) infrastructure, we can see what's [17:03](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1023s) happening in the infrastructure. We use [17:05](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1025s) this information to self-improve the [17:06](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1026s) factory. [17:08](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1028s) We want it to learn, we want it to get [17:09](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1029s) better about how it writes code. [17:13](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1033s) We want to see where um our engineers [17:17](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1037s) can improve their skills. There's so [17:19](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1039s) much moving quickly in AI. It's kind of [17:22](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1042s) like a constant race to keep up with the [17:24](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1044s) latest [17:25](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1045s) um [17:26](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1046s) you know, tips and techniques. [17:28](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1048s) We can actually point agents at sessions [17:30](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1050s) and see where are the gaps and how [17:32](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1052s) people are using [17:34](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1054s) our factory. Where is there a skill that [17:36](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1056s) we should be building because the agent [17:38](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1058s) made a mistake. What skills are no [17:40](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1060s) longer relevant? Maybe the code has [17:43](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1063s) changed to a degree where a skill that [17:45](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1065s) we wrote six months ago is now obsolete. [17:48](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1068s) So, that continues verification. [17:50](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1070s) Semi-online [17:52](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1072s) is where we kind of see a big advantage [17:54](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1074s) to actually owning all the [17:56](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1076s) infrastructure ourselves. [18:02](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1082s) So, [18:04](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1084s) say it for the third time, it's worth [18:05](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1085s) repeating. [18:07](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1087s) Sandboxes are great. [18:09](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1089s) It's cool to run an agent and have it [18:10](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1090s) open a PR. [18:12](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1092s) We really think about our software [18:13](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1093s) engineering practices and processes and [18:15](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1095s) we want to encode those in automation. [18:21](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1101s) I'm Ryan. I'm one of the engineers here [18:22](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1102s) at WorkOS. We have a booth right [18:24](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1104s) opposite the speaking area. If you are [18:26](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1106s) building software factories, I would [18:27](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1107s) love to talk to you. I would really love [18:29](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1109s) to hear how you're handling [18:30](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1110s) authorization. This is something I don't [18:32](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1112s) talk about because we haven't figured it [18:34](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1114s) out yet for ourselves, but maybe you all [18:37](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1117s) you all have some insights. So, please [18:38](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1118s) come by the booth, talk to me, would [18:40](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1120s) love to chat. Thank you. [18:55](https://www.youtube.com/watch?v=HvboD89DyQ8&t=1135s) >> [music]