Contents
  1. Customer delivery and product learning
  2. Discovery of the operating problem
  3. Workflow and decision authority
  4. Value agreement and delivery boundaries
  5. Customer variation and reusable capability
  6. Integration into existing work
  7. Operational acceptance
  8. Adoption and realized customer value
  9. Sustainable operating ownership
  10. Field evidence and product decisions
  11. Check understanding
  12. Open questions
  13. Selected talks
  14. References
  15. Talk library
← All topics

Forward Deployed Engineering

Forward deployed engineering connects software delivery to the work a customer needs to accomplish. Its central challenge is establishing a useful, authorized, maintainable change under real operating constraints. The resulting evidence serves two purposes: improving that customer’s workflow and informing what the shared product should become.

Customer delivery and product learning

Forward deployed engineering (FDE) is hands-on engineering close to customer operations: understanding a problem, implementing a solution, and learning from its use. Organizational arrangements vary. The AWS delivery account, for example, places embedded delivery capability within partner organizations and identifies customer self-sufficiency as an intended outcome.

The mandate must specify how FDEs collaborate with customer operators, product engineering, support, and commercial teams. Operators understand daily work; commercial commitments establish expectations; engineering and support need explicit implementation and operating responsibilities. These boundaries cannot be inferred from the title: historical FDE work has included platform operation, integration, custom solutions, and customer enablement.

Two destinations for field evidence

Customer improvement and product improvement are distinct outputs.

The two loops are unrolled: operating evidence informs a revised customer workflow and shared improvements tested in subsequent delivery.
Read the diagram as text
  • Customer problem.
  • Implementation.
  • Operating evidence.
  • Revised workflow.
  • Product decision.
  • Shared improvement.
  • Subsequent deployment.
  • Customer problemImplementation: Shapes.
  • ImplementationOperating evidence: Produces.
  • Operating evidenceRevised workflow: Local improvement.
  • Operating evidenceProduct decision: Reusable need.
  • Product decisionShared improvement: If prioritized.
  • Shared improvementSubsequent deployment: Applied.

Two outputs make the delivery mandate concrete.

  • Customer deliveryA working agent that fits the customer’s process and can be understood and maintained.
  • Product learningChanges that make subsequent deployments easier, including replacing recurring bespoke work with supported capabilities.

An illustrative support-triage engagement makes these responsibilities tangible. A support team requests automatic assignment of incoming requests. The intended outcome spans intake, triage, resolution, and recording the result; assigning a category is only one step. This example supplies operating decisions, not customer performance measurements.

Discovery of the operating problem

Discovery investigates users, work, constraints, and desired outcomes before committing to a solution. A requested feature is a starting hypothesis. The GOV.UK discovery guide explicitly allows discovery to end with a different intervention or a decision not to build.

Contextual inquiry combines observation of everyday work with questions about what is happening. Follow recent cases using the worker’s actual records and tools, including exceptions and workarounds. Asking for explanations reveals reasoning but can change behavior; silent observation preserves more natural activity. The contextual research method makes that tradeoff explicit.

A discovery ledger for the support example separates what happened from what it might mean.
ObservationInterpretationAssumption to testDisconfirming evidence
A request moves between queues.Ownership may be unclear.Better classification would prevent transfers.Correctly classified requests also move.
An operator copies account details.Missing context adds work.The source records are accessible.Access restrictions prevent retrieval.

Jobs to Be Done frames discovery around progress sought in particular circumstances. For the support operator, the job may be finding an accountable resolver without losing the customer’s explanation. That includes functional progress and social concerns, such as avoiding an unsupported promise to another team.

A testable problem statement names the affected worker, failure, and consequence: operators cannot identify the responsible team when account context is missing, causing transfers and repeated investigation. This directs discovery toward context availability and ownership rules, rather than prematurely committing to a classifier.

The intervention should survive comparison with simpler alternatives.

  • Repair the processIf teams disagree about ownership, establish responsibility before automating assignment.
  • Use deterministic behaviorIf account type uniquely determines the destination, a rule may be easier to explain and maintain.
  • Use model assistanceInterpret varied request language where that helps, while preserving review for costly mistakes.

Workflow and decision authority

A workflow comprises the activities and decisions carrying work toward an outcome. It includes mental tasks, waiting, parallel activity, and exchanges between organizations—not just API calls. The AHRQ definition makes human work part of the system boundary.

A stakeholder is a person, team, or organization with an interest in the system and whether it meets their needs. Purchasers and direct users are only part of that population. Decision rights specify which stakeholders can authorize particular changes.

Follow the request across handoffs

Example

Triage speed leaves downstream waiting visible.

An example request branches on missing information and resolver authority. Each terminal record must distinguish resolved work from pending information or escalation; recording alone does not mean resolution.
Read the diagram as text
  • Intake. Support: capture request.
  • Check information. Operator: inspect records.
  • Await information. Customer response pending.
  • Assign destination. Operator: apply ownership policy.
  • Resolver queue. Specialist capacity required.
  • Escalate. Supervisor decision pending.
  • Record actual status.
  • IntakeCheck information: Request received.
  • Check informationAwait information: Incomplete.
  • Check informationAssign destination: Sufficient.
  • Assign destinationResolver queue: Assigned.
  • Resolver queueEscalate: Beyond authority.
  • Resolver queueRecord actual status: Resolved.
  • Await informationRecord actual status: Pending.
  • EscalateRecord actual status: Escalated.
Assign decisions explicitly; one person may hold several roles.
RoleContribution or authority
Daily usersDemonstrate real cases and explain whether proposed outcomes fit the work.
Process ownerAccept workflow changes and assign routine exception handling.
SponsorAuthorize commitments and resolve priorities beyond the delivery team’s authority.
Data and security approversAuthorize access and restrictions; user enthusiasm supplies neither.
Integration ownerMaintain data meaning, interfaces, and support contacts.
Support operatorOwn incident response, escalation, and authority to suspend unsafe operation.

Map observed practice alongside the documented process, then validate discrepancies with participants. A swimlane map assigns activities to responsible roles; the figure instead names owners in node details. Queues and handoffs expose where work accumulates, but a process map alone establishes neither time savings nor value.

Conflicting incentives need resolution before implementation. A sponsor may want more automation while operators inherit checking work and support inherits failures. Ask whether the receiving roles have time and authority to perform their new duties. Agreement to a demonstration is not an agreement to operate its result.

Value agreement and delivery boundaries

A value hypothesis states which change should improve which outcome for which work. Task correctness, workflow performance, and business value are distinct: correct routing might reduce transfers, but shorter resolution requires the destination team to act. Define success before selecting a model, using actual domain examples rather than public benchmark rankings.

A guardrail is an outcome that must remain acceptable while another improves. For triage, resolution speed might improve only while routing errors, repeated transfers, and operator burden remain within agreed limits. Guardrails constrain optimization; maximizing a usage count or minimizing human involvement is not the objective.

A proposed delivery agreement turns ambition into reviewable commitments.
AgreementSupport-triage application
Population and boundaryOne named queue and defined request types; exclude account changes and customer messages.
Baseline and windowMeasure existing routing, resolution, transfers, and total operator effort over an agreed period.
AcceptanceRecord outcome targets, guardrail limits, evidence requirements, and the accepting owner before rollout.
Dependencies and decisionsName access, integration, and staffing owners; specify who may expand, revise, or stop delivery.

A vertical slice is a narrow release spanning a complete user outcome. Here it could read an eligible request, propose a destination, obtain operator confirmation, and record the accepted assignment. That is more useful than separately completing a classifier, dashboard, and connector without joining them into usable work. Story mapping organizes such releases across activities and exceptions.

Evaluations are systematic checks against intended-use criteria. They should distinguish the agent’s claimed success from the resulting environment state. Evals develops scenario selection and judgment methods; locally, the acceptance agreement determines which outcomes matter and who judges them.

Constraints change the deployment boundary.
ConstraintDelivery decision
Data or access unavailableSecure the dependency, narrow the population, or stop.
Restricted deployment or legacy interfaceAgree an operable integration before promising automation.
Latency or cost exceeds the feature budgetReassess candidates against explicit acceptable-output criteria.
Approval lead time or staffing exceeds the planRevise commitments; technical feasibility does not supply operating capacity.

A new request changes more than the implementation backlog. Extending assignment into automatic resolution introduces new behavior, consequences, and acceptance evidence. Restate the user, need, reason, and testable acceptance criteria before treating that extension as part of the existing commitment.

Customer variation and reusable capability

Two customers can request automatic routing while requiring different behavior. One permits assignment by account tier; another requires supervisor approval for the same tier. Different record schemas create a separate integration difference. Shared wording does not establish shared policy, while different wording can conceal the same underlying capability.

Separate kinds of variation before choosing an implementation boundary.
KindMeaningExample
ConfigurationValues within supported behavior.Queue identifiers and permitted categories.
AdapterTranslation to a customer interface.Map account identifiers and assignment fields.
Bespoke logicBehavior maintained for a particular customer.A locally required approval sequence.
Product capabilitySupported behavior reusable across customers.A configurable policy-aware routing component.
Platform serviceShared capability serving engineering teams.A maintained integration or deployment service.

A practical reuse decision weighs recurrence, policy meaning, interface stability, maintenance burden, strategic fit, and a willing owner. This is an engineering judgment, not a universal scoring rule. Generalizing unstable policy can create more exceptions than it removes; retaining a bounded local implementation can be the more maintainable choice.

Platform engineering builds shared capabilities and interfaces for multiple engineering users. A repeated customer need becomes a platform candidate only when that shared service boundary is useful. AI Platform Engineering covers its architecture and operating model; field delivery should not silently inherit responsibility for operating a new internal platform.

Reuse is only one possible disposition.

  • Keep localPreserve differing requirements rather than forcing them into one ambiguous behavior.
  • Change the processRemove an unnecessary handoff instead of automating it.
  • DeclineReject a request when its expected benefit does not justify its error costs and downstream effects.

Integration into existing work

A system of record is the designated authoritative source for a particular business fact. A data contract records the producer–consumer agreement around that data. Establish field meanings, identifiers, freshness expectations, quality, ownership, and support contacts—not just JSON types. The Open Data Contract Standard supplies a broader checklist; completing it does not prove compliance.

Workflow automation coordinates activities, state, and people through software. Keep explicit control flow, tool boundaries, and information restrictions in code; use model assistance where interpretation is useful. Workflow Automation develops execution and recovery patterns. The FDE’s local responsibility is agreeing how those mechanisms fit the customer’s work.

Proposal, authority, and external outcome

Example

A proposal is not a committed change.

Proposed design: rejection prevents execution. A timeout can leave the external outcome unknown; the recovery owner checks authoritative status before a retry under an idempotent contract.
Read the diagram as text
  • Read source.
  • Model proposal.
  • Validate permissions and fields.
  • Operator review.
  • Write record.
  • Verify outcome.
  • No write.
  • Recovery owner.
  • Read sourceModel proposal: Data: source context.
  • Model proposalValidate permissions and fields: Data: proposed fields.
  • Validate permissions and fieldsOperator review: Control: valid and permitted.
  • Validate permissions and fieldsNo write: Control: invalid or denied.
  • Operator reviewWrite record: Control: approved.
  • Operator reviewNo write: Control: rejected.
  • Write recordVerify outcome: Data: operation status.
  • Verify outcomeRecovery owner: Control: unknown or failed.

Integration discovery must expose what a demonstration conceals.

  • Access and test conditionsObtain approved representative fixtures and test the real permission boundary. Valid arguments do not authorize an action; server access controls remain necessary.
  • Manual implementation laborRecord data preparation, copied records, manual launches, corrections, and exception handling. These are delivery or operating costs even when the interface appears automatic.
  • Instructions and acceptanceState intended behavior directly, then preserve the criteria behind manual tuning as evaluations. Examples and graders do not replace an explicit behavioral specification.

Human review means inspecting a proposed action before deciding whether it may proceed. Show concrete parameters, not merely a tool name, and enforce the decision in the execution path. An approval interface that the agent can bypass does not establish authority.

Customer-system integration also translates meaning. In the VoiceOps account, generated customer intent and resolution status are mapped into customer relationship management records, with operators validating and editing prepopulated fields. That illustrates a useful workflow placement, but does not establish that every downstream write waits for confirmation.

Operational acceptance

Operational acceptance applies the delivery agreement to representative work and consequential exceptions. Customer operators should judge outcomes and record reasons for rejection. Those explanations expose domain criteria that a bare approval rate hides, and can become reusable failure categories.

Review quality itself needs testing. In a proctoring study, trained reviewers accepted some fabricated AI alerts on legitimate historical sessions. Later evidence-focused guidelines accompanied improved rejection, although different sessions and timing prevent attributing the change solely to those guidelines. A review step is an imperfect safeguard whose behavior depends on its design.

Permission changes with evidence

Example

The deployment persists while operating permission changes.

1 / 4 · Current: demonstration

Selected cases establish possibility.

Prior statuses remain history, not simultaneous permissions. Shadow operation requires blocked external effects; copied inference requests alone do not isolate agent writes.
Read the diagram as text
  • Triage deployment.
  • Demonstration only.
  • Shadow only. Live inputs; no live changes.
  • Limited reliance.
  • Suspended.
  • Triage deploymentDemonstration only: Initial restriction.
  • Demonstration onlyShadow only: Isolation approved.
  • Shadow onlyLimited reliance: Acceptance authorized.
  • Limited relianceSuspended: Stop condition.
  1. Current: demonstration. Selected cases establish possibility. Active: Triage deployment, Demonstration only. New: Triage deployment, Demonstration only.
  2. Current: shadow. Compare without changing work. Active: Triage deployment, Demonstration only, Shadow only. New: Shadow only.
  3. Current: limited. Owner authorizes bounded reliance. Active: Triage deployment, Demonstration only, Shadow only, Limited reliance. New: Limited reliance.
  4. Current: suspended. Intervention blocks further actions. Active: Triage deployment, Demonstration only, Shadow only, Limited reliance, Suspended. New: Suspended.
Proposed acceptance cases test operating consequences, not just fluent responses.
ConditionRequired observation
Missing accessNo unauthorized read or write; the request reaches an accountable operator.
Stale recordThe system avoids treating obsolete information as current authority.
Ambiguous requestUnresolved uncertainty reaches review rather than an invented destination.
Unavailable integrationThe workflow reports incomplete work instead of claiming completion.
Incorrect proposalReviewers can detect and reject it before the external effect.

Limited rollout permits reliance for a restricted population after acceptance; wider reliance remains conditional on outcomes and operating capacity. Rollback restores an earlier software or configuration version for future work. It cannot retract an already delivered harmful instruction, so the acceptability of live exposure must be decided before relying on reactive monitoring.

Rehearse recovery with the receiving operator. Compensation is a new business action that counters completed work, not an erasure of history; it can fail and may not restore the original state. Record progress and escalate irrecoverable effects. Evals covers scenario coverage; acceptance additionally requires authority and practical ability to respond.

Readiness includes intervention that still works after deployment.

  • Reach active workCheck shutdown controls at subsequent decisions, including inside spawned agents; a session-start check leaves ongoing work unaffected.
  • Preserve the reason for changesRecord which failure prompted a prompt revision and which behavior it should correct.

Adoption and realized customer value

Adoption is sustained, appropriate use in the intended workflow. It requires workable interactions, confidence, accountability, and allocation of duties. Normalization Process Theory explains why introducing a tool can add work when people repeat its output checks rather than replace old activities.

Non-use needs a cause before it needs a remedy. Separate lack of exposure or access from awkward workflow fit, justified rejection of unreliable outputs, abandonment after failed attempts, and continued dependence on the implementer. Observe cases and ask for reasons; forcing acceptance can suppress useful evidence about the system.

Compare removed work with newly inherited duties.
RoleWork reducedWork that remains or appears
Support operatorTyping summaries and selecting fields.Checking evidence, correcting fields, confirming results.
Operational supportSome manual investigation.Monitoring failures, intervention, and recovery.
Implementation teamRepeated manual preparation after automation.Integration maintenance and unresolved exception work.

Measure across all eligible work, including rejected suggestions, failures, manual assistance, and escalations. A support-assistant study assessed access to assistance through resolved issues per hour alongside handling time, resolution, and customer satisfaction; its main analysis was not restricted to accepted suggestions. That avoids selecting only successful interactions as the population.

The VoiceOps speaker reported average after-call administrative work falling from 6.3 to 3.1 minutes. These are post-call work averages, not call duration or model latency. Without sample size, observation window, or comparison controls, the account illustrates a measurable workflow target but cannot establish the causal saving for another deployment.

A value ledger keeps operational improvement separate from financial realization.
ClaimEvidence needed
Time releasedComparable work takes less total effort, including checking and rework.
Usable capacityReleased time supports additional valuable work without worsening outcomes.
Spending reductionActual expenditure falls; spare capacity alone does not establish this.
Net benefitSubtract implementation, integration, review, support, and recurring operating costs without double counting.

Attribution depends on a credible comparison. Representative observation, phased introduction, and suitable comparison groups can help, but changing workload, selected participants, and simultaneous process changes remain threats. METR’s follow-up productivity work illustrates how participation and task selection can undermine an estimate. Evals develops the methods; report uncertainty rather than converting a pilot association into a guaranteed return.

Expansion is justified when the intended operation improves within its guardrails and the added capacity has a valuable use. Faster engineering or more generated output remains an intermediate result when another part of the business is the binding constraint. Revise or stop when review burden and ongoing costs consume the expected gain.

Sustainable operating ownership

Handoff is an accepted transfer of operating knowledge, authority, and responsibility. Google’s SRE engagement account provides a useful analogy: documentation, system instruction, access, and hands-on exercises support progressive transfer while the development team remains available. Sending documents alone does not establish receiving capability.

A runbook describes how to perform a recurring operational task or respond to a known failure. It needs prerequisites, diagnostic evidence, actions, and escalation contacts. Incident findings should update these instructions and response workflows, so the handoff remains connected to operating experience rather than becoming a frozen description.

Name receiving owners and demonstrate their readiness.
ResponsibilityReceiving ownerEvidence of readiness
Routine work and exceptionsCustomer operationsComplete ordinary and exceptional cases independently.
Incidents and interventionService supportDiagnose failure, suspend work, and activate escalation.
AccessAuthorized administratorManage approved credentials and understand revocation limits.
CostsNamed service ownerAccount for provider, infrastructure, and support costs.
Evaluations and upgradesMaintaining engineering teamOwn categorized cases and assess changed behavior.
Configuration changesDesignated change ownerReview definition changes separately from approving runtime actions.

A practical acceptance exercise has the receiving team run routine work, diagnose an unavailable dependency, and recover or escalate without the original implementer directing each step. Record missing knowledge and authority, assign maintainers for customer-specific documentation and known limits, then repeat the failed portions. This is a proposed handoff test, not a claim about every FDE engagement.

The receiving owner may be a customer team or an internal service team. Customer enablement can reduce repeated vendor-led work without eliminating all FDE responsibilities. Continuing support, withdrawal, and eventual retirement should therefore depend on accepted responsibilities and ongoing usefulness, rather than an assumption that every engagement ends with vendor departure.

Field evidence and product decisions

A useful field record contains the underlying problem, workflow context, observed frequency, workaround, outcome evidence, customer variation, and maintenance burden. Preserve the customer’s circumstances while grouping related needs. A request for another control may reveal a documentation gap, an unclear interaction, or a missing capability; implementing the requested control immediately can hide that distinction.

Productization turns a local solution into a supported capability with an owner and lifecycle. Decagon reports moving promotional-policy changes from custom engineering into configuration, debugging, and testing by its deployment team. The useful change is who can safely perform the work and what the product supports—not merely where the original implementation was written.

A finding has several possible destinations

Example

Recurrence prompts assessment, not automatic generalization.

Classify the underlying need before choosing local work, correction, or shared investment. Subsequent deployments test the chosen response.
Read the diagram as text
  • Field evidence.
  • Classify need.
  • Local or process change.
  • Documentation or fix.
  • Assess shared capability.
  • Retain local or decline.
  • Owned implementation.
  • Later-deployment checks.
  • Field evidenceClassify need: Underlying problem.
  • Classify needLocal or process change: Local circumstances.
  • Classify needDocumentation or fix: Existing behavior inadequate.
  • Classify needAssess shared capability: Recurring capability gap.
  • Assess shared capabilityRetain local or decline: Poor shared fit.
  • Assess shared capabilityOwned implementation: Accepted investment.
  • Local or process changeOwned implementation: Accepted change.
  • Documentation or fixOwned implementation: Accepted correction.
  • Owned implementationLater-deployment checks: Verify benefit.

Close the learning loop with evidence and an explicit disposition.

  • Preserve the failureTurn reviewed production failures into categorized regression cases with a maintaining owner.
  • Communicate the decisionReturn the rationale to field engineers, including when work stays local or the requested solution is declined.
  • Test subsequent deliveryCheck whether the supported capability reduces repeated work while preserving each customer’s requirements; retire duplicated local implementations only after that fit is established.

Open questions

  1. Reliable reuse boundaries remain difficult to establish before maintaining several deployments. Similar requests can hide incompatible policies, while excessive local work duplicates support effort. Progress would be demonstrated by a shared capability that preserves differing policies and measurably reduces later implementation and maintenance work.

  2. Net FDE value remains hard to attribute when pilots select willing users or unusually tractable work. Hidden assistance and changing workloads can inflate apparent gains. Progress requires outcomes across a predefined eligible population, including non-use and failed work, with implementation and operating effort recorded.

  3. Durable handoff remains uncertain when receiving teams rarely encounter serious failures. A successful exercise may not predict operation after personnel, policies, or integrations change. Progress would include repeated independent recovery exercises and documented responsibility changes after real incidents.

  4. Human review must balance catching consequential mistakes against introducing enough work to erase the benefit. Approvals can also become misleading feedback labels. Progress would show reviewers rejecting seeded errors while preserving usable workflow performance, with evidence kept separate from model-influenced agreement.

Follow the curated reading path through the speakers and demonstrations behind this entry.

Explore more talks

The rest of the library, beyond the curated path. Cited talks support this entry; reviewed transcripts were processed in full. Metadata candidates have not been reviewed as sources or verified as topic members.

20 matching talks

TalkSpeakerEventYear
Tuomas Artman, Gergely OroszAI Engineer Europe 20262026
Sandipan BhaumikAI Engineer Europe 20262026
Yegor Denisov-BlanchAI Engineer Code 20252025
Joel BeckerAI Engineer Code 20252025
Vaidas RazgaitisAI Engineer World's Fair 20262026
Rachel Lee Nabors (RL Nabors)AI Engineer World's Fair 20262026
Omar KhattabAI Engineer World's Fair 20252025
Liam McGarrigleAI Engineer Europe 20262026
Angel Ortmann LeeAI Engineer World's Fair 20262026
Phil HetzelAI Engineer Europe 20262026
Agents Need Feature Flags

Cited in this entry

Sachin GuptaAI Engineer World's Fair 20262026
Jared JoselowitzAI Engineer World's Fair 20262026
Eno ReyesAI Engineer World's Fair 20252025
Anish Agarwal, Matthew SchoenbauerAI Engineer World's Fair 20252025
Vasuman MozaAI Engineer World's Fair 20262026
Kevin BaiAI Engineer World's Fair 20262026
Pauline BrunetAI Engineer World's Fair 20262026
Eno ReyesAI Engineer World's Fair 20262026
Leo MehrAI Engineer World's Fair 20262026
Vinoo GaneshAI Engineer World's Fair 20262026

References

Coverage and source review
Processed transcripts
19 processed in full · 5 in the curated path
Automated source review
Passed
Metadata candidates
6 unreviewed; not verified topic membership
Corpus version
1bd8e407b26a07b33815594e1b2db5f41827119a2b3cb6fbf240f9fc571fc767

Automated review checks source support; it is not publication approval.

A synthesis of selected conference talks and technical references. Citations link to the source material; they do not imply that every talk on this subject is included.

  1. Introducing Forward Deployed Engineering for Partners: Winning the Future of Enterprise AI

    Plain-language FDE introduction and an example of organizational arrangements beyond a product vendor's internal implementation team.

  2. The Dirty Secret of Forward Deployed Engineering

    Use the speaker's 'FDE vintage' concept to ask which kinds of work a candidate has actually performed instead of treating the title as a uniform skill set.

  3. Introducing Agent Development: How Decagon is redefining forward deployment

    FDE operating loop, customization, customer ownership, and product learning; June 2026 first-person organizational account.

  4. You Can't Prompt the Room: The Last Skill AI Won't Replace

    Use a high-level story map to organize user stories beneath workflow stages, then select a first release spanning those stages.

  5. How the discovery phase works — GOV.UK Service Manual

    Problem discovery, disconfirming evidence, organizational readiness, and delivery boundaries.

  6. Contextual research and observation — GOV.UK Service Manual

    First-use explanation of contextual observation and practical discovery of actual customer work.

  7. Analyse a research session — GOV.UK Service Manual

    Discovery evidence records and the distinction between observed behavior and inferred meaning.

  8. Jobs to Be Done Theory — Christensen Institute

    Brief introduction to Jobs to Be Done as a discovery lens, rather than a separate marketing tutorial.

  9. User Needs + Defining Success — People + AI Guidebook

    Evidence that discovery may reject AI; local choice between automation, assistance, and deterministic behavior.

  10. What is workflow? — AHRQ Workflow Assessment for Health IT Toolkit

    Plain-language workflow definition that includes human decisions and parallel activity.

  11. Stakeholder — NIST CSRC Glossary

    First-use stakeholder explanation; glossary definitions attributed to NIST SP 800-160v1r1 and ISO/IEC/IEEE 15288:2015.

  12. You Can't Prompt the Room: The Last Skill AI Won't Replace

    Give subject matter experts access to customers and a role in deciding what gets built, without requiring everyone to become a product manager.

  13. Guidelines for Managing Projects: How to organise, plan and control projects

    Scope definition, stakeholder analysis, change control, and benefits realization sections; supports a lightweight customer delivery agreement.

  14. Tools — Model Context Protocol specification 2025-06-18

    Versioned primary protocol specification; tool definition, errors, and security considerations.

  15. Open Data Contract Standard v3.0.0

    First-use data-contract explanation and an authoritative checklist for integration conversations.

  16. NIST AI RMF: Monitoring, incident response, and recovery

    AI RMF 1.0, MEASURE 2.4 and 3.1–3.3; MANAGE 2.3–2.4 and 4.1–4.3. Delayed-label handling and operational sequence are explicitly engineering applications.

  17. Flowchart — AHRQ Workflow Assessment for Health IT Toolkit

    Current-state mapping and first-use explanation of swimlanes as lanes assigning activities to responsible roles.

  18. Collective Action — Normalization Process Theory

    Adoption as incorporation into everyday work; before-and-after role analysis and newly introduced checking burdens.

  19. The Production AI Playbook: Deploying Agents at Enterprise Scale

    Define business success and build a representative evaluation dataset before comparing models; reuse that dataset to assess provider upgrades.

  20. How to Quantify AI ROI in Software Engineering (Stanford Study / 120k Devs)

    The proposed framework increases a primary engineering-output metric while keeping guardrail metrics healthy rather than maximizing them.

  21. Story Mapping Quick Reference — Jeff Patton

    Workflow mapping and selection of a narrow but complete customer outcome.

  22. Demystifying evals for AI agents

    Primary engineering report; evaluation structure, grader types, and capability versus regression suites.

  23. Frontier results, on device - RL Nabors, Arize

    Use 'prototype big, deploy small': establish feasibility with a capable model, define acceptable outputs, then test candidates from smallest upward and select the SAGE model—'small and good enough.'

  24. You Can't Prompt the Room: The Last Skill AI Won't Replace

    Express the persona, need, and reason for a user story, add acceptance criteria that can yield tests, and connect related stories into a coherent specification.

  25. Meet the AI agent engineer — Sierra

    Concrete field implementation, customer-specific policy variation, systems-of-record integration, and reusable capability.

  26. CNCF Platforms White Paper

    Brief distinction between field implementation and shared platform investment; contextual link to /topics/ai-platform-engineering.

  27. On Engineering AI Systems that Endure The Bitter Lesson

    Put essential control flow, tool definitions, information-flow restrictions, and function composition in code where explicit structure is required.

  28. FinOps terminology: ownership, depreciation and utilization

    Capitalization; Depreciation; Fixed Cost; Cost Allocation; Total Cost of Ownership; Activity Based Costing; Shared cost.

  29. On Engineering AI Systems that Endure The Bitter Lesson

    Use natural-language definitions to state core behavior and evaluations to preserve the criteria behind repeated manual tuning; neither replaces the other.

  30. Building Your Own Secure AI Workflows: Human-in-the-Loop Automation with n8n

    Place approval in the execution path so the agent cannot bypass it by choosing to call the underlying tool directly.

  31. Building Your Own Secure AI Workflows: Human-in-the-Loop Automation with n8n

    Show the proposed action's concrete parameters in readable form, rather than only its tool name.

  32. VoiceOps-fying Low-Latency Intelligence Extraction from Messy Audio Streams — Dippu Kumar Singh

    Map model output to the customer's CRM schema and retain an operator verification step.

  33. The maturity phases of running evals

    During initial vibe checking, collect a good/bad judgment and a written justification from a human reviewer, preferably a subject matter expert.

  34. When Machines Mislead: Human Review of Erroneous AI Cheating Signals

    Primary study, methods, results, and limitations; supports testing whether human oversight actually detects incorrect proposals.

  35. Shipping AI to a Million Patients Without an A/B Test

    An already-delivered clinical utterance cannot be undone, so reactive rollout monitoring cannot substitute for evidence gathered before exposure.

  36. Compensating Transaction pattern

    Context and problem; Solution; Issues and considerations; travel-booking example.

  37. Agents Need Feature Flags

    Ship agent-wide and per-tool kill switches first, and ensure in-flight work checks them at the next decision point.

  38. The Production AI Playbook: Deploying Agents at Enterprise Scale

    Prompt change management should record the failure that motivated each change and the behavior the change is intended to correct.

  39. Generative AI at Work

    Real support-domain workflow and outcome measurement; version 2, deployment description and empirical analysis.

  40. VoiceOps-fying Low-Latency Intelligence Extraction from Messy Audio Streams — Dippu Kumar Singh

    Target after-call work (ACW): the speaker reports average administrative processing falling from 6.3 to 3.1 minutes after introducing AI summarization and extraction.

  41. The Government Efficiency Framework

    Customer-value accounting: time released, usable capacity, spending reductions, and implementation costs.

  42. We are Changing our Developer Productivity Experiment Design — METR

    Concrete limitations of customer-value attribution, eligible-work selection, and time accounting.

  43. How to Quantify AI ROI in Software Engineering (Stanford Study / 120k Devs)

    Engineering outcomes can provide a more interpretable AI-impact signal than revenue, but translating capacity into value requires explicit assumptions.

  44. The Evolving SRE Engagement Model — Google SRE

    Operational handoff as transfer of knowledge and responsibility, with training and transition support.

  45. Ship Production Software in Minutes, Not Months

    Treat the RCA as input to a learning cycle that updates runbooks and response workflows rather than as the final incident artifact.

  46. RFC 7009: OAuth 2.0 Token Revocation

    Sections 2.1 and 2.2 of RFC 7009; revocation-versus-undo is a consequence of this protocol scope.

  47. The Production AI Playbook: Deploying Agents at Enterprise Scale

    Turn reviewed failures into regression cases and give the growing evaluation library an owner and problem categories.

  48. Building Your Own Secure AI Workflows: Human-in-the-Loop Automation with n8n

    Runtime human review does not itself provide a change-review process for shared workflow definitions.

  49. The Dirty Secret of Forward Deployed Engineering

    Customer enablement can become an additional FDE responsibility, allowing customers to perform work previously done by deployed engineers.

  50. Taste & Craft: A Conversation with Tuomas Artman, CTO of Linear, and Gergely Orosz of The Pragmatic Engineer

    Investigate and group the underlying customer problems before choosing a solution, even when AI makes individual requests cheap to implement.

  51. How Forward Deployed Engineering is done at Cognition

    Cognition frames forward deployed engineering as expanding product-market fit through both customer delivery and feedback that changes the product.

  52. gRPC lifecycle: cancellation is not rollback

    RPC life cycle: Deadlines/Timeouts; RPC termination; Cancelling an RPC and its Warning.

  53. Shadow tests — Amazon SageMaker AI

    Concrete explanation of shadow operation and its role between offline testing and live reliance.

  54. How Forward Deployed Engineering is done at Cognition

    Use customer problems as a field-derived evaluation set, and distinguish recurring enterprise needs from individual exceptions before promoting workarounds into features.

  55. How Forward Deployed Engineering is done at Decagon

    Enterprise deployment includes configuring behavior, brand voice, permitted actions, and human handoffs, alongside backend integrations.

  56. How Forward Deployed Engineering is done at Decagon

    Decagon describes forward deployed engineering and product engineering as sharing the same organizational structure because enterprise pain points frequently become product features.

  57. Agents Need Feature Flags

    Flags require ongoing drills, lifecycle ownership, and interaction testing.

  58. Build AI Systems for Discernment, Not Approval - Angel Ortmann Lee, Duolingo

    The human-AI interaction loop is cyclical: interfaces that encourage rubber-stamping can turn model-influenced approvals into misleading evaluation and training labels.