Customer delivery and product learning
Forward deployed engineering (FDE) is hands-on engineering close to customer operations: understanding a problem, implementing a solution, and learning from its use. Organizational arrangements vary. The AWS delivery account, for example, places embedded delivery capability within partner organizations and identifies customer self-sufficiency as an intended outcome.
The mandate must specify how FDEs collaborate with customer operators, product engineering, support, and commercial teams. Operators understand daily work; commercial commitments establish expectations; engineering and support need explicit implementation and operating responsibilities. These boundaries cannot be inferred from the title: historical FDE work has included platform operation, integration, custom solutions, and customer enablement.
Two destinations for field evidence
Customer improvement and product improvement are distinct outputs.
Read the diagram as text
- Customer problem.
- Implementation.
- Operating evidence.
- Revised workflow.
- Product decision.
- Shared improvement.
- Subsequent deployment.
- Customer problem → Implementation: Shapes.
- Implementation → Operating evidence: Produces.
- Operating evidence → Revised workflow: Local improvement.
- Operating evidence → Product decision: Reusable need.
- Product decision → Shared improvement: If prioritized.
- Shared improvement → Subsequent deployment: Applied.
Two outputs make the delivery mandate concrete.
- Customer delivery — A working agent that fits the customer’s process and can be understood and maintained.
- Product learning — Changes that make subsequent deployments easier, including replacing recurring bespoke work with supported capabilities.
An illustrative support-triage engagement makes these responsibilities tangible. A support team requests automatic assignment of incoming requests. The intended outcome spans intake, triage, resolution, and recording the result; assigning a category is only one step. This example supplies operating decisions, not customer performance measurements.
Discovery of the operating problem
Discovery investigates users, work, constraints, and desired outcomes before committing to a solution. A requested feature is a starting hypothesis. The GOV.UK discovery guide explicitly allows discovery to end with a different intervention or a decision not to build.
Contextual inquiry combines observation of everyday work with questions about what is happening. Follow recent cases using the worker’s actual records and tools, including exceptions and workarounds. Asking for explanations reveals reasoning but can change behavior; silent observation preserves more natural activity. The contextual research method makes that tradeoff explicit.
| Observation | Interpretation | Assumption to test | Disconfirming evidence |
|---|---|---|---|
| A request moves between queues. | Ownership may be unclear. | Better classification would prevent transfers. | Correctly classified requests also move. |
| An operator copies account details. | Missing context adds work. | The source records are accessible. | Access restrictions prevent retrieval. |
Jobs to Be Done frames discovery around progress sought in particular circumstances. For the support operator, the job may be finding an accountable resolver without losing the customer’s explanation. That includes functional progress and social concerns, such as avoiding an unsupported promise to another team.
A testable problem statement names the affected worker, failure, and consequence: operators cannot identify the responsible team when account context is missing, causing transfers and repeated investigation. This directs discovery toward context availability and ownership rules, rather than prematurely committing to a classifier.
The intervention should survive comparison with simpler alternatives.
- Repair the process — If teams disagree about ownership, establish responsibility before automating assignment.
- Use deterministic behavior — If account type uniquely determines the destination, a rule may be easier to explain and maintain.
- Use model assistance — Interpret varied request language where that helps, while preserving review for costly mistakes.
Workflow and decision authority
A workflow comprises the activities and decisions carrying work toward an outcome. It includes mental tasks, waiting, parallel activity, and exchanges between organizations—not just API calls. The AHRQ definition makes human work part of the system boundary.
A stakeholder is a person, team, or organization with an interest in the system and whether it meets their needs. Purchasers and direct users are only part of that population. Decision rights specify which stakeholders can authorize particular changes.
Follow the request across handoffs
ExampleTriage speed leaves downstream waiting visible.
Read the diagram as text
- Intake. Support: capture request.
- Check information. Operator: inspect records.
- Await information. Customer response pending.
- Assign destination. Operator: apply ownership policy.
- Resolver queue. Specialist capacity required.
- Escalate. Supervisor decision pending.
- Record actual status.
- Intake → Check information: Request received.
- Check information → Await information: Incomplete.
- Check information → Assign destination: Sufficient.
- Assign destination → Resolver queue: Assigned.
- Resolver queue → Escalate: Beyond authority.
- Resolver queue → Record actual status: Resolved.
- Await information → Record actual status: Pending.
- Escalate → Record actual status: Escalated.
| Role | Contribution or authority |
|---|---|
| Daily users | Demonstrate real cases and explain whether proposed outcomes fit the work. |
| Process owner | Accept workflow changes and assign routine exception handling. |
| Sponsor | Authorize commitments and resolve priorities beyond the delivery team’s authority. |
| Data and security approvers | Authorize access and restrictions; user enthusiasm supplies neither. |
| Integration owner | Maintain data meaning, interfaces, and support contacts. |
| Support operator | Own incident response, escalation, and authority to suspend unsafe operation. |
Map observed practice alongside the documented process, then validate discrepancies with participants. A swimlane map assigns activities to responsible roles; the figure instead names owners in node details. Queues and handoffs expose where work accumulates, but a process map alone establishes neither time savings nor value.
Conflicting incentives need resolution before implementation. A sponsor may want more automation while operators inherit checking work and support inherits failures. Ask whether the receiving roles have time and authority to perform their new duties. Agreement to a demonstration is not an agreement to operate its result.
Value agreement and delivery boundaries
A value hypothesis states which change should improve which outcome for which work. Task correctness, workflow performance, and business value are distinct: correct routing might reduce transfers, but shorter resolution requires the destination team to act. Define success before selecting a model, using actual domain examples rather than public benchmark rankings.
A guardrail is an outcome that must remain acceptable while another improves. For triage, resolution speed might improve only while routing errors, repeated transfers, and operator burden remain within agreed limits. Guardrails constrain optimization; maximizing a usage count or minimizing human involvement is not the objective.
| Agreement | Support-triage application |
|---|---|
| Population and boundary | One named queue and defined request types; exclude account changes and customer messages. |
| Baseline and window | Measure existing routing, resolution, transfers, and total operator effort over an agreed period. |
| Acceptance | Record outcome targets, guardrail limits, evidence requirements, and the accepting owner before rollout. |
| Dependencies and decisions | Name access, integration, and staffing owners; specify who may expand, revise, or stop delivery. |
A vertical slice is a narrow release spanning a complete user outcome. Here it could read an eligible request, propose a destination, obtain operator confirmation, and record the accepted assignment. That is more useful than separately completing a classifier, dashboard, and connector without joining them into usable work. Story mapping organizes such releases across activities and exceptions.
Evaluations are systematic checks against intended-use criteria. They should distinguish the agent’s claimed success from the resulting environment state. Evals develops scenario selection and judgment methods; locally, the acceptance agreement determines which outcomes matter and who judges them.
| Constraint | Delivery decision |
|---|---|
| Data or access unavailable | Secure the dependency, narrow the population, or stop. |
| Restricted deployment or legacy interface | Agree an operable integration before promising automation. |
| Latency or cost exceeds the feature budget | Reassess candidates against explicit acceptable-output criteria. |
| Approval lead time or staffing exceeds the plan | Revise commitments; technical feasibility does not supply operating capacity. |
A new request changes more than the implementation backlog. Extending assignment into automatic resolution introduces new behavior, consequences, and acceptance evidence. Restate the user, need, reason, and testable acceptance criteria before treating that extension as part of the existing commitment.
Customer variation and reusable capability
Two customers can request automatic routing while requiring different behavior. One permits assignment by account tier; another requires supervisor approval for the same tier. Different record schemas create a separate integration difference. Shared wording does not establish shared policy, while different wording can conceal the same underlying capability.
| Kind | Meaning | Example |
|---|---|---|
| Configuration | Values within supported behavior. | Queue identifiers and permitted categories. |
| Adapter | Translation to a customer interface. | Map account identifiers and assignment fields. |
| Bespoke logic | Behavior maintained for a particular customer. | A locally required approval sequence. |
| Product capability | Supported behavior reusable across customers. | A configurable policy-aware routing component. |
| Platform service | Shared capability serving engineering teams. | A maintained integration or deployment service. |
A practical reuse decision weighs recurrence, policy meaning, interface stability, maintenance burden, strategic fit, and a willing owner. This is an engineering judgment, not a universal scoring rule. Generalizing unstable policy can create more exceptions than it removes; retaining a bounded local implementation can be the more maintainable choice.
Platform engineering builds shared capabilities and interfaces for multiple engineering users. A repeated customer need becomes a platform candidate only when that shared service boundary is useful. AI Platform Engineering covers its architecture and operating model; field delivery should not silently inherit responsibility for operating a new internal platform.
Reuse is only one possible disposition.
- Keep local — Preserve differing requirements rather than forcing them into one ambiguous behavior.
- Change the process — Remove an unnecessary handoff instead of automating it.
- Decline — Reject a request when its expected benefit does not justify its error costs and downstream effects.
Integration into existing work
A system of record is the designated authoritative source for a particular business fact. A data contract records the producer–consumer agreement around that data. Establish field meanings, identifiers, freshness expectations, quality, ownership, and support contacts—not just JSON types. The Open Data Contract Standard supplies a broader checklist; completing it does not prove compliance.
Workflow automation coordinates activities, state, and people through software. Keep explicit control flow, tool boundaries, and information restrictions in code; use model assistance where interpretation is useful. Workflow Automation develops execution and recovery patterns. The FDE’s local responsibility is agreeing how those mechanisms fit the customer’s work.
Proposal, authority, and external outcome
ExampleA proposal is not a committed change.
Read the diagram as text
- Read source.
- Model proposal.
- Validate permissions and fields.
- Operator review.
- Write record.
- Verify outcome.
- No write.
- Recovery owner.
- Read source → Model proposal: Data: source context.
- Model proposal → Validate permissions and fields: Data: proposed fields.
- Validate permissions and fields → Operator review: Control: valid and permitted.
- Validate permissions and fields → No write: Control: invalid or denied.
- Operator review → Write record: Control: approved.
- Operator review → No write: Control: rejected.
- Write record → Verify outcome: Data: operation status.
- Verify outcome → Recovery owner: Control: unknown or failed.
Integration discovery must expose what a demonstration conceals.
- Access and test conditions — Obtain approved representative fixtures and test the real permission boundary. Valid arguments do not authorize an action; server access controls remain necessary.
- Manual implementation labor — Record data preparation, copied records, manual launches, corrections, and exception handling. These are delivery or operating costs even when the interface appears automatic.
- Instructions and acceptance — State intended behavior directly, then preserve the criteria behind manual tuning as evaluations. Examples and graders do not replace an explicit behavioral specification.
Human review means inspecting a proposed action before deciding whether it may proceed. Show concrete parameters, not merely a tool name, and enforce the decision in the execution path. An approval interface that the agent can bypass does not establish authority.
Customer-system integration also translates meaning. In the VoiceOps account, generated customer intent and resolution status are mapped into customer relationship management records, with operators validating and editing prepopulated fields. That illustrates a useful workflow placement, but does not establish that every downstream write waits for confirmation.
Operational acceptance
Operational acceptance applies the delivery agreement to representative work and consequential exceptions. Customer operators should judge outcomes and record reasons for rejection. Those explanations expose domain criteria that a bare approval rate hides, and can become reusable failure categories.
Review quality itself needs testing. In a proctoring study, trained reviewers accepted some fabricated AI alerts on legitimate historical sessions. Later evidence-focused guidelines accompanied improved rejection, although different sessions and timing prevent attributing the change solely to those guidelines. A review step is an imperfect safeguard whose behavior depends on its design.
Permission changes with evidence
ExampleThe deployment persists while operating permission changes.
Selected cases establish possibility.
Read the diagram as text
- Triage deployment.
- Demonstration only.
- Shadow only. Live inputs; no live changes.
- Limited reliance.
- Suspended.
- Triage deployment → Demonstration only: Initial restriction.
- Demonstration only → Shadow only: Isolation approved.
- Shadow only → Limited reliance: Acceptance authorized.
- Limited reliance → Suspended: Stop condition.
- Current: demonstration. Selected cases establish possibility. Active: Triage deployment, Demonstration only. New: Triage deployment, Demonstration only.
- Current: shadow. Compare without changing work. Active: Triage deployment, Demonstration only, Shadow only. New: Shadow only.
- Current: limited. Owner authorizes bounded reliance. Active: Triage deployment, Demonstration only, Shadow only, Limited reliance. New: Limited reliance.
- Current: suspended. Intervention blocks further actions. Active: Triage deployment, Demonstration only, Shadow only, Limited reliance, Suspended. New: Suspended.
| Condition | Required observation |
|---|---|
| Missing access | No unauthorized read or write; the request reaches an accountable operator. |
| Stale record | The system avoids treating obsolete information as current authority. |
| Ambiguous request | Unresolved uncertainty reaches review rather than an invented destination. |
| Unavailable integration | The workflow reports incomplete work instead of claiming completion. |
| Incorrect proposal | Reviewers can detect and reject it before the external effect. |
Limited rollout permits reliance for a restricted population after acceptance; wider reliance remains conditional on outcomes and operating capacity. Rollback restores an earlier software or configuration version for future work. It cannot retract an already delivered harmful instruction, so the acceptability of live exposure must be decided before relying on reactive monitoring.
Rehearse recovery with the receiving operator. Compensation is a new business action that counters completed work, not an erasure of history; it can fail and may not restore the original state. Record progress and escalate irrecoverable effects. Evals covers scenario coverage; acceptance additionally requires authority and practical ability to respond.
Readiness includes intervention that still works after deployment.
- Reach active work — Check shutdown controls at subsequent decisions, including inside spawned agents; a session-start check leaves ongoing work unaffected.
- Preserve the reason for changes — Record which failure prompted a prompt revision and which behavior it should correct.
Adoption and realized customer value
Adoption is sustained, appropriate use in the intended workflow. It requires workable interactions, confidence, accountability, and allocation of duties. Normalization Process Theory explains why introducing a tool can add work when people repeat its output checks rather than replace old activities.
Non-use needs a cause before it needs a remedy. Separate lack of exposure or access from awkward workflow fit, justified rejection of unreliable outputs, abandonment after failed attempts, and continued dependence on the implementer. Observe cases and ask for reasons; forcing acceptance can suppress useful evidence about the system.
| Role | Work reduced | Work that remains or appears |
|---|---|---|
| Support operator | Typing summaries and selecting fields. | Checking evidence, correcting fields, confirming results. |
| Operational support | Some manual investigation. | Monitoring failures, intervention, and recovery. |
| Implementation team | Repeated manual preparation after automation. | Integration maintenance and unresolved exception work. |
Measure across all eligible work, including rejected suggestions, failures, manual assistance, and escalations. A support-assistant study assessed access to assistance through resolved issues per hour alongside handling time, resolution, and customer satisfaction; its main analysis was not restricted to accepted suggestions. That avoids selecting only successful interactions as the population.
The VoiceOps speaker reported average after-call administrative work falling from 6.3 to 3.1 minutes. These are post-call work averages, not call duration or model latency. Without sample size, observation window, or comparison controls, the account illustrates a measurable workflow target but cannot establish the causal saving for another deployment.
| Claim | Evidence needed |
|---|---|
| Time released | Comparable work takes less total effort, including checking and rework. |
| Usable capacity | Released time supports additional valuable work without worsening outcomes. |
| Spending reduction | Actual expenditure falls; spare capacity alone does not establish this. |
| Net benefit | Subtract implementation, integration, review, support, and recurring operating costs without double counting. |
Attribution depends on a credible comparison. Representative observation, phased introduction, and suitable comparison groups can help, but changing workload, selected participants, and simultaneous process changes remain threats. METR’s follow-up productivity work illustrates how participation and task selection can undermine an estimate. Evals develops the methods; report uncertainty rather than converting a pilot association into a guaranteed return.
Expansion is justified when the intended operation improves within its guardrails and the added capacity has a valuable use. Faster engineering or more generated output remains an intermediate result when another part of the business is the binding constraint. Revise or stop when review burden and ongoing costs consume the expected gain.
Sustainable operating ownership
Handoff is an accepted transfer of operating knowledge, authority, and responsibility. Google’s SRE engagement account provides a useful analogy: documentation, system instruction, access, and hands-on exercises support progressive transfer while the development team remains available. Sending documents alone does not establish receiving capability.
A runbook describes how to perform a recurring operational task or respond to a known failure. It needs prerequisites, diagnostic evidence, actions, and escalation contacts. Incident findings should update these instructions and response workflows, so the handoff remains connected to operating experience rather than becoming a frozen description.
| Responsibility | Receiving owner | Evidence of readiness |
|---|---|---|
| Routine work and exceptions | Customer operations | Complete ordinary and exceptional cases independently. |
| Incidents and intervention | Service support | Diagnose failure, suspend work, and activate escalation. |
| Access | Authorized administrator | Manage approved credentials and understand revocation limits. |
| Costs | Named service owner | Account for provider, infrastructure, and support costs. |
| Evaluations and upgrades | Maintaining engineering team | Own categorized cases and assess changed behavior. |
| Configuration changes | Designated change owner | Review definition changes separately from approving runtime actions. |
A practical acceptance exercise has the receiving team run routine work, diagnose an unavailable dependency, and recover or escalate without the original implementer directing each step. Record missing knowledge and authority, assign maintainers for customer-specific documentation and known limits, then repeat the failed portions. This is a proposed handoff test, not a claim about every FDE engagement.
The receiving owner may be a customer team or an internal service team. Customer enablement can reduce repeated vendor-led work without eliminating all FDE responsibilities. Continuing support, withdrawal, and eventual retirement should therefore depend on accepted responsibilities and ongoing usefulness, rather than an assumption that every engagement ends with vendor departure.
Field evidence and product decisions
A useful field record contains the underlying problem, workflow context, observed frequency, workaround, outcome evidence, customer variation, and maintenance burden. Preserve the customer’s circumstances while grouping related needs. A request for another control may reveal a documentation gap, an unclear interaction, or a missing capability; implementing the requested control immediately can hide that distinction.
Productization turns a local solution into a supported capability with an owner and lifecycle. Decagon reports moving promotional-policy changes from custom engineering into configuration, debugging, and testing by its deployment team. The useful change is who can safely perform the work and what the product supports—not merely where the original implementation was written.
A finding has several possible destinations
ExampleRecurrence prompts assessment, not automatic generalization.
Read the diagram as text
- Field evidence.
- Classify need.
- Local or process change.
- Documentation or fix.
- Assess shared capability.
- Retain local or decline.
- Owned implementation.
- Later-deployment checks.
- Field evidence → Classify need: Underlying problem.
- Classify need → Local or process change: Local circumstances.
- Classify need → Documentation or fix: Existing behavior inadequate.
- Classify need → Assess shared capability: Recurring capability gap.
- Assess shared capability → Retain local or decline: Poor shared fit.
- Assess shared capability → Owned implementation: Accepted investment.
- Local or process change → Owned implementation: Accepted change.
- Documentation or fix → Owned implementation: Accepted correction.
- Owned implementation → Later-deployment checks: Verify benefit.
Close the learning loop with evidence and an explicit disposition.
- Preserve the failure — Turn reviewed production failures into categorized regression cases with a maintaining owner.
- Communicate the decision — Return the rationale to field engineers, including when work stays local or the requested solution is declined.
- Test subsequent delivery — Check whether the supported capability reduces repeated work while preserving each customer’s requirements; retire duplicated local implementations only after that fit is established.
Open questions
Reliable reuse boundaries remain difficult to establish before maintaining several deployments. Similar requests can hide incompatible policies, while excessive local work duplicates support effort. Progress would be demonstrated by a shared capability that preserves differing policies and measurably reduces later implementation and maintenance work.
Net FDE value remains hard to attribute when pilots select willing users or unusually tractable work. Hidden assistance and changing workloads can inflate apparent gains. Progress requires outcomes across a predefined eligible population, including non-use and failed work, with implementation and operating effort recorded.
Durable handoff remains uncertain when receiving teams rarely encounter serious failures. A successful exercise may not predict operation after personnel, policies, or integrations change. Progress would include repeated independent recovery exercises and documented responsibility changes after real incidents.
Human review must balance catching consequential mistakes against introducing enough work to erase the benefit. Approvals can also become misleading feedback labels. Progress would show reviewers rejecting seeded errors while preserving usable workflow performance, with evidence kept separate from model-influenced agreement.
























