Contents
  1. Business outcomes and process boundaries
  2. Actual work and automation choices
  3. Cases and authoritative state
  4. Triggers and permitted transitions
  5. Accepted handoffs and responsibility
  6. Model interpretation and enforced decisions
  7. Exceptions and resolution paths
  8. Human review and action authorization
  9. Review capacity and service commitments
  10. Integration and confirmed effects
  11. Partial completion and business recovery
  12. Process time, quality, and total operating work
  13. Adoption, operating ownership, and controlled change
  14. Check understanding
  15. Open questions
  16. Selected talks
  17. References
  18. Talk library
← All topics

Workflow Automation

Workflow automation coordinates people, rules, models, and existing systems to complete business work. Its central decision is where automation should act, where it should assist, and where responsibility must remain with people. Useful automation improves confirmed outcomes after waiting, review, exceptions, and recovery are counted.

Business outcomes and process boundaries

A workflow comprises the activities, decisions, waiting, and exchanges that carry work toward an outcome. Its process boundary identifies where work begins, who participates, and what establishes completion. Human reasoning and organizational exchanges belong inside that boundary when the outcome depends on them.

An internal equipment request provides a running example. An employee needs a kit at a specified location for a specified period. The following requirements define this example; rejection and cancellation are separate outcomes from fulfillment.

Automation finishes before fulfillment

Example

A completed reservation leaves receiving work outstanding.

The equipment process continues beyond its automated portion through accepted fulfillment, delivery, and confirmation.
Read the diagram as text
  • Employee submits request. Process begins.
  • Interpret and validate. Automation prepares an eligible proposal.
  • Confirm permitted reservation. Automated step finishes.
  • Fulfillment team accepts. Team takes responsibility.
  • Deliver equipment.
  • Receiver confirms receipt. Business completion.
  • Employee submits requestInterpret and validate: Enter automation.
  • Interpret and validateConfirm permitted reservation: Checks and authorization pass.
  • Confirm permitted reservationFulfillment team accepts: Exit automation; offer work.
  • Fulfillment team acceptsDeliver equipment: Accepted assignment.
  • Deliver equipmentReceiver confirms receipt: Confirm agreed outcome.
BoundaryEquipment-request requirement
Unit and startOne employee request, beginning at submission.
ParticipantsEmployee, service desk, approving manager, inventory service, and fulfillment team.
Acceptance criteriaThe agreed kit reaches the agreed destination and the receiving party confirms receipt.
Excluded workPurchasing additional stock and collecting equipment after the requested period.

Acceptance criteria are observable conditions for treating work as successful. A reservation can satisfy an automated step while delivery remains unfinished. Even ticket closure may record an administrative transition: Zendesk, for example, distinguishes solved tickets from automatically closed tickets. Neither status alone establishes customer-confirmed resolution.

Actual work and automation choices

The current process is what people actually do, including variations. Observe requests and walk through them with operators; compare those observations with event logs. A swimlane is a process map grouped by responsible participant. Workflow and decision authority develops this mapping practice. Here, the map supports choosing what to remove, simplify, automate, or retain.

Include ordinary requests, missing information, nonstandard equipment, and withdrawals. Record volumes, waiting, handling, duplicate entry, informal checks, and rework before changing the process. Verify whether logs capture each activity; supplement incomplete records through observation. Compare current and proposed responsibilities, including new review and maintenance work. Tables suffice for these categorical choices.

WorkControl choice and evidenceConsequence, burden, and owner
Reenter request detailsRemove duplicate entry when the receiving system can consume the original record.Preserve field meanings; service-desk staff verify the redesigned transfer.
Check standard eligibilityUse deterministic rules: explicitly specified logic applied to current employee and equipment facts.Wrong inputs can still produce wrong decisions. The policy owner defines eligibility and exceptions.
Interpret request textUse model assistance where wording varies; require evidence for extracted equipment, destination, and dates.Misinterpretation can reserve the wrong kit. Clarification or review must cost less than the work it replaces.
Resolve an undocumented exceptionRetain a human decision until policy and accountable authority are established.A model cannot supply missing organizational authority. The responsible manager decides the permitted treatment.
Reserve eligible equipmentPermit bounded execution after required facts, checks, and approvals exist.Verify cancellation and recovery before expanding scope. The operating owner remains accountable.

A fixed workflow specifies its paths in application code and may contain model calls. An agent delegates meaningful next-action choices to a model. Use dynamic choice where observations genuinely require different actions; it adds cost and uncertainty. Agent Engineering explains this distinction. Missing policy is a reason to resolve the policy, not expand model discretion.

Cases and authoritative state

A case is one instance of the process. Its business state describes established progress and outstanding obligations. An execution attempt records one run's activity; several attempts may serve the same unfinished case.

A system of record is the designated authoritative source for a particular fact. The case application owns request status, inventory owns reservations, and the receiving process supplies receipt evidence. An agent's summary does not replace these records. Integration into existing work explains authority across systems.

One case, accumulating evidence

Example

State advancement requires new evidence.

1 / 3 · Waiting

No fulfillment acceptance yet.

R17 persists while its status changes. Accepted work and confirmed receipt establish different milestones.
Read the diagram as text
  • Request R17.
  • Awaiting fulfillment.
  • Team acceptance record.
  • Accepted for fulfillment.
  • Matching receiving confirmation. Confirms the agreed delivery.
  • Fulfilled.
  • Request R17Awaiting fulfillment: Initial status.
  • Request R17Team acceptance record: Retains evidence.
  • Team acceptance recordAccepted for fulfillment: Establishes acceptance.
  • Request R17Matching receiving confirmation: Retains evidence.
  • Matching receiving confirmationFulfilled: Establishes completion.
  1. Waiting. No fulfillment acceptance yet. Active: Request R17, Awaiting fulfillment. New: Request R17, Awaiting fulfillment.
  2. Accepted. Acceptance evidence replaces the waiting status. Active: Request R17, Team acceptance record, Accepted for fulfillment. New: Team acceptance record, Accepted for fulfillment.
  3. Fulfilled. Receipt evidence is added; acceptance evidence remains. Active: Request R17, Team acceptance record, Matching receiving confirmation, Fulfilled. New: Matching receiving confirmation, Fulfilled.

For Request R17, retain its identity and revision, accountable owner, requested kit and destination, evidence references, pending decisions, and fulfillment evidence. Keep operation attempts separately associated with R17. A successful reservation attempt cannot establish delivery; absent receiving confirmation leaves fulfillment unresolved.

Triggers and permitted transitions

A trigger starts or resumes processing: a request, record change, timer, or human decision. A transition changes business state. A guard is a condition checked before allowing that transition. Parallel prerequisites require all specified conditions, rather than whichever finishes first.

Recognizing duplicate delivery and checking current eligibility solve different problems. A distinct approval can still be stale. Stripe's webhook contract illustrates why delivery order and timestamps cannot substitute for event identity and current resource state.

Delivery does not imply relevance

Example

New events can still concern obsolete work.

Duplicate recognition prevents repeated event handling. Current-state checks separately reject stale or withdrawn requests.
Read the diagram as text
  • Receive event.
  • Check event identity.
  • No repeated handling.
  • Check current case. Read revision and withdrawal state.
  • No advancement. Record why the event is inapplicable.
  • Evaluate transition guards.
  • Receive eventCheck event identity: Event identifier.
  • Check event identityNo repeated handling: Already handled.
  • Check event identityCheck current case: Not already handled.
  • Check current caseNo advancement: Stale or withdrawn.
  • Check current caseEvaluate transition guards: Current and applicable.

Both prerequisites must hold

Example

One completed branch leaves the join incomplete.

The example requires approval and inventory confirmation for the current revision; otherwise the case waits.
Read the diagram as text
  • Manager approval.
  • Inventory confirmation.
  • AND: both current and valid.
  • Ready.
  • Await prerequisites.
  • Manager approvalAND: both current and valid: Required.
  • Inventory confirmationAND: both current and valid: Required.
  • AND: both current and validReady: Both valid.
  • AND: both current and validAwait prerequisites: Either missing or invalid.
Current state and eventExample guardDisposition
Awaiting prerequisites; approval arrivesManager approval and inventory confirmation both match the current revision.Advance only when both hold; otherwise keep waiting.
Ready; destination changesThe proposed effect still matches the approved request.Invalidate the old approval and obtain review of the changed proposal.
Pending; response deadline expiresNo required response has been recorded.Escalate the unresolved case; expiry does not establish fulfillment.
Open; employee withdrawsWithdrawal precedes any new fulfillment action.Block new actions. Reconcile already dispatched operations before declaring cancellation complete.

These cancellation and deadline rules are the equipment process's policy. Concurrent updates also need an enforced check against the state used to decide: a conditional write can reject an intervening change instead of silently overwriting it.

Accepted handoffs and responsibility

A handoff is an accepted transfer of work and responsibility. The recipient needs the knowledge, access, and authority to act. Sustainable operating ownership explains this receiving capability. At case level, specify who remains responsible until acceptance and what happens after rejection or silence.

The transfer offer should identify the case and revision, relevant evidence, requested action, authorized recipient, response deadline, and escalation route. In this example, the service desk retains responsibility until fulfillment accepts. Acceptance records a new responsible team; it does not establish delivery.

Ownership while work is offered

Example

Delivery alone does not transfer responsibility.

Example policy keeps the desk accountable until acceptance. Rejection or silence requires reassignment; acceptance still precedes fulfillment.
Read the diagram as text
  • Request R17.
  • Offered. Owner: desk.
  • Notification delivered. Owner: desk.
  • Accepted. Owner: fulfillment.
  • Rejected. Owner: desk.
  • Acceptance timed out. Owner: desk.
  • Desk reassigns.
  • Completion verified.
  • Request R17Offered: Offer work.
  • OfferedNotification delivered: Delivery confirmed.
  • Notification deliveredAccepted: Recipient claims work.
  • Notification deliveredRejected: Recipient declines.
  • OfferedAcceptance timed out: No acceptance by deadline.
  • RejectedDesk reassigns: Transfer failed.
  • Acceptance timed outDesk reassigns: Escalate unclaimed work.
  • AcceptedCompletion verified: Fulfillment evidence confirmed.

Acknowledgment means explicitly claiming the work. PagerDuty separates acknowledgment from resolution and can resume escalation when acknowledgment expires. Apply the same distinction to accepted but stalled requests: retain an owner, record the unresolved obligation, and escalate against the response commitment.

Model interpretation and enforced decisions

An employee requests a presentation kit at the North office for September 10–12 but omits the delivery room. The model can propose equipment, location, and dates with references to the request text. The room remains unresolved. A step contract defines these permitted interpretations and missing-information outcomes before execution.

BoundaryRequired check
StructureRequired fields and types match the schema; unknown values use the declared missing-value representation.
MeaningDates are ordered correctly, the location exists, and extracted values agree with the request. See semantic validation.
Business policyExecutable checks establish eligibility and required prerequisites before the tool acts.
AuthorityThe executing identity may perform this operation on this resource. Enforce permission downstream, across every available execution path.

Interpretation precedes permission

Example

Useful proposals can still lack prerequisites.

Data feeds interpretation. Control checks route incomplete, ineligible, unauthorized, and executable proposals differently.
Read the diagram as text
  • Request and source records.
  • Model proposal. Values, evidence, unresolved fields.
  • Validate meaning and policy.
  • Request clarification.
  • Stop or seek policy decision.
  • Check execution authority.
  • Hold for authorization.
  • Execute permitted action.
  • Request and source recordsModel proposal: Data: evidence.
  • Model proposalValidate meaning and policy: Data: candidate.
  • Validate meaning and policyRequest clarification: Control: facts missing.
  • Validate meaning and policyStop or seek policy decision: Control: policy fails.
  • Validate meaning and policyCheck execution authority: Control: eligible.
  • Check execution authorityHold for authorization: Control: not authorized.
  • Check execution authorityExecute permitted action: Control: authorized.

Abstention means declining to decide with the available evidence. Missing information can request clarification; conflicting interpretations can require qualified review. A confidence statement alone does not validate autonomous use. Measure both the fraction accepted automatically and errors among accepted cases; selective automation explains that tradeoff.

Bounded dynamic choice can search permitted alternatives, such as another eligible kit. Each alternative must preserve the request's requirements and pass the same checks; changing the user's need to satisfy a constraint is not successful recovery.

Exceptions and resolution paths

An exception is a case that cannot proceed under normal conditions. An exception queue is a managed collection awaiting action. Triage determines the reason, urgency, and responsible role. UiPath's distinction between business and application exceptions illustrates why unchanged invalid input and a temporarily unavailable application need different remedies.

Example reasonOwner and next actionCondition for progression
Missing delivery roomService desk requests clarification.Required information is supplied and validated.
Nonstandard policy requestPolicy owner decides whether an exception is permitted.An explicit decision establishes the allowed treatment.
Ambiguous equipment interpretationQualified reviewer examines the original evidence.Interpretation is resolved or clarification is requested.
Inventory service unavailableIntegration operator performs bounded recovery.Service is available and current prerequisites are rechecked.
Reservation response lostRecovery owner checks the receiving system.The original operation's outcome is established or remains explicitly unresolved.

Each queued exception needs a reason, evidence, urgency, assigned owner, next action, response expectation, and closure condition. Record dispositions such as corrected, rejected, awaiting information, or unresolved. Returning to normal processing requires fresh guards. This table supplies the routing paths directly; a second diagram would repeat them.

Repeated model calls cannot restore an unavailable service or establish whether an external write committed. Investigation should gather the evidence needed for the relevant remedy. Periodic operational review can detect cases that receive repeated investigation without useful progress.

Human review and action authorization

Human review examines a proposal against relevant criteria. Reviewers need domain expertise, independent evidence, adequate time, and authority to resolve the case. Operations and quality-assurance staff can help define those criteria from real interactions. Separation of duties keeps construction and acceptance from depending entirely on the same builder's judgment.

Show current and proposed values, supporting records, unresolved assumptions, and consequences. Provide approve, correct, reject, clarify, and escalate choices. Concrete action parameters matter: the n8n workshop exposes email fields and formats calendar times for inspection. Readability helps review but does not establish that the underlying time or destination is correct.

Approval belongs to one proposal

Example

Changed actions require renewed authorization.

Review yields a decision. The execution gate checks whether the approved action remains valid; corrections create another proposal.
Read the diagram as text
  • Proposal P7 and evidence.
  • Human review.
  • Corrected proposal P8. Requires review.
  • No execution.
  • Final binding and state check.
  • Renewed review required.
  • Execute approved P7.
  • Proposal P7 and evidenceHuman review: Inspect.
  • Human reviewCorrected proposal P8: Correct.
  • Human reviewNo execution: Reject, clarify, or escalate.
  • Human reviewFinal binding and state check: Approve P7.
  • Final binding and state checkRenewed review required: Changed or expired.
  • Final binding and state checkExecute approved P7: Binding and conditions valid.

Authorization permits a particular effect. Bind it to the action and relevant current facts; material changes require renewed review. Review interfaces and approval security develop the distinction. A final execution gate must enforce the binding.

Mandatory approval belongs in the execution path. The n8n example intercepts an ordinary tool invocation before it acts, rather than depending on the model to request review voluntarily. Sampling completed actions can reveal defects, but it cannot prevent effects already performed.

Automation bias is undue reliance on automated suggestions. Research using fabricated cheating alerts in previously certified exam sessions tested whether reviewers rejected erroneous assistance. Instructions requiring independent suspicious behavior increased estimated rejection, but also increased rejection of genuine operational alerts. The studies used different sessions and periods, limiting causal attribution. Review quality needs testing beyond approval rates.

Review capacity and service commitments

Backlog is accumulated unresolved work; queue age is the time an individual case has waited. Human approval can become the limiting stage when automation produces more work than reviewers can inspect. Nominal sign-off capacity is not evidence of substantive review.

d=λp,c=nahd=\lambda p,\qquad c=\frac{na}{h} Here, dd is review demand in cases/hour, λ\lambda incoming cases/hour, pp the review fraction, nn reviewers, aa available minutes per reviewer per hour, and hh minutes per review. This workload approximation holds these inputs constant; bursts and varied case effort need additional analysis.

Faster intake reverses queue direction

Example

Unchanged review capacity cannot absorb the increased demand.

Same reviewers, different intake

Intake increases from 40 to 60 cases/hour.

Scroll sideways if the figure extends beyond the screen.

0123405101520Elapsed time (hours)Queued work (cases)Before: 12 reviews/hourAfter: 18 reviews/hour
  • 1. Before: 12 reviews/hour
  • 2. After: 18 reviews/hour
Read coordinates and regions as data

X: 04 hours; Y: 020 cases, increasing up. Axes scaled independently; screen angles and distances are not comparable.

Before: 12 reviews/hour (polyline)

(0, 6); (1, 3); (2, 0); (3, 0); (4, 0)

After: 18 reviews/hour (polyline)

(0, 6); (1, 9); (2, 12); (3, 15); (4, 18)

Both scenarios start with six queued cases. Constant review demand is 12 or 18 cases/hour; capacity is 15. The fluid approximation drains the first queue after two hours while the second grows by three cases/hour. Staffing, review fraction, and effort stay fixed.

Assume intake increases from 40 to 60 cases/hour, with 30% requiring review. Two reviewers each provide 45 available minutes/hour, and each review takes six minutes. Demand rises from 12 to 18 reviews/hour against capacity of 15. The figure starts both scenarios with six queued cases.

L=λWL=\lambda W Little's law relates average work present LL, arrival rate λ\lambda, and average time inside the same boundary WW, when finite long-run averages exist. For a waiting-only boundary, WW excludes handling. It neither guarantees individual deadlines nor supplies a finite steady waiting time for the continually growing queue shown here.

A service commitment specifies an expected response or completion time. Aging increases attention to waiting cases so lower-priority work is not indefinitely postponed. Luma's compute scheduling illustrates urgency relative to a waiting-time objective; applying that idea to people requires local urgency rules and realistic reviewer availability.

Backpressure reduces upstream intake when downstream work accumulates. Depending on service obligations, reduce automatic routing, reserve capacity for urgent cases, or pause expansion. Do not silently discard old requests to improve queue statistics.

Integration and confirmed effects

An integration contract connects case identity to receiving-system identifiers, field meanings, freshness, permissions, expected changes, and support ownership. A postcondition is a condition required after successful execution. For a reservation, it includes the correct kit, destination, period, and request revision—not merely a well-formed response.

Available evidenceWhat it establishesWhat remains
Local proposalThe application has prepared an intended change.Receiving-system execution.
HTTP 202 responseThe request was accepted for processing.Processing and its eventual result; consult the status monitor.
Fresh reservation lookupThe receiving system records the matching reservation.Fulfillment and receiving-party confirmation.
Administrative closureThe case reached a configured workflow status.Whether the required business outcome was independently confirmed.

Robotic process automation operates an application's user interface. An API may expose structured status and resource identifiers; UI automation depends on locating and operating controls. Playwright's actionability checks establish readiness to click, not a completed server-side effect. Both paths need postcondition checks. Computer Use covers UI execution in depth.

Interface changes add maintenance work. Microsoft's documented AI-assisted selector repair asks an operator to recapture, review, and test the intended element during debugging. It is not automatic runtime recovery. Repairing the locator still leaves the business result to verify.

Successful writes can coexist with stale downstream reads. A reported credit workflow wrote a new score but a failed cache invalidation left the next agent using the old value. Validate schema changes and acceptance conditions at the consuming boundary; reject incompatible data rather than propagating it. The evidence table makes these distinct acceptance boundaries visible without another flow diagram.

Partial completion and business recovery

A reservation can commit before its acknowledgment is lost. The caller then lacks knowledge of the outcome. Uncertain effects explains this boundary: recorded completions support replay, but an external effect may occur before completion is recorded.

Reconciliation checks authoritative state to resolve that uncertainty. Preserve the original operation identifier and correlate returned records with the pending request. A timeout alone does not establish failure. Stripe's guidance illustrates keeping an operation pending while provider evidence resolves ambiguous execution.

External reality and caller knowledge

Example

Losing the response does not erase the reservation.

1 / 4 · Submit

Intent is recorded.

The committed effect remains while the caller is uncertain. A fresh correlated lookup resolves its knowledge.
Read the diagram as text
  • Request R17.
  • Reservation operation O17.
  • Inventory service.
  • Submission recorded.
  • Reservation committed. External state.
  • Caller outcome unknown.
  • Caller confirms reservation.
  • Request R17Reservation operation O17: Tracks.
  • Reservation operation O17Submission recorded: Local evidence.
  • Reservation operation O17Inventory service: Reservation request.
  • Inventory serviceReservation committed: Commits.
  • Submission recordedCaller outcome unknown: Response lost.
  • Reservation committedCaller confirms reservation: Fresh lookup.
  1. Submit. Intent is recorded. Active: Request R17, Reservation operation O17, Inventory service, Submission recorded. New: Request R17, Reservation operation O17, Inventory service, Submission recorded.
  2. Commit. Inventory creates the reservation. Active: Request R17, Reservation operation O17, Inventory service, Submission recorded, Reservation committed. New: Reservation committed.
  3. Lose acknowledgment. The effect persists; caller knowledge is incomplete. Active: Request R17, Reservation operation O17, Inventory service, Submission recorded, Reservation committed, Caller outcome unknown. New: Caller outcome unknown.
  4. Reconcile. Confirmation replaces uncertainty. Active: Request R17, Reservation operation O17, Inventory service, Submission recorded, Reservation committed, Caller confirms reservation. New: Caller confirms reservation.

Idempotency means repeated attempts produce no additional effect beyond the intended operation. It requires a receiving-system contract, not merely a logged key. Stripe, for example, matches parameters and returns the first executed result for a reused key; keys may be pruned after at least 24 hours. Reuse after pruning can create a new operation.

Established situationPermitted recovery
Nonexecution confirmedRetry only after current prerequisites and authority hold.
Matching effect confirmedRecord the evidence and resume subsequent work without recreating the effect.
Outcome unknown; deduplication appliesAn unchanged retry may use the original key within the service's contract and current authorization.
Outcome still unknown; repetition unsafeKeep an assigned recovery owner, investigation deadline, and unresolved disposition.
Effect confirmed but no longer wantedChoose an authorized correction, compensation, or manual remedy.

Compensation is new business work that counteracts completed work. Canceling a reservation may release equipment but incur a charge; it need not restore the original state. Preserve concurrent changes, record compensation progress, and verify its result. Compensation can itself fail. An irreversible effect needs an acceptable remedy or escalation, rather than a claim that history was erased.

Suppose revision 7's reservation is uncertain when revision 8 changes the destination. Reconcile the original operation before replacing it. Approval for revision 7 cannot silently authorize revision 8 or a corrective mutation. Preserve the action binding across callbacks and retries, and obtain current authority for the required remedy.

Durable continuation preserves completed work outside a running worker so another attempt can resume. Journaled steps can skip recorded completions, such as a charge before failed receipt delivery. They do not eliminate the gap between an external effect and its recorded acknowledgment. Harness Engineering covers the execution machinery.

Process time, quality, and total operating work

Measure the whole flow from agreed intake to confirmed completion. A faster step may leave the limiting stage unchanged. Keep elapsed time separate from labor: simultaneous work by two people adds handling effort without necessarily adding the same amount of elapsed time.

MeasureDefinition and boundary
Lead timeElapsed time from request submission to confirmed fulfillment; report unresolved cases separately.
Active handling timePerson-minutes spent doing the work, excluding unattended waiting.
ThroughputCorrectly completed requests per reporting period.
First-pass yieldCases meeting quality requirements without reruns, returns, or repair, divided by cases entering the defined process.
ReworkAdditional handling needed to correct or repeat work; eventual success must not conceal it.

For this process, define unassisted completion share as correctly fulfilled cases without human intervention divided by eligible incoming cases in the same cohort. State eligibility and follow-up duration; keep pending cases visible. This differs from first-pass quality: a human-assisted case can succeed on its first pass.

Operating-work ledgerBefore and after measurement
Ordinary handlingRecord manual intake, interpretation, entry, and coordination time.
Review and correctionCount inspection, rejection, and repair rather than treating generated output as finished work.
Exceptions and duplicate checkingInclude clarification, overrides, parallel spreadsheets, and repeated verification.
Integration and recoveryInclude support effort, broken interfaces, reconciliation, and corrective actions.
Downstream workCount effort transferred to receiving teams; leave unobserved amounts unknown.

Cost per correctly completed case divides attributable cohort cost by confirmed correct completions. Include cost incurred on failed and unresolved cases, plus review, integration support, and recovery. Compare similar case types and inspect slow cases as well as averages. A cost ledger supports this analysis; no measured equipment-request distribution is available here.

Report incorrect actions per executed action separately from unnecessary escalations per independently judged automatically handleable case. Track missed required escalations too. Unknown reference judgments remain unknown, and zero denominators make rates undefined. Live evidence and causal improvement explains how to attribute changes rather than infer causation from usage.

A study of 5,172 support workers reported a 15% average increase in issues resolved per hour with response assistance. Workers could edit or ignore suggestions, and effects differed by experience and skill. This is evidence about assisted support at one firm, not autonomous fulfillment or a complete operating-cost calculation.

Adoption, operating ownership, and controlled change

Adoption is sustained appropriate use in the intended process. Operators need participation, training, workable manual paths, and incentives compatible with careful decisions. Track bypasses, overrides, and duplicate checking. Adoption and realized customer value explains how apparent use can coexist with additional operating work.

Operating modeAuthority and evidence
Shadow proposalsCompare candidate decisions without applying their effects. Isolating inference responses does not automatically disable an agent's tools.
Assisted usePeople inspect and act on suggestions; observe correction effort and the usefulness of the assistance.
Bounded automatic executionPermit specified cases and actions after functional checks and recovery paths are established; expand gradually.

Evidence governs the next operating period

Example

Operational review can retain, change, or suspend scope.

Owners review outcomes and exceptions. Changes require testing; suspension preserves pending work for reconciliation.
Read the diagram as text
  • Completed cases and outcomes.
  • Exceptions and operator overrides.
  • Accountable owner review.
  • Retain current scope.
  • Test proposed change.
  • Suspend automatic effects. Reconcile pending work.
  • Next operating period.
  • Revise proposal.
  • Completed cases and outcomesAccountable owner review: Outcome evidence.
  • Exceptions and operator overridesAccountable owner review: Failure and burden evidence.
  • Accountable owner reviewRetain current scope: Scope remains justified.
  • Accountable owner reviewTest proposed change: Improvement warrants testing.
  • Accountable owner reviewSuspend automatic effects: Pause criterion met.
  • Retain current scopeNext operating period: Continue.
  • Test proposed changeNext operating period: Tests pass; change approved.
  • Test proposed changeRevise proposal: Evidence insufficient.

Assign owners for policy, integrations, review queues, incidents, and change approval. Transfer operating knowledge and access with hands-on exercises, while builders remain available during transition. Ownership is incomplete if the receiving team cannot investigate failures or operate the fallback.

Expand only when representative outcomes, review capacity, and recovery evidence support the larger scope. Define pause conditions for harmful actions, unmanageable queues, or unavailable completion evidence. Returning to a safer mode must retain ownership of in-flight cases. A rollback of software does not settle their business obligations.

Approval of one runtime action does not approve edits to the workflow itself. Review policy and integration changes separately before promotion. The n8n workshop explicitly distinguishes these responsibilities. Use completed cases, exceptions, and operator corrections to propose changes, then test the changed behavior before its next operating period.

Open questions

  1. Adaptive review routing must balance errors among automatically handled cases with reviewer overload. Changing case mix and scarce labels make fixed thresholds fragile. Progress would demonstrate bounded error rates and acceptable queue ages across representative workload shifts, rather than improving one measure by silently reducing coverage.

  2. Whole-process gains remain difficult to establish when automation changes who performs checking and recovery. Assisted-support productivity evidence does not supply a complete operating-cost ledger. Progress would compare matched case populations with measured downstream labor, unresolved work, quality, and sustained use over a sufficient follow-up period.

  3. Recovery across changing requests remains difficult when an old operation may complete after a new revision arrives. Provider evidence can be delayed or incomplete. Progress would demonstrate resolution of the original operation, authorized corrective work, and fulfillment of the current request without duplicate effects across lost responses and delayed updates.

Follow the curated reading path through the speakers and demonstrations behind this entry.

Explore more talks

The rest of the library, beyond the curated path. Cited talks support this entry; reviewed transcripts were processed in full. Metadata candidates have not been reviewed as sources or verified as topic members.

22 matching talks

TalkSpeakerEventYear
Cornelia DavisAI Engineer World's Fair 20262026
Martin Harrysson, Natasha ManiarAI Engineer Code 20252025
Eliza Cabrera, Jeremy SilvaAI Engineer World's Fair 20252025
Dan MasonAI Engineer World's Fair 20252025
Rachna SrivastavaAI Engineer World's Fair 20252025
Dan BjornnAI Engineer World's Fair 20262026
Steven WillmottAI Engineer Europe 20262026
Elizabeth Fuentes LeoneAI Engineer World's Fair 20262026
Anju KambadurAI Engineer Summit 20252025
Dan FarrellyAI Engineer World's Fair 20262026
Ben SteinAI Engineer World's Fair 20252025
Jeremy Silva, Chris HernandezAI Engineer World's Fair 20252025
Keegan McCallumAI Engineer World's Fair 20252025
Anant ShankhdharAI Engineer World's Fair 20262026
Sandipan BhaumikAI Engineer Europe 20262026
Vinoth GovindarajanAI Engineer World's Fair 20262026
Sachin GuptaAI Engineer World's Fair 20262026
Nikhil AbrahamAI Engineer World's Fair 20252025
Mike SpitzAI Engineer Europe 20262026
Anna Marie BenzonAI Engineer World's Fair 20262026
Alex VolkovAI Engineer World's Fair 20262026
Shirsha ChaudhuriAI Engineer Summit 20252025

References

Coverage and source review
Processed transcripts
27 processed in full · 6 in the curated path
Automated source review
Passed
Metadata candidates
1 unreviewed; not verified topic membership
Corpus version
1bd8e407b26a07b33815594e1b2db5f41827119a2b3cb6fbf240f9fc571fc767

Automated review checks source support; it is not publication approval.

A synthesis of selected conference talks and technical references. Citations link to the source material; they do not imply that every talk on this subject is included.

  1. What Does Done Even Mean? Agents and Paperclip's Liveness Model - Dotta, Paperclip

    Exhaustive human review can become a queue bottleneck and verification theater when agents produce work faster than people can inspect it.

  2. AHRQ: What is workflow?

    Official ambulatory health IT toolkit; supports a first-use definition and mapping tasks, people, and organizational handoffs.

  3. You Can't Prompt the Room: The Last Skill AI Won't Replace

    Express the persona, need, and reason for a user story, add acceptance criteria that can yield tests, and connect related stories into a coherent specification.

  4. Zendesk: What is the difference between a solved ticket and a closed ticket?

    Question-and-answer section and status comparison table, excluding the page's appended AI-generated FAQ.

  5. AHRQ: Mapping and Redesigning Workflow

    Module 10, workflow maps, redesign reflection questions, implementation guidance, and Appendix 10A. Supports actual-work discovery and operational adoption.

  6. Can Oncology Workflows Run Without Human Touch? - Anant Shankhdhar, Risa Labs

    Route retrieval through an API or RPA path, normalize both outputs, and apply a shared deterministic eligibility gate.

  7. Spec-Driven Testing for Agents With A Brain the Size of A Planet — Steven Willmott, Safe Intelligence

    Spec-driven validation, also called spec-driven testing, defines the role or task independently of the agent and supplements examples with explicit behavioral constraints.

  8. OMG: Case Management Model and Notation, Version 1.1

    Sections 4.1–4.3, case management concepts and model instantiation. Supports cases, evidence-bearing state, and human decisions.

  9. Agents Don't Do Standups: Building the Post-Engineer Engineering Org

    Start with knowledgeable engineers and low-risk work, establish verifiable tasks and working guardrails, then expand gradually.

  10. Anthropic: Building effective agents

    Architectural definitions and when to use agents. Brief contextual support for /topics/agent-engineering#section-0.

  11. Your Agent Didn’t Fail. Your Harness Did.

    Delivery alone is insufficient: a named system of record must persist the fact and support replay into future work.

  12. Your Agent Didn’t Fail. Your Harness Did.

    Internal acceptance does not prove the intended result appeared at the user-visible boundary.

  13. W3C: State Chart XML—State Machine Notation for Control Abstraction

    Sections 3.4, 3.5, and 3.7. Authoritative vocabulary for triggers, guards, transitions, and parallel prerequisites.

  14. Stripe: Receive events in your webhook endpoint

    Event ordering, API versioning, and duplicate-event handling. Concrete support for treating incoming notifications separately from current business state.

  15. OWASP Transaction Authorization Cheat Sheet

    Sections 1.1, 1.4–1.5; 2.1–2.3; 2.5–2.10, particularly modification invalidation and the final execution gate.

  16. What Does Done Even Mean? Agents and Paperclip's Liveness Model - Dotta, Paperclip

    Use explicit task transitions and enforced dependencies, with invariants that preserve productive progress, stop only for real blockers, and bound loops.

  17. Temporal Activity Execution

    What is an Activity Execution?; task-loss, Start-To-Close timeout and retry discussion; Cancellation.

  18. RFC 9110: HTTP Semantics

    Sections 13, 13.1.1, and 15.3.3. Supports submission versus completion and checking stale update assumptions.

  19. The Evolving SRE Engagement Model — Google SRE

    Operational handoff as transfer of knowledge and responsibility, with training and transition support.

  20. What Does Done Even Mean? Agents and Paperclip's Liveness Model - Dotta, Paperclip

    Make blockers, reviewers, approvers, and audited human choices first-class parts of task progression.

  21. PagerDuty: Incidents

    Incident statuses, assignment, acknowledgment, and incident timeline; operational vocabulary for agent-to-human escalation.

  22. JSON Schema: Objects

    Official JSON Schema object reference; properties, required and additionalProperties examples.

  23. OWASP Input Validation Cheat Sheet

    Goals of Input Validation; Input Validation Strategies; Implementing Input Validation. Brief vocabulary bridge to /topics/structured-outputs.

  24. Stop AI Agent Hallucinations: 5 Techniques + Production Patterns - Elizabeth Fuentes Leone, AWS

    Neuro-symbolic guardians enforce rules in executable checks before tool execution rather than relying on model compliance with prompt text.

  25. OWASP LLM06:2025 Excessive Agency

    Official OWASP risk definition, mitigation strategies, and email example. Complements transcript examples with a reusable three-part permission analysis.

  26. SelectiveNet: A Deep Neural Network with an Integrated Reject Option

    Section 2 Selective Prediction Problem Formulation, section 3 confidence-score approaches, and section 4 SelectiveNet.

  27. Stop AI Agent Hallucinations: 5 Techniques + Production Patterns - Elizabeth Fuentes Leone, AWS

    Use steering for recoverable constraints while retaining denial for hard prerequisites; the described Agent Control integration also separates rule updates from agent deployment.

  28. UiPath Orchestrator: Business Exception Vs Application Exception

    Exception definitions, telephone-number example, and transaction-status behavior. Supports routing exceptions according to their remedy.

  29. UiPath Orchestrator: Review Requests

    Displaying review requests, updating revision status, and removing requests. Concrete example of an assigned exception worklist with explicit dispositions.

  30. Your agent architecture has a half-life of 6 months

    Use a scheduled high-level check to conditionally invoke a more expensive triage agent.

  31. Your agent architecture has a half-life of 6 months

    Add a periodic reviewer that examines execution history, including which tasks and sub-agents actually ran.

  32. The Build-Operate Divide: Bridging Product Vision and AI Operational Reality

    Bring operations and CX teams into development early to define good behavior and help construct golden sets.

  33. The Z/L Continuum: Should AI Engineers Still Read Code?

    Having the same agent write code, inspect its output, and write its tests creates a self-grading problem.

  34. Building Your Own Secure AI Workflows: Human-in-the-Loop Automation with n8n

    Show the proposed action's concrete parameters in readable form, rather than only its tool name.

  35. Building Your Own Secure AI Workflows: Human-in-the-Loop Automation with n8n

    Place approval in the execution path so the agent cannot bypass it by choosing to call the underlying tool directly.

  36. When Machines Mislead: Human Review of Erroneous AI Cheating Signals

    Methods, Table 1, and Figures 2 and 4. Copy-typing concerns entering text from an external source; proctors review evidence of possible test misconduct.

  37. Amazon Builders’ Library: Avoiding insurmountable queue backlogs

    Backlog scenarios, protection at each layer, and sending backpressure upstream. Supports explaining why faster intake can worsen downstream service.

  38. Karl Sigman: Notes on Little’s Law

    Section 1.1, Theorem 1.1, Proposition 1.1, and section 1.2. Supports review-capacity reasoning with consistent boundaries and units.

  39. Dream Machine: Scaling to 1m users in 4 days — Keegan McCallum, Luma AI

    Use product-defined service-level objectives to age jobs, then rank urgency by the fraction of each job's SLO already consumed.

  40. Design by Contract Introduction

    Sections 2–3, client/supplier obligations and dictionary example; section 4, tank example; section 5, Invariants.

  41. Playwright: API testing and shared authentication state

    Establishing preconditions; Validating postconditions; Reusing authentication state; Context request vs global request.

  42. Playwright: Auto-waiting

    Official actionability and auto-retrying assertion documentation.

  43. Microsoft Power Automate: AI-assisted UI automation repair

    UI automation maintenance conditions and repair workflow. Concrete example of model assistance bounded by operator validation.

  44. From Chaos to Choreography: Multi-Agent Orchestration Patterns That Actually Work — Sandipan Bhaumik

    A shared cache can break agent-to-agent data consistency even when the underlying database write succeeds.

  45. From Chaos to Choreography: Multi-Agent Orchestration Patterns That Actually Work — Sandipan Bhaumik

    Enforce explicit data contracts at agent handoff boundaries so invalid output fails before it propagates downstream.

  46. Temporal Activity Definition

    Section 'Idempotency', including the three-step activity example, worker-crash scenario, and service-enforced idempotency keys.

  47. Resolving an ambiguous payment request

    Network errors, Server errors and Idempotency; metadata correlation during reconciliation.

  48. Stripe retry keys and retention boundaries

    Idempotent requests: response caching, parameter matching, key retention and execution-start exceptions.

  49. Compensating Transaction pattern

    Context and problem; Solution; Issues and considerations; travel-booking example.

  50. Your Agent Didn’t Fail. Your Harness Did.

    Approval must remain bound to one specific action and its scope, identity, arguments, and lifetime; expiration should terminate the approval path.

  51. Two Roads to Durable Agents: Replay vs. Snapshot — Eric Allam, Co-founder, Trigger.dev

    The presented replay model caches completed side-effect steps so a retry can skip them and reach the failed operation.

  52. Lean Enterprise Institute: Value Stream Mapping

    Definition, current and future states, process data boxes, and whole-flow perspective. Supports boundaries and distinguishing elapsed completion time from individual-step speed.

  53. ASQ: Quality Glossary—First pass yield

    First pass yield entry. Supports measuring rework and explicitly defining the entering-case denominator.

  54. Leadership in AI-Assisted Engineering

    The DX AI Measurement Framework separates utilization, impact, and cost, and treats speed and quality as joint outcome concerns.

  55. ReviewDebt: a practical framework for scoring every pull request — Sachin Gupta, eBay

    ReviewDebt names the gap between generated code and code humans have reviewed, trusted, and understood; the speaker proposes that repository grounding and organizational feedback loops make this gap compound.

  56. Normalization Process Theory: Collective Action

    Collective Action components 3.1–3.4. Supports the contextual reminder at /topics/forward-deployed-engineering#section-7 and accounting for added operating work.

  57. Leadership in AI-Assisted Engineering

    Apply Eli Goldratt's Theory of Constraints to find the workflow bottleneck; the speaker gives legacy-code reverse engineering as a concrete target.

  58. scikit-learn confusion_matrix

    Definition of C[i,j]; binary-classification interpretation; y_true, y_pred, labels and normalize parameters.

  59. Brynjolfsson, Li, and Raymond: Generative AI at Work

    Inspected PDF identifies arXiv v2; abstract and introduction describe deployment, worker discretion, outcome components, and heterogeneous effects.

  60. Amazon SageMaker AI: Shadow tests

    Shadow-test definition and response routing. Concrete illustration of observing candidate behavior without adopting its responses.

  61. Build Dynamic Products, and Stop the AI Sideshow

    The 'crawl, walk, run' framework progresses from embedded enhancements to contextual assistance and then cross-product autonomy, building on earlier functionality.

  62. NIST AI RMF Core

    MEASURE 2.13 and 3–4; MANAGE 1–4. Supports decision responsibilities and reassessment throughout operation.

  63. Building Your Own Secure AI Workflows: Human-in-the-Loop Automation with n8n

    Runtime human review does not itself provide a change-review process for shared workflow definitions.

  64. You Can't Prompt the Room: The Last Skill AI Won't Replace

    Use a high-level story map to organize user stories beneath workflow stages, then select a first release spanning those stages.

  65. Build AI Systems for Discernment, Not Approval - Angel Ortmann Lee, Duolingo

    The When Machines Mislead case study inserted fake copy-typing alerts into legitimate historical exam sessions and found that skilled proctors accepted half of those alerts.