AI Engineer NYC 2026 Online Talks
The Agent Works. Does Anyone Trust It? — Michele Grant, Leading at Scale
Read the talk
The Agent Works. Does Anyone Trust It?
Michele Grant’s acquisition-integration example shows how measurable value, named decision owners and inspectable engineering evidence let an organization grant—and withdraw—an agent’s authority.
From a talk by Michele Grant
At a glance
Ideas worth remembering
Baseline report duration, capacity actually redeployed and detection time separately. Faster reporting alone does not establish earlier intervention or net business value.
Assign both executive accountability and daily intervention authority. Each consequential decision needs a human owner, required evidence and an escalation path.
Run financial calculations in deterministic, approved code. Preserve input records, tool execution, versions and sign-off; escalate ledger discrepancies without changing the number.
Greater autonomy requires checking what the agent failed to flag. Independent sampling helps distinguish better performance from weaker review, and supports holding or reducing authority as well as promotion.
Go-live does not establish trust
A system can go live without anyone relying on it or realizing a dollar of the value used to justify it. Michele Grant, creator of Leading at Scale, opens with that gap. Her experience in enterprise transformation and acquisition integration makes trust an operating problem: teams need change management and risk management before launch so they can actually use the system afterward.
For an agent, trust requires delivering the value the business budgeted for while remaining within its approved risk appetite, policies and regulatory guardrails. Three questions organize the work: What value does it create? Who is accountable? What evidence permits greater autonomy? A successful pilot answers whether the software functions. A core business workflow adds the question of whether it produces value under leadership’s controls.
Risk principles still need agent-specific decisions. Leadership must specify permitted actions, ownership, recorded evidence and escalation triggers. Engineers implement much of this behavior, but deciding how much authority the business will delegate is an executive operating responsibility.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A matching report leaves the business question open
The running example is an illustrative $15 billion acquisition targeting $400 million in net synergies. After the deal closes, the integration management office, or IMO, coordinates 40 workstreams, from rationalizing vendor contracts to consolidating ERP systems. Each initiative has a business owner accountable for its savings. These figures frame the scenario; they are not disclosed deployment results.
Every week, IMO analysts collect updates from the 40 leads, reconcile numbers against the general ledger, compare actuals with the plan and assemble an executive deck. The audience is a steering committee containing the CFO, chief risk officer, business unit heads and integration lead. The report connects operational progress to the people who can intervene.
An agent automates the first draft: it ingests weekly updates, reconciles the ledger, calculates variances against the plan and flags initiatives falling behind. During a four-week pilot, its drafts match the analysts’ reports line for line. That establishes the scenario’s technical success. It leaves the steering committee with a separate decision: what additional evidence would justify relying on this agent to help run the integration?
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Measure faster reporting, usable capacity and earlier detection separately
Before the agent touches production data, the steering committee needs an agreed written value case. Grant separates three measures because a faster report does not necessarily release useful capacity or reveal trouble sooner.
- Reporting cycle time. If compiling the weekly report currently takes five business days, set a concrete target such as two days. This measures how quickly the reporting process finishes.
- Analyst capacity redeployed. Four analysts spending half their week on the deck represents 80 hours a week of potentially released capacity. It becomes value when the integration lead assigns that time to workstreams missing their synergy targets. The value case must also subtract compute, licensing and human review costs.
- Detection time. Measure the interval between the first sign of a slip in the data and the steering committee seeing it. Finishing the deck in two days cannot help if the agent misses the slip.
The causal distinction matters. Shortening report preparation changes a process duration. Reassigning analysts changes where the organization spends its effort. Detecting a material issue earlier changes when leadership can act. Each needs its own pre-deployment baseline; otherwise, a successful-looking report cannot establish which business outcome improved. Grant’s deployment rule is firm: if the IMO cannot state the baseline, do not deploy.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Separate operating ownership, oversight and assurance
The next question is who owns the risk. Grant applies the three lines model, separating the people responsible for running the workflow from those challenging its controls and those providing independent assurance.
- Management owns the controls. The integration lead remains the accountable executive and owns the agent’s output just as she owned the manual deck.
- Risk and compliance oversee and challenge. Their review should reflect the agent’s exposure. An internal draft checked by a senior analyst warrants a different standard from an autonomous system publishing figures directly to the board.
- Internal audit provides assurance. Material risk determines when and how the agent enters the annual audit plan.
Executive accountability also needs a daily operator. Here, the senior analyst reviews the draft before the committee receives it. That employee must understand the agent’s operational limits, interpret its output in context, have authority to override or reverse it and have a clear kill switch for unexpected behavior. Assigning a reviewer without the power to intervene would leave the role incomplete.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give each consequential decision a human owner
A one-page human work standard records where the agent’s authority stops. At each fork in the process, it names the decision, the human role authorized to make it, the evidence required and the escalation path. The acquisition example makes those assignments concrete.
- Interpret the variance. The senior analyst decides whether a difference reflects accounting timing or a real cash shortfall.
- Choose the remedy. The workstream lead decides how to address a shortfall.
- Declare the initiative at risk. The integration lead decides whether to formally raise its status to the steering committee.
The agent prepares analysis for all three decisions; the people making them own the consequences. Keeping those judgments with their existing owners lets the agent enter use without acquiring their authority. An unassigned decision creates a different problem: if nobody owns declaring an initiative off track, the agent can produce correct flags that lead to no action. A missing owner is a reason to delay deployment.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Keep financial proof in code, telemetry and sign-off
Greater autonomy needs evidence built into the system. The first artifact is an auditable decision log. For each workstream evaluated, it records the raw input data, executed tool calls, prompt and model versions, generated output and human sign-off. Together, these records let a reviewer inspect what the agent used, what it did and who accepted the result.
Two engineering decisions give that log substance. Core financial calculations and reconciliation rules run in deterministic, finance-approved code. The language model’s written summary explains the result; it does not establish that the calculation was correct. The supporting evidence is which records were retrieved, which deterministic tools ran and who signed the review.
A disagreement between reported savings and the general ledger shows how this division affects behavior. The agent leaves the number untouched, flags the discrepancy and escalates it. Generating a plausible explanation must not become permission to alter the financial figure.
The second artifact is a model inventory record containing the accountable executive, data lineage, source-system dependencies, validation results, escalation paths and kill switch runbook. It also documents regression triggers: changes to a prompt, base model or source API require full revalidation before production use resumes. Validation therefore attaches to a particular configuration and its dependencies, rather than granting permanent approval to the agent’s name.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reduce routine review without losing sight of missed issues
The third artifact, an autonomy scorecard, lets the steering committee grant authority in four stages. Shadow mode keeps the existing analyst-built report while the agent runs alongside it for comparison. In the supervised-draft stage, the agent prepares the report and the senior analyst reviews every line item.
What changes as review narrows? The comparison below separates routine review from the checks on work the agent considers clean. At stage three, the analyst reviews flagged initiatives plus a random sample of clean ones. At stage four, routine review covers only high-risk anomalies, while periodic independent sampling continues. “Exceptions only” describes the analyst’s routine workload; independent checking still has a job.
The danger at stage four is a material problem the agent never flags. Reviewing only the flagged set cannot reveal those omissions. Independent sampling examines work outside that set, helping track missed material issues and their severity. It also helps interpret a falling override rate: fewer overrides may reflect better output, or they may reflect shallower human review.
The scorecard puts these control measures beside reporting cycle time, analyst hours redeployed and detection time. This gives the committee a basis to promote the agent, hold its current authority or step it back to supervised review. Grant supplies the review structure, rather than numeric promotion thresholds or a sampling schedule; those decisions still need to be made for the workflow’s risks.
What humans inspect at each autonomy stage
Read the diagram as text
- 1. Shadow mode. Human review coverage — The analyst builds the existing report. The agent runs alongside it, and the team compares the reports.
- 2. Supervised draft. Human review coverage — The agent compiles the draft. The senior analyst reviews every line item.
- 3. Exceptions and sample. Human review coverage — The analyst reviews flagged initiatives and a random sample of clean initiatives.
- 4. Exceptions only. Human review coverage — Routine review covers high-risk anomalies. Periodic independent sampling checks for missed material issues.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Earlier evidence gives operating leaders time to act
The ending returns to who actually captures the acquisition’s $400 million in targeted synergies. Procurement, supply chain and operating leaders do that work through integration. The agent’s contribution is to give them verified evidence early enough to intervene while savings remain achievable. Grant describes that benefit as evidence arriving weeks ahead of schedule—a prospective benefit of the scenario, rather than a measured outcome of the four-week pilot.
That connects the three questions to a business decision. The value case defines what improvement counts. Named owners turn analysis into action. Logs, validation and sampling let leadership decide how much review the agent still needs. A working agent earns trust at scale when the organization can make those decisions with evidence—and reduce its authority when that evidence changes.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Transcript & slides
Loading transcript and slides…