Your Company Has No Model of Itself. Agents Can Build One - Joao Wenzel, Citadel Securities

A project is green in the monthly status deck. Someone has also left a comment on the slide about an outstanding dependency. Ask an agent on March 16 whether the project is on track, and it can retrieve the slide, find a couple of reassuring Slack messages and confidently report green. The answer a manager needs would preserve both observations and explain why the dependency warrants a closer look.

João Wenzel of Citadel Securities calls these “watermelon projects.” The difficulty is that each source answers a slightly different question. A deck records what someone was prepared to report at a particular time. A Slack thread five days later may carry a fresher assessment, but even there, someone can say the project is on track and get a dashboard screenshot in reply that calls the claim into question.

Wenzel borrows the idea of a state estimator from robotics. The system maintains a belief about the project and revises it as observations arrive. How much an observation counts depends on when it was made, whether its source is trustworthy and whether other sources agree. Agents propose changes to the record. Code and people decide which changes to apply.

Even extracting the official status takes some care. Unpacking a PowerPoint file gives the team XML, but a script then has to decide which shades of green carry the intended meaning. A vision model can interpret the slide visually, yet struggle to distinguish 31 from 81 in a screenshot someone pasted into the presentation. The team uses both the XML and the vision model’s output so that reading a slide needn’t depend on either method alone.

Each month, the agent takes the previous month’s record and the current slide together. When a project changed names in March, continuity in its participants and goals helped the agent infer that the new entry still referred to Project North Star. Treating each deck independently would have split one project’s history at precisely the point where someone reading it needed an explanation.

The new record distinguishes the reported status, the evidence, the agent’s assessment and its reason. That lets green remain the official report while yellow records the assessment prompted by the open dependency. Corrections go into the next record. Keeping earlier months intact preserves what the system could reasonably have concluded at the time.

A go-live target was Q2 in January, then Q3, then October. All three dates belong in the project history, but they carry very different weight when someone asks about delivery today. Across three years of the team’s records, Wenzel reported that, on average, half of delivery dates changed within two months. Of the dates that changed, 90 percent moved later. Attaching a date to each claim gives the agent a basis for discounting an old estimate even before a replacement arrives.

The team initially tried giving a model the entire collection of materials in one pass. It missed details. They switched to a workflow in which specialist agents look for particular kinds of evidence and other agents review their findings. Wenzel reported that this more than doubled the number of updates the team considered insightful for a project. Whether those observations helped identify future trouble required a separate test.

To test a warning, the team can reconstruct the record as of February, ask the agent which projects deserve scrutiny and then check the labels that arrived afterward. Preserving the February record matters here. A dependency that became obvious in March cannot be allowed to help the agent make its February call.

In the team’s back-test, green projects the agent flagged were twice as likely to turn yellow or red as other green projects. The measured outcome was a subsequent status change. That makes the flag useful for deciding which project to discuss first, while leaving open what will go wrong and whether a delivery date will actually slip.

When the evidence leaves a question open, Wenzel’s team seeks feedback where the stakeholder already works, whether in the app, Slack or email. What the system asks matters as much as where it asks. Confirming a fact settles one output. Asking about the rule a stakeholder used to reach that verdict can improve every subsequent assessment to which the rule applies. The record distinguishes what an agent inferred from what a person later confirmed.

Wenzel uses stakes and confidence to decide when an interruption is worthwhile. A confident, low-stakes update can go through automatically. An uncertain, low-stakes update can wait for more evidence. For consequential judgments, the team still considers whom to ask and when to ask them an open design problem.

The team reads Slack threads with their attachments and participants, recording when a claim was made, whom and which project it concerns, and a link to the evidence. Pull requests show what changed. For engineers who opt in, coding-agent session traces can also reveal how the original request evolved, which tests failed and which decisions the developer made before opening the pull request.

One failure came from treating a channel that appeared quiet as evidence that activity had stopped. Wenzel compares it to losing a speedometer reading on the highway. The team began tracking both the last time it tried to collect information from a source and the last time collection succeeded. A gap between those timestamps changes what a manager can infer from silence.

Citadel Securities now uses disagreements between the deck and the evidence to open leadership discussions. A manager can start with a project whose go-live has slipped twice and whose engineers are discussing failing tests, even while its slide stays green. The same records feed a weekly read on what stuck and what went quiet. Extending this approach to clients exposes a harder identity problem than the North Star rename. Wenzel said the team still lacks a clear way to establish that four records across four systems refer to one client when they share no canonical identifier.