Can AI Agents Beat Wall Street at Predicting Earnings? - Ashwin Paranjape, Samaya AI

Suppose an agent picks a stock and the price goes up. How would you tell whether it got the reasoning right? Ashwin Paranjape, founding AI lead at Samaya AI, came to that question after building tools for financial research. His team could retrieve information from messy customer data, give an agent access to multiple sources within its context window, and evaluate the research it produced. A stock price made for a frustrating next test. Even people following a company often struggle to explain why its stock moved. After a bad call, what would an engineer know to fix?

A colleague suggested predicting quarterly earnings. A company reports figures such as revenue, gross margin, operating income and earnings per share, and gives guidance for the coming quarter. Analysts make their own forecasts. Many sell-side analysts publish theirs, and those estimates are pooled into a consensus. When the next report arrives, each figure has a reference point. A result above consensus is a beat. A result below it is a miss.

The reporting cycle gives an engineer three things to work with. The reported figure settles the numerical question. Consensus shows how well professionals did with the information available beforehand. And the earnings call and subsequent commentary discuss what drove the surprise. If an agent called a beat, the engineer can check whether the development it spotted figures in the company’s explanation.

Where does the reported number enter the experiment? Take a forecast made seven days, or five trading sessions, before earnings. The model runs inside a harness, the software that gives it tools and context. Those tools can return information available up to the forecast minute. The agent researches the company and issues its estimate. Only on earnings day does the evaluator bring in the reported figure to score that estimate.

Samaya started with its production harness, which already served company documents, prices, consensus estimates and news. The team modified both document retrieval and the structured-data tools to enforce a point-in-time cutoff through an authentication token. That puts the decision about which records to return in software outside the model. Even if the agent asks for a later document, the tool enforces the same cutoff.

The open web posed a practical difficulty. A page available today is hard to reconstruct as it appeared at a particular minute in the past. The team approximated web coverage by adding timestamped news streams that the harness could filter. It also supplied expert guidance on how to use the available context to make a forecast.

A tool cutoff handles one way an agent might know the answer. A historical earnings release could also have been in a model’s training data. Samaya therefore chose reports from July 2026 onward, after the June knowledge cutoff of the newest model it tested. The evaluation covered more than 450 earnings reports and compared seven frontier models.

The first score measured the distance between forecast and actual, scaled by the company’s historical volatility. Published consensus had an error of 1.5 on that scale. But companies beat the published estimate more often than they miss it. An agent could gain some ground simply by allowing for that tendency. Correcting consensus for the bias brought its error down to 1.18, giving the agents a tougher benchmark.

Where did an agent clear that tougher benchmark? Across the earnings metrics taken together, prediction error tended to fall with newer models. Revenue gave a clearer answer when examined on its own. It required the fewest adjustments of the metrics tested, and Astra had significantly lower revenue error than bias-corrected consensus.

• Hit rate. If a company reports above consensus, any forecast above consensus gets the direction right. Predicting a slight beat earns the same credit as predicting a large one. To see how much of that success could come from a company’s track record, Samaya looked back over its previous eight quarters. The baseline called whichever outcome, beat or miss, had occurred more often. Newer models began to outperform that rule even on the harder below-consensus calls.

• Surprise correlation. When an agent predicts a large beat, does the company tend to deliver a large beat? Do the largest predicted misses also correspond to large actual misses? Correlating predicted and actual departures from consensus uses both direction and magnitude, giving the comparison more statistical power. On this measure, three leading models significantly outperformed the baseline. The experiment could distinguish the leaders from the baseline more confidently than it could distinguish them from one another.

One agent was forecasting earnings for a semiconductor manufacturer that supplied data centers. A clue turned up one step upstream, at a small UK wafer maker that supplied the manufacturer. Information from the wafer maker pointed to stronger revenue in its customer’s data-center business. The agent raised its revenue assumption for that part of the business.

How far did that observation move the final number? The agent carried its revised revenue assumption through the manufacturer’s profit-and-loss statement. Its forecast landed well above both published and bias-corrected consensus, close to the figure the manufacturer ultimately reported. A piece of news about one business had changed the agent’s estimate for a particular line of business at another.

What happens when the agent finds a clue and leaves it out of the calculation? For Ross Stores, the clue was a tariff refund that another retailer had booked. A similar refund at Ross could lift gross margin. The agent recognized the peer’s refund and chose to leave that possibility out of its Ross forecast. Its gross-margin estimate stayed very close to consensus. Ross reported a much higher margin. Better retrieval alone would have done little here. The agent had already found the item that called for a judgment.

Across the broader set of forecasts, Paranjape also examined calls well above or below consensus. The models tended to outperform when the surprises were larger. Those calls give a researcher a specific decision to investigate in the light of the earnings commentary. What evidence justified moving that far from the analysts’ estimate, and did the company’s result bear it out?

The wafer supplier’s contribution suggests a way to test what the harness adds. How well could the model forecast using only what it had learned during training? Samaya removed external data access. Even a model with a June 2026 training cutoff, which could have learned from documents published shortly before the evaluation period, forecast poorly in that condition.

How much did freshness matter once the model could do research? In a second comparison, the tools supplied information only up to 11 weeks before the earnings call. The agent still had the previous quarter’s results, management guidance and the analyst consensus available then. Losing access to more recent developments cost about as much forecasting performance as the gap between Sonnet and Fable 5.1. Among the factors Samaya examined, access to up-to-date information mattered most.

With expert guidance, the agents spent more minutes researching a forecast and consumed more tokens than they did without it. For Astra, guidance also led to a significant reduction in prediction error.

Did the agents need Wall Street’s number as a starting point? The team removed direct access to consensus. Since a news story could still quote the estimate, the document extraction tools also redacted mentions of consensus from news. Older models performed substantially better when they could see the estimate. For Astra, the difference between the visible and hidden conditions was statistically insignificant. Paranjape interpreted that result as evidence that a capable model could reconstruct the consensus view from the remaining information.

At the time of the talk, Samaya had made earnings forecasting available for alpha testing and was working with customers to find out whether their internal data improved forecasts. The team was also comparing model reasoning with human reasoning. A decision like the one at Ross gives that question a practical focus. When a peer books a refund, might an analyst and an agent bring different evidence to bear on whether to change the estimate?

Post-training and continual learning were open research directions. For Ross, the agent could revisit the evidence it had, the adjustment it declined to make and the margin the company reported. The question is whether experience with a miss like that can improve the judgment it makes before the next report comes out.

For more worked forecasts, Samaya’s accompanying blog post continues the case studies. Frontier Finance is the resource Paranjape points engineers to for evaluating their own financial research harnesses. The two QR codes are on the closing resource slide.