Ashwin Paranjape

Ashwin Paranjape is the founding AI lead at Samaya AI. He designs the tools that financial research agents use to search company filings, news and analyst estimates. He also led the design of FrontierFinance, an open benchmark built around 220 questions written by financial experts. Each question comes with source-backed criteria that let researchers see which parts of an analyst’s request an agent answered and what it missed.

Finding a fact that fits the conversation

Ashwin studied computer science and engineering at IIT Bombay, then earned a master’s degree and a PhD in computer science at Stanford, where Christopher Manning advised his doctoral research. With Abigail See, he co-led the team that built Chirpy Cardinal, Stanford’s entry in the 2020 Alexa Prize. Ashwin worked on answering questions in conversation and designed the software so teammates could develop different parts of the bot independently. Abigail worked on neural response generation. Chirpy finished second, and Ashwin stayed on to mentor the next team. Problems he encountered in conversations with Alexa users helped shape his doctoral research.

One problem was how to introduce a fact while acknowledging what the other person had just said. Ashwin and Christopher Manning studied how people made that transition, then tested a way to rank chatbot replies by how specifically they responded to the preceding conversation. Human evaluators preferred the resulting replies 60% of the time in the study’s comparison.

A related problem arose when training a chatbot to look up facts. Many Wikipedia passages could fit the topic of a conversation, but few might support the particular reply in a training example. For Hindsight, Ashwin and his collaborators gave a second search model access to that reply during training so it could find a supporting passage. Learning from that guide improved both the passages the ordinary retriever found and how closely the chatbot grounded its own replies in them.

Retrieval for financial research

Ashwin joined Samaya in October 2022 and began by training retrieval models on the problem of searching customers’ document collections. He also co-authored Lost in the Middle, a study led by his intern Nelson Liu. In one experiment, the researchers kept the question and documents the same and moved the document containing the answer. Models often answered correctly when it appeared first or last, yet accuracy fell when it appeared in the middle.

Ashwin later designed the tools that let a financial agent consult several sources during an investigation. In Samaya’s deployment for Morgan Stanley’s Institutional Securities Group, analysts can search the firm’s own research alongside external sources, with access controls on internal material and citations back to the underlying documents.

Testing an earnings forecast

Ashwin initially proposed testing the agents on stock prices. His colleague Arash Alidoust suggested quarterly earnings because a price move can be hard to explain even after the fact. For earnings, analysts publish estimates in advance, the company reports a number, and management often explains the gap. That gave Ashwin, Arash and Maithra Raghu a way to check both a forecast and the reasoning behind it. For their study of 456 earnings releases, the team restricted the agents’ tools to information available five trading sessions before each release. The best model’s revenue forecasts had lower error than analyst consensus even after the team adjusted the baseline for companies’ historical tendency to beat estimates.

One miss Ashwin examined shows what the error score alone could not explain. An agent discovered that a peer of Ross Stores had booked a tariff refund. It considered the clue but left a possible refund out of its gross-margin forecast. Its estimate stayed close to consensus, and Ross reported a much higher margin. The search had surfaced a reason to revisit the forecast. The consequential decision was how much weight to give it.

Ashwin and his colleagues also tested whether a smaller model could learn to make better research decisions. For Samaya Sterling, they trained the agent on examples of different search strategies, then used rubric scores on the resulting answers to reward its more successful attempts. A separate model wrote the final report. On one question about AT&T’s and Verizon’s ability to cover interest payments, the trained agent requested operating income and interest expense by name, filled in missing quarters and checked quarters where the ratio could mislead. Across FrontierFinance, which the team kept out of training, Sterling satisfied 58.9% of the grading criteria, up from 42.5% for the starting model.