Ryan Stevens

Ryan Stevens joined Ramp as its first machine learning engineer and now leads its applied science team as senior director of applied science. His team grew from running a daily credit risk model in a notebook to supporting more than 100 production models. For Stack, Ramp’s bookkeeping agent, Ryan helped design a test that followed the same business across two monthly closes. A rule for matching bills could still be useful the next month. An adjustment to one bill could cause an error if the agent applied it again.

From a credit risk notebook to Turbo

Ryan earned a PhD in economics at New York University, where his research interests included industrial organization and applied econometrics. He then worked as a data scientist on Meta’s Ads Core ML team before joining Ramp. Ramp’s first credit risk model took nearly four months to deploy. A scheduled notebook was enough to put it to work, but a failure midway through a run meant starting again from the first cell.

As more business decisions depended on those predictions, Ryan’s team used Airflow to run forecasts once their inputs were ready. At first, each forecast waited for the entire data build, which could take four to six hours. The team changed the trigger so that a forecast could run as soon as its own input table was ready. That brought daily credit risk and spending forecasts forward by hours.

With Turbo, the internal platform Ryan documented with Ryne Carbone, a model author could specify the inputs, prediction target, model type and schedule in one file. Ryan saw a payoff for the engineers who had to keep those models running. Every new model inherited the same failure alert, and an on-call agent could use the history of similar incidents to diagnose a failed job and propose a fix for an engineer to review.

What a bookkeeping agent should remember

Ryan treats agent evaluations as backtests. To compare two versions of Stack, his team needed to give both the same books and know what an accountant would accept as finished work. For a reconciliation, that meant accounting for discrepancies between a ledger and the system holding the bills. The team graded the finished reconciliation and left the agent free to choose its tools. Ryan also co-authored the paper introducing APEX-Accounting, a benchmark Mercor built with Ramp. Accounting and bookkeeping professionals wrote and solved the 160 tasks in its private evaluation set, which spans ten simulated businesses. They also specified how to grade each result.

On Ramp’s development benchmark, the team tested each of Stack’s skills individually. They removed all the skills, then added them back one at a time to see which instructions improved the finished work. Most reduced performance. The team kept the two that helped and cut the largest from 14,000 characters to 5,000.

Memory required a different test. The team had expected Stack to infer a reusable procedure from a previous task and its correct answer. But in one simulated construction business, an adjustment that made sense for an earlier billing period led Stack astray when January’s bill arrived. The team revised how Stack wrote its notes to capture which accounts to check and how to calculate an estimate. By advancing the same business one accounting period, they could measure whether the note helped with the next bill. On the roll-forward test, memory raised Stack’s mean score from 56.0% to 61.5%.

AI spending, contract work and payroll

Ryan also studies how firms change their spending when they adopt AI. In Payrolls to Prompts, he tracked payments to online labor marketplaces such as Upwork and Fiverr and to AI providers from the third quarter of 2021 through the third quarter of 2025. Firms that had relied most heavily on those marketplaces before ChatGPT’s release adopted paid AI services earlier and reduced their marketplace spending. Among those firms, each dollar of reduced marketplace spending was associated with about three cents of additional spending on AI providers.

With Ara Kharazian and Lisa Simon, Ryan examined a different question by matching payments to workforce records for 21,559 U.S. firms. For companies with the highest initial AI spending per employee, the researchers estimated that headcount averaged about 10% above what it would have been over the first two years after adoption. They used later adopters in the same spending-intensity group to make that comparison. A firm can cut what it spends on freelancers even as it adds employees to its payroll.