Why do research if the next model will be better?
Ramp’s model usage is growing while the cost of doing the same work is falling. Alexander Shevchenko reports that the number of LLM use cases at the company grew 123% over the preceding year. Some major features were spending more on tokens overall, yet getting cheaper per unit of work as new models arrived.
That creates an awkward question for an applied-research team at an application company: why not wait? If the next model will be faster, cheaper, and more capable, how much engineering should go into overcoming today’s limitations?
Shevchenko goes back to 2023. Building an agent around GPT-4 meant constraining its behavior, keeping its context small, and worrying that a different word in a prompt might change the result. A model that took a wrong turn often kept going. Ramp shipped agents anyway because its finance customers had work to do before the next model release.
By 2026, a model could handle much more context and choose among tools with far less scaffolding. That made the basic agent easier to build. It also changed where additional work could pay off. Ramp Labs looks for expensive or slow steps inside an otherwise capable agent, then asks whether a narrower model, advance preparation, or more parallel attempts can improve the experience. The question begins at 21:32.
Let the planner ask a faster specialist
Ramp Sheets began as a general-purpose spreadsheet agent. Finance and accounting users spend much of their day in spreadsheets, so the team could reuse its harness and model capabilities elsewhere in Ramp. But a common request exposed a bottleneck: drop in several spreadsheets and ask for a discounted-cash-flow valuation or another financial scenario.
Before doing the financial work, the agent spent time planning and exploring the sheets. Claude Opus 4.6 could do that exploration, but its reasoning took time and consumed expensive tokens. The frontier model was being used both to decide what to do and to answer the many narrower questions needed to make that decision.
Ramp kept the frontier model as the orchestrator and trained a specialist for financial question answering. The planner could hand a retrieval question to that faster model, get the answer, and continue. Working with Prime Intellect, the team post-trained Qwen3.5-35B-A3B into the model labeled FastAsk on the slide.
The comparison places accuracy and time on separate axes, so the benefit is visible in one picture.
At 25:45, Ramp reports 66.25% exact-task accuracy for FastAsk, compared with 61.88% for Opus 4.6, at close to Haiku-level time per task. These are results for the task comparison shown, not a general ranking of the models.
The useful unit of specialization is the subtask. A faster question-answering model can improve the whole spreadsheet agent without taking over the planning job. Shevchenko says this work had reached production.
Prepare the month before the accountant opens it
Some financial tasks are predictable before anyone asks the agent to perform them. When the month ends, an accountant will need to reconcile the books. The ledger, aging report, and open invoices are already identifiable. An agent can retrieve them on demand, but the user then waits while it builds an understanding of material the system could have prepared earlier.
Ramp is exploring KV cartridges to move that preparation offline. A transformer’s key-value cache holds representations of context that it can reuse while generating an answer. The Cartridges research from Hazy Research trains a smaller cache on a document collection in advance, using synthetic conversations about that collection. The resulting cache can be loaded for later requests. This is background for the technique; the published research’s benchmark results are separate from Ramp’s application.
Shevchenko describes building a cartridge overnight, before the accountant starts the next morning. The timing change matters more than a generic promise of a longer context window: the agent can begin reconciliation with the relevant information already represented in its cache.
The slide at 27:18 moves the ledger-reading work out of the user’s session. Its bars explain the sequence; they are not a measured latency scale.
Cartridges remain an area of exploration in this talk. Shevchenko also accepts that a frontier lab may eventually offer the capability through an API. That would make Ramp’s experience useful immediately: the team would already know which contexts to prepare and where the technique helps. Learning how to use a capability can retain value even when another company later supplies it.
Search in parallel, then challenge the findings
Security scanning offers a different opportunity. A single unsuccessful attempt costs tokens, while one successful attempt can uncover a real vulnerability. Ramp used its background coding agent, Inspect, to run roughly 10,000 scanning sessions against its backend.
The work does not end when those agents file tickets. Their outputs need a second round of work before a security team can act on them:
- Search: agents inspect different parts of the system for potential vulnerabilities.
- Deduplicate: agents merge reports that describe the same underlying issue.
- Challenge and reproduce: agents try to establish whether the reported behavior is reachable, consequential, and reproducible.
This is agentic map-reduce: distribute the investigation, then use further agents to turn the results into a smaller set of useful findings. Shevchenko reports seven high-severity issues, all patched. One involved Lua injection through an incoming email. The failure required following data through several components that appeared safe in isolation. Ramp’s published investigation gives the full pipeline and the email example.
The same setup lets the team compare models by the findings they recover for their cost. A cheaper model can be less successful per attempt and still be useful when the savings buy more attempts.
At 30:00, the slide compares detection on seven known findings, with 25 attempts per model per finding: 52.0% for GPT-5.5, 36.6% for Kimi K2.6, and 20.6% for DeepSeek V4 Pro. The two open-weight models are labeled roughly five times cheaper per token.
Those rates do not show that the models are interchangeable. They show why model choice belongs inside the design of the search. The strongest model finds more per attempt; cheaper attempts can cover more ground within a fixed budget. Reproduction and severity review still have to separate a useful discovery from a persuasive ticket.
Turn the research into cheaper work
The three examples reduce different kinds of waiting and expense. The specialist shortens repeated question answering. Cartridges prepare context before the user arrives. The swarm spends parallel attempts where a single successful discovery can justify many failures.
Shevchenko connects that work to two products:
- Token spend management attributes usage beyond an API key or employee. Semantic analysis can group what people are actually doing with models, then connect those categories to spending.
- Ramp Router applies routing strategies to improve latency and token cost, including ways to move suitable work to cheaper service tiers.
The final research example returns to the risk of building something that the next model makes obsolete. PorTAL, short for Portable Task Adaptation for LoRA, explores learning a task representation that can be transferred to another base model. Rather than simply copying an adapter between incompatible models, the method keeps a shared learned representation and fits a small alignment for the new base. The aim is to avoid paying the full specialization cost again whenever the model changes.
Other projects apply the same research effort to practical operations: creating and revising Datadog monitors, and grouping user sessions into an open-ended taxonomy of model usage. That taxonomy feeds back into spend analysis.
Waiting for the frontier would have left Ramp’s 2023 customers without useful agents. Shevchenko’s current work makes the same decision at a smaller scale: find the step that costs the user time now, improve it, and learn enough that the next model release becomes an upgrade rather than a reason to start over. Watch the closing projects at 32:20.