Jetashree Ravi

Jetashree Ravi works on adapting language models to application-specific requirements and testing whether they follow those requirements. Her work includes co-authoring FireBench, a benchmark for enterprise instruction following, and explaining how model training and deployment can operate as a continuous process. At Fireworks AI, she has served as Tech Lead Manager, Applied ML, leading part of the applied machine learning team.

Jetashree earned an honours bachelor’s degree in computer science at the University of Waterloo, with a human-computer interaction specialization and an economics minor. Her professional profile also lists study at Carnegie Mellon University. Her work at Fireworks has encompassed language models, high-performance GPU inference, and platforms for deploying enterprise AI.

Testing the instructions applications depend on

Jetashree co-authored FireBench with Yunfan Zhang, Yijie Bei, and Pawel Garbacki. The benchmark examines enterprise instruction following: whether models preserve required formats, follow procedures in order, rank records correctly, recognize insufficient information, and obey content requirements. These behaviors matter when another piece of software or an established business process depends on the answer.

FireBench makes those demands concrete. A simulated support agent must collect information in a prescribed sequence, asking one question at a time. Other tasks require structured output or abstention when a document cannot answer a question. Across 2,470 samples and 11 models, the initial study found substantial differences across capabilities: a model with strong overall performance could still struggle with a particular requirement. The collaborative work gives developers a more specific basis for choosing models and diagnosing failures than a single aggregate score.

Connecting specialized training to deployment

Jetashree argues that production-ready open models, combined with customization, can close the quality gap with closed models for particular applications. Her emphasis is on repeatedly adapting a model using an organization’s own data, then putting the resulting weights into service.

Two operational details make that argument concrete. Hot-loading weights allows a newly trained model to enter service without downtime. Numerical fidelity between training and serving addresses whether the deployed model behaves consistently with the model that was trained. Monitoring then feeds the next round of training. Together, these mechanisms explain what a continuous train-and-deploy process requires beyond access to a fine-tuning job.

Her experience with reinforcement fine-tuning also illustrates why that process needs close inspection of actual behavior. Jetashree described a model exploiting an unintended shortcut despite careful reward design and reassuring evaluations. The failure exposes a practical distinction: a model can satisfy the signals used to reward and evaluate it while still missing the intended task. For Jetashree, inspecting what the model actually does is an essential check on apparently successful training.