Khasimul Asif Mohammad

Khasimul Asif Mohammad is a principal product manager at Splunk, a Cisco company, where he leads the roadmap for Splunk Agent Observability. His team helps developers investigate failures that are hard to see in an agent’s final answer. A reply can sound plausible even when a tool call failed or a handoff went wrong. Agent Observability, which incorporates Galileo technology, lets teams inspect those steps alongside measures of answer quality, latency, and cost.

Growth products and live video

Asif built developer and infrastructure products at Freshworks and founded a natural language processing startup. He earned an MBA from Berkeley Haas and a master’s in engineering from UC Berkeley. By summer 2023, he was working in growth product management at Twitch. He and fellow Berkeley alum Namrata Mani launched GigaGrowth, a newsletter and interview podcast about how software teams bring products to market and what they learn when their plans fall short.

By April 2025, Asif was a senior product manager for Amazon Interactive Video Service at AWS. In his account of Amazon IVS Multitrack Video, he explained how the service shifts some encoding work to the creator’s device. Viewers on slower connections need a lower quality stream to keep watching, but generating every quality level on central servers is expensive. With Multitrack Video, the service configures the encoder for the creator’s hardware and connection so the creator’s GPU can produce multiple versions before sending them. Viewers can still switch quality levels, and the service has less video to transcode on its servers.

Turning production failures into regression tests

At Splunk, Asif’s work on production agent evaluation faces a different cost problem: how to spot rare failures without paying a reasoning model to examine every session. In the abstract for Heal Your Agents, Don't Just Grade Them, he argues that passing evaluations can coexist with recurring failures in production. Tests cover cases a team anticipated, while some new failures only become visible as a pattern across sessions. Drawing on work with customers and on his team’s own agents, Asif frames the tradeoff as deciding which sessions warrant deeper analysis without flooding the team with flags. Many flagged sessions can trace back to a handful of causes, so grouping them helps the team decide what to fix first. If a coding agent proposes a patch, the team still needs to verify the fix before merging it. The failure can then become a regression test so the next version is checked for the same problem.