The Death of Developer Advocates — Stephanie Jarmak, Sourcegraph

The title is deliberately theatrical. Jarmak, an astronomer a year earlier, arrived as an “agent advocate” after her developer-advocate manager submitted the talk and then went on vacation rather than deliver his own eulogy. Her career change also supplies an early example of the shift she wants DevRel teams to notice: she went from zero GitHub commits to 12,000 and became a maintainer of an open-source multi-agent orchestration framework. Agents did not merely change how an established engineer worked; they helped a research scientist become a user and builder of developer tools. 0:30

Her short history of the role explains what should survive. Software evangelism in the 1980s largely sent a product message outward. Developer advocacy in the 2010s became a two-way loop: advocates understood developers, helped them succeed, and carried their needs back into product development. As developers gained purchasing influence inside companies, developer experience also became part of go-to-market strategy. 2:00

What changes is the set of users inside that loop. Engineers increasingly orchestrate fleets of agents rather than performing every step alone, while people without conventional engineering backgrounds can use agents to operate developer tools. Jarmak’s claim is therefore narrower than the title: developer advocacy must adapt because both the developer’s job and the population capable of development work are changing. 3:00

The agent occupies two positions that DevRel usually treats separately. As a user, it reads documentation, calls APIs, selects tools, encounters errors, and tries to recover. As a recommender, it answers product questions, proposes libraries, and may install a dependency directly into a developer’s workflow. The machine’s experience with a tool can therefore influence both successful use and whether a human ever considers the product. 4:19

This preserves the bottom-up adoption model familiar to DevRel, but inserts a new decision-maker into it. A developer can still recommend a tool to colleagues; now an assistant can surface the tool in a Q&A session or quietly embed a framework while completing a task. Agent advocacy must consequently inspect two different outcomes: whether the agent can operate the product efficiently, and whether it connects the product to the right user problem. 4:49

Jarmak’s first concrete mechanism is CodeScaleBench, a benchmark containing hundreds of tasks modeled on the software development lifecycle. She ran agents both with and without Sourcegraph’s code-navigation MCP tool, then collected thousands of traces. The comparison asks whether the tool helps, while the traces answer the more actionable question: when it does not help, where does the interaction break down? 5:36

One trace showed an agent using a read tool. Based on patterns learned elsewhere, it guessed a parameter name resembling read_line when the available interface expected something resembling start_line. The tool description had not corrected that expectation, so the call failed. The error message was useful enough for the agent to repair its call, but recovery consumed an entire additional turn. 6:36

That example separates recoverability from efficiency. An actionable error prevented terminal failure, which is good. A clearer tool description could have prevented the error in the first place, which is better. Buyers may evaluate an agent-facing tool not only by whether the task eventually succeeds, but also by the tokens, turns, latency, and recovery work required to reach success. Instrumented traces make those costs visible at the level where documentation and interface changes can address them. 7:06

The recommendation side needs a different experiment. Jarmak created prompts representing situations in which Sourcegraph might reasonably appear, then checked whether chatbots and agents mentioned or recommended it. Prompt design matters because a comparison-shopping query and a developer describing pain are different stages of discovery. Measuring only the first can make a product look much more visible than it is. 8:06

When prompts explicitly shopped for code-intelligence tooling, Sourcegraph appeared about 65 percent of the time. When the prompt described the underlying problem—shared-library changes repeatedly breaking downstream services because the team could not see all consumers—it received zero mentions. Instead, one response suggested that developers create a wiki page. Yet cross-repository visibility is precisely the sort of capability Jarmak expected the product to be associated with. 8:35

The gap turns vague discoverability into a testable hypothesis: perhaps the company’s content describes its category better than the pains and use cases it addresses. Jarmak says Sourcegraph planned website content changes and could rerun the experiment to look for lift among agents using web search. This would measure how agents interpret current public information, rather than waiting for future training data to absorb revised messaging. 9:35

Current product information remains difficult because training data is necessarily stale. In Jarmak’s pilot, a later rerun did not correct an outdated association with Cody, one of Sourcegraph’s older products; it recommended Cody even more. She offers a plausible mechanism rather than a controlled causal result: old model outputs add more old content to the internet, which can reinforce an obsolete product story. The example’s limitation matters—the talk reports a small rerun, not a general measurement of how generated content changes recommendation systems. 10:35

Her response is to create stronger current signals. An llms.txt page can offer an authoritative source that teams hope agents will find, but it does not solve the problem alone: the agent must still use retrieval tools, fetch current information, and preserve provenance. Product pages should also provide fresh examples and concise material an agent can carry into a recommendation. Jarmak says agents respond well to structured artifacts such as charts and FAQs, though the talk does not provide a comparative evaluation establishing their effect. 11:05

Discoverability also has a distribution layer. Products should appear in the marketplaces and MCP registries where agents look for capabilities. After discovery, the path into a working environment must be short. A tool that requires three demos and an email exchange with sales is unlikely to be recommended for immediate use, because the agent would have to propose a long human process rather than complete the task. 12:05

Jarmak’s practical content advice can be grouped into three distinct jobs:

• Describe the pain: Publish enough use-case language for an agent to connect a user’s problem with the product.
• Supply current evidence: Keep examples and structured product information fresh enough to compete with obsolete material.
• Shorten adoption: Put the product where agents discover tools and remove unnecessary steps between discovery and use.

Together, these measures reduce the distance from “I have this problem” to “this tool can enter my workflow.” 11:35

The work does not fit neatly into one department, which is familiar territory for DevRel. Jarmak divides agent advocacy into “flavors” that organizations can mix according to available skills and the product’s current needs. The categories describe different ownership points in the same experience rather than three independent programs. 13:17

• Engineering flavor: Build the interfaces through which agents use the product, including MCP servers, evaluations, and instrumentation.
• Product flavor: Own the end-to-end agent experience, translate evaluation results into product work, and maintain rubrics for how agents encounter interfaces and content.
• Marketing flavor: Measure how agents enter the funnel, find the product, and bring developers with them through recommendations.

The organizational seams remain fuzzy because the agent’s journey crosses all three. 13:47

The ending returns to what survives the supposed death. Enablement now serves developers who orchestrate agents and the agents themselves. That means machine-readable content, agent-friendly APIs, and interfaces that both participants can use. The goal is not to replace human documentation with machine documentation, but to make the same product legible to a human developer and the software acting on that developer’s behalf. 14:48

Community remains a human-to-human function, but agents introduce new governance questions. If participants bring assistants into a Discord community and those assistants record or process conversations, community builders must consider privacy and data handling. The talk identifies the concern without prescribing a complete policy, leaving consent, retention, and access controls as implementation work for each community. 15:18

The feedback loop expands in two directions. Advocates still carry the developer’s experience back to the company, including the experience of developers working through agents. They can also run experiments across thousands of agent executions, collecting traces at a scale that would be difficult to reproduce through developer interviews alone. Those traces add behavioral evidence; they do not remove the need to understand the human whose goal the agent is serving. 15:48

Credibility splits by audience. Jarmak warns against sending developers generic AI-generated prose because people recognize and dislike it. She also observes that agents may respond differently to highly structured machine-facing content, even when humans find its style tedious. The practical lesson is not that low-quality content becomes acceptable; it is that human trust and machine legibility require different tests. 16:18

Her closing analogy is the curb cut. Curb cuts were built for wheelchair users but also help people moving suitcases and anything else on wheels. In the same way, clearer tool descriptions, actionable errors, current documentation, easy installation, and shorter onboarding can improve the agent path while also making the human path easier. The agent is one more user in the room, but the human remains at the other end of the work. 16:47

Jarmak ends with two immediate experiments. DevRel teams can point a coding agent at their documentation, inspect the resulting transcript, and begin an agent-experience report. Go-to-market teams can assemble pain- and category-oriented prompts, then distinguish mere mentions from genuine recommendations. These are small probes, not a complete strategy, but they reveal friction and invisibility that ordinary web analytics may miss. 17:17