What We Learned Deploying AI within Bloomberg’s Engineering Organization

How do you use AI to improve engineering when your responsibility spans data centers, connectivity, developer productivity, SRE tooling, telemetry and incident response? That is the scope of Lei Zhang’s Technology Infrastructure organization at Bloomberg. The problem extends well beyond helping someone write the next function: it includes keeping the infrastructure and software behind the Bloomberg Terminal reliable.

Zhang reports more than 9,000 engineers, mostly software engineers, and more than 500 employees focused on customer-facing AI products. Those teams build and operate the software that helps Terminal users make consequential financial decisions.

The technical estate is correspondingly varied. Zhang describes Bloomberg’s private network and JavaScript codebase as among the world’s largest. The Terminal itself supports thousands of applications, called functions: email, news, fixed-income price-to-yield and spread calculations, and trading workflows. Different functions require different technologies, so an engineering tool has to work across a heterogeneous environment.

Bloomberg also contributes to the infrastructure it uses, including KServe and Envoy AI Gateway. Against that backdrop, adopting AI internally becomes a practical question: where can these tools improve the work of building and maintaining a large software estate?

Bloomberg began exploring AI coding tools roughly two years before the talk. The number of available products made choosing a direction difficult, and the team did not yet know which capabilities would improve productivity and stability. It formed a dedicated team, released an initial set of capabilities and let engineers iterate with them. Deployment was a way to learn what the tools could actually contribute.

Developer-productivity surveys revealed a useful boundary. Engineers reported faster proofs of concept, test creation and one-time scripts, but the reported gains fell off beyond greenfield work. Generating something new was an obvious strength; improving an existing system was a harder problem.

Speed is both the benefit and the risk. In an estate containing hundreds of millions of lines of code, faster generation also makes it easier to accumulate software that someone must understand and maintain. Zhang tentatively describes complexity as growing polynomially or exponentially with software assets—a warning about compounding interactions, rather than a measured scaling law. The accompanying joke about vibe coding makes the same point: the ability to create more code does not establish that the additional code is worth owning.

The next step was to broaden the target from coding to software engineering, including maintenance and migrations that developers often prefer not to do. Consider a patch ticket: instead of receiving only a request to update a piece of software, the engineer also receives a pull request containing the proposed fix and an explanation of why the patch takes that form. That is the workflow Bloomberg wants its uplift agents to support.

The agents scan the codebase, identify applicable patches and produce changes. Zhang describes broader deployment as an ongoing effort. Bloomberg previously used a regex-based refactoring tool, which worked within limits; he reports better results with LLM-based tooling, without giving a quantified comparison. The intended improvement is not simply more generated code, but a maintenance task arriving with much of its implementation work already done.

That proposal still needs deterministic verification. Tests and good linters provide checks independent of the model’s explanation. Where those checks are missing, a plausible patch can be difficult to apply with confidence. The rationale helps a reviewer understand the change, but it does not replace verification.

Zhang reports that average open pull requests and time to merge increased after AI-tool deployment. His explanation is that code production accelerated while people still had to review and merge the changes; he supplies no effect size or observation window. This exposes a bottleneck downstream of generation: producing a proposed fix is not the same as completing the maintenance work. It also shifts the engineer’s attention toward specifying what the change must achieve, rather than prescribing every implementation step.

Incident handling offers another way to improve both productivity and stability. Bloomberg is developing and deploying incident-response agents that can inspect code, telemetry, feature flags and call traces quickly. Zhang contrasts this broad inspection with a human responder becoming attached to an initial explanation. His description of an unbiased lens concerns that contrast with human fixation; it is not evidence that models are free of bias.

The organizational problem appears as soon as many teams pursue the same opportunity. Zhang offers an illustrative scenario: ten teams want to build pull-request review bots, while twenty want to build incident-response agents. Without coordination, each team can recreate integrations and solve overlapping problems.

For incident response, those integrations are central to the system. Model Context Protocol (MCP) servers connect agents to metrics and log dashboards, network topology, service dependencies, alarms, triggers and service-level objectives. These sources provide the operational context needed to investigate a failure. Bloomberg’s infrastructure and AI organizations therefore partnered on a paved path for building MCP servers, rather than leaving every team to construct its own approach.

The platform model sits between two familiar organizational choices. At one end, business units can build their own infrastructure and accept responsibility for it. At the other, teams must use a tightly abstracted central platform. Bloomberg takes an intermediate approach: enablement teams provide a preferred path that makes the right actions easy and the wrong ones difficult.

The paved path separates three needs: choosing a model, finding an existing integration and shipping a tool.

The gateway lets teams explore which model fits a problem while giving the platform organization visibility into usage. The directory makes duplication easier to avoid: a team can discover an existing MCP server and partner with its owners. The PaaS supports the software development lifecycle and runtime, including authentication and cybersecurity responsibilities. Those security capabilities belong to Bloomberg’s platform; they are not automatic guarantees of MCP. Zhang does not identify the complete internal platform implementation with the open-source gateway project mentioned earlier.

A proof of concept and a production service should face different requirements. Bloomberg wants demonstrations to be easy because freedom to try ideas encourages creativity. Production requires quality controls because stability and reliability are core business requirements. Zhang describes this paved path as deployed for the broader engineering organization: a shared way to experiment and operate tools, not a requirement that every team build the same agent.

Tools and infrastructure address only part of adoption. Bloomberg also used an established organizational strength: an onboarding program that Zhang says has operated for more than twenty years. It prepares both entry-level and senior hires before they join their teams. AI coding instruction was added to that program, alongside guidance on using the tools with Bloomberg’s principles and technologies.

The mechanism is social as well as educational. New hires learn a practice before inheriting a team’s existing habits. When they join, they can ask why the team is not using the approach they just learned, prompting even senior colleagues to reconsider their workflow. Zhang describes onboarding as an effective change mechanism and reports greater familiarity and comfort with the tools, along with a more nuanced understanding of where they add value.

Bloomberg’s champ and guild programs provide another existing route for adoption. These communities bring together people with shared technical interests across organizational boundaries. Zhang says the programs have existed for more than ten years; Bloomberg used them to bootstrap an engineering AI productivity community roughly two years before the talk. Bringing interested engineers together supports shared learning and helps teams discover overlapping efforts organically.

The communities also help turn shared interests into completed work when team priorities differ. A requesting team may need a capability that a platform team cannot prioritize immediately. Bloomberg uses two approaches:

• InnerSource contributions: Engineers contribute to another team’s software to help deliver the needed capability.
• Visiting engineers: An engineer temporarily joins the other team, potentially for six months or a year, to complete the work.

Both approaches give a team a way to help build what it needs instead of depending entirely on another team’s backlog.

Adoption is not evenly distributed through the organization. Zhang reports that internal data shows stronger AI adoption among individual contributors than among leadership. He gives no numerical rates. Technical leads and managers may therefore be guiding teams without enough practical experience of the tools those teams use. Their existing engineering knowledge remains valuable, but Bloomberg is rolling out leadership workshops to fill the missing experience and help leaders guide development effectively.

Generative AI changes the cost function of software engineering. Some work becomes cheaper, while other work becomes more expensive, changing the decision about whether a task is worth undertaking at all. That is the opportunity Zhang finds most exciting: engineers and engineering leaders can revisit decisions whose economics have changed, rather than merely accelerate the workflows they already have.

The deciding criterion remains the quality of the resulting engineering. Cheaper implementation creates room to reconsider what should be built, maintained or improved, but it does not choose those goals. Zhang closes by asking engineers to return to basic principles: what constitutes high-quality software engineering, and how can these tools serve that purpose?