Small Bets, Big Impact: Building GenBI at a Fortune 100

Asaf Bord began drafting his presentation with o3 in August. A couple of weeks before the conference, he returned to refresh it, but GPT-5 took over and, in his account, messed up the slides. He finished them manually. His opening disclaimer—that the presentation was not created with generative AI—comes from an ordinary problem with ambitious tools: getting an initial result is different from reliably improving it.

GenBI combines generative AI with business intelligence: an agent helps people answer business questions using data, much as a BI professional would. Its appeal is data democratization. Instead of waiting for a BI team to locate a report, explain its meaning, and learn enough about the requester's business context to help, users could get that assistance directly.

Northwestern Mutual operates in financial services, life insurance, and wealth management, with more than 160 years of institutional history. It has abundant data, funding, use cases, and technical talent. Those advantages coexist with a strong reason for caution: the company makes commitments that can last for decades. Bord describes life-insurance relationships extending 20, 40, or even 80 years. Under the principle of “generational responsibility,” stability matters because clients depend on it. The engineering question is how to innovate without compromising that obligation.

The initial proposal faced four connected uncertainties:

• Unfamiliar design: Bord describes GenBI built in this fashion as uncharted territory for the project, without an established implementation to follow.
• Messy data: The team deliberately wanted actual enterprise data, where accumulated complexity would expose the real engineering problems.
• Trust: Both users and leadership needed confidence in answer accuracy despite familiar concerns about generative AI.
• Investment: Leadership had to fund research before the team could specify exactly how the system would work or what the finished product would look like.

The budget question therefore depended on the technical questions: how could a risk-averse organization justify investing before those uncertainties were resolved?

Using synthesized or pre-cleaned data would have made demonstrations easier, but it would also have postponed the problems that separate a proof of concept from production. The team used real data to encounter those problems during research, while the system's design and expected behavior were still being discovered. Success in that environment provided more confidence that a capability could survive outside the lab.

Working with actual data also brought in the people who used it every day. Subject-matter experts supplied real business questions and the answers people had previously given, creating material for evaluations as well as a way to judge whether outputs made sense. Their participation changed adoption, too. Rather than encountering a finished lab demonstration and being asked to trust it, business users helped shape the research. When parts became useful, those users began asking how quickly the team could package them for practical use.

Management trust did not begin with an unrestricted research budget. Bord jokes that receiving a million dollars for a speculative idea was something he woke up from, not a description of the project's funding. The team worked in a sandbox and excluded actual client data. Using real enterprise data did not mean exposing client records or immediately releasing the tool to everyone.

The rollout depended on users' ability to verify the answers and provide useful feedback:

For BI experts, the initial tool would accelerate work they already understood, analogous to GitHub Copilot. Business managers could follow because their familiarity with the data made them more likely to recognize an error and report it rather than abandon the tool. Executive use remained an aspiration: Bord said the system was not accurate enough, offered no arrival date, and allowed that it might never meet that standard. The release boundary was the user's capacity to supervise the system.

The first product boundary was equally deliberate: do not begin by generating SQL. Writing the right query can be difficult even for a person who understands the data. Instead, retrieve information that already exists in certified reports and dashboards. The organization had already invested effort in validating those assets.

Bord says some BI teams estimated that 80% of their work involved directing people to the right report and helping them use it. That made report discovery a useful product in its own right. The agent could deliver the same asset a BI professional would have supplied, but through a faster, more interactive exchange. Setting that expectation with both users and management narrowed the initial trust problem: the system's job was to find and explain existing information, not invent a new analysis.

The funding process mirrored the product design. Each increment needed to give leadership visibility into what it was funding and produce something the business could use. Leadership could stop because a phase failed, because it had already delivered enough value, or because the next phase was too uncertain. The project did not require a commitment to the entire end-state vision.

The early roadmap built capabilities in sequence:

1. Research the fundamentals. Explore natural-language-to-SQL, question understanding, and response generation. This was research into eventual capabilities, distinct from the decision to keep generated SQL out of the first product.
2. Determine useful metadata and context. Establish what a BI agent needs to understand data. Those requirements differ from ordinary chat or retrieval over unstructured documents.
3. Apply the metadata lessons beyond the agent. Understanding how to extract and describe metadata also helps human data users and informs the company's separate semantic-layer initiative.
4. Package semantic discovery. Use the question's context to find relevant information, creating a data finder and data-owner finder that can stand on its own.

Bord estimates that finding what data exists and who owns it can take two to four weeks in an enterprise like Northwestern Mutual. Discovery therefore offered value before the system could independently answer complex analytical questions.

The next layer would retrieve information and support light pivoting of the data. Enterprise readiness would then add an understanding of different users, their likely questions, and their access permissions. A fully fledged GenBI agent remained further out: it would generate SQL, retrieve additional data, and perform more sophisticated joins to answer questions beyond existing reports. Each layer would supply both working components and knowledge for the next.

Bord describes the steps as six-week sprints, each ending in a tangible deliverable that the business could choose to productize. Those decision points made progress visible while allowing the team to adjust its direction as it learned. They also limited sunk-cost pressure: prior spending did not have to become an argument for funding every remaining phase.

The same structure preserved a buy-versus-build option. Databricks Genie was one candidate Bord thought the organization might eventually adopt. If an external product became the better choice, the internal research would still have produced evaluation questions, benchmarks, and knowledge of failure modes. The team could test a vendor against its own expectations and probe beyond a polished demonstration. Stopping development would not erase what the organization had learned.

Why not paste a database schema into ChatGPT? In Bord's environment, schemas were messy and did not convey enough business meaning. Governance also needed to be part of the architecture. He considered it harder to impose that governance from outside a general chat product—or a third-party analytics system—than to build it into the system itself. Whether a vendor could meet the requirements remained open.

Above a data and metadata layer, the architecture named four agents:

Governance, trust, and orchestration surround this pipeline, with a contextual interface above it. A business question first reaches the orchestrator, which coordinates the work.

The metadata agent consults the catalog and system documentation to establish what the user is asking about and which information is relevant. Only then does the RAG agent search for an existing report, drawing from certified reports approved for use and refined for accuracy. Retrieval is therefore conditioned on business meaning, rather than starting from an unexplained schema or an arbitrary collection of documents.

If there is no suitable report, or the closest report does not quite answer the question, the SQL agent attempts a more precise or elaborate query. An imperfectly matched report can still help: its query supplies a starting point instead of forcing generation from scratch. Bord compares this to a few-shot example that is unusually close to the desired result.

For example, suppose an existing report totals sales by region for a full year, while the user asks for the first quarter. The refinement can preserve the selected measure and grouping while narrowing the date range:

-- Existing report query
SELECT region, SUM(sales_amount) AS total_sales
FROM sales
WHERE sale_date >= DATE '2025-01-01'
  AND sale_date < DATE '2026-01-01'
GROUP BY region;

-- Proposed refinement: first-quarter totals
SELECT region, SUM(sales_amount) AS total_sales
FROM sales
WHERE sale_date >= DATE '2025-01-01'
  AND sale_date < DATE '2025-04-01'
GROUP BY region;

The sales table and dates illustrate the mechanism: reuse the report's existing analytical structure and change what the question requires. The second query is a proposed refinement; retrieval of a certified starting point does not itself validate every subsequent edit.

In the described flow, the query is then executed against the database. Its results go to the BI agent, which turns them into an answer to the business question rather than merely returning rows. Conversation history creates a reuse path: when a follow-up concerns the same data, the system need not rediscover the entire context.

The architecture also supports the funding model. Components can be packaged as separate products, delivering useful capabilities before the full pipeline is complete. After each productization decision, the team can stop or continue. The complete diagram brings together the catalog, certified reports, SQL execution, BI interpreter, final answer, and conversation-history route.

Bord reports that report routing occupied about 20% of a 10-person BI team's capacity, and the RAG agent automated around 80% of that routing work. This is a different account from the earlier estimate about report assistance across unspecified BI teams. Here, the denominator is the capacity of one 10-person team.

Those percentages imply about 16% of total team capacity, or 1.6 full-time equivalents of work, automated. The arithmetic is:

Bord's reference to roughly two full-time people describes the original routing workload, not two jobs eliminated. He does not supply a measurement period or a method for verifying realized time savings.

Metadata research supplied another route to measurable value. In the semantic-layer project, the team ran an A/B test using a battery of questions against a database with good metadata and one without it. Bord reports better LLM performance with enriched metadata, but gives no numerical uplift or scoring details. The experiment made a normally diffuse proposal—invest in more documentation and better descriptions—something leadership could evaluate through answer quality.

At the time of the talk, the team was experimenting with a data-pivoting bot. Starting from an existing dashboard or report, it would let users change the time horizon, views, segmentations, and groupings in real time. The aim was to avoid requiring a person to make each adjustment for a business stakeholder. Next steps included evaluating GenBI tools such as Databricks Genie and undertaking more rigorous catalog enrichment with metadata and documentation informed by the research.

These remained distinct from a completed, autonomous GenBI agent. Report discovery, metadata lessons, and the developing pivoting capability could justify investment even if the organization never built the full system end to end. Bord credits the value accumulated along the way with sustaining leadership support quarter after quarter.

The surrounding market, in Bord's view, has room for products devoted to data preparation, task-specific models and applications, copilots embedded where users already work, and model security. These opportunities address the supporting work needed to make AI useful inside an enterprise.

His final question concerns software economics. Bord posits a worker becoming 10× more effective as a premise for discussing pricing, not as a measured GenBI result. If one person can accomplish much more, which unit should a software vendor charge for?

The choice matters when employee count becomes a weaker proxy for the amount of work software enables.

Bord points to Salesforce Data Cloud's usage pricing as an example. Its contemporaneous pricing used consumption credits, with separate storage charges and optional add-ons; the example concerns Data Cloud, not a universal replacement of Salesforce seat pricing. His broader question extends even to products that contain no generative AI themselves. If AI changes how much their users can accomplish—and what those users can do with the time they recover—pricing by employee count may capture something different from pricing by the work those employees get done.