Why Can't Anyone Answer Questions About the Business?

How do you answer a question about your customers when the dashboard doesn’t contain the field you need? Garrett Galow introduces this problem from inside WorkOS, where he leads product. WorkOS supplies enterprise platform features, including password and enterprise identity-provider login; he names Cursor, Anthropic and OpenAI as customers. Studio, the subject of this demonstration, is an internal productivity tool built to help colleagues answer questions about the business.

Engineers, go-to-market teams and support staff all need to understand how customers use the product. Retool apps and existing dashboards answer questions someone anticipated, but a request for one extra field or a deeper breakdown creates more work. Someone with SQL skills or data access must receive the question, understand its context and return an answer. The requester waits, checks the result and often asks for another layer of detail. The exchange ends up in Slack as a one-off answer, ready to be repeated next time.

Studio makes that investigation self-service. It provides an internal workspace where employees can answer questions and build apps or dashboards themselves, so a useful investigation can become something colleagues reuse.

Galow begins the live demonstration with a prepared prompt about content attribution. WorkOS buys podcast advertisements and Google ads that bring visitors to its marketing site, blog and documentation. The next question is what those visitors read before becoming customers. WorkOS calls its customers teams, so the request asks which content leads to the most new team creations.

Studio first determines which internal resources it can use. It has connections to Linear, Notion and Snowflake; for this question, it investigates Snowflake, where WorkOS stores the relevant data. The agent examines schemas and tables and runs multiple queries to assemble an answer.

While that investigation runs, Galow explains the request path. Employees can start from either the Studio dashboard or a Slack bot. An API parses the request and runs an agent built with LangGraph, using Opus as the model. An integration proxy connects the agent to Snowflake, Linear and Notion, while a guidance layer explains how to use those systems.

That guidance supplies business meaning the database alone may not make clear: how a customer is represented and which joins connect the relevant tables across a sprawling Snowflake estate. The agent uses the tools to return answers or update widgets. Convex persists state across sessions, allowing the workspace to retain information after the immediate interaction ends.

The results include the homepage, pricing page, blog posts, changelogs and documentation. They give Galow a view of content associated with team signups; the demonstration does not establish a causal attribution method. Having received an answer, he asks Studio to build a table that shows the data over different time slices. That would make the investigation useful in weekly syncs and shareable with teammates.

Studio calls the resulting tool a widget. It bundles sandboxed code for the user interface, API interactions and the query needed to power the tool. While the live widget generates, Galow opens a version he prepared earlier. It shows content and team signups across time spans, reruns the query when used and can expand to fullscreen for viewing or sharing.

The newly generated version looks similar but is not identical: it includes content-category filters. Those filters let someone narrow the view to blog posts, for example, and investigate which posts are associated with signups. The workflow has moved from answering a question once to providing a reusable interface for deciding where to focus future content work.

The next example comes from support. WorkOS Radar protects against bots and bad actors, which creates a recurring customer question: why did a particular user get blocked? Previously, answering that could involve navigating existing dashboards or sharing SQL queries. Galow opens an internal Studio widget with the necessary queries or API connections already attached and searches for his personal email address. The widget runs a real database query.

Its conversation history exposes the iteration behind the finished interface. The first attempt had a problem; Galow asked Studio to continue. A later request fixed a visual bug in the type column so the values appeared in one clean column. The resulting table shows his Cursor login activity and whether attempts were blocked, including a deliberate self-block in a test environment.

Support staff also use Studio directly in Slack to retrieve a customer’s sessions and investigate what went wrong. A one-off issue can remain a question and answer. When the same question recurs, the team can turn it into a shared widget. This lets support develop its own tooling without repeatedly asking a platform or data team to build and modify dashboards.

Making Studio useful required more than giving a model access to tools. Galow groups the reliability work into sequencing, layering and validation. Sequencing controls how the agent approaches each new request:

1. Check that the tools are connected correctly.
2. Determine whether the question includes enough context; ask clarifying questions if it does not.
3. Follow a checklist to choose the appropriate tools.
4. Inject the relevant tool guidance when the agent decides to invoke that tool.

For Snowflake, this guidance is substantial: it describes how teams connect to environments and the resources they use. Loading it at invocation time avoids filling the context window with lengthy instructions for every available tool before knowing which ones the question needs.

Layering combines the base Studio prompt, defaults and organization rules for particular tool configurations. When someone edits a tool, its existing context should remain intact. Studio also instructs the model to distrust potentially outdated knowledge about WorkOS products and consult primary sources, including the current documentation. That matters when product behavior changes faster than the model’s training knowledge.

Validation happens before a generated query becomes part of a dashboard. Studio runs Snowflake queries and checks whether they return data before embedding them in widgets. A query can be valid SQL yet return no rows, leaving the original question unanswered. Executing and inspecting the query catches that failure before the interface makes it look like a finished answer.

WorkOS also runs evals in both staging and production, treating the two environments consistently so that development exercises the experience teammates will use. Galow does not detail the evaluation design or report scores.

The first audience question asks whether WorkOS had to clean up its Snowflake estate before building Studio. Galow says it did not. He describes one customer-to-user relationship that requires four joins: new employees previously learned to copy a particular join block, whereas Studio receives that knowledge in its guidance. Descriptive column names help the model interpret the schema, but explicit relationship instructions still matter.

Radar illustrates why. Its data distinguishes login attempts from detections, and the guidance explains how those tables join. Studio can then query the underlying data directly. This implementation has no RAG database; it uses tool invocation with added context.

Another questioner, who uses a similar approach to explaining joins, raises the harder issue: what happens when an incorrect generated query becomes a shared widget and everyone starts treating its output as truth? Galow responds with a trust-but-verify approach rather than describing a formal audit process.

The examples he gives are semantic rules that a plausible query can omit:

• Deletion state: exclude deleted entities when the question concerns current usage.
• Active status: filter to the relevant active records.
• Counting scope: a count grouped by customer ID still needs those filters to answer the intended business question.

These rules belong in the context supplied to the agent. Galow reports that doing so removed many problems for WorkOS and describes larger remaining errors as easier to spot. He supplies no measured accuracy. A query returning rows establishes that it ran; it does not establish that it counted the right population.

A widget can combine data from multiple connected tools into one interface, and WorkOS is adding more connections. The question of how such a widget refreshes reveals an important boundary: the generated artifact is JavaScript that makes the underlying service calls through the tools. Refreshing a widget does not ask the LLM to repeat its investigation. It executes the code and re-queries the data. The model returns when someone requests a change, such as adding a column.

User inputs become query arguments handled by that code. In the email-search example, the application does not need a model to reinterpret the email every time someone searches. The following JavaScript illustrates that boundary with an injected query function and result renderer:

function createLoginSearch(queryAttempts, renderRows) {
  let currentEmail = null;

  async function refresh() {
    if (currentEmail === null) {
      throw new Error("Enter an email before refreshing.");
    }

    const rows = await queryAttempts({ email: currentEmail });
    renderRows(rows);
    return rows;
  }

  async function search(email) {
    const value = email.trim();
    if (!value) {
      throw new Error("Email is required.");
    }

    currentEmail = value;
    return refresh();
  }

  return { search, refresh };
}

Both search and refresh use the same query function; neither requires model inference. This is the runtime reliability Galow emphasizes: ordinary inputs follow a stable code path. Whether the query expresses the correct business definition remains a separate question.

At the time of the demonstration, Studio integrations are per user: each employee connects their own Snowflake, Linear and Notion accounts. Galow describes that setup as inconvenient and points to a broader access problem: an employee might reasonably need selected Salesforce data without having a Salesforce account. WorkOS Pipes powers Studio’s third-party integrations underneath.

WorkOS is building organization connectors so one person can configure a connection and set default access rules. Galow’s proposed Linear example grants read-only access by default, with edit or administrative access determined by roles in Studio. That permission layer is work in progress during the talk. The connector screen shows existing Linear, Notion and Snowflake connections, not completed role-based controls.

The final question asks about Opus costs and caching. Galow returns to the distinction between building a widget and using it: generated widgets execute code, so each refresh does not incur another LLM charge. For the questions and generation steps that do use the model, WorkOS is willing to pay for answer quality.

Galow says Opus performs sufficiently better for WorkOS that switching to cheaper alternatives would sacrifice an unacceptable amount of quality. He gives no model version, comparative benchmark, cost figures or caching strategy. The economic choice is nevertheless clear: spend on the investigation and construction when quality matters, then let the resulting code serve repeated use without repeating the model call.