Can Your AI Analyst Keep a Secret? - Aaron Linsky, Bridgewater & Nikhil Benesch, turbopuffer

An AI analyst receives a question: why have token prices fallen so much in the last month? The model begins investigating and searches for facts to guide its next steps. Public web pages can enter that investigation regardless of which employee asked the question. Internal emails, meeting notes and research reports require a different decision: is this particular user allowed to see this particular document?

That is the search problem Aaron Linsky, head of product engineering at Bridgewater Associates, and Nikhil Benesch, CTO of turbopuffer, develop in this talk. A document can grant access to an individual, a group, or some combination of them. The analyst needs relevant evidence drawn from the documents its user can access. Finding nearby vectors is only part of that job; the search also has to respect a different permitted collection for each user.

Bridgewater’s research process starts with observations about markets and economies and turns tested ideas into systems that trade. Its 1980 bond system was a set of if-then statements on a yellow legal pad. The implementation moved through Excel, Scala and Python, while the learning process kept humans at its center: perceive what is happening, form a hypothesis, investigate it, systemize and integrate successful ideas, then industrialize them so they run every day.

Moving machines toward the center of that process makes access to research material essential. Bridgewater has spent decades building DeLorean, its bitemporal macro time series database. The Vault supplies the unstructured side: about 100 million documents that Bridgewater indexes, organizes and secures. It combines research from 135 external sources with internal writing, meeting recordings and reports. External sources have their own access restrictions, so bringing material into one database does not make it available to everyone.

Investors and AI agents both read from the Vault, and the new content they create goes back into it. Search therefore serves an expanding research collection. In Linsky’s simplified permissions example, user 1 belongs to groups B and C. A document may be accessible through a direct grant to that user or through one of those groups. A vector query has to combine similarity with those access conditions.

The first implementation appeared to work. Then the growing document collection overwhelmed it—Linsky’s description is that it “fell over.” He looked for other providers and tested turbopuffer. The failure establishes the practical requirement for the rest of the talk: permission logic that looks simple in a query must remain affordable when the collection reaches tens of millions of documents.

turbopuffer stores data directly in object storage such as S3, GCS or Azure Blob Storage. Data begins cold there; queries cause the needed data to move into SSD or memory caches. An untouched namespace—or an untouched region within a namespace—can stay in cheaper storage. Frequently accessed data becomes hot and can serve faster searches. Benesch attributes the system’s cost advantage to this arrangement, with the performance comparison to traditional disk-based search systems explicitly depending on the data being hot.

The engine supports keyword, regular-expression and vector search. Benesch focuses on vector search to explain the indexing problem, beginning with the simplest possible implementation: store everything in a list and inspect every item for every query. His engineering preference is to earn complexity with napkin math before introducing it.

Assume each document occupies only one kilobyte. Even an embedding can consume that much: 1,024 dimensions at one byte per dimension already amounts to roughly a kilobyte, before larger document contents enter the calculation. At the assumed scan rate of 20 gigabytes per second, 100 million one-kilobyte documents take about five seconds to inspect, even when all the data is resident in memory. That misses the roughly 100-millisecond response time Benesch says search users commonly want.

At 10,000 documents, the same calculation gives about half a millisecond, so a scan can be perfectly reasonable. Scale changes the decision. At 10 million or 100 million documents, the representation must let a query avoid most of the collection. Fetching the entire collection from S3 makes the scan still less practical: Benesch estimates a minute or more just to pull that data from object storage. Caching helps the cost of each access; an index reduces how much needs accessing.

An embedding represents a concept numerically so that similar concepts can lie near one another. Red shoes and maroon sandals make an intuitive pair; a concrete block is much less similar. The teaching example reduces vectors to two dimensions so they can be plotted. Those axes are not established semantic labels such as size and color—Benesch offers those only as an intuition. Real embeddings commonly have 1,024 dimensions, and may have thousands.

What work does the index let the query avoid? In the two-dimensional example, red shoes and maroon sandals occupy the upper-left region, while concrete block lies toward the lower right. A query marked Q for red shoes can search its local neighborhood instead of examining the whole plot. Nearby vectors supply plausible results because proximity represents conceptual similarity. The spatial relationship explains the saving: searching around Q avoids spending the same effort on distant concepts.

turbopuffer builds a vector index automatically once the stored data is large enough to justify it. A query might then inspect only 10,000 vectors, which Benesch estimates can take a millisecond or two. Permissions disturb this useful shortcut. In the next version of the example, plus signs mark documents the user may access and minus signs mark documents they may not. A neighborhood can be close to Q yet contain few usable results.

Two straightforward strategies choose a different starting point:

• Post-filtering: Find nearby vectors first—perhaps the closest 10,000—then discard documents whose permissions do not match.
• Pre-filtering: Find every document the user can access first, then scan those documents to determine which vectors are closest to the query.

Why can a nearest-neighbor search miss the nearest accessible results? Suppose the user can access only documents in the lower-right region of the example. Searching around Q first spends its effort on forbidden points. Expanding outward may eventually reach accessible documents, but the search has a time budget. It can stop after a couple of seconds with only 40 of the requested top 100 results—or with none. A user searching for red shoes may know that an accessible maroon-sandals document exists and still fail to retrieve it. This is the recall problem: relevant, permitted documents are missing from the returned set.

Pre-filtering avoids spending similarity calculations on forbidden documents, but the permitted set can itself be enormous. If Aaron can access 50 million of the Vault’s 100 million documents, scanning the permitted half brings back the original bandwidth problem. The filter index has found the right population; it has not used vector proximity to make that population cheap to search. One strategy risks running out of useful candidates, while the other risks doing far too much work.

turbopuffer combines the two approaches through a clustered vector index. Similar vectors are grouped together. The permission indexes also identify which clusters contain at least one document the user may access. That gives the query a useful middle level between individual documents and the entire collection: it can eliminate a whole cluster when none of that cluster’s documents could become a permitted result.

Which clusters survive for Aaron? In Benesch’s example, clusters 5 and perhaps 2 contain documents he can access, while clusters 3 and 0 contain none. The latter can be discarded before similarity search. Among the surviving clusters, the engine searches the closest ones and scans sequentially inside them. A surviving cluster means that at least one document qualifies; it does not grant access to every document in the cluster. The combination still has to return documents that satisfy the permission filter.

The branching below answers where work disappears. Permissions eliminate clusters with no accessible documents; vector proximity then narrows the remaining candidates. The sequential scan happens inside those selected clusters, rather than across every accessible document in the Vault. This is what turbopuffer calls native filtering: use permissions to guide the vector search, while retaining document-level filtering within the search.

At the time of the talk, clusters contain about 200 documents. Benesch describes that size as a tuning choice and expects it may increase to maximize throughput. The essential mechanism is the combination of permission-aware cluster selection and local scanning; 200 is the reported configuration, not a necessary constant of the design.

The reported result is 120 milliseconds at P90 across the deployment’s namespaces, for a collection of about 100 million documents. P90 describes the latency at the ninetieth percentile, rather than a maximum for every query. The vector count is inconsistent in the spoken result: Benesch says both 450 million and 400 million, so the supported scale is roughly 400–450 million vectors. The application-facing question remains much simpler than the indexing work: how does user 1’s membership in B and C become a search filter? The displayed example uses viewers Contains 1 and groups ContainsAny B C.

The logical rule can be written compactly. What makes a document eligible for this user?

allowed(document) =
    document.viewers contains 1
    OR document.groups contains any of [B, C]

The two branches preserve the Vault example’s access paths: a direct grant to the user, or a grant through one of the user’s groups. The vector query supplies what the user is looking for; this predicate supplies which documents may participate in the answer.

The application expresses that permission logic in turbopuffer’s filter language, and the engine handles query planning underneath it. This division lets the application retain its conceptual access model without implementing the clustered search strategy itself. Benesch describes searches in the roughly 100–200-millisecond range even at 100-million-document scale.

Nested groups remain a limitation. Benesch says they make the problem tricky and invites teams with those structures to work through them with turbopuffer; he does not develop their handling in this talk. The simple B-and-C example therefore teaches the direct translation for flat memberships, while leaving more complicated group relationships as an integration problem.

The ending widens the requirement from document visibility to what Benesch calls an “isolation-native search engine.” Fine-grained permissions operate within namespaces, but organizations can also need controls over where data resides and who controls its protection:

• Regional placement: Keep data in a particular region to meet security or regulatory obligations.
• Network placement: Keep data inside the organization’s own VPC.
• Encryption control: Support keys controlled by the organization, potentially extending that control to its customers.

Benesch presents turbopuffer as having solutions at each of these levels, though this talk explains the mechanics of permission filtering rather than those deployment controls. Bridgewater’s case makes the retrieval requirement concrete: an analyst needs a large shared research collection, yet its useful search space changes with the user. Native filtering makes those access rules part of deciding where to search, allowing the engine to skip work before the document scan becomes prohibitively large.