Jo Kristian Bergum

Jo Kristian Bergum is chief executive of Hornet.dev, which is building a retrieval engine for AI agents. His work connects large-scale search engineering with a practical question: how can an agent find, inspect, and retain useful evidence without exhausting its context or spending a model turn on every small operation?

From large-scale search to agentic retrieval

Before Hornet, Bergum worked as chief scientist at Vespa. His authored and coauthored technical writing spans product search and recommendation in 2019, transformer-based ranking and billion-scale vector search in 2021, and subsequent work on hybrid search, retrieval-augmented generation, and visual document retrieval. Across these topics, he examines what it takes to turn a retrieval model into a usable system: storing its representations, finding candidate documents, computing relevance scores, and moving data between services.

His September 2024 article on scaling ColPali with Vespa makes those concerns concrete. ColPali represents images of document pages through many visual embeddings, allowing search to account for tables, figures, and layout alongside text. That richer representation creates storage and computation costs: each page contains more than a thousand vectors, which must be compared with the query’s token vectors.

Bergum described an approach combining binary representations, a cheaper Hamming-distance similarity calculation, and phased retrieval and ranking. Candidate selection narrows the collection before expensive scoring, while ranking runs beside the indexed data. Keeping those operations together avoids transferring large quantities of page vectors to an external ranking service for every query. The article also reported the Vespa team’s optimization of the Hamming-based calculation and evaluated the ranking-quality tradeoff on DocVQA. It presented an architecture intended to scale to billions of documents, rather than evidence of a completed billion-document deployment.

At Hornet, his focus shifts toward repeated searches inside an agent’s reasoning loop. The company is developing schema-first APIs for iterative and parallel retrieval, with deployment options intended to place retrieval beside customers’ agents and data. Bergum’s writing examines both the search engine and the harness—the software that controls how an agent searches, reads, verifies, and carries evidence into its next turn.

What changes when agents do the searching

• Agent queries create a different workload. In his analysis of public BrowseComp-Plus trajectories, Bergum examined 19,279 search calls across 830 GPT-5 sessions. The median session used 24 searches, and queries frequently included quoted phrases, dates, and search operators. He argues that systems tuned for short human queries can be poorly matched to these longer, more constrained requests. If an API accepts an operator without enforcing it, the agent may continue reasoning as though a filter was applied.

• Retrieval mistakes affect later decisions. Each search result becomes context for the next query. Missing a useful document early can change the snippets the agent sees, the explanation it develops, and the searches it tries afterward. Bergum therefore treats retrieval as a sequence of dependent decisions: performance on the original question alone does not explain how a retriever behaves throughout the agent’s loop.

• The harness determines how much work fits into a turn. His code-mode comparison held the model, corpus, lexical retriever, scoring, and session limits constant while changing how retrieval operations were expressed. Tool mode allowed one search, read, verify, or curate operation per model turn. Code mode allowed a bounded Python program to fan out searches, fetch documents, scan text, and curate evidence before the next turn. The program ran against a read-only SDK in a constrained sandbox.

On 100 BrowseComp-Plus questions using gpt-5.4-mini, the Hornet experiment increased gold-document recall from 0.265 to 0.437 while reducing prompt tokens by 51% and model cost by 44%. Recall measured the fraction of labeled gold documents retained in the final curated evidence set, averaged across questions; it did not measure final-answer accuracy.

The mechanism explains why the result is useful. Code mode performed more searches in fewer model turns, so the harness rendered session memory into the prompt less often. The model spent more output tokens writing code, but that cost was smaller than the savings in repeated input context. The comparison covered one model, corpus, and lexical retrieval method. Within that setup, it demonstrated a defining concern of Bergum’s current work: improving access to evidence depends on how the search engine and agent harness work together.