Agentic Search for Context Engineering

An agent can draw on code files, databases, the web and memory. Which pieces should enter its context window so it can answer the current question? That is the opening problem in Leonie Monigatti’s workshop. Working at Elastic, the company behind Elasticsearch, she focuses on the mechanism inside the arrow between available context and selected context: search tools.

Her framing—that context engineering is about eighty percent agentic search—is a personal hot take, not a measured allocation of engineering work. Its practical point is that choosing context depends on how the agent can find it. A larger collection of sources does little good if the available search interface cannot retrieve the relevant material.

The familiar starting point is a fixed RAG pipeline: use the user’s message almost verbatim as a vector-search query, retrieve chunks, and send those chunks alongside the message to the LLM. This makes retrieval mandatory even when the question needs no additional context. Unnecessary results can confuse the model. Conversely, a single retrieval step can be insufficient: a returned chunk may reveal a fact that requires a second search.

Agentic RAG turns retrieval into a callable tool. The agent can decide whether to search, inspect the response, rewrite the query and search again. Extending that design to context engineering adds more sources: local code, scratchpads and plan.md; skill folders; enterprise databases; the web; and long-term memory. Monigatti deliberately leaves open whether that memory belongs in files or a database.

These interfaces expose different operations. A semantic-search tool accepts a topic; a general database tool may accept a complete SQL query. Both retrieve context, but they demand different work from the agent.

A terminal interface cuts across these boundaries. LangChain calls it a shell tool, Anthropic a bash tool, and OpenClaw an exec tool. Through it, an agent can run ls and grep, invoke a database CLI, write a script that connects to a database, or use curl against an HTTPS endpoint. The same interface can support file exploration, database access and web requests.

That versatility raises the central design question: is a shell enough, or should the agent also have specialized search tools? Search itself remains difficult. Keyword search, dense embeddings, sparse embeddings, multi-vector embeddings and indexing choices address different retrieval needs. Choose the search stack around the required search behavior and latency, rather than assuming that one interface supplies every useful retrieval capability.

The apparent simplicity of agentic search hides several failure points: the agent must choose a tool, generate its parameters, interpret the response and answer. Monigatti highlights three recurring failures from work with teams building Elasticsearch agents:

• No tool call: the agent answers from parametric knowledge when it should retrieve context.
• Wrong tool: it chooses an unsuitable source. One colleague struggled to make an agent search a database instead of the web.
• Wrong parameters: it selects the right operation but generates arguments that do not express the intended search.

Improve descriptions incrementally. Start with the tool’s core purpose. If additional tools or parameters make selection unreliable, add conditions explaining when to use it and when not to. Then describe relationships: load a particular skill before executing a query, or obtain confirmation before performing an operation. If the description still does not guide the agent reliably, reinforce the instruction in the system prompt.

Parameter complexity is part of the failure surface. Adding filters and result limits increases the number of decisions; generating an entire query also requires knowledge of a language and the underlying schema. Models vary in how well they handle that responsibility, so expressive interfaces may need more assistance than simple lookup tools.

The workshop demonstrations follow three interfaces over conference-session data: semantic retrieval from a local Elasticsearch cluster, general database query execution, and shell search over local files with a semantic CLI extension. The database already contains chunked sessions; ingestion and chunking are outside the demonstration.

LangChain supplies the agent abstractions, shell tool and examples for skill loading. The first agent uses GPT-5.4 nano. Its system prompt asks it to decide whether additional context is needed before answering and explains the indexed schema. The embedded text combines each session’s title and description. Day, time, room and speaker are metadata: in this setup they can be filtered, but they are not part of the semantic representation.

Jina Embeddings V5 embeds the query at search time through the Elasticsearch vector store. The tool calls similarity_search with a fixed k=3, so each invocation can return only three results. LangChain’s @tool decorator derives the tool name from the Python function name and its description from the docstring. A factory makes that small interface explicit:

from langchain.tools import tool


def make_semantic_search(vector_store):
    @tool
    def semantic_search(query: str) -> str:
        """Find conference sessions related to a topic."""
        documents = vector_store.similarity_search(query, k=3)
        return "\n\n".join(
            f"{doc.page_content}\nMetadata: {doc.metadata}"
            for doc in documents
        )

    return semantic_search

The short description is sufficient for this demonstration because the agent has only one search tool to choose from.

A direct search for regulatory constraints finds Bilge’s session on engineering AI systems under sovereignty constraints, along with talks by Tejas and Pedro. Monigatti then combines the LLM, system prompt and tool into an agent, intentionally omitting memory. Asked which sessions discuss regulatory constraints in AI systems, the agent searches, rewrites its query, receives similar results and answers with the relevant talks. The loop works for this conceptual question.

A successful topic search leaves substantial gaps. An audience member suggests asking for something absent from the database. Exact keywords and metadata filters offer other ways to break this tool: filters exist in the data model, but the tool has not exposed them. Monigatti asks which sessions she should attend to learn about GEPA. The semantic tool returns material about DeepMind’s Gemma models, harness engineering and another unrelated session, despite a known GEPA session in the dataset. She speculates about tokenization similarity between GEPA and Gemma, but does not establish the cause.

The next interface accepts a whole database query instead of a topic. Monigatti pairs that more expressive tool with a skill-loading tool, then switches to GPT-5.4 mini because she expects query generation to be harder. Initially, the system prompt stays unchanged.

ES|QL, Elasticsearch’s piped query language, supports filtering, transformation and analysis. A direct query finds Samuel’s GEPA session. Wrapping the client’s query method exposes those capabilities to the agent. This time the wrapper also catches exceptions and returns the error as a tool response, giving the agent a chance to revise its query instead of terminating the interaction:

from langchain.tools import tool


def make_query_tool(client):
    @tool
    def execute_esql(query: str) -> dict:
        """Execute ES|QL against the indexed conference schedule."""
        try:
            response = client.esql.query(query=query)
            return {
                "columns": response["columns"],
                "values": response["values"],
            }
        except Exception as exc:
            return {"error": str(exc)}

    return execute_esql

Returning errors supplies feedback for correction; it does not guarantee that every unsuccessful search will produce an error.

Asked the GEPA question again, the agent generates a query that looks plausible but uses SQL-style % wildcards. In ES|QL pattern matching, the wildcard is *. The demonstrated %GEPA% pattern searches for literal percent signs around GEPA, so it returns no rows.

An empty result is not proof that the requested information is absent. It can also mean that a syntactically acceptable query expressed the wrong search. The exception handler cannot repair this by itself because there is no database exception to return.

A richer tool description or system prompt could explain the wildcard rule. For a language with more documentation than a few lines, Monigatti instead introduces an agent skill. Although official Elastic skills are available, she uses a short custom skill. Its name and description enter the system prompt; its body loads only when needed. This progressive disclosure keeps detailed instructions out of the initial context.

The custom skill covers basic query structure, double-quoted string literals and asterisk wildcards. LangChain’s skill-loading tool and middleware handle loading. The execution tool’s description now requires the Elasticsearch ES|QL skill before query execution, and the system prompt repeats that dependency. The resulting sequence is:

1. Load the ES|QL skill into context.
2. Generate the query using its syntax rules.
3. Execute the query against the conference index.
4. Use the returned session information to answer.

On the next GEPA attempt, the agent follows that order, uses asterisk wildcards and identifies the session scheduled for 10:40.

The same interface can answer an analytical question: how many sessions are on April 8? After loading the skill, the agent generates a date filter and a count aggregation. The database aggregation returns 27 sessions for April 8 in the workshop’s indexed dataset.

Returning every matching session and asking the LLM to count would consume context and rely on the model’s counting ability. The database can perform the calculation and return the compact result instead. This is a broader benefit of query execution: the retrieval layer can compute the answer’s supporting facts, rather than merely supply documents from which the model must derive them.

For the filesystem demonstration, the conference data moves into a session-data directory. Session-type subfolders, such as keynotes and workshops, contain one file per session, with its title, metadata and description. Monigatti returns to GPT-5.4 nano, which she considers sufficient for this navigation task, and replaces the database-schema explanation in the prompt with the file layout.

Terminal access also permits destructive operations. Monigatti warns that the demonstrated LangChain shell tool has no default safeguards and recommends a sandbox. After importing and instantiating the tool, she illustrates its commands parameter with an echo command that prints a greeting. That simple test establishes the interface: the agent supplies shell commands and receives their output.

The GEPA lookup now becomes a sequence of file operations. The agent inspects the directory structure, searches with grep using an initial first-fifty limit, then searches the full dataset after finding one session. It finds the same session again, reads the complete matching file and uses its contents to recommend the session. The final file read matters: locating a matching word is only the first step toward obtaining the metadata and description needed for an answer.

grep matches text and regular expressions, but an agent can approximate conceptual search by generating related terms. Asked again about regulatory constraints, the agent first searches for regulat, then expands into compliance, constraints, GDPR, governance and sovereignty. The basic operation looks like this:

grep -RniE 'regulat|compliance|constraints|GDPR|governance|sovereignty' session_data/

Repeated searches eventually locate Bilge’s session. The agent has supplied the semantic expansion; grep still performs lexical matching.

This can work, but enumerating terms is an awkward general search strategy. Monigatti illustrates the limitation with movies about animal superheroes: should the agent list every possible animal to find all relevant films? The problem is not whether it can generate another synonym, but how much searching it must do to cover a concept.

Semantic CLIs add another operation behind the same terminal interface. Monigatti names LlamaIndex’s SemTools, LightOn’s ColGREP, which uses multi-vector embeddings, and JinaGrep. For JinaGrep, she installs the CLI and updates the existing system prompt to explain its availability, usage and examples.

JinaGrep also has classification and reranking modes, but this demonstration uses semantic search. The prompt distinguishes the two search paths: retain grep for exact matches and use JinaGrep for semantic or fuzzy queries. Asked about regulatory constraints, the agent explores the folders and then successfully invokes JinaGrep. In this demonstration, its first JinaGrep search finds Bilge’s session while requesting a top K of 10. The returned set includes other sessions, and the agent uses the results to answer.

The demonstrations suggest a mixed toolset rather than a single universal search tool. Monigatti describes the balance using two interface-design ideas:

• Low floor: specialized tools make routine operations easy. A customer lookup by ID or a simple semantic search exposes few parameters, can suit a smaller model and avoids unnecessary iterations.
• High ceiling: shell access and complete-query execution let the agent handle complex or unexpected requests that narrow tools cannot express. Their flexibility can require more attempts before the agent reaches a useful answer.

When query behavior is unknown, begin with a general-purpose tool and log what the agent does. Monigatti treats repeated sequences of four or five tool calls for a question as a reason to investigate whether the interface is too difficult—not as a universal limit. Inspect what those calls are trying to accomplish, then consider exposing that recurring operation through a specialized tool.

In her OpenClaw experiment, she logged database interactions through exec and later asked the agent to identify patterns. It recommended dedicated database search tools. The design process is therefore empirical: start with an expressive interface, observe the behavior and failures, and add purpose-built interfaces for the operations that recur.

The Q&A first asks how much the toolset depends on model capability. Monigatti recalls a substantial reduction in parameter errors with stronger models in internal testing, but does not recall the figures. A stronger model can help with general-purpose tools; it does not remove the need to handle mistakes.

Latency raises a separate architectural question: should a system retain a simple RAG path for fast answers and use agentic retrieval only for harder questions? Monigatti does not offer a routing algorithm. She emphasizes that ordinary RAG remains effective for many applications, while selecting between retrieval paths may itself require decision-making logic. The existence of an agentic alternative is not, by itself, a reason to replace a working fixed pipeline.

Returning to the GEPA failure, an audience member proposes deterministically replacing SQL-style percent signs with ES|QL asterisks. Monigatti clarifies that a skill was not necessary to fix that particular example: a simple instruction against percent-sign wildcards worked while she built the demo. Her concern is what happens next. Repeatedly adding isolated syntax instructions can eventually recreate the query language’s documentation inside the system prompt. The skill provides a place to load that knowledge when needed; her response does not implement or evaluate the proposed deterministic repair.

Could an agent use both database and shell tools, then validate results across them? Monigatti agrees that the single-search-tool demonstrations simplify the real problem of selecting among several interfaces. She points to Testing if “bash is all you need”, an experiment comparing database, Bash, filesystem and hybrid approaches. Her recollection highlights their complementary strengths: database operations for analytical questions, file search for quick lookup, and shell-based checking of database results. The published experiment’s hybrid conclusion is about more consistent accuracy through cross-checking, rather than a uniquely higher maximum score.

More retrieval also introduces more irrelevant material. Asked whether semantic search should use a conservative score threshold, Monigatti notes that in the final JinaGrep example she considered only the first result relevant. Models can often reject irrelevant results while composing an answer, but those results may remain in context and confuse later turns. Threshold choice therefore depends both on immediate relevance judgment and on how the application manages retrieved material over a long conversation.

An audience question then raises search subagents and relevance evaluation, mentioning Ragas. Monigatti has not experimented with subagents herself. She tentatively recalls Claude Code delegating niche product questions to a specialist helper and sees that division of expertise as interesting, but offers no tested design or performance recommendation.

Progressive skill loading postpones context growth, but a long session can still accumulate many full skill bodies. The final substantive question asks when and how to clear them. Monigatti passes the question to her Elastic colleague Joe, who describes using a file store to offload content as context grows. Loaded material and messages can be replaced with references to their stored locations.

In Joe’s account, the agent retains skill names, descriptions and file-store locations, loads a body when needed, and offloads it as the conversation progresses. He applies the same approach to context compaction and previous tool results: keep the material in the file store and provide tools such as grep to recover it. Search then serves both sides of context management—bringing useful material into the active window and making offloaded material retrievable again.