How Instacart transformed its search and discovery using an LLM-driven approach

A grocery shopper arrives with a list of tens of items. Most replenish products they have bought before; a few are things they want to try for the first time. Search has to support both activities without making an already long shopping trip harder. In Instacart's account, most purchases come through search, so quickly finding a familiar product and discovering something new are responsibilities of the same interface.

Discovery also matters beyond the individual purchase. It gives advertisers a way to introduce products and gives the platform an opportunity to increase basket sizes. A search engine that efficiently retrieves an item but offers no useful next action satisfies only part of the shopping journey.

Two kinds of query expose weaknesses in models trained on engagement data:

• Broad queries, such as snacks, match many products. Products that are not shown cannot collect the engagement signals that would help them rank higher. This creates a cold-start problem within an apparently well-covered query.
• Highly specific queries, such as unsweetened plant-based yogurt, occur too infrequently to provide much training evidence. The intent is narrow, but the model has few examples from which to learn it.

Instacart had made progress retrieving more candidates, improving recall. Precision—whether those candidates actually fit the request—remained difficult.

A physical pasta aisle illustrates a different gap. A shopper encounters unfamiliar pasta brands alongside sauce and other ingredients needed for a meal. Online, customers could find the product they requested, yet related products remained difficult to discover. They had to issue another search, then another, to assemble what the store aisle naturally brought together. Sparse engagement data made that related-product experience difficult to build with the existing methods.

Gudla begins with query understanding, the upstream layer whose interpretations guide retrieval and ranking. It contains several models, including normalization, tagging and category classification. The product-category classifier maps a query into the catalog taxonomy: watermelon, for example, can map to fruits, organic food and broader food categories. Instacart's taxonomy contained about 10,000 labels, roughly 6,000 of them commonly used. Because one query can receive several labels, this is a multi-label classification task.

The previous system combined a fastText-based neural model for semantic relationships with an NPMI statistical co-occurrence fallback. These approaches worked well for head and torso queries—the frequent and moderately frequent searches—but covered the tail poorly. The team also tried BERT-based models. Those produced some improvement, but the shortage of engagement evidence still constrained learning, and the gains did not meet the team's expectations for the added latency. Increasing model sophistication had not removed the underlying data problem.

The first LLM experiment supplied queries and the taxonomy, then asked for the most relevant categories. The outputs looked sensible to the team. An online A/B test was less encouraging: plausible category assignments were not necessarily the assignments Instacart shoppers needed.

The query protein makes the mismatch concrete. A general model interpreted it through foods such as chicken and tofu. Instacart shoppers often meant protein shakes, bars or supplements. The missing input was shopping behavior. The team therefore supplied each query's top-K converting categories as additional context, giving the LLM a commercially grounded set of candidates to work with. Candidate ranking and downstream validation remained part of the process.

The gain was also visible in specificity. For Vernors soda, the previous model returned a fruit-flavored-soda category. The LLM identified ginger ale, a more precise category that helped downstream retrieval and ranking. For tail-query product-category classification, Gudla reports gains of more than 18 percentage points in precision and more than 70 percentage points in recall.

The prompt itself was compact: top-converting categories supplied context, and guidelines specified the expected output. Its simplicity did not eliminate the surrounding processing. It made the model's part of the task easier by supplying useful candidates and a clearer output contract.

Understanding a request does not guarantee that a particular retailer can fulfill it. Instacart serves retailers with very different catalog sizes, so the same query can succeed at one store and return nothing at another. Broadening one percent milk to milk can expose available choices while leaving the purchase decision to the shopper. The previous engagement-trained rewrite model again worked better on head and torso queries than on the data-poor tail.

The LLM approach generated distinct kinds of rewrite rather than treating every alternative as equivalent. For avocado oil, the talk illustrates three relationships:

The last entry is a generated synonym candidate to validate, not a basis for treating avocado extract and avocado oil as interchangeable products. Keeping these relationships distinct matters because broadening a request and preserving its exact meaning are different operations.

On the team's human evaluation data, more capable third-party LLMs improved query-rewrite quality. In online testing, the team reported a large reduction in queries returning no results. That changed the user experience directly: searches that previously ended at an empty page could now offer products to consider.

Instacart's query distribution made batch processing useful: a large head and torso of recurring searches sits alongside a long tail. The serving design separated expensive generation from the live request:

1. Offline: precompute LLM outputs for head and torso queries and cache the results.
2. Online, on a cache hit: retrieve the stored output instead of generating it again.
3. Online, on a cache miss: fall back to the existing models for queries outside the cached set.

The work performed during a live search was therefore usually a lookup for the precomputed queries, while the existing system handled uncovered requests.

Gudla reports improved coverage with little impact on search latency. For the extreme tail, the team was trying a distilled Llama 8B model as a replacement for the existing-model fallback. That was a proposed improvement to the uncovered-query path; the described serving system still used the existing models there.

Beyond individual prediction quality, a collection of query-understanding models creates coordination problems. For Humm, Instacart's brand tagger correctly recognized a kombucha brand, while its spell corrector changed the query to hummus. The same request thus acquired incompatible interpretations before reaching the rest of the search stack. Gudla presents consolidation into a small or large language model as a way to make these outputs more consistent, as well as reduce the complexity of managing many models.

A shared model also creates an opportunity to interpret more than an isolated query. Additional context might reveal that the customer is gathering ingredients for a recipe, allowing the system to generate content for that broader shopping mission. This is the transition from Gudla's query-understanding work to Tenneti's discussion of discovery on the results page.

Tenneti starts where the exact-match experience ends. Once customers added an item to their cart, they often had to search again or leave the page to continue shopping. Discovery content offered two ways forward. If swordfish returned no exact results, the page could suggest seafood alternatives; tilapia was one example shown. If sushi already had plenty of results, complementary groups such as Asian cooking ingredients or Japanese drinks could appear near the bottom of the page. Tenneti reports improved engagement and revenue per search for these discovery experiences.

The content had to add something beyond the existing results, rather than repeat products already shown. It also had to reflect Instacart's domain. In the stated shopping context, dishes refers to cookware, while Thanksgiving dishes refers to food. A useful generator must preserve that difference before it can suggest the next thing to buy.

The initial generation prompt asked for two shopping lists: complementary items and substitutes for the query. Product managers and other internal reviewers liked the results. After launch, however, users engaged less than the team expected. The answers were plausible in everyday terms but did not consistently match what customers were trying to do. Tenneti returns to the earlier protein example: bars and shakes were the shopping intent, while the model suggested chicken, turkey and tofu.

The team augmented the prompt with behavioral context in two forms:

• Top-converting categories, together with query-understanding annotations such as a detected brand or dietary attribute.
• Subsequent queries that users issued after the current query.

The first supplied evidence about how to interpret the request. The second supplied evidence about where shoppers went next, which was directly relevant to generating a useful continuation.

Tenneti reports that adding this behavioral metadata improved both engagement and revenue. The prompt still asked the model to generate related shopping content, but it now included evidence of how Instacart customers actually shopped.

Discovery used a similar separation between generation and serving. Calling an LLM during every search was considered impractical because of latency and, sometimes, cost. Instead, the team processed historical search logs in batches and stored the query, generated content metadata and even products that could appear in a carousel. At request time, recommendations came from a quick feature-store lookup. The stored result included material needed to present the discovery experience, not just an abstract related-query suggestion.

The simple batch-and-lookup outline concealed substantial iteration. One challenge was aligning generation with business outcomes such as revenue. The team changed both prompts and the metadata supplied to the LLM to improve that alignment. Another was deciding which generated content to show and in what order. Generating a relevant set of possibilities did not settle the ranking problem.

Traditional predicted click-through-rate and conversion-rate models—pCTR and pCVR—did not work for this discovery content. The team used strategies including diversity-based reranking to encourage engagement. The new surface therefore needed work on its selection and ordering logic as well as its generation prompt.

The third implementation challenge was evaluation. It required two separate judgments: whether the generated content was correct rather than hallucinated, and whether it met Instacart's product requirements. World knowledge helped with predictions, especially for tail queries, but combining it with domain knowledge was what produced the business gains the speakers described. Semantic plausibility alone was an incomplete test of usefulness.

Evaluation covered both discovery content and query-understanding predictions, and it proved more difficult and more important than the team had anticipated. They used LLM-as-judge evaluation to support that work. Tenneti's closing admission is that they recognized the importance of this step late: evaluation needed to be a central part of developing the system.

An audience member asks whether the approach extends to long conversational requests containing multiple items, rather than searches focused on one item. Gudla points to Ask Instacart, which mapped natural-language requests into search intent. The existing recall and ranking stack then retrieved the products. The answer describes an intent-translation layer feeding conventional search.

Asked what the team learned from that experience, Gudla again emphasizes injecting Instacart context and building a robust automated evaluation pipeline. He then identifies a further requirement: context must survive the handoff from the language model to downstream systems.

In his Mother's Day example, extracting only perfumes loses the intended restriction to women's perfumes. The retrieval system can then return perfumes of all kinds, even though the original request contained a more specific shopping context. Preserving that context through retrieval and ranking is part of fulfilling the request; passing along the generic item term alone is not enough.