Distill the LLM, Don't Serve It: Search & Personalization at DoorDash — Raghav Saboo, DoorDash

A shopper who has just adopted a puppy might begin with a search for puppy food. But the job is getting through the first week: food, a crate, a leash, training pads. Those needs can unfold through suggested searches and collections across several sessions and days. In this talk, Raghav Saboo, a staff machine learning engineer working on search and personalization at DoorDash, explains how shared semantic representations help different models follow that larger journey.

The expansion beyond restaurants into grocery, retail, pets and gifting makes this problem harder. Engagement optimization can learn which products people tend to click or buy. It does not automatically explain what an item means in a particular context, or what a shopper intends to accomplish. Saboo identifies semantic understanding as the bottleneck and develops four building blocks in order: supervision, catalog semantics, consumer memory and steerable content generation.

Consider the query “gluten-free pasta.” Regular spaghetti can rank well because it sells well. Gluten-free white bread can enter the candidate set because it shares part of the query. Both miss the request in different ways: the spaghetti violates the dietary constraint, while the bread misses the product type. Graded relevance gives the system a more useful target. True gluten-free pasta is highly relevant; chickpea pasta is presented as a possible substitute; popular regular spaghetti fails the stated constraint.

Two existing sources of supervision each have a cost:

• Human annotation: People can judge query–item relevance, but labeling is expensive and slow. A rapidly changing catalog also makes those judgments stale.
• Behavioral signals: Clicks and purchases are abundant, but exposure, position, price, promotions and earlier model choices all influence them. An item selling well is therefore a reason to investigate its relevance, rather than a complete answer.

DoorDash starts with a human-labeled seed set of query–item pairs on a three-level relevance scale: 0, 1 and 2. Suspicious disagreements receive extra attention. If an item labeled irrelevant nevertheless has strong add-to-cart or conversion behavior, a stronger LLM reevaluates it with more granular prompts. Query-to-category models provide another check: a query mapping to valid categories can prompt a label adjustment when the item belongs to those categories. These checks build the high-precision golden dataset used to train the scalable labeler.

A lightweight LLM is then fine-tuned on that dataset and run offline across the catalog to generate graded query–item labels. Retrieval and ranking learn from the same target. This is the first concrete form of the title’s advice: spend on reasoning during preparation, then transfer what it learned into models that can serve quickly and cheaply.

Retrieval needs more than a general sense that two things are related. At e-commerce scale, standard embeddings can place exact matches, substitutes and complementary products too close together. For the pasta query, that distinction determines whether the candidate set contains products satisfying the request or merely products associated with its words.

DoorDash uses two-stage contrastive training on the graded labels. The first stage shapes the overall embedding geometry with two-tower encoders and a multilevel supervised contrastive loss. The resulting model then exposes its own difficult cases: negatives ranked too high and strong positives ranked too low. Those cases are relabeled and used in a second stage of curriculum training. Training becomes more focused because the first model identifies the distinctions it still cannot make.

What changes between the two training stages? The flow below shows how a model’s ranking mistakes become the next training material. Saboo describes overlap between relevant and moderately relevant items after stage one, followed by substantially better separation after stage two. He reports a 2.3% improvement in relevance NDCG, a metric for the quality of an ordered result list. The talk’s percentage improvements do not specify experimental windows, sample sizes, baseline details or whether the changes are relative percentages or percentage points; they describe DoorDash’s reported outcomes rather than a portable expectation.

The ranking model receives a related change. An ordinal relevance tower learns from the LLM’s graded labels alongside existing towers predicting clicks, add-to-cart events and conversions. The towers share lower layers, so their training updates teach a common representation about both semantic fit and engagement. The relevance tower predicts probabilities across relevance levels; a value function blends those predictions with engagement objectives for each product surface. The system can therefore learn the difference between gluten-free pasta and popular regular spaghetti while retaining the business signals needed to order useful results.

The pasta example now has a causal path through the architecture: judgments distinguish constraint satisfaction from popularity; an offline labeler expands those judgments; retrieval learns finer separation; and ranking learns an explicit relevance prediction alongside conversion predictions. The observable result reported here is improved aggregate relevance, rather than a measured before-and-after result for that individual query. Keeping the production retrieval and ranking architecture also lets DoorDash test these improvements while retaining its serving and business objectives.

At DoorDash’s scale, the catalog contains a few billion store-level items. A unique item identifier distinguishes records but says nothing about meaning. A human-curated taxonomy supplies meaning, yet can be too coarse and rigid: placing products under sauces and hot sauces does not express the finer relationships among them. Semantic IDs provide a short hierarchical code whose granularity can be controlled and whose structure can be used by downstream models.

The beginning of a semantic ID identifies a broad neighborhood; later tokens express finer distinctions. In the hot-sauce example, products share their first and second prefixes, then split into specialties such as Mexican, Caribbean and Korean hot sauces. Saboo describes that structure as emerging from data with zero labels. Comparing prefixes gives models a way to recognize shared meaning at different resolutions, without requiring every distinction to be a manually maintained taxonomy branch.

That shared representation supports several distinct uses:

• Cross-category relationships: Chips, salsa and guacamole may occupy different taxonomy branches, while a semantic neighborhood can connect them as parts of one shopping mission.
• Cold starts: New store items can enter models through encoded semantic tokens used as sparse ID features.
• Tail coverage: Items with little interaction data can inherit signal from semantically related items, reducing the need to wait for their own exposure and purchase volume.
• Catalog audits: Comparing learned semantic groupings with human labels can reveal disagreements worth inspecting.

Semantic IDs reportedly improved ranking MRR by 4–5%, with conversion gains as well. MRR measures how early the first relevant result appears. The same representation also supports query reformulation: a search for Sriracha can lead to a suggestion for chili garlic sauce because the queries map into a catalog-grounded neighborhood. Inventory matters here. A plausible suggested query has little value if DoorDash has no products to satisfy it; connecting the query graph to catalog semantics makes the suggestion useful for shopping.

Once items have useful representations, the next question is what the system knows about the shopper. User embeddings are valuable, but they do not necessarily explain why someone has a preference or intent. An LLM also cannot readily consume an existing user embedding without additional work to learn its representation. Consumer memory supplies an explicit semantic counterpart that can be inspected and reused across recommendation models and LLM applications.

The framework combines three kinds of context. Saboo calls these three timescales, although the third describes how information is supplied rather than how long it lasts:

• Long-term memory: Preferences inferred from orders, searches, browsing and support interactions.
• Real-time context: The current cart and active searches within a session.
• Stated preferences: Constraints and preferences explicitly supplied through agentic interactions, such as Ask DoorDash.

Long-term memory is organized into blocks that can grow as the system learns more: dietary preferences, dining preferences and substitution preferences, for example. Separating this representation from downstream applications means each application does not have to reinterpret the shopper’s history for itself. Each consumer’s memory is materialized in several forms: compact, human-readable text; embeddings of memory blocks for retrieval and ranking; and graph or hierarchical relationships connecting consumers with brands, taxonomies and revealed preferences.

A context graph is useful because shopping relationships are sparse and often require several hops. Connecting a consumer to extracted memory concepts creates relationships that direct consumer–item interactions may not provide, particularly at fine taxonomy levels. Saboo reports that graph-based embeddings outperform DoorDash’s existing taxonomy-based embeddings for retrieval. The resulting composite memory objects already feed personalized collections, Ask DoorDash sessions, and features for retrieval and ranking.

The fourth building block brings the earlier representations together. Semantic IDs, memory blocks and relevance supervision can feed LLMs, smaller language models and traditional models. Their outputs can take different shapes: ranked semantic ID lists, carousel titles and supporting copy. Personalized collections on store pages are a concrete application of that shared vocabulary.

Historically, store collections came from a fixed library or product attributes. The newer approach generates collections for individual consumers. An offline LLM process takes consumer memory and semantic IDs and synthesizes the title, subtitle and items. Control matters as much as generation: the system can shape collections around occasions and moments, then use its existing retrieval and ranking stack for item hydration and collection ranking when serving shoppers.

Where does the expensive generation end, and the serving stack begin? The diagram separates batch collection creation from item hydration and ranking. That division lets LLMs produce a richer collection concept while established models handle the delivery of the shopping experience. The talk specifies offline generation but does not detail its refresh cadence, which limits how precisely we can explain the timing of responses to new occasions or session changes.

The visible changes are specific to the shopper:

• Plant-based affinity: A store page can show plant-based pantry rows.
• Cat-only households: Collections can focus on cat dry food.
• Pantry restocking: Session context can bias the experience toward pantry staples.

In early pets-category tests, Saboo reports order-rate increases close to 1% and a 6% increase in active users. These are the business outcomes of the collection experience, distinct from the earlier retrieval and ranking metrics.

The ending returns to three decisions. Discovery needs an understanding of item meaning and shopper intent alongside engagement. Expensive reasoning can be captured offline in labels, semantic IDs and memory, then served through smaller, faster models. And those representations should be shared: the same work can support retrieval, ranking and content generation rather than being rebuilt separately for every feature.

Saboo’s closing architectural judgment is that an online LLM call is often not the product architecture you need. DoorDash’s examples give that judgment substance: a relevance label can teach both candidate retrieval and final ranking; a semantic code can improve ranking and suggest an inventory-backed search; consumer memory can personalize an agent session and a store collection. The useful output of reasoning is something the rest of the system can keep using.