360Brew: LLM-based Personalized Ranking and Recommendation — Hamed Firooz and Maziar Sanjabi, LinkedIn AI

When someone opens a feed to read an article or searches for a job, the system must find content relevant to that person’s interests. Search and online shopping pose similar matching problems, yet their rankers are often trained and optimized separately. Each new architecture then has to make its way through multiple development and deployment processes. Could one model handle those tasks together?

That is the starting point for 360Brew, the personalization project Hamed Firooz and Maziar Sanjabi describe. The aim is a foundation model that understands a member’s journey across LinkedIn well enough to support its different personalization tasks. Three capabilities would make that consolidation useful:

• Zero-shot recommendation: Describe a new task, its user and its candidate entity, then ask about relevance—without first collecting a task-specific dataset and deploying another ranker.
• In-context learning: Give the model a few interactions, or a description of a member’s interests, to support personalization when behavioral history is sparse.
• Instruction following: Let members specify niche interests or topics they want to explore, and use those instructions to guide recommendations.

These are different ways to adapt the same model: change the task description, provide examples, or express a preference directly.

The first implementation step is promptification: turn profiles, history and recorded interactions into text the LLM can consume. The prompt begins with an instruction defining the task, adds information about the member, supplies past interactions, and finally introduces a new item. The prediction question is what the member will do with that item. This makes the task instruction part of the input rather than fixing the task entirely in a separate model’s architecture.

For a feed-engagement task, the structure can be expressed as a prompt like this:

Task: Predict whether the member will engage with the candidate post.

Member profile:
Software engineer interested in database performance.

Past interactions, oldest to newest:
- Liked a post about query optimization.
- Was shown a post about office furniture; did not engage.

Candidate post:
How database indexes change query execution plans.

Question:
Will this member engage with the candidate post?

The profile, observed responses and candidate all remain explicit. A general-purpose LLM can already do something with that information, Sanjabi explains, but adequate recommendation quality requires training for the problem.

The development pipeline separates maximizing quality from maximizing serving efficiency:

1. Start with an open-source model. Apply upcycling to control model size and the tradeoff between throughput and quality.
2. Train for the domain and tasks. Run continued pretraining, fine-tuning, instruction fine-tuning and alignment.
3. Establish a large teacher. Sanjabi describes Brew-XL here as a 150-billion-parameter model optimized for quality.
4. Distill into a smaller student. He gives a 3B model as an example of a size that can be made practical for serving.

The large model is an intermediate asset: recommendation traffic demands too many predictions to treat the highest-quality teacher as the default online serving model.

The number of stages makes automation part of the model-development method. Repeatedly moving through training, alignment and distillation would otherwise make iteration cumbersome. Sanjabi says the team also tried avoiding the large teacher, but directly training a small model did not work as well. First establish the capability at large scale, then transfer it into a cheaper model is the recipe their experiments supported.

There are three distinct ways to improve the model. The first is training-data scale. Recommendation systems may have six months, a year or more of behavioral logs, depending on what they record. In the experiment Sanjabi presents, increasing the amount of training data improves performance. That makes the available interaction history a resource for continued model development, not merely an input to individual predictions.

The second lever is model capacity. Sanjabi reports improved performance in a Mixtral architecture experiment spanning 7B to 8×22B. This is especially useful for the XL teacher, where the immediate objective is quality before compression.

The third lever is context length, which determines how much of a member’s history can accompany a prediction. Supplying more history initially improves performance in the presented experiment, but the curve eventually turns downward. Sanjabi attributes that decline to the tested model’s difficulty generalizing to longer contexts, rather than to the additional history becoming less informative. A larger context window therefore creates an opportunity to use more evidence; the model still has to use that evidence effectively.

Cold-start members test whether the model can personalize without a rich interaction record. The presented cohorts include members with fewer than five interactions and members with fewer than 100. Sanjabi reports that 360Brew’s advantage over the production models grows as the number of interactions decreases. He interprets this as a benefit of the LLM’s world knowledge: the model can bring useful prior understanding to profiles and content when behavioral evidence is limited.

Sanjabi reports that, on four tasks whose surfaces were excluded from training, 360Brew matched or exceeded some production models trained specifically for those tasks. The relevant comparison is against specialized models with exposure to their own task, not against untrained baselines. These are model-quality comparisons; they do not establish online business gains or replacement of the production rankers. The proposed operational benefit is faster rollout: a new feature or surface could begin with the shared model instead of waiting for a dedicated recommendation system.

Serving changes the objective. Firooz describes recommendation workloads requiring tens of thousands of queries per second and roughly 400–500 ms latency. Meeting that demand requires combining smaller models, sparsification and quantization rather than relying on model size alone.

The distillation process reduces capacity in stages. Firooz describes an illustrative progression from a large teacher through 8B, 3B and 1B students, with each student learning from the preceding model. His account uses both 250B and 150B for the starting teacher, so the exact teacher size in this serving recipe is inconsistent; the earlier Brew-XL description uses 150B. The stable result is the comparison between procedures: staged distillation worked better for the team than jumping directly from 150B to 1B. Starting small also performed poorly, which Firooz attributes to insufficient capacity and reasoning ability for the tasks.

Pruning follows the same gradual pattern. Transformer redundancy creates opportunities to remove attention heads, MLPs or layers; reducing activation and parameter precision is another compression option. But removing too much structure at once can damage quality. The team instead alternates a small pruning step with distillation, then repeats. Distillation provides a recovery stage before the next reduction.

In the presented pruning experiment, Firooz describes gradual pruning as approaching no information loss, while aggressive initial pruning caused up to a 1% reduction in model quality. The talk does not specify the quality metric or whether that percentage is relative or a percentage-point change. The practical comparison is between a sequence of small reductions with recovery and one large initial reduction.

Quantization reduces the cost of activations and model parameters, but the team found that applying FP8 uniformly across every layer significantly hurt quality. Their solution is mixed precision, with special treatment for the output that becomes the ranking score.

For this system, Firooz reports that using FP16, BF16 or FP8 for the LM head caused numerical distinctions to collapse, impaired calibration and made recommended items harder to distinguish. Ranking depends on differences between candidate probabilities, so preserving those differences can matter even when much of the model tolerates lower precision. The FP32 head is a system-specific finding, not a claim that every ranking model needs the same precision policy.

Attention is another major serving cost. A recommendation task gives the implementation structure it can exploit: not every history item needs to attend to every other item. Rather than preserve all possible attention links, the model can use a sparse pattern tailored to the information needed for prediction.

The same principle applies to multi-item scoring. Instead of processing one candidate at a time, the system can score tens or hundreds together—Firooz gives 50 and 500 as examples. The crucial constraint is that the candidate items must not attend to one another. Putting them in the same request should not make one candidate’s prediction depend on the contents of the other candidates.

Across four or five successive releases, Firooz reports a 7× reduction in latency and a 30× increase in queries handled per GPU. These are combined gains from the serving optimizations, not measurements of sparsity alone. The talk does not specify the hardware, workload, batch and context settings, or latency percentile needed to reproduce the comparison. The chart captures the intended outcome: more work from each GPU while each query waits less.

The prepared presentation closes by pointing to technical reports and papers sharing lessons from the project. The audience questions then fill in several practical details: how generalization is checked, how multi-item scoring executes, and how history selection interacts with serving.

An audience member asks the natural follow-up to the unseen-task results: after so much fine-tuning, how does the team know the model will work on the next five tasks? Firooz describes a suite of approximately 50–60 internal and external benchmarks. IFEval checks retained instruction following, while LinkedIn tasks excluded from training test transfer to new domains. Those checks address different capabilities: following instructions is not itself evidence of recommendation quality on a new surface. The answer supplies ongoing empirical tests, rather than a guarantee about future tasks.

The next question moves from LinkedIn’s infrastructure to a smaller website. An audience member introduces Microsoft’s NLWeb, then asks what a real-estate site with thousands of listings could use as an out-of-the-box recommender. Sanjabi says he is unfamiliar with NLWeb and does not evaluate it. He also does not identify a ready-made model for the listing-site problem. Making a foundation model useful beyond large companies is part of the project’s ambition, but the answer does not offer an immediately available solution. The audience member recommends investigating NLWeb, and Sanjabi agrees to look.

Does scoring multiple items mean generating multiple answers through several decoding steps? Firooz clarifies that the system keeps the work in prefill. It does not use multi-step generation or speculative decoding for this operation. The candidate items are sequenced together in the input, and the attention pattern prevents them from attending to one another.

The team implemented custom attention-mask kernel work in SGLang and vLLM. Firooz describes query segments containing up to 500 candidates, with candidates attending to member history and profile information rather than to other candidates. These are the team’s serving integrations described in the talk, not a claim that a current default installation exposes the same behavior.

Using the earlier feed example, add a second candidate about office furniture. At the level of whole prompt regions, the relevant cross-region connections are:

Candidate query                  Profile + history   Other candidate
Database-index post              allowed             blocked
Office-furniture post            allowed             blocked

Both candidates can use the same evidence about the engineer’s interests and past responses. Neither can use the other candidate as additional evidence. This is the separation the mask must preserve; the table abstracts away token-level attention within each region. No engagement outcome has been assigned—the operation organizes the evidence available when the model computes the scores.

A member’s history can include job applications, job postings and many other entities. Fitting that history into context raises a selection problem before inference begins. Firooz describes experiments with three approaches:

• Retrieval: Use a RAG system to find historical items closest to the current query.
• Chronological ordering: Supply interactions according to their order in time.
• Time-based weighting: Experiment with weight decay over chronological history.

For most of the team’s applications, chronological ordering was sufficient. Firooz connects that result to recommendation’s preference for freshness: recent activity is often particularly useful evidence of what a member wants now.

Selection is not only about relevance or recency. The context also contains a distribution of positive and negative examples, and the team must decide how to balance them and how much history to include. Those are familiar ML engineering questions inside a new input format: converting interactions to text does not remove sampling decisions.

Sanjabi adds a serving constraint: a more sophisticated history-selection strategy must avoid breaking KV caching. Changing which historical items appear, or how they are ordered, can interfere with the cache reuse the serving system depends on. An alternative to chronological history therefore has to earn its place on both prediction quality and serving behavior.

The final question asks how the team manages so many combinations of model and serving changes. Firooz’s answer is concrete: experiment results are automatically pushed into an Excel sheet. A developer investigating quantization can change its parameters and trigger the rest of the flow end to end with a button. Automation makes it practical to compare variants repeatedly, rather than manually shepherding each experiment through disconnected stages.

Most of that automation was built in-house, but it integrates open-source tools including Lightning, vLLM and SGLang. The engineering emphasis is on making those components work well together and optimizing the entire flow. The development system is what lets the team repeatedly move from a quality improvement to compression, serving evaluation and a recorded result.