Why LLM Recommenders Will Be AI's Biggest Consumer App — Devansh Tandon, Meta

[music]

Welcome everyone. Thank you for coming

out to the LM Rexus track. Um I'll be

sharing the first talk. Uh my talk is

titled tokens and engagement out

training LM recommenders. Um, and I want

to make two big arguments today. The

first is that recommendation systems

scale just like LMS do and that the

field is very early in that scaling

curve. Um, and the second is that the LM

recommener is going to be one of the

biggest consumer applications of AI.

Um, okay. So, quickly about me. Uh, I

currently work at Meta on research and

product. Uh I lead a team called Meta

Recommendations Research uh which is

this group that's training frontier

models, LLMs and recommenders that power

Instagram, Facebook ads, the meta family

of apps. Uh before this I was at Google

for a long time uh working on a lot of

the key ML teams including deep mind and

YouTube. Last year I gave a talk at AI

engineer called teaching Gemini to speak

YouTube about two ideas semantic IDs and

generative retrieval which we also wrote

two papers about which I've linked here.

Um it was really fun and it led to a lot

of discussions and collaborations. Uh

the ideas behind semantic ID and

generative retrieval have really taken

off in the industry over the last year.

Um and they've moved from more research

to now scaled production systems. And

I've seen exciting launches and papers

from YouTube, Meta, Spotify, Door Dash

across the industry. And we have a

couple of examples of that later today.

This year I want to talk about four uh

sections. Recommendation scaling curves,

a framework of four, recommendation

paradigm scurves that we are climbing as

an industry. um sharing the recommener

recipe and finally this consumer AI app

framework.

Um let's start with scaling curves. So I

wanted to start with this landmark

scaling curve paper from 2020 which

feels like a lifetime ago. This is when

Daario was still at OpenAI and Anthropic

didn't exist yet. But the core idea that

this paper shared is the power law of

scaling. As you increase model size,

data, the amount of compute flops

trained uh for model training, the loss

falls on this log linear scale. And this

clean and predictable curve is what

really set off the race for the AI

frontier because you can forecast what

model quality and capability

improvements will look like. And this is

what's underwriting the massive capex

investments and the AI buildout today.

It turns out that recommendation systems

follow a very similar scaling law. In

fact, before this wave of LLMs, Rexs

were the largest production ML models in

big tech companies. And they're still

some of the largest models that are

served at a scale of a billion plus

daily active users.

And they follow this similar power loss

scaling curve. On the x-axis, you have

data, compute, and model size. And on

the Y ais you would see falling loss or

in this chart uh an improvement in

recommendation quality. In offline evals

it's net entropy or AU gains and then

when it's translated to a real

production launch it's engagement impact

revenue impact at some of the biggest

consumer app scale.

Here's a real example from Meta that

demonstrates these power loss scaling

curves. uh there's a paper the first is

a paper HSTU from 2024 and the second is

a followup from this year both

demonstrate that as we scale model size

compute and data we see this clear

improvement in offline eval of

recommendation quality

scaling curves aren't just academic

research they're driving real product

impact at scale for some of the biggest

consumer businesses in the world here's

a couple of examples I have from Meta's

recent earnings reports. Um, Instagram

reels had a strong quarter, 30%

year-on-year watch time. And,

uh, the optimizations we made to improve

the quality of recommendations included

simplifying our ranking architecture to

enable efficient model scaling and

longer interaction histories to identify

a person's interests.

uh we doubled the length of user

interaction sequences used for training

Instagram and increased the richness of

each user interaction.

So these are direct parallels to the

power loss scaling curves for LLMs.

And I want to introduce this idea of a

flywheel of tokens in engagement out

which is what's powering all of these

Rex model scaling. You train a model.

You then run inference on it which is

the tokens in that recommendation model

results in better content

recommendations. It drives consumer

engagement, daily active users time

spent. It translates to monetization and

ads or subscription which pays for the

next model training run. And so every

step on the scaling curve is one loop

around this flywheel. And a lot of

consumer apps are spinning this core

flywheel at the heart of their business.

So we have a long way to scale these

recommener systems. I want to talk about

the four paradigms that I see the

industry progressing through. The first

scurve was more traditional Rexus where

this scurve focused more on feature

engineering and user and content

embeddings. Most production systems are

still sitting on this curve. they're

running some type of two tower sparse

network rankers scaling the embedding

models. I don't think this curve is

going to go away but model development

here will be accelerated with auto

research and things like feature

engineering will be handled by agents uh

rather than real ML engineers. The next

curve is kind of LM inspired models

where you are scaling models ideally end

to end. HSTU and one wreck papers are

examples in this paradigm. I think this

is

where the leading recommener systems in

the industry are largely operating

today. I think the next scurve will be

this paradigm of LLM native where you

adapt a base model that understands and

can reason uh and adapt it for

recommendation tasks. The tiger and plum

papers are some examples of this

paradigm. And I think the final paradigm

that I start to see emerging is agentic

where LLMs will start to orchestrate Rex

systems in a loop. I think there's a

parallel here with coding agents. So the

LM native models are like improving the

core capabilities of the model going

from opus 45 to 48 versus the agentic

curve will be like improving the coding

harness behind claw code or codeex. Um

and so instead of just having a single

forward pass through the recommener, you

can imagine a loop where agents plan,

retrieve, rank and then critique the

recommendations. They can refine them by

calling models again and finally deliver

the recommendations. This I think is an

interesting area of research. Now

let me jump into

so here here's kind of the framework of

all the four Rex paradigms. I think

companies are scaling across each of

these curves in parallel.

Most of recommendations I think lives in

LM inspired today and is trying to

graduate into LM native. Uh but then a

lot of companies are still using

traditional models uh and climbing that

scurve.

Let me shift gears a bit to share the

recipe of how to actually build a LLM

recommener.

I think it's pretty simple. It's three

steps. Uh you start with tokenizing your

content and creating a language for your

domain.

Then you want to adapt the LLM so that

it understands both English and your

domain language and becomes this

bilingual model. Finally, you can prompt

this model with user information and it

will directly decode recommendations

from your content corpus.

Let's go a bit deeper. This is the LLM

recommener as a five layer cake. Um,

we'll start at the bottom. Uh, that's

semantic ID where you're converting your

content corpus into tokens that the LLM

can understand and reason over. Then you

have kind of the base LLM foundation

model. This can be an open weights model

or an internal first party model. Then

the core training stages. Pre-training

is around bridging English and these

recommener tokens. Post-raining is about

steering the model towards

recommendation tasks like predicting

engagement or reasoning over

recommendations.

And then finally, you can just do some

light surface specific fine-tuning to

deploy it on a product surface. The

exciting thing about this paradigm is

most of the compute is shared across all

of the product surfaces. So you don't

have to train individual models from

scratch for every product surface.

I'll go a bit deeper into each stage for

semantic ids. I think this has seen

incredible adoption. A lot of teams are

just replacing their hash ID with the

SID in traditional models and seeing

good impact. I think there's two big

reasons to tokenize content. The first

is it gives you the stable

representation for models to learn over

rather than a constantly shifting hash

that the model can only memorize. And

the second is compression. You want to

be reasoning over these long sequences

of user interactions. And if you don't

compress the content, for example, a

threeminut Instagram real video would be

10,000 tokens and it will just fill up

the content context window too quickly.

So you have to to compress it into about

10 tokens. And so here I have some

examples of Instagram reels about

tennis. You can see that the semantic

token shares the prefix of the first

three tokens because they're very

similar reels. You can imagine the first

token representing sports um and the

second two tokens representing tennis

and then the final token making these

videos individual.

Once you have a semantic ID, you can

train it to understand both English and

semantic ID. And so the task I have on

the left for pre-training here is an

example of where you prompt with a video

with semantic ID ABC has the description

blank and the output is a shot that was

instantly iconic from Wimbledon. Here

you're teaching the model to connect

these semantic tokens with synthetic

English natural language text. The

example on the right is about reasoning

over sequences of semantic IDs. And so

in a user's interaction history, you can

mask some parts of the sequence and the

model learns to predict them and

understand what videos are watched

together in sequence.

Here I have an example of post training

where we're teaching the model how to

rerank content. So the input is a bunch

of user information and 30 candidate

videos that are then ranked to be the

top five recommendations from this LLM

ranker. What's really interesting here

is that you can see the chain of thought

reasoning of this model. And because

this model knows both English and

recommendations, you can simply just

inspect the model and understand why it

made the decisions that it did. In this

example, the model understands the

user's topic interests like comedy,

food, DIY, wellness. It understands the

engagement style and what creators this

user has an affinity towards.

and then it reranks the content based on

this chain of thought reasoning.

I think this is super exciting because

once you have a model that can

understand both English and

recommendations, it opens up new product

surfaces and new experiences where users

can steer their feed. Here's an example

from your algorithm on Instagram where

users can talk to the algorithm while

they're consuming content. It's a

screenshot from scrolling through reels

or when you click in you can understand

what the Instagram algorithm thinks

about you and your interests and then

you can add or remove interest and talk

to it in natural language. And so we're

going to see this shift I think from

blackbox recommendations algorithms to

giving users more control over their

algorithm and algorithms becoming more

interactive and steerable.

I'm really excited that users can direct

it towards their own goals that's

expressed in language rather than just

likes or comments. Um, and I think this

can this foundation model can also start

to explain its recommendations.

And so for this example, I've added an

interest that I want to follow the FIFA

World Cup at this time. And the model

would get both my user history and this

new input and then be able to decode

recommendations that are personalized to

me like this free cake that Messi scored

recently.

I think these interactive recommenders

are going to be a really interesting new

product surface and we're seeing this

across the industry. We have some

examples of prompted playlists from

Spotify, custom feeds from YouTube, Ask

Door Dash, and we'll be hearing more

from speakers about these.

Um, finally, I want to talk about kind

of the framework of tokens in engagement

out this flywheel that I started with um

of model training, inference, consumer

engagement, and then monetization.

This is actually the same flywheel

that's shared by content feeds and the

AI chat apps. And this is a lens that

you can use to evaluate any consumer

app. What'll make an app successful on

the training ROI side is how well can it

translate compute into a frontier model.

On the inference side, how well can the

inference tokens translate into

engagement and then monetization.

And so if you try to compare content

feeds and AI chat apps, I think that LM

recommenders are actually structurally

more token efficient than the AI chat.

So on the left you have content feeds

like Instagram, Facebook, Tik Tok and

YouTube. On the right you have the big

AI chat apps like Gemini, ChatGpt and

Claude.

Content feeds are currently using models

that are around 1 to 10 billion active

parameters. The chat apps are serving

much larger models 10 to 100 billion

active parameters. Um the output for the

content feed is a semantic ID token

which is a pointer or an address to

existing content because the content

supplied on these content feeds is

uploaded by creators. There's a very

healthy creator economy and so the

supply of content is effectively free or

it's uploaded by creators. For the AI

chat apps, they have to decode every

token of content themselves and the

amount of tokens output in every turn of

an LM chat interaction is a few thousand

tokens. Uh, and the the big difference

here is that every token has to be

manufactured by the app at inference

time. And so what that means is if you

compare these two apps on how much

inference cost and compute is spent to

generate an hour of consumer engagement,

there's a huge structural gap where

content feeds are significantly cheaper

up to 100 times or more cheaper than AI

chat apps because they're decoding

pointers to content rather than content

itself.

Um, finally I want to kind of end with

why I think LM Rexus is one of the most

significant consumer AI applications. If

you look at the top apps by daily active

users, these are the top 10 apps. Four

out of 10 of them are content feeds. And

so this is a really significant consumer

uh application. If you look at the

content feeds, almost all of the

consumer app growth on both the

engagement and monetization side is

driven by the recommener and ads models.

And these are going to be entirely

transformed by LLM recommenders.

It's a very large and very token

efficient application of AI for consumer

apps. We're going to see a lot of new

product experiences come through with

steerable and interactive

recommendations, explanation of

recommendations, and just putting more

users in control of their experience on

these apps. I think we're going to see

some really exciting research on

consumer agents and recommendation

agents that come out over the next year

or so. Um and so this is why I think

this is a super exciting area of both

research and product uh at this

intersection of LLM and recommendations.

That's all. Thank you so much.