AI Engineer World's Fair 2025
One model to rule recommendations: Netflix's Big Bet
Read the talk
One foundation model for Netflix’s many recommendation problems
Netflix’s recommendation architecture centralizes learning from interaction histories, then reuses the resulting representations across ranking models, embedding stores and specialized applications.
From a talk by Yesu Feng
Before you start: Familiarity with embeddings, transformer hidden states and supervised learning losses will help with the model architecture and training examples.
A homepage is many recommendation problems
A Netflix homepage has to decide both which rows to show and which titles belong in each row. Its two-dimensional layout combines row ranking with item ranking: comedy and action rows sit alongside trending releases and Netflix-exclusive titles. A useful recommendation system must coordinate those choices, not simply produce one ordered list of movies.
The variety extends beyond rows. Movies and TV shows share the content space with games and live streaming. Different pages impose different structures: the homepage, search and kids homepage serve distinct needs, while a mobile feed is linear rather than two-dimensional. Historically, these differences produced specialized models—some ranking videos, others ranking rows, some finding unwatched shows and others supporting titles a member was already watching. Their objectives differed, but much of their work overlapped.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Learn once from the same interaction facts
Independent models repeatedly transformed the same user interaction history into slightly different features. One might count actions; another count them within several time windows; another compute similarities between previously watched titles and a candidate title. A sequence model might instead consume a sequence of unique show IDs. Label engineering was duplicated too. The underlying facts were shared, but the derived features accumulated variations that made them difficult to maintain.
Expanding the business by building another model for every new use case made this maintenance problem an innovation problem. Some feature and label components were already shared, yet each new model still largely started from scratch. Yesu Feng places this situation around the beginning or middle of the pandemic, roughly four years before the talk. The architectural question became: could Netflix centralize the learning of user representations?
Netflix’s recommendation foundation model, abbreviated FM, rests on two hypotheses. First, scaling semi-supervised learning with a transformer should improve personalization, much as scaling improves language models. Second, integrating that shared model into downstream applications should spread improvements across recommendation surfaces. The intended benefit is both better learning and a reusable foundation for application development.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The token is an interaction event
The first design decision sits below the transformer: how to clean and tokenize interaction history. As in language modeling, choices at this layer propagate through the entire model. But the input unit is different. A language token can be represented by a vocabulary ID; a recommendation token represents a user interaction event with multiple fields. Reducing it to a title ID alone would discard information about the interaction itself.
Token granularity determines how much history fits into the context window. The slide maps raw interaction history to a shorter tokenized history, making that compression visible. Netflix iterated on the abstractions and interfaces around tokenization so it could adjust the representation to different applications. Pretraining and application-specific fine-tuning can therefore use slightly different tokenization rather than requiring one fixed treatment of every event.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From event fields to stable user representations
The model proceeds through four layers: event representation, embedding and feature transformation, the transformer, and the training objective. At the input, Feng organizes the event fields around three questions:
| Question | Information represented |
|---|---|
| When? | Event time and its encoding |
| Where? | Location, locale, country, device, page and row |
| What? | Target title or entity, interaction type and duration |
“Where” includes both physical context and the interface where the action occurred. “What” includes both the object and what the member did with it. These fields make an event richer than a title appearing in a sequence.
The embedding layer must decide which information to retain and combine learned ID embeddings with semantic content information. An ID embedding learned from interactions has a cold-start problem: a newly introduced title was absent during training, so its ID has no learned behavioral representation. Semantic information supplies a complementary description that does not depend solely on that title’s interaction history.
The transformer’s hidden-state output becomes the user representation. That changes the design question from merely predicting the next event to producing a useful, long-term representation of a member. A profile and its interaction history continually change, so stability matters alongside responsiveness.
There are several places to shape that representation. Outputs can be aggregated across sequence positions, across transformer layers, or adapted explicitly to a downstream objective through fine-tuning. These are consequential choices: the last hidden state is not the only possible interface between the shared model and an application. Feng identifies the aggregation and adaptation questions without prescribing one universal recipe.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Predict more than the next title
At the objective layer, the event’s multiple fields create multiple possible output sequences. Predicting the next entity ID resembles next-token prediction and can use softmax or sampled softmax. But a title is only one part of the next event. Other targets include the action type; entity metadata such as type, URL or language; action duration; device; and the time of the next play. The application determines which of these predictions is useful.
These fields can enter training in different ways:
- Prediction targets: Use multitask learning with multiple heads or hierarchical predictions.
- Loss weights: Give some events more influence than others.
- Rewards: Use event-derived feedback to guide the desired behavior.
- Loss masks: Restrict learning to a selected subset of events.
For example, the following Python function expresses the loss-weighting and masking option for a title-prediction head. next_title_logits contains predictions at each sequence position, next_title_ids contains the targets, and include_event selects the behavior category that should contribute. event_weights controls the contribution of included events.
python
import torch
import torch.nn.functional as F
def weighted_title_loss(
next_title_logits: torch.Tensor,
next_title_ids: torch.Tensor,
include_event: torch.Tensor,
event_weights: torch.Tensor,
) -> torch.Tensor:
# logits: [batch, sequence, titles]
# targets, mask, weights: [batch, sequence]
per_event_loss = F.cross_entropy(
next_title_logits.transpose(1, 2),
next_title_ids,
reduction="none",
)
weights = event_weights.to(per_event_loss.dtype)
weights = weights * include_event.to(per_event_loss.dtype)
denominator = weights.sum().clamp_min(
torch.finfo(per_event_loss.dtype).eps
)
return (per_event_loss * weights).sum() / denominator
The distinction is useful: an event field need not become another thing to predict. It can instead specify which behavior the existing prediction objective should emphasize.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Scale the learning, then account for serving
Feng reports continuing recommendation-quality gains over roughly two to two-and-a-half years as the model grew from a few million or order-ten-million parameters to order-one-billion parameters, with training data scaled alongside it. He does not give a named quality metric or numerical lift in the discussion. Netflix’s stopping point reflects recommendation systems’ strict latency and cost requirements: further growth would require distilling the larger model back into something suitable for serving, rather than assuming the largest trained model should answer every online request.
Three techniques borrowed from language modeling help make that learning more useful. The first is multi-token prediction. Feng points to DeepSeek as an example, without identifying a particular paper, and describes multi-head and multi-label implementations. The purpose is to make the model less focused on the immediately next action. Predicting farther ahead also addresses the time gap between training and serving and encourages learning about longer-term behavior and satisfaction.
Feng reports a notable metric improvement from multi-token prediction, without specifying the metric or magnitude. The second technique is multi-layer representation: layer-wise supervision, self-distillation and aggregation of outputs from multiple layers can help produce better, more stable user representations. This returns to the earlier question of which transformer outputs should become the shared representation.
The third technique is long-context handling. Feng lists truncated sliding windows, sparse attention, training on progressively longer sequences and parallelization strategies. These methods address the cost of learning from extended histories: the objective is to use more of the available behavior efficiently, rather than treating context length as an isolated model setting.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Three ways applications consume the foundation model
Before the foundation model, the personalization stack contained many independently developed features and models, each serving one or more applications. The shared model consolidates much of the data and representation layer, including both user and content representations. Application models remain, but become thinner layers built on top of the shared learning. One foundation model does not mean one identical serving path for every application.
The first consumption pattern embeds the pretrained foundation model as a subgraph inside a downstream neural network. An application that already has a sequence-transformer tower can replace that component with the pretrained subgraph. Learned content embeddings can also become embedding lookup layers. This lets an application reuse the model within its own computation graph.
The second pattern publishes content, entity and member embeddings to a centralized embedding store. Downstream consumers retrieve representations rather than integrating the foundation model itself. Member embeddings introduce two operational decisions: how frequently to refresh them and how to keep them stable. The store also broadens access beyond recommendation models, allowing analysts and data scientists to consume embeddings directly.
The third pattern extracts the model and fine-tunes it for a specific application. Where online latency is especially strict, distillation can produce a smaller serving model.
| Consumption pattern | What the application reuses |
|---|---|
| Subgraph integration | Pretrained computation and embedding lookups |
| Embedding store | Published member and content representations |
| Fine-tuning or distillation | An application-adapted model |
These interfaces preserve specialization while avoiding the need to relearn the entire representation independently for every surface.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Measure adoption separately from experiment wins
Feng describes application adoption and A/B-test wins accumulated over the preceding year and a half. His chart separates two quantities: blue bars count applications incorporating the foundation model; green bars count A/B-test wins. One application can produce multiple winning experiments, so those bars are not interchangeable measures of adoption. The discussion supplies no exact counts, effect sizes or experimental conditions.
Feng regards the combination of recommendation gains and infrastructure consolidation as validation of the original bet. Scaling improves the shared model, while centralizing learning makes its benefits available to multiple applications. The development benefit is concrete: a new application can fine-tune the foundation model to launch its first experience instead of assembling a full model from scratch.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Represent heterogeneous entities and generate collections
At the time of the talk, the next directions extend both what the model represents and how applications adapt it:
- Universal entity representations: Explore semantic IDs and related approaches so a shared representation can cover increasingly heterogeneous content types.
- Generative collection retrieval: Generate a collection rather than recommend only an individual video. Multi-step decoding creates opportunities to incorporate business rules and diversity during generation.
- Prompt tuning: Train soft tokens that can be swapped at inference time to change the foundation model’s behavior, enabling faster adaptation.
Feng presents these as research directions. In particular, soft-token prompt tuning is a promising approach the team is exploring, distinct from the deployed subgraph, embedding-store and fine-tuning consumption patterns.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What transfers beyond the recommendation surface?
The audience first pushes the idea of shared preferences beyond entertainment recommendations, asking whether they might reveal purchasing choices or political opinions—even predict the next US president. After requesting clarification, Feng gives a narrower answer: Netflix is expanding across entity types and moving toward capturing tastes both on and off its platform. He does not confirm the speculative political-prediction capability.
A subsequent question asks where graph models and reinforcement learning fit, and whether they improve performance. Feng explains that a dedicated graph-model team works on a knowledge graph covering the entertainment ecosystem, including content both on and off Netflix. Embeddings from those graph models provide semantic information for cold start—the semantic complement to learned title IDs described earlier.
Reinforcement learning connects to collection generation. Feedback from user actions is sparse, but those rewards can guide how an entire collection is generated. Feng identifies this use for reward signals without giving a quantitative accuracy or performance improvement.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Embeddings flow through the model—but not every source is integrated
The next question makes the embedding flow explicit. The unified model exposes its learned embeddings as features for downstream models, while also consuming upstream representations such as graph neural network embeddings. Centralized learning therefore sits between existing sources of semantic information and the applications that need personalized representations.
The final question asks how deeply the model reads the video itself: does it rely on interactions and metadata, or use individual frames and short clips? Feng says another Netflix content group already has relevant embeddings, but granular clip- or view-level representations have not yet been incorporated into this model. He expects movement in that direction. The boundary is practical: having an embedding somewhere in the organization is different from integrating it into the shared recommendation representation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
Further reading
Original DeepSeek report describing its language-model architecture and multi-token prediction training objective.
Research on predicting several future tokens with independent output heads sharing a model trunk, cited by Netflix's companion article.
Read the complete timestamped transcript
- 0:00
[on-hold music] Uh, good afternoon.
- 0:16
Uh, thank you, uh, Eugene, for the, uh, introduction. Uh, so today, uh, I'm going to share our big bet at Netflix, uh, on personalization, namely to use one foundation model to cover all the recommendation use cases.
- 0:32
Uh, at Netflix, we have diverse recommendation needs. Uh, this is an example homepage of, uh, one of profile on Netflix.
- 0:42
Uh, it's a 2D layout, rows and items. Diversity comes at, uh, at least the three levels. Uh, there is first level about row. We have diverse rows. We have genres, for example, rows on comedies, rows on action movies.
- 0:57
We have rows about, uh, new trending, uh, just the release titles. We also have rows about, for example, titles only available on Netflix. Um, so that's the first dimension.
- 1:09
Second dimension is, of course, the, the items or entities. Uh, in addition to traditionally movie and TV shows, now we have games, we have live streaming, and, uh, we are going to add more.
- 1:21
So our content space is ex-expanding to, uh, very heterogeneous content types.
- 1:29
The third level is page, right? So we have homepage, we have search page, we have a kids homepage, which is tailored very differently toward kids' interest. Uh, mobile feed is a linear page.
- 1:43
It's not a 2D layout, uh, so on and so forth. So page-- different pages are also very diverse. Uh, what happened traditionally was that these lead to naturally, um, many specialized models that got developed over the years.
- 2:00
Uh, some models rank videos, some rank rows, some focus on, for example, shows user have not watched yet. Some, uh, focus on shows what a user are already engaging.
- 2:13
Uh, and many of those models are-- were built independently over the years. They may have different objectives, uh, but have a lot of overlaps as well.
- 2:24
Uh, so naturally, this lead to duplications. Uh, uh, duplications in our label engineering as well as feature engineering. Take the, uh, feature engineering as example. Uh, we have this very commonly used factual data about user interaction history.
- 2:41
Uh, the, the factual data is the same, but over the years, many features are developed, uh, derived out of the same facts data like counts of different actions, counts of actions within various time window or other kind of, uh, slice and dice dimensions, similarity between the user's history titles against the target titles.
- 3:02
Unique-- Lastly, like, uh, just a sequence of, uh, unique show IDs, uh, to be used as a sequence feature into the model. So this list can go on and on.
- 3:13
And a lot of those features, uh, because they are developed independently into each model, they have slight variations, but become very-- but largely, uh, very similar, so become very hard to, uh, maintain.
- 3:27
So the challenging, uh, the challenge, uh, back then was, uh, is this scalable? Uh, obviously not. If we keep expanding our landscape of content type or business use cases, it's not manageable to spin up new models for each i, uh, individual use cases.
- 3:45
Uh, there's not much leverage. Uh, there are some shared components on building the feature label, but still, by and large, each model, uh, basically, uh, spinned up independently. And that also impact our, uh, innovation velocity in, in the terms that you don't reuse as much as you can.
- 4:04
Instead, you just spin up new models, uh, pretty much from scratch. So this was the situation about four years ago, uh, at the beginning or middle of the pandemic.
- 4:16
So the question we asked at that time was, uh, can we centralize the learning of user representation in one place?
- 4:25
So the answer is yes, and we had this key hypothesis that about foundation model based on transformer architecture. Uh, concretely, two hypothesis here. One hypothesis is that through scale up semi-supervised learning, personalization can be improved.
- 4:40
Uh, the scaling law also applies to recommendation system as it applies to LLM. Uh, second is that by integrating the foundation model into all systems, we can create high leverage.
- 4:51
We can simultaneously improve all the downstream canvas-facing, uh, models at the same time. So we'll see in the, uh, following slides how we validate those hypothesis.
- 5:03
Uh, I, I'll break up the, uh, overview into two sub-sessions. First, about data, data in the training, and then later, uh, second about application and serving.
- 5:14
So, um, about data and training. So starting from data, a very interesting aspect of building such foundation model, autoregressive transformer, is that there's a lot of analogy, uh, but also differences sometimes, uh, between this and the LLM.
- 5:29
So we can transfer a lot of learnings, inspirations from LLM, uh, uh, development. Uh, if we start from the very bottom layer, which is basically da-data cleaning and tokenization, uh, people work with LLM understand tokenization decisions have profound impact in your model quality.
- 5:49
So, uh, although it's the bottom layer, the decision you made there can percolate through all the downstream layers and manifest as either your model quality problem or model quality plus.
- 6:01
So, uh, this applies to recommendation, uh, foundation model as well. Uh, instead of... Uh, there are some differences very importantly. Instead of language tokens, which is just the one ID, here for, uh, if we want to translate the user interaction his- history or sequence, each of the token is a event, a interaction event from the user, right?
- 6:23
But that event has many facets or many fields, so it's not just the one ID you can represent. There are a lot of rich information about the event. So how you-- All of those fields can play a role in making the decision of tokenization.
- 6:36
Uh, I think that's what we need to consider very carefully. Um, what is the granularity of tokenization, and trade-off that versus the context window, for example. Um, and through many iterations, we reach the right, I think reach the right abstraction and interfaces that we can use to, uh, adjust our tokenization to different use cases.
- 6:57
For example, you can imagine we have a tokeni- one version of tokenization used for pre-training. For fine-tuning against a specific application, we apply slightly different tokenization.
- 7:08
Um, so moving up from the tokenization layer, uh, then becomes the model layers. Uh, at high level, uh, from bottom to top, we go through the, uh,
- 7:23
event representation, uh, embedding layer, transformer layer, and the objective layer.
- 7:29
So event representation, as we just briefly touched upon, uh, many information in the event. But at a high level, you can break it down by when, where, and what.
- 7:40
When that event happened, that's about time encoding. And where it happened, it's about physical location, your locale, country, so on and so forth, but also about device, about the, uh, canvas or whi- which row, which page this action happened.
- 7:55
Uh, and then, uh, what basically is about the target, uh, entity or the title, which title you interacted with, what is the interaction, how long, and, uh, any that kind of information associated with the action.
- 8:09
So, um, that's where the-- we need to decide what information we need to keep, what we should drop, so on and so forth. Uh, moving one layer above, uh, the embedding feature transformation layer, uh, one thing that needs to be pointed out is that for recommendation, we need to combine ID embedding learning with other semantic content information.
- 8:32
Um, if you only have ID embedding learned from scratch in the model, then you have problem with cold start, meaning the titles the model hasn't seen during training, it doesn't know how to deal with it at inference time.
- 8:44
So we need to have, uh, semantic content information to be, uh, uh, comp- complementary to those ID embeddings. Uh, this is not a problem for LLM, but very commonly encountered a cold start problem for rec- rec- recommendation system.
- 8:59
Uh, uh, transformer layer, I think there's no need to talk too much into this in terms of architecture choices, optimization, so on and so forth. The only thing that, uh, I want to point out is that, uh, we are using the hidden state output from this layer as our user representation, which is one of the primary goal
- 9:15
of the foundation model, is to learn a good long-term user representation. Then, uh, we need to put this into context, then co- things to consider are, for example, how stable is our user- user representation, given our user profile, user interaction history keep changing.
- 9:31
How do we guarantee or maintain the stability of that representation? And what kind of aggregation we should use. You can think of broadly aggregate across the time dimension in terms of sequence dimension or aggregate, uh, uh, across the layers.
- 9:47
You have multiple self-attention layer, how do you aggregate that?
- 9:50
Um, and then lastly, do we need to do explicit adaptation of the, uh, representation based on our downstream objective to fine-tune it?
- 10:00
Um, so then we move to last, uh, the very top layer, uh, objective loss function. This is also very interesting in the sense that it's much richer than LLM.
- 10:10
Because you can see first we use, uh, instead of one sequence, but multiple sequence to represent the output, because you can have a sequence of, uh, entity IDs. That's your, like, uh, next token prediction, s- softmax or sample softmax.
- 10:24
But then we have ma- many other facets or field of each event that can be also used as a target, right? So it could be for things like, uh, action type, it could be some aspect of the entity's metadata like entity type, URL, language, so on and so forth, and also about your action like a prediction of
- 10:44
the duration or, uh, the device where the action happened, or the time when the next, uh, user play will happen. So those are all legitimate, uh, targets or labels.
- 10:56
Depends on your use case. You can use them to do the fine-tuning. Now, instead of-- Uh, so you can cast the problem as a multitask learning problem, uh, multi-head or hierarchical prediction, but you can also use them just as your weights, your rewards, or your mask on the loss function.
- 11:12
So in terms of to adapt the model to zooming into one subcategory of, uh, user behavior you want to, you want the model to learn. Okay. So that's about the model architecture that I want to talk about. [coughs]
- 11:26
Um, so does it scale? The first question, a part of the first hypothesis we want to answer is that does a sca- a scaling law apply? And I think the answer is yes.
- 11:36
So this is over the, uh, roughly two to, two to two and a half years we were scaling up, and then we constantly still see the gain, uh, from only on the order of ten million profile or a few million profile to now on the order of one billion, um, model parameters.
- 11:56
We scale up the data accordingly. Um, now we stop here because we can still keep going, but, uh, as you may- I realize that recommendation system usually have much stringent latency cost requirement.
- 12:12
So scaling up, scaling up more require us to also distill back. Yeah, but certainly, I think this is not the end of the scaling law.
- 12:21
Uh, before we wrapping up the data and training sys- uh, discussion, I would like to highlight some of the learnings I think quite interesting we borrow from LLM. This is not exhaustive list, but, uh, uh, I think, uh, very interesting to me, uh, the top three.
- 12:36
One is to- uh, multi-token prediction. You may have seen this in the DeepSeek paper, so on and so forth. So you can, uh, implementation-wise, you can use multi-head, multi-label, so on, uh, different implementation flavor.
- 12:47
But the goal is really to force the model to be less myopic, more robust to serving time shift because you have a time gap between your training and serving, and also force the model to target long-term user satisfaction and long-term user behavior instead of just focus on next action.
- 13:05
Um, I-- we have observed a very notable, uh, metrics improvement by doing that. Uh, the second is multi-layer representation, which, uh, I touched upon on the profile representation. So, this is also translated from LLM, uh, side of techniques of layer-wise supervision, self-distillation, or multi-layer output aggregation.
- 13:25
The goal here is really to make better and more stable user representation.
- 13:30
Um, lastly, uh, this is also should be no surprise, long context window handling from truncated sliding window to sparse attention to progressively training, uh, longer and longer sequences, uh, to eventually all of the parallel strategies.
- 13:46
So this is about more efficient training and maximize the learning.
- 13:51
Okay. So, uh, shift gear to talk about the serving and applications. Uh, before the foundation model FM, uh, this is, uh, roughly the algo stack we have for personalization.
- 14:03
Many data, many features, many models independently developed, each serving multiple or one canvases or applications we call.
- 14:12
Now, with the foundation model, we consolidate largely the data and the representation layer, especially the user representation as well as content representation in the personalization domain. Uh, model layer as well, because model now, each application model now are built on top of FM, so become a thinner layer instead of a very standalone, full-fledged model trained from scratch.
- 14:37
So how do the various models utilize the foundation model? Um, there are three main approaches, uh, or consumption patterns. Uh, the first is foundat- foundation model can be integrated as a subgraph within the downstream model.
- 14:51
Uh, additionally, the content embeddings learned from the foundation model can be integrated as the embedding lookup layers. So downstream model is a neural network, uh, it may already have initially some of the sequence Transformer, uh, tower or, uh, graph, and then using a pre-trained foundation model subgraph to directly replace that.
- 15:14
Uh, second is that, uh, we can push out embeddings. This is no surprise from both content side and entity embedding as well as member embeddings. Uh, the only, uh, the main concern here, of course, is how we want to ref- how frequently we want to refresh the member embeddings and how we make sure they are stable,
- 15:33
uh, and push them to the centralized embedding store. And this, of course, allow far more, uh, wider use cases than just the personalization because people, analytics, data scientists can also just, uh, fetch those embeddings directly to do the things that they want.
- 15:51
Finally, user can, uh, extract the models and fine-tune it for specific applications. Uh, either fine-tune or they need to do distillation to meet the, uh, online serving, uh, requirement, um, especially for those with a very strict latency requirement.
- 16:09
To wrap up, uh, I want to show at high level the wings we accumulated over the last one year and a half, uh, by incorporating FM into various places.
- 16:20
So, the blue bar represent how many, uh, applications have FM incorporated. The green bar represent the A/B test wings, because in any application, we may have multiple A/B tests going on there to have wings.
- 16:34
So we see-- we indeed see high leverage of FM to bring about both A/B test wings as well as infrastructure consolidation.
- 16:44
Uh, so I think the big bet, uh, big bets are validated. Uh, it is a scalable solution, uh, in terms of both, both in terms of a scalable, scaled up the model with improved quality, as well as making the whole infra consolidated and the scale, uh, to new applications to be much easier.
- 17:03
High leverage because it's a centralized learning. Innovation velocity also is faster because we allow, uh, a lot of newly, uh, launched applications to directly fine-tune the foundation model to, uh, launch the first experience. [lips smack]
- 17:19
Uh, so the current directions, um, one is that, um, we want to have universal representation for heterogeneous entities. This is, uh, as you can guess, the semantic ID and along those lines, because we want to cover that,
- 17:34
uh, as Netflix expand into very different, very heterogeneous content types. Uh, second is generative retrieval for collection recommendation, right? So instead of just recommending a single video, be generative at inference time and serving time.
- 17:47
Because you have a multi-step decoding, a lot of the consideration about busi- business rules or diversity, for example, can be naturally handled in the decoding process. Lastly, faster adaptation through prompt tuning.
- 18:00
Uh, so this is also borrowed from LLM. Can we just train some soft tokens so that at inference time, we can directly swapping in and out those soft tokens to prompt the FM to behave differently?
- 18:13
So that is also a very promising direction that we are getting into. All right, that concludes my talk. Thank you for your attention. Any questions? [clapping]
- 18:27
Thank you, Yesu. Um, if you have any questions, may I invite you to come to the mics in front, um, while we get our next speakers from Mr. Carter to get set up.
- 18:36
Oh, yeah. Uh, hi, thank you for the talk. Uh, since you get billions of users, so except the recommendation system, you-- maybe it can do much more, right? So what's your thought on that, since I can just ask it to p- to predict who's the next president in the United States?
- 18:55
Thank you.
- 18:56
Um, so I actually don't... Uh, could you explain a little bit what do you mean by beyond recommendation? Do you mean the other personalization or other things? Yeah.
- 19:06
Um, yeah.
- 19:08
Okay.
- 19:08
Since you get kind of billion users preference, so actually the, that, that preference is also been leaning into what things they're buying or who they will vote for the next president.
- 19:19
Mm.
- 19:19
So do you think your foundation model has that capability to, to expand not only recommendation, what videos they want to look or what others they like, or what's their o- opinions on any- anything else?
- 19:30
Thank you.
- 19:31
Yes. So I think we are expanding to different, uh, I think entity type and also capture,
- 19:38
uh, users' taste from both on and off our platform. I think that's the general trend that we're going to.
- 19:45
Yes.
- 19:45
Go for it.
- 19:46
Okay.
- 19:46
We're all done.
- 19:48
Great. Thank you. This was really helpful. Um, question on... And you might not be able to share it, um-
- 19:54
Yeah
- 19:54
... for IP reasons, but whatever you can. Uh, thoughts on graph models? Didn't, uh, didn't hear a lot of that in your talks. Graphs and, uh, reinforcement learning. Any utilization there?
- 20:04
Mm.
- 20:05
Any benefits you saw? Any boost in, in performance and accuracy? [laughs]
- 20:08
Yeah, that's a very good question. I think we have actually, uh, a dedicated team, sub-team doing graph model, uh, especially around our knowledge graph to cover the content space, both on and o- off our platform, in the whole e- enter- entertainment ecosystem.
- 20:25
So we use actually a lot of, uh, embeddings, for example, from the graph model to cold start. That's where I see, I show those semantic embeddings. That's where it comes from.
- 20:36
In terms of reinforcement learning, yes, as well, especially where we consider sparse reward that we have on users, from users' action, uh, pretty much sparse. But we want to use them to guide how, for example, we generate the whole collection.
- 20:52
That's where we need to consider how to use those reward to guide those, uh, process. Yeah.
- 20:58
I think we can take one more question. I'm sorry.
- 21:00
Can I ask a two-part question?
- 21:02
Sure. I will be here, and so we can also follow up. Yeah.
- 21:05
One, one question.
- 21:06
So, uh, do you also use these unified representations as embedding features to downstream models?
- 21:13
Y- you had a slide how you use the unified model.
- 21:15
Yeah. So, uh, the-- We-- So for the embeddings learning within our model, we also expose to downstream to direct consume them. Uh, we also have a-- To train our unified re- uh, embedding, we also have some upstream, like just the, for example, the GAN embeddings-
- 21:35
Right
- 21:35
... that those are also consumed too, to do that.
- 21:38
Questions. Last one. Is it fast? Yes. [laughs]
- 21:42
Hi. Hello.
- 21:44
Yeah.
- 21:44
Uh, in your embeddings-
- 21:45
Yeah
- 21:45
... are you just using, when someone does an action or... Sorry. For the-- In these embeddings, are you just using metadata over the video to understand what they like?
- 21:55
Or are you actually using like frame by frame of the video or second clips?
- 22:00
Uh, not yet. We do have that, uh, f- from some other content group of our organization, but I think the trend will go there. So we are not yet, uh, into very granular sub, like clips level or view level.
- 22:15
We have those embeddings, but not quite yet to incorporate. Yeah.
- 22:18
Thank you.
- 22:19
No problem.
- 22:19
Thank you, Yesu. Uh, please another round of applause for Yesu. [upbeat music]