[
  {
    "startMs": 1309,
    "text": "[music]"
  },
  {
    "startMs": 12639,
    "text": "Welcome everyone. Thank you for coming"
  },
  {
    "startMs": 14639,
    "text": "out to the LM Rexus track. Um I'll be"
  },
  {
    "startMs": 19439,
    "text": "sharing the first talk. Uh my talk is"
  },
  {
    "startMs": 22560,
    "text": "titled tokens and engagement out"
  },
  {
    "startMs": 24800,
    "text": "training LM recommenders. Um, and I want"
  },
  {
    "startMs": 28160,
    "text": "to make two big arguments today. The"
  },
  {
    "startMs": 30400,
    "text": "first is that recommendation systems"
  },
  {
    "startMs": 33040,
    "text": "scale just like LMS do and that the"
  },
  {
    "startMs": 35840,
    "text": "field is very early in that scaling"
  },
  {
    "startMs": 37680,
    "text": "curve. Um, and the second is that the LM"
  },
  {
    "startMs": 40480,
    "text": "recommener is going to be one of the"
  },
  {
    "startMs": 42719,
    "text": "biggest consumer applications of AI."
  },
  {
    "startMs": 47200,
    "text": "Um, okay. So, quickly about me. Uh, I"
  },
  {
    "startMs": 50160,
    "text": "currently work at Meta on research and"
  },
  {
    "startMs": 52239,
    "text": "product. Uh I lead a team called Meta"
  },
  {
    "startMs": 54559,
    "text": "Recommendations Research uh which is"
  },
  {
    "startMs": 57840,
    "text": "this group that's training frontier"
  },
  {
    "startMs": 59520,
    "text": "models, LLMs and recommenders that power"
  },
  {
    "startMs": 62719,
    "text": "Instagram, Facebook ads, the meta family"
  },
  {
    "startMs": 65760,
    "text": "of apps. Uh before this I was at Google"
  },
  {
    "startMs": 68560,
    "text": "for a long time uh working on a lot of"
  },
  {
    "startMs": 71040,
    "text": "the key ML teams including deep mind and"
  },
  {
    "startMs": 73520,
    "text": "YouTube. Last year I gave a talk at AI"
  },
  {
    "startMs": 76320,
    "text": "engineer called teaching Gemini to speak"
  },
  {
    "startMs": 78880,
    "text": "YouTube about two ideas semantic IDs and"
  },
  {
    "startMs": 82560,
    "text": "generative retrieval which we also wrote"
  },
  {
    "startMs": 85119,
    "text": "two papers about which I've linked here."
  },
  {
    "startMs": 87280,
    "text": "Um it was really fun and it led to a lot"
  },
  {
    "startMs": 89200,
    "text": "of discussions and collaborations. Uh"
  },
  {
    "startMs": 91680,
    "text": "the ideas behind semantic ID and"
  },
  {
    "startMs": 93680,
    "text": "generative retrieval have really taken"
  },
  {
    "startMs": 96640,
    "text": "off in the industry over the last year."
  },
  {
    "startMs": 98880,
    "text": "Um and they've moved from more research"
  },
  {
    "startMs": 101200,
    "text": "to now scaled production systems. And"
  },
  {
    "startMs": 103680,
    "text": "I've seen exciting launches and papers"
  },
  {
    "startMs": 106399,
    "text": "from YouTube, Meta, Spotify, Door Dash"
  },
  {
    "startMs": 110479,
    "text": "across the industry. And we have a"
  },
  {
    "startMs": 111920,
    "text": "couple of examples of that later today."
  },
  {
    "startMs": 115759,
    "text": "This year I want to talk about four uh"
  },
  {
    "startMs": 118560,
    "text": "sections. Recommendation scaling curves,"
  },
  {
    "startMs": 121840,
    "text": "a framework of four, recommendation"
  },
  {
    "startMs": 124000,
    "text": "paradigm scurves that we are climbing as"
  },
  {
    "startMs": 126159,
    "text": "an industry. um sharing the recommener"
  },
  {
    "startMs": 129520,
    "text": "recipe and finally this consumer AI app"
  },
  {
    "startMs": 133520,
    "text": "framework."
  },
  {
    "startMs": 135360,
    "text": "Um let's start with scaling curves. So I"
  },
  {
    "startMs": 137599,
    "text": "wanted to start with this landmark"
  },
  {
    "startMs": 139760,
    "text": "scaling curve paper from 2020 which"
  },
  {
    "startMs": 142239,
    "text": "feels like a lifetime ago. This is when"
  },
  {
    "startMs": 144160,
    "text": "Daario was still at OpenAI and Anthropic"
  },
  {
    "startMs": 146319,
    "text": "didn't exist yet. But the core idea that"
  },
  {
    "startMs": 148640,
    "text": "this paper shared is the power law of"
  },
  {
    "startMs": 150959,
    "text": "scaling. As you increase model size,"
  },
  {
    "startMs": 154879,
    "text": "data, the amount of compute flops"
  },
  {
    "startMs": 157280,
    "text": "trained uh for model training, the loss"
  },
  {
    "startMs": 160160,
    "text": "falls on this log linear scale. And this"
  },
  {
    "startMs": 163200,
    "text": "clean and predictable curve is what"
  },
  {
    "startMs": 165280,
    "text": "really set off the race for the AI"
  },
  {
    "startMs": 167200,
    "text": "frontier because you can forecast what"
  },
  {
    "startMs": 169519,
    "text": "model quality and capability"
  },
  {
    "startMs": 171120,
    "text": "improvements will look like. And this is"
  },
  {
    "startMs": 173440,
    "text": "what's underwriting the massive capex"
  },
  {
    "startMs": 176080,
    "text": "investments and the AI buildout today."
  },
  {
    "startMs": 179280,
    "text": "It turns out that recommendation systems"
  },
  {
    "startMs": 181360,
    "text": "follow a very similar scaling law. In"
  },
  {
    "startMs": 184480,
    "text": "fact, before this wave of LLMs, Rexs"
  },
  {
    "startMs": 187120,
    "text": "were the largest production ML models in"
  },
  {
    "startMs": 189440,
    "text": "big tech companies. And they're still"
  },
  {
    "startMs": 191360,
    "text": "some of the largest models that are"
  },
  {
    "startMs": 193200,
    "text": "served at a scale of a billion plus"
  },
  {
    "startMs": 195040,
    "text": "daily active users."
  },
  {
    "startMs": 197440,
    "text": "And they follow this similar power loss"
  },
  {
    "startMs": 199519,
    "text": "scaling curve. On the x-axis, you have"
  },
  {
    "startMs": 201440,
    "text": "data, compute, and model size. And on"
  },
  {
    "startMs": 204159,
    "text": "the Y ais you would see falling loss or"
  },
  {
    "startMs": 207200,
    "text": "in this chart uh an improvement in"
  },
  {
    "startMs": 210000,
    "text": "recommendation quality. In offline evals"
  },
  {
    "startMs": 213360,
    "text": "it's net entropy or AU gains and then"
  },
  {
    "startMs": 216959,
    "text": "when it's translated to a real"
  },
  {
    "startMs": 218400,
    "text": "production launch it's engagement impact"
  },
  {
    "startMs": 221200,
    "text": "revenue impact at some of the biggest"
  },
  {
    "startMs": 223920,
    "text": "consumer app scale."
  },
  {
    "startMs": 226319,
    "text": "Here's a real example from Meta that"
  },
  {
    "startMs": 229680,
    "text": "demonstrates these power loss scaling"
  },
  {
    "startMs": 231760,
    "text": "curves. uh there's a paper the first is"
  },
  {
    "startMs": 234879,
    "text": "a paper HSTU from 2024 and the second is"
  },
  {
    "startMs": 239040,
    "text": "a followup from this year both"
  },
  {
    "startMs": 241680,
    "text": "demonstrate that as we scale model size"
  },
  {
    "startMs": 244000,
    "text": "compute and data we see this clear"
  },
  {
    "startMs": 246080,
    "text": "improvement in offline eval of"
  },
  {
    "startMs": 248000,
    "text": "recommendation quality"
  },
  {
    "startMs": 251599,
    "text": "scaling curves aren't just academic"
  },
  {
    "startMs": 253280,
    "text": "research they're driving real product"
  },
  {
    "startMs": 255200,
    "text": "impact at scale for some of the biggest"
  },
  {
    "startMs": 257600,
    "text": "consumer businesses in the world here's"
  },
  {
    "startMs": 260079,
    "text": "a couple of examples I have from Meta's"
  },
  {
    "startMs": 261759,
    "text": "recent earnings reports. Um, Instagram"
  },
  {
    "startMs": 264960,
    "text": "reels had a strong quarter, 30%"
  },
  {
    "startMs": 266880,
    "text": "year-on-year watch time. And,"
  },
  {
    "startMs": 270880,
    "text": "uh, the optimizations we made to improve"
  },
  {
    "startMs": 273040,
    "text": "the quality of recommendations included"
  },
  {
    "startMs": 275360,
    "text": "simplifying our ranking architecture to"
  },
  {
    "startMs": 277360,
    "text": "enable efficient model scaling and"
  },
  {
    "startMs": 280000,
    "text": "longer interaction histories to identify"
  },
  {
    "startMs": 282160,
    "text": "a person's interests."
  },
  {
    "startMs": 284400,
    "text": "uh we doubled the length of user"
  },
  {
    "startMs": 286960,
    "text": "interaction sequences used for training"
  },
  {
    "startMs": 288800,
    "text": "Instagram and increased the richness of"
  },
  {
    "startMs": 291280,
    "text": "each user interaction."
  },
  {
    "startMs": 293680,
    "text": "So these are direct parallels to the"
  },
  {
    "startMs": 296080,
    "text": "power loss scaling curves for LLMs."
  },
  {
    "startMs": 299360,
    "text": "And I want to introduce this idea of a"
  },
  {
    "startMs": 302160,
    "text": "flywheel of tokens in engagement out"
  },
  {
    "startMs": 304479,
    "text": "which is what's powering all of these"
  },
  {
    "startMs": 306479,
    "text": "Rex model scaling. You train a model."
  },
  {
    "startMs": 309360,
    "text": "You then run inference on it which is"
  },
  {
    "startMs": 311199,
    "text": "the tokens in that recommendation model"
  },
  {
    "startMs": 314240,
    "text": "results in better content"
  },
  {
    "startMs": 316800,
    "text": "recommendations. It drives consumer"
  },
  {
    "startMs": 318800,
    "text": "engagement, daily active users time"
  },
  {
    "startMs": 321039,
    "text": "spent. It translates to monetization and"
  },
  {
    "startMs": 323600,
    "text": "ads or subscription which pays for the"
  },
  {
    "startMs": 326160,
    "text": "next model training run. And so every"
  },
  {
    "startMs": 328880,
    "text": "step on the scaling curve is one loop"
  },
  {
    "startMs": 331759,
    "text": "around this flywheel. And a lot of"
  },
  {
    "startMs": 333840,
    "text": "consumer apps are spinning this core"
  },
  {
    "startMs": 335840,
    "text": "flywheel at the heart of their business."
  },
  {
    "startMs": 339039,
    "text": "So we have a long way to scale these"
  },
  {
    "startMs": 340400,
    "text": "recommener systems. I want to talk about"
  },
  {
    "startMs": 342160,
    "text": "the four paradigms that I see the"
  },
  {
    "startMs": 344320,
    "text": "industry progressing through. The first"
  },
  {
    "startMs": 347039,
    "text": "scurve was more traditional Rexus where"
  },
  {
    "startMs": 350400,
    "text": "this scurve focused more on feature"
  },
  {
    "startMs": 352320,
    "text": "engineering and user and content"
  },
  {
    "startMs": 354000,
    "text": "embeddings. Most production systems are"
  },
  {
    "startMs": 356639,
    "text": "still sitting on this curve. they're"
  },
  {
    "startMs": 358080,
    "text": "running some type of two tower sparse"
  },
  {
    "startMs": 360320,
    "text": "network rankers scaling the embedding"
  },
  {
    "startMs": 362880,
    "text": "models. I don't think this curve is"
  },
  {
    "startMs": 365199,
    "text": "going to go away but model development"
  },
  {
    "startMs": 367120,
    "text": "here will be accelerated with auto"
  },
  {
    "startMs": 369039,
    "text": "research and things like feature"
  },
  {
    "startMs": 371039,
    "text": "engineering will be handled by agents uh"
  },
  {
    "startMs": 373919,
    "text": "rather than real ML engineers. The next"
  },
  {
    "startMs": 377360,
    "text": "curve is kind of LM inspired models"
  },
  {
    "startMs": 380240,
    "text": "where you are scaling models ideally end"
  },
  {
    "startMs": 382479,
    "text": "to end. HSTU and one wreck papers are"
  },
  {
    "startMs": 385600,
    "text": "examples in this paradigm. I think this"
  },
  {
    "startMs": 388160,
    "text": "is"
  },
  {
    "startMs": 389840,
    "text": "where the leading recommener systems in"
  },
  {
    "startMs": 391680,
    "text": "the industry are largely operating"
  },
  {
    "startMs": 393360,
    "text": "today. I think the next scurve will be"
  },
  {
    "startMs": 396400,
    "text": "this paradigm of LLM native where you"
  },
  {
    "startMs": 399120,
    "text": "adapt a base model that understands and"
  },
  {
    "startMs": 402080,
    "text": "can reason uh and adapt it for"
  },
  {
    "startMs": 404800,
    "text": "recommendation tasks. The tiger and plum"
  },
  {
    "startMs": 407199,
    "text": "papers are some examples of this"
  },
  {
    "startMs": 408639,
    "text": "paradigm. And I think the final paradigm"
  },
  {
    "startMs": 411520,
    "text": "that I start to see emerging is agentic"
  },
  {
    "startMs": 414240,
    "text": "where LLMs will start to orchestrate Rex"
  },
  {
    "startMs": 416800,
    "text": "systems in a loop. I think there's a"
  },
  {
    "startMs": 418960,
    "text": "parallel here with coding agents. So the"
  },
  {
    "startMs": 421199,
    "text": "LM native models are like improving the"
  },
  {
    "startMs": 424160,
    "text": "core capabilities of the model going"
  },
  {
    "startMs": 426080,
    "text": "from opus 45 to 48 versus the agentic"
  },
  {
    "startMs": 429520,
    "text": "curve will be like improving the coding"
  },
  {
    "startMs": 431360,
    "text": "harness behind claw code or codeex. Um"
  },
  {
    "startMs": 434800,
    "text": "and so instead of just having a single"
  },
  {
    "startMs": 436560,
    "text": "forward pass through the recommener, you"
  },
  {
    "startMs": 439120,
    "text": "can imagine a loop where agents plan,"
  },
  {
    "startMs": 442319,
    "text": "retrieve, rank and then critique the"
  },
  {
    "startMs": 445120,
    "text": "recommendations. They can refine them by"
  },
  {
    "startMs": 447440,
    "text": "calling models again and finally deliver"
  },
  {
    "startMs": 449919,
    "text": "the recommendations. This I think is an"
  },
  {
    "startMs": 451919,
    "text": "interesting area of research. Now"
  },
  {
    "startMs": 454880,
    "text": "let me jump into"
  },
  {
    "startMs": 457680,
    "text": "so here here's kind of the framework of"
  },
  {
    "startMs": 459440,
    "text": "all the four Rex paradigms. I think"
  },
  {
    "startMs": 462240,
    "text": "companies are scaling across each of"
  },
  {
    "startMs": 465199,
    "text": "these curves in parallel."
  },
  {
    "startMs": 468160,
    "text": "Most of recommendations I think lives in"
  },
  {
    "startMs": 470240,
    "text": "LM inspired today and is trying to"
  },
  {
    "startMs": 472479,
    "text": "graduate into LM native. Uh but then a"
  },
  {
    "startMs": 475440,
    "text": "lot of companies are still using"
  },
  {
    "startMs": 477120,
    "text": "traditional models uh and climbing that"
  },
  {
    "startMs": 479759,
    "text": "scurve."
  },
  {
    "startMs": 482400,
    "text": "Let me shift gears a bit to share the"
  },
  {
    "startMs": 485360,
    "text": "recipe of how to actually build a LLM"
  },
  {
    "startMs": 487440,
    "text": "recommener."
  },
  {
    "startMs": 490479,
    "text": "I think it's pretty simple. It's three"
  },
  {
    "startMs": 492720,
    "text": "steps. Uh you start with tokenizing your"
  },
  {
    "startMs": 495440,
    "text": "content and creating a language for your"
  },
  {
    "startMs": 497280,
    "text": "domain."
  },
  {
    "startMs": 498879,
    "text": "Then you want to adapt the LLM so that"
  },
  {
    "startMs": 501440,
    "text": "it understands both English and your"
  },
  {
    "startMs": 503599,
    "text": "domain language and becomes this"
  },
  {
    "startMs": 505680,
    "text": "bilingual model. Finally, you can prompt"
  },
  {
    "startMs": 508720,
    "text": "this model with user information and it"
  },
  {
    "startMs": 511759,
    "text": "will directly decode recommendations"
  },
  {
    "startMs": 514159,
    "text": "from your content corpus."
  },
  {
    "startMs": 516719,
    "text": "Let's go a bit deeper. This is the LLM"
  },
  {
    "startMs": 519039,
    "text": "recommener as a five layer cake. Um,"
  },
  {
    "startMs": 521599,
    "text": "we'll start at the bottom. Uh, that's"
  },
  {
    "startMs": 524080,
    "text": "semantic ID where you're converting your"
  },
  {
    "startMs": 526160,
    "text": "content corpus into tokens that the LLM"
  },
  {
    "startMs": 529519,
    "text": "can understand and reason over. Then you"
  },
  {
    "startMs": 531760,
    "text": "have kind of the base LLM foundation"
  },
  {
    "startMs": 534480,
    "text": "model. This can be an open weights model"
  },
  {
    "startMs": 536800,
    "text": "or an internal first party model. Then"
  },
  {
    "startMs": 540480,
    "text": "the core training stages. Pre-training"
  },
  {
    "startMs": 542640,
    "text": "is around bridging English and these"
  },
  {
    "startMs": 544959,
    "text": "recommener tokens. Post-raining is about"
  },
  {
    "startMs": 548480,
    "text": "steering the model towards"
  },
  {
    "startMs": 551440,
    "text": "recommendation tasks like predicting"
  },
  {
    "startMs": 553839,
    "text": "engagement or reasoning over"
  },
  {
    "startMs": 555519,
    "text": "recommendations."
  },
  {
    "startMs": 557360,
    "text": "And then finally, you can just do some"
  },
  {
    "startMs": 559040,
    "text": "light surface specific fine-tuning to"
  },
  {
    "startMs": 561760,
    "text": "deploy it on a product surface. The"
  },
  {
    "startMs": 564240,
    "text": "exciting thing about this paradigm is"
  },
  {
    "startMs": 566399,
    "text": "most of the compute is shared across all"
  },
  {
    "startMs": 568800,
    "text": "of the product surfaces. So you don't"
  },
  {
    "startMs": 570880,
    "text": "have to train individual models from"
  },
  {
    "startMs": 572720,
    "text": "scratch for every product surface."
  },
  {
    "startMs": 576080,
    "text": "I'll go a bit deeper into each stage for"
  },
  {
    "startMs": 578720,
    "text": "semantic ids. I think this has seen"
  },
  {
    "startMs": 580640,
    "text": "incredible adoption. A lot of teams are"
  },
  {
    "startMs": 582560,
    "text": "just replacing their hash ID with the"
  },
  {
    "startMs": 585680,
    "text": "SID in traditional models and seeing"
  },
  {
    "startMs": 587680,
    "text": "good impact. I think there's two big"
  },
  {
    "startMs": 589920,
    "text": "reasons to tokenize content. The first"
  },
  {
    "startMs": 592320,
    "text": "is it gives you the stable"
  },
  {
    "startMs": 593839,
    "text": "representation for models to learn over"
  },
  {
    "startMs": 596320,
    "text": "rather than a constantly shifting hash"
  },
  {
    "startMs": 598720,
    "text": "that the model can only memorize. And"
  },
  {
    "startMs": 601279,
    "text": "the second is compression. You want to"
  },
  {
    "startMs": 603600,
    "text": "be reasoning over these long sequences"
  },
  {
    "startMs": 605600,
    "text": "of user interactions. And if you don't"
  },
  {
    "startMs": 608800,
    "text": "compress the content, for example, a"
  },
  {
    "startMs": 610560,
    "text": "threeminut Instagram real video would be"
  },
  {
    "startMs": 612399,
    "text": "10,000 tokens and it will just fill up"
  },
  {
    "startMs": 614959,
    "text": "the content context window too quickly."
  },
  {
    "startMs": 617279,
    "text": "So you have to to compress it into about"
  },
  {
    "startMs": 619839,
    "text": "10 tokens. And so here I have some"
  },
  {
    "startMs": 622160,
    "text": "examples of Instagram reels about"
  },
  {
    "startMs": 623600,
    "text": "tennis. You can see that the semantic"
  },
  {
    "startMs": 625920,
    "text": "token shares the prefix of the first"
  },
  {
    "startMs": 628320,
    "text": "three tokens because they're very"
  },
  {
    "startMs": 630240,
    "text": "similar reels. You can imagine the first"
  },
  {
    "startMs": 632000,
    "text": "token representing sports um and the"
  },
  {
    "startMs": 635600,
    "text": "second two tokens representing tennis"
  },
  {
    "startMs": 637920,
    "text": "and then the final token making these"
  },
  {
    "startMs": 639600,
    "text": "videos individual."
  },
  {
    "startMs": 642399,
    "text": "Once you have a semantic ID, you can"
  },
  {
    "startMs": 644800,
    "text": "train it to understand both English and"
  },
  {
    "startMs": 648399,
    "text": "semantic ID. And so the task I have on"
  },
  {
    "startMs": 651040,
    "text": "the left for pre-training here is an"
  },
  {
    "startMs": 652640,
    "text": "example of where you prompt with a video"
  },
  {
    "startMs": 655920,
    "text": "with semantic ID ABC has the description"
  },
  {
    "startMs": 658480,
    "text": "blank and the output is a shot that was"
  },
  {
    "startMs": 660959,
    "text": "instantly iconic from Wimbledon. Here"
  },
  {
    "startMs": 663440,
    "text": "you're teaching the model to connect"
  },
  {
    "startMs": 664880,
    "text": "these semantic tokens with synthetic"
  },
  {
    "startMs": 668000,
    "text": "English natural language text. The"
  },
  {
    "startMs": 670079,
    "text": "example on the right is about reasoning"
  },
  {
    "startMs": 672480,
    "text": "over sequences of semantic IDs. And so"
  },
  {
    "startMs": 675519,
    "text": "in a user's interaction history, you can"
  },
  {
    "startMs": 677680,
    "text": "mask some parts of the sequence and the"
  },
  {
    "startMs": 680560,
    "text": "model learns to predict them and"
  },
  {
    "startMs": 682640,
    "text": "understand what videos are watched"
  },
  {
    "startMs": 685519,
    "text": "together in sequence."
  },
  {
    "startMs": 687760,
    "text": "Here I have an example of post training"
  },
  {
    "startMs": 689519,
    "text": "where we're teaching the model how to"
  },
  {
    "startMs": 691040,
    "text": "rerank content. So the input is a bunch"
  },
  {
    "startMs": 693600,
    "text": "of user information and 30 candidate"
  },
  {
    "startMs": 696320,
    "text": "videos that are then ranked to be the"
  },
  {
    "startMs": 699360,
    "text": "top five recommendations from this LLM"
  },
  {
    "startMs": 702160,
    "text": "ranker. What's really interesting here"
  },
  {
    "startMs": 704560,
    "text": "is that you can see the chain of thought"
  },
  {
    "startMs": 707519,
    "text": "reasoning of this model. And because"
  },
  {
    "startMs": 709680,
    "text": "this model knows both English and"
  },
  {
    "startMs": 711760,
    "text": "recommendations, you can simply just"
  },
  {
    "startMs": 714079,
    "text": "inspect the model and understand why it"
  },
  {
    "startMs": 716320,
    "text": "made the decisions that it did. In this"
  },
  {
    "startMs": 718640,
    "text": "example, the model understands the"
  },
  {
    "startMs": 720399,
    "text": "user's topic interests like comedy,"
  },
  {
    "startMs": 723519,
    "text": "food, DIY, wellness. It understands the"
  },
  {
    "startMs": 726720,
    "text": "engagement style and what creators this"
  },
  {
    "startMs": 729360,
    "text": "user has an affinity towards."
  },
  {
    "startMs": 732560,
    "text": "and then it reranks the content based on"
  },
  {
    "startMs": 735279,
    "text": "this chain of thought reasoning."
  },
  {
    "startMs": 738720,
    "text": "I think this is super exciting because"
  },
  {
    "startMs": 740560,
    "text": "once you have a model that can"
  },
  {
    "startMs": 741920,
    "text": "understand both English and"
  },
  {
    "startMs": 744079,
    "text": "recommendations, it opens up new product"
  },
  {
    "startMs": 747360,
    "text": "surfaces and new experiences where users"
  },
  {
    "startMs": 750399,
    "text": "can steer their feed. Here's an example"
  },
  {
    "startMs": 753040,
    "text": "from your algorithm on Instagram where"
  },
  {
    "startMs": 756480,
    "text": "users can talk to the algorithm while"
  },
  {
    "startMs": 758880,
    "text": "they're consuming content. It's a"
  },
  {
    "startMs": 761040,
    "text": "screenshot from scrolling through reels"
  },
  {
    "startMs": 763600,
    "text": "or when you click in you can understand"
  },
  {
    "startMs": 765920,
    "text": "what the Instagram algorithm thinks"
  },
  {
    "startMs": 767839,
    "text": "about you and your interests and then"
  },
  {
    "startMs": 769839,
    "text": "you can add or remove interest and talk"
  },
  {
    "startMs": 771839,
    "text": "to it in natural language. And so we're"
  },
  {
    "startMs": 774720,
    "text": "going to see this shift I think from"
  },
  {
    "startMs": 776880,
    "text": "blackbox recommendations algorithms to"
  },
  {
    "startMs": 780160,
    "text": "giving users more control over their"
  },
  {
    "startMs": 782160,
    "text": "algorithm and algorithms becoming more"
  },
  {
    "startMs": 784560,
    "text": "interactive and steerable."
  },
  {
    "startMs": 786959,
    "text": "I'm really excited that users can direct"
  },
  {
    "startMs": 789040,
    "text": "it towards their own goals that's"
  },
  {
    "startMs": 791120,
    "text": "expressed in language rather than just"
  },
  {
    "startMs": 793600,
    "text": "likes or comments. Um, and I think this"
  },
  {
    "startMs": 797920,
    "text": "can this foundation model can also start"
  },
  {
    "startMs": 800800,
    "text": "to explain its recommendations."
  },
  {
    "startMs": 803920,
    "text": "And so for this example, I've added an"
  },
  {
    "startMs": 806399,
    "text": "interest that I want to follow the FIFA"
  },
  {
    "startMs": 807920,
    "text": "World Cup at this time. And the model"
  },
  {
    "startMs": 810079,
    "text": "would get both my user history and this"
  },
  {
    "startMs": 813440,
    "text": "new input and then be able to decode"
  },
  {
    "startMs": 816079,
    "text": "recommendations that are personalized to"
  },
  {
    "startMs": 817760,
    "text": "me like this free cake that Messi scored"
  },
  {
    "startMs": 820399,
    "text": "recently."
  },
  {
    "startMs": 824320,
    "text": "I think these interactive recommenders"
  },
  {
    "startMs": 827440,
    "text": "are going to be a really interesting new"
  },
  {
    "startMs": 828959,
    "text": "product surface and we're seeing this"
  },
  {
    "startMs": 830480,
    "text": "across the industry. We have some"
  },
  {
    "startMs": 832480,
    "text": "examples of prompted playlists from"
  },
  {
    "startMs": 834079,
    "text": "Spotify, custom feeds from YouTube, Ask"
  },
  {
    "startMs": 837120,
    "text": "Door Dash, and we'll be hearing more"
  },
  {
    "startMs": 839199,
    "text": "from speakers about these."
  },
  {
    "startMs": 843199,
    "text": "Um, finally, I want to talk about kind"
  },
  {
    "startMs": 846639,
    "text": "of the framework of tokens in engagement"
  },
  {
    "startMs": 849360,
    "text": "out this flywheel that I started with um"
  },
  {
    "startMs": 853519,
    "text": "of model training, inference, consumer"
  },
  {
    "startMs": 856320,
    "text": "engagement, and then monetization."
  },
  {
    "startMs": 859120,
    "text": "This is actually the same flywheel"
  },
  {
    "startMs": 861600,
    "text": "that's shared by content feeds and the"
  },
  {
    "startMs": 864560,
    "text": "AI chat apps. And this is a lens that"
  },
  {
    "startMs": 867680,
    "text": "you can use to evaluate any consumer"
  },
  {
    "startMs": 869600,
    "text": "app. What'll make an app successful on"
  },
  {
    "startMs": 872160,
    "text": "the training ROI side is how well can it"
  },
  {
    "startMs": 875120,
    "text": "translate compute into a frontier model."
  },
  {
    "startMs": 878480,
    "text": "On the inference side, how well can the"
  },
  {
    "startMs": 880800,
    "text": "inference tokens translate into"
  },
  {
    "startMs": 883199,
    "text": "engagement and then monetization."
  },
  {
    "startMs": 886000,
    "text": "And so if you try to compare content"
  },
  {
    "startMs": 888560,
    "text": "feeds and AI chat apps, I think that LM"
  },
  {
    "startMs": 892079,
    "text": "recommenders are actually structurally"
  },
  {
    "startMs": 894000,
    "text": "more token efficient than the AI chat."
  },
  {
    "startMs": 897440,
    "text": "So on the left you have content feeds"
  },
  {
    "startMs": 899199,
    "text": "like Instagram, Facebook, Tik Tok and"
  },
  {
    "startMs": 901199,
    "text": "YouTube. On the right you have the big"
  },
  {
    "startMs": 902959,
    "text": "AI chat apps like Gemini, ChatGpt and"
  },
  {
    "startMs": 905680,
    "text": "Claude."
  },
  {
    "startMs": 907440,
    "text": "Content feeds are currently using models"
  },
  {
    "startMs": 909360,
    "text": "that are around 1 to 10 billion active"
  },
  {
    "startMs": 912079,
    "text": "parameters. The chat apps are serving"
  },
  {
    "startMs": 914320,
    "text": "much larger models 10 to 100 billion"
  },
  {
    "startMs": 916320,
    "text": "active parameters. Um the output for the"
  },
  {
    "startMs": 919519,
    "text": "content feed is a semantic ID token"
  },
  {
    "startMs": 922720,
    "text": "which is a pointer or an address to"
  },
  {
    "startMs": 924959,
    "text": "existing content because the content"
  },
  {
    "startMs": 927199,
    "text": "supplied on these content feeds is"
  },
  {
    "startMs": 929040,
    "text": "uploaded by creators. There's a very"
  },
  {
    "startMs": 931040,
    "text": "healthy creator economy and so the"
  },
  {
    "startMs": 933360,
    "text": "supply of content is effectively free or"
  },
  {
    "startMs": 936079,
    "text": "it's uploaded by creators. For the AI"
  },
  {
    "startMs": 939279,
    "text": "chat apps, they have to decode every"
  },
  {
    "startMs": 941600,
    "text": "token of content themselves and the"
  },
  {
    "startMs": 945839,
    "text": "amount of tokens output in every turn of"
  },
  {
    "startMs": 949120,
    "text": "an LM chat interaction is a few thousand"
  },
  {
    "startMs": 952880,
    "text": "tokens. Uh, and the the big difference"
  },
  {
    "startMs": 956720,
    "text": "here is that every token has to be"
  },
  {
    "startMs": 958240,
    "text": "manufactured by the app at inference"
  },
  {
    "startMs": 960399,
    "text": "time. And so what that means is if you"
  },
  {
    "startMs": 962560,
    "text": "compare these two apps on how much"
  },
  {
    "startMs": 966399,
    "text": "inference cost and compute is spent to"
  },
  {
    "startMs": 968480,
    "text": "generate an hour of consumer engagement,"
  },
  {
    "startMs": 970959,
    "text": "there's a huge structural gap where"
  },
  {
    "startMs": 973360,
    "text": "content feeds are significantly cheaper"
  },
  {
    "startMs": 976000,
    "text": "up to 100 times or more cheaper than AI"
  },
  {
    "startMs": 978880,
    "text": "chat apps because they're decoding"
  },
  {
    "startMs": 980639,
    "text": "pointers to content rather than content"
  },
  {
    "startMs": 982480,
    "text": "itself."
  },
  {
    "startMs": 984959,
    "text": "Um, finally I want to kind of end with"
  },
  {
    "startMs": 987279,
    "text": "why I think LM Rexus is one of the most"
  },
  {
    "startMs": 990959,
    "text": "significant consumer AI applications. If"
  },
  {
    "startMs": 994000,
    "text": "you look at the top apps by daily active"
  },
  {
    "startMs": 996399,
    "text": "users, these are the top 10 apps. Four"
  },
  {
    "startMs": 998720,
    "text": "out of 10 of them are content feeds. And"
  },
  {
    "startMs": 1002000,
    "text": "so this is a really significant consumer"
  },
  {
    "startMs": 1005920,
    "text": "uh application. If you look at the"
  },
  {
    "startMs": 1008639,
    "text": "content feeds, almost all of the"
  },
  {
    "startMs": 1010800,
    "text": "consumer app growth on both the"
  },
  {
    "startMs": 1012880,
    "text": "engagement and monetization side is"
  },
  {
    "startMs": 1015040,
    "text": "driven by the recommener and ads models."
  },
  {
    "startMs": 1018000,
    "text": "And these are going to be entirely"
  },
  {
    "startMs": 1019360,
    "text": "transformed by LLM recommenders."
  },
  {
    "startMs": 1022880,
    "text": "It's a very large and very token"
  },
  {
    "startMs": 1025039,
    "text": "efficient application of AI for consumer"
  },
  {
    "startMs": 1027600,
    "text": "apps. We're going to see a lot of new"
  },
  {
    "startMs": 1029760,
    "text": "product experiences come through with"
  },
  {
    "startMs": 1032640,
    "text": "steerable and interactive"
  },
  {
    "startMs": 1033919,
    "text": "recommendations, explanation of"
  },
  {
    "startMs": 1036079,
    "text": "recommendations, and just putting more"
  },
  {
    "startMs": 1038160,
    "text": "users in control of their experience on"
  },
  {
    "startMs": 1040640,
    "text": "these apps. I think we're going to see"
  },
  {
    "startMs": 1043039,
    "text": "some really exciting research on"
  },
  {
    "startMs": 1044959,
    "text": "consumer agents and recommendation"
  },
  {
    "startMs": 1046880,
    "text": "agents that come out over the next year"
  },
  {
    "startMs": 1048640,
    "text": "or so. Um and so this is why I think"
  },
  {
    "startMs": 1051280,
    "text": "this is a super exciting area of both"
  },
  {
    "startMs": 1054000,
    "text": "research and product uh at this"
  },
  {
    "startMs": 1056320,
    "text": "intersection of LLM and recommendations."
  },
  {
    "startMs": 1061120,
    "text": "That's all. Thank you so much."
  }
]