[
  {
    "startMs": 1309,
    "text": "[music]"
  },
  {
    "startMs": 12640,
    "text": ">> Joining us on stage is the co-founder"
  },
  {
    "startMs": 14840,
    "text": "and chief science officer at Hugging"
  },
  {
    "startMs": 16760,
    "text": "Face, Thomas Wolf."
  },
  {
    "startMs": 20305,
    "text": ">> [music]"
  },
  {
    "startMs": 26200,
    "text": "[music]"
  },
  {
    "startMs": 32000,
    "text": ">> Hello everyone."
  },
  {
    "startMs": 33560,
    "text": "Hello Olive, nice to have you on stage."
  },
  {
    "startMs": 36040,
    "text": ">> Hi, nice to meet you. Thanks for having"
  },
  {
    "startMs": 37640,
    "text": "me, yeah."
  },
  {
    "startMs": 38640,
    "text": ">> So I think you're on for a treat today"
  },
  {
    "startMs": 40400,
    "text": "because you just saw a GLM"
  },
  {
    "startMs": 43360,
    "text": "uh which is current number"
  },
  {
    "startMs": 46240,
    "text": "two on the artificial intelligence"
  },
  {
    "startMs": 48000,
    "text": "leaderboard. I take Fable out because"
  },
  {
    "startMs": 50440,
    "text": "nobody can use it. And now we have"
  },
  {
    "startMs": 52080,
    "text": "number four. So you basically you will"
  },
  {
    "startMs": 53600,
    "text": "have all the top models, at least the"
  },
  {
    "startMs": 55760,
    "text": "top open source model in a row. And"
  },
  {
    "startMs": 57920,
    "text": "we're very lucky to have"
  },
  {
    "startMs": 59520,
    "text": "Olive who has a"
  },
  {
    "startMs": 61880,
    "text": "pretty amazing path in life."
  },
  {
    "startMs": 64720,
    "text": "Uh so she came to the US, Pennsylvania."
  },
  {
    "startMs": 66960,
    "text": "She was studying, doing PhD at uh"
  },
  {
    "startMs": 69400,
    "text": "NYU uh in the lab of Jan LeCun"
  },
  {
    "startMs": 73480,
    "text": "working on J Pa, but we decided we won't"
  },
  {
    "startMs": 75360,
    "text": "talk about J Pa today, right?"
  },
  {
    "startMs": 77520,
    "text": "Something for another day. Um and then"
  },
  {
    "startMs": 80280,
    "text": "instead of joining Hugging Face, which"
  },
  {
    "startMs": 81960,
    "text": "was in New York also at that time she"
  },
  {
    "startMs": 84680,
    "text": "decided to go join MiniMax."
  },
  {
    "startMs": 88040,
    "text": "So for those who who maybe don't know"
  },
  {
    "startMs": 90600,
    "text": "all the all the neo labs around the"
  },
  {
    "startMs": 93400,
    "text": "world and you're you're forgiven because"
  },
  {
    "startMs": 95320,
    "text": "I think there's like 64 neo labs right"
  },
  {
    "startMs": 97680,
    "text": "now. MiniMax is one of the top of what"
  },
  {
    "startMs": 100040,
    "text": "we call the AI dragons in China."
  },
  {
    "startMs": 103240,
    "text": "So these are the new there's there's"
  },
  {
    "startMs": 105080,
    "text": "Deep Seek which is very well known now,"
  },
  {
    "startMs": 107680,
    "text": "Moonshot who does Kimi Z and GLM that"
  },
  {
    "startMs": 110760,
    "text": "you just saw and now we have a MiniMax."
  },
  {
    "startMs": 113240,
    "text": "They're all extremely good, extremely a"
  },
  {
    "startMs": 115520,
    "text": "team"
  },
  {
    "startMs": 116600,
    "text": "uh fighting for the first spot."
  },
  {
    "startMs": 119000,
    "text": "Uh so the the the latest uh release of"
  },
  {
    "startMs": 121960,
    "text": "MiniMax was M3 uh just earlier earlier"
  },
  {
    "startMs": 125040,
    "text": "in June, which was the the top model at"
  },
  {
    "startMs": 127000,
    "text": "the time, top open-source model."
  },
  {
    "startMs": 129080,
    "text": "Uh very impressive. There's a lot of"
  },
  {
    "startMs": 131040,
    "text": "very interesting things about this"
  },
  {
    "startMs": 132520,
    "text": "model, so we'll quickly dive in them."
  },
  {
    "startMs": 135120,
    "text": "And then talk a little bit about uh"
  },
  {
    "startMs": 137320,
    "text": "what's what's what's specific about"
  },
  {
    "startMs": 138840,
    "text": "MiniMax, what's what's great there."
  },
  {
    "startMs": 142040,
    "text": "So uh maybe Olive to to start a little"
  },
  {
    "startMs": 144800,
    "text": "bit. Can you Can you give us, you know,"
  },
  {
    "startMs": 146640,
    "text": "a"
  },
  {
    "startMs": 147600,
    "text": "a little bit of your your view of of M3,"
  },
  {
    "startMs": 150200,
    "text": "what you like about this model, how was"
  },
  {
    "startMs": 152000,
    "text": "the release?"
  },
  {
    "startMs": 153200,
    "text": ">> Mhm."
  },
  {
    "startMs": 154080,
    "text": "Yeah, M3 we released M3 earlier this"
  },
  {
    "startMs": 156920,
    "text": "month, and it is a smaller model with"
  },
  {
    "startMs": 160320,
    "text": "400 around 400 billion total parameters"
  },
  {
    "startMs": 163520,
    "text": "and 20 billion activated. Um"
  },
  {
    "startMs": 166120,
    "text": "but it is very capable in terms of both"
  },
  {
    "startMs": 168840,
    "text": "coding performances, and also it"
  },
  {
    "startMs": 171200,
    "text": "understands vision. So um"
  },
  {
    "startMs": 174400,
    "text": "that's uh what"
  },
  {
    "startMs": 176400,
    "text": "open-source models don't usually have."
  },
  {
    "startMs": 178920,
    "text": "It's that they can"
  },
  {
    "startMs": 180600,
    "text": "the model can only deal with coding, but"
  },
  {
    "startMs": 183000,
    "text": "it can also understand videos, um"
  },
  {
    "startMs": 185320,
    "text": "images, and it has a super uh long"
  },
  {
    "startMs": 187800,
    "text": "context of 1 million."
  },
  {
    "startMs": 189760,
    "text": "Um with our new architecture called MSA,"
  },
  {
    "startMs": 192400,
    "text": "MiniMax Sparse Attention. So we"
  },
  {
    "startMs": 195600,
    "text": "we really put these three things"
  },
  {
    "startMs": 197600,
    "text": "together uh because we know that they"
  },
  {
    "startMs": 201000,
    "text": "are they will be very important in"
  },
  {
    "startMs": 203160,
    "text": "future AI applications. Coding"
  },
  {
    "startMs": 205720,
    "text": "capabilities, agentic capabilities,"
  },
  {
    "startMs": 208080,
    "text": "longer context, and multimodal"
  },
  {
    "startMs": 210959,
    "text": "understanding. Um yeah, I think that"
  },
  {
    "startMs": 213280,
    "text": "would be very interesting about the"
  },
  {
    "startMs": 214840,
    "text": "model."
  },
  {
    "startMs": 215560,
    "text": ">> Yeah, so so there's a lot to unpack"
  },
  {
    "startMs": 217519,
    "text": "unpack in this model, and it's um"
  },
  {
    "startMs": 219800,
    "text": "it's it's still, I think, the only top"
  },
  {
    "startMs": 221840,
    "text": "five model open-source model that is"
  },
  {
    "startMs": 223640,
    "text": "actually multimodal, so we need to talk"
  },
  {
    "startMs": 225280,
    "text": "about that. But maybe first about the"
  },
  {
    "startMs": 227040,
    "text": "long context, because that was also the"
  },
  {
    "startMs": 228640,
    "text": "first one that really had this real 1"
  },
  {
    "startMs": 231360,
    "text": "million token long context is actually"
  },
  {
    "startMs": 233960,
    "text": "functional and you guys had also the the"
  },
  {
    "startMs": 236800,
    "text": "Minimax pass attention"
  },
  {
    "startMs": 239160,
    "text": "which is this one technique to to make"
  },
  {
    "startMs": 240960,
    "text": "that efficient that you also published"
  },
  {
    "startMs": 243880,
    "text": "and and share extensively. So can you"
  },
  {
    "startMs": 245640,
    "text": "can you talk a little bit about this?"
  },
  {
    "startMs": 247160,
    "text": "Maybe how the project went from"
  },
  {
    "startMs": 249440,
    "text": "from the attention how to make this long"
  },
  {
    "startMs": 251000,
    "text": "context."
  },
  {
    "startMs": 252080,
    "text": ">> Yeah, I would say the story about long"
  },
  {
    "startMs": 254160,
    "text": "context went back to even Minimax M1 and"
  },
  {
    "startMs": 257920,
    "text": "Minimax 01 where the model was actually"
  },
  {
    "startMs": 261519,
    "text": "was able to perform tasks of 10 million"
  },
  {
    "startMs": 265080,
    "text": "token context."
  },
  {
    "startMs": 266080,
    "text": ">> 10 million?"
  },
  {
    "startMs": 266880,
    "text": ">> 10 million, yes."
  },
  {
    "startMs": 268919,
    "text": "But then it was not an agentic model,"
  },
  {
    "startMs": 271320,
    "text": "right? It was just 10 for example, you"
  },
  {
    "startMs": 273480,
    "text": "dump in a book, it would be able to give"
  },
  {
    "startMs": 275600,
    "text": "reviews on it and stuff like that. So"
  },
  {
    "startMs": 278880,
    "text": "what we realized was that, you know,"
  },
  {
    "startMs": 281160,
    "text": "longer context actually unlocks a lot of"
  },
  {
    "startMs": 283760,
    "text": "capabilities especially when interacting"
  },
  {
    "startMs": 287000,
    "text": "with users and now when, you know, the"
  },
  {
    "startMs": 289800,
    "text": "agent is interacting with the whole"
  },
  {
    "startMs": 291480,
    "text": "environment and getting all the tool"
  },
  {
    "startMs": 293120,
    "text": "responses,"
  },
  {
    "startMs": 294560,
    "text": "getting multi rounds"
  },
  {
    "startMs": 297520,
    "text": "the like shorter context wouldn't be"
  },
  {
    "startMs": 300360,
    "text": "enough to perform the complex task. So"
  },
  {
    "startMs": 304840,
    "text": "for this version we said, \"Oh, we have"
  },
  {
    "startMs": 307440,
    "text": "to have our longer context backs.\""
  },
  {
    "startMs": 310640,
    "text": "So what we pursued was with our Minimax"
  },
  {
    "startMs": 314040,
    "text": "sparse attention."
  },
  {
    "startMs": 315919,
    "text": "Um which you know, was the architecture"
  },
  {
    "startMs": 320240,
    "text": "that was scalable"
  },
  {
    "startMs": 322320,
    "text": "and had a simple design. So I would say"
  },
  {
    "startMs": 326120,
    "text": "from a higher level, right? It has an"
  },
  {
    "startMs": 328560,
    "text": "index branch"
  },
  {
    "startMs": 330240,
    "text": "that, you know, selects"
  },
  {
    "startMs": 333400,
    "text": "on a higher level what is what matters"
  },
  {
    "startMs": 335720,
    "text": "more in the context and then we have a"
  },
  {
    "startMs": 338160,
    "text": "sparse attention branch that calculates"
  },
  {
    "startMs": 340360,
    "text": "performs the calculation on the selected"
  },
  {
    "startMs": 342320,
    "text": "blocks to actually performs the task."
  },
  {
    "startMs": 345480,
    "text": "Um and so"
  },
  {
    "startMs": 347760,
    "text": "yeah, like that we really designed um an"
  },
  {
    "startMs": 351320,
    "text": "elegant architecture so that we can"
  },
  {
    "startMs": 353320,
    "text": "scale the length and then scale the"
  },
  {
    "startMs": 355520,
    "text": "model size in the future with that."
  },
  {
    "startMs": 357600,
    "text": ">> That's beautiful. I like how for those"
  },
  {
    "startMs": 359640,
    "text": "who've been in the field for quite some"
  },
  {
    "startMs": 361080,
    "text": "time, we we had a lot of work on"
  },
  {
    "startMs": 362600,
    "text": "attention, right? This N square and"
  },
  {
    "startMs": 364520,
    "text": "there was a lot of linear attention."
  },
  {
    "startMs": 366280,
    "text": ">> Yeah."
  },
  {
    "startMs": 366760,
    "text": ">> And then some that somehow all of these"
  },
  {
    "startMs": 368440,
    "text": "disappeared at some point when flash"
  },
  {
    "startMs": 370080,
    "text": "attention came around. We discovered we"
  },
  {
    "startMs": 371880,
    "text": "just needed more efficient camera. Now,"
  },
  {
    "startMs": 373920,
    "text": "I like how we come back to thinking, you"
  },
  {
    "startMs": 375760,
    "text": "know,"
  },
  {
    "startMs": 376760,
    "text": "first principle, what is attention? How"
  },
  {
    "startMs": 378600,
    "text": "can we make that more efficient? So, 1"
  },
  {
    "startMs": 381320,
    "text": "million token is crazy, right? GPT-2 was"
  },
  {
    "startMs": 384040,
    "text": "1,024 and and everyone was like, \"Oh,"
  },
  {
    "startMs": 386240,
    "text": "that's really big. We we we never need"
  },
  {
    "startMs": 388200,
    "text": "more.\""
  },
  {
    "startMs": 389160,
    "text": "Where do you see this coming? Like,"
  },
  {
    "startMs": 390800,
    "text": "going in the future? Like, Jeff Dean was"
  },
  {
    "startMs": 392480,
    "text": "pitching me the other day a trillion"
  },
  {
    "startMs": 393919,
    "text": "token attention. You think we should go"
  },
  {
    "startMs": 396120,
    "text": "to"
  },
  {
    "startMs": 396840,
    "text": "a trillion token attention?"
  },
  {
    "startMs": 398360,
    "text": ">> So, that's definitely something we can"
  },
  {
    "startMs": 399840,
    "text": "explore towards, right? Ultra lengths of"
  },
  {
    "startMs": 403480,
    "text": "the context. Definitely, it's something"
  },
  {
    "startMs": 406000,
    "text": "that's very exciting to explore with."
  },
  {
    "startMs": 407800,
    "text": "And something that architecture design"
  },
  {
    "startMs": 409840,
    "text": "along with hardware um"
  },
  {
    "startMs": 412200,
    "text": "would require a lot of research onto"
  },
  {
    "startMs": 413960,
    "text": "that. Yeah."
  },
  {
    "startMs": 414919,
    "text": ">> You think there's still a lot of"
  },
  {
    "startMs": 415960,
    "text": "low-hanging fruits? So, typically today"
  },
  {
    "startMs": 418200,
    "text": "we saw Open AI really reducing I mean,"
  },
  {
    "startMs": 420680,
    "text": "we don't know how as of firm, but like"
  },
  {
    "startMs": 422640,
    "text": "reducing their their inference bill by"
  },
  {
    "startMs": 424600,
    "text": "half, by probably having some more"
  },
  {
    "startMs": 426440,
    "text": "efficient processing around tensions of"
  },
  {
    "startMs": 428360,
    "text": "any type. You think there's still a lot"
  },
  {
    "startMs": 430120,
    "text": "of low-hanging fruit that can be getting"
  },
  {
    "startMs": 432600,
    "text": "um"
  },
  {
    "startMs": 433440,
    "text": "how we can process that. So, so one one"
  },
  {
    "startMs": 435240,
    "text": "thing we're still very interesting about"
  },
  {
    "startMs": 436640,
    "text": "M3 is how cheap it is in particular"
  },
  {
    "startMs": 438800,
    "text": "because of this part attention and part"
  },
  {
    "startMs": 440480,
    "text": "because of its small one, but it's also"
  },
  {
    "startMs": 442280,
    "text": "very efficient."
  },
  {
    "startMs": 443160,
    "text": ">> Right."
  },
  {
    "startMs": 443600,
    "text": ">> You think we can go even way further?"
  },
  {
    "startMs": 445960,
    "text": "And maybe how did you guys invented uh"
  },
  {
    "startMs": 449080,
    "text": "Min Max Fast Attention? Was it an agent"
  },
  {
    "startMs": 451040,
    "text": "coming up with the idea? Was it a human"
  },
  {
    "startMs": 452840,
    "text": "still coming up with the idea? Tell us a"
  },
  {
    "startMs": 454760,
    "text": "little bit about"
  },
  {
    "startMs": 455880,
    "text": ">> Yeah. Um, so we do think there's still a"
  },
  {
    "startMs": 459280,
    "text": "lot of work that can get into"
  },
  {
    "startMs": 461520,
    "text": "architecture and inference optimization"
  },
  {
    "startMs": 463800,
    "text": "so that the model can be more efficient,"
  },
  {
    "startMs": 466240,
    "text": "especially if there are tasks that are"
  },
  {
    "startMs": 468040,
    "text": "very task sensitive but require very"
  },
  {
    "startMs": 471200,
    "text": "strong capabilities, right? And for that"
  },
  {
    "startMs": 473680,
    "text": "those kind of tasks we really want model"
  },
  {
    "startMs": 475480,
    "text": "to be efficient."
  },
  {
    "startMs": 477000,
    "text": "Um and who came up with this part of"
  },
  {
    "startMs": 479920,
    "text": "actually I think an intern from our team"
  },
  {
    "startMs": 482200,
    "text": "worked on that. That yeah, an intern."
  },
  {
    "startMs": 485440,
    "text": "Uh that"
  },
  {
    "startMs": 486880,
    "text": "doesn't usually happen in a lot of labs"
  },
  {
    "startMs": 489960,
    "text": "because I think in some labs interns"
  },
  {
    "startMs": 491720,
    "text": "don't have access to"
  },
  {
    "startMs": 493680,
    "text": "the data, the work, and stuff. Uh but"
  },
  {
    "startMs": 496800,
    "text": "yeah, we are open to anyone who would"
  },
  {
    "startMs": 499000,
    "text": "like to contribute to our models. So, um"
  },
  {
    "startMs": 501360,
    "text": "the architecture was actually designed"
  },
  {
    "startMs": 503000,
    "text": "by an intern."
  },
  {
    "startMs": 504360,
    "text": ">> It's very good. Still some work for"
  },
  {
    "startMs": 505919,
    "text": "interns here. Good good news. Um that's"
  },
  {
    "startMs": 509520,
    "text": "also a good segue to also how Min Max is"
  },
  {
    "startMs": 511720,
    "text": "working internally. So, so we were"
  },
  {
    "startMs": 513320,
    "text": "discussing before coming on stage I was"
  },
  {
    "startMs": 515440,
    "text": "saying everyone can propose a project."
  },
  {
    "startMs": 518280,
    "text": "Can you tell us a little bit about how"
  },
  {
    "startMs": 519919,
    "text": "you are organized, how you do your"
  },
  {
    "startMs": 521360,
    "text": "research?"
  },
  {
    "startMs": 522159,
    "text": ">> Mhm. Mhm. I think that is very different"
  },
  {
    "startMs": 524960,
    "text": "from uh"
  },
  {
    "startMs": 526720,
    "text": "even in school or even in earlier"
  },
  {
    "startMs": 530120,
    "text": "you know, the earlier tech companies is"
  },
  {
    "startMs": 532040,
    "text": "pretty pretty different. It's that um"
  },
  {
    "startMs": 535560,
    "text": "what we"
  },
  {
    "startMs": 536680,
    "text": "what we make sure is that we have good"
  },
  {
    "startMs": 539120,
    "text": "foundation and good um infrastructure so"
  },
  {
    "startMs": 542360,
    "text": "that anyone can play with the model and"
  },
  {
    "startMs": 545200,
    "text": "can think of what they can improve with"
  },
  {
    "startMs": 547240,
    "text": "the model."
  },
  {
    "startMs": 548360,
    "text": "And then after model releases when they"
  },
  {
    "startMs": 550800,
    "text": "are free, right? They can play with the"
  },
  {
    "startMs": 553240,
    "text": "model. They can think of their own"
  },
  {
    "startMs": 555280,
    "text": "evaluations. They can find their own"
  },
  {
    "startMs": 557240,
    "text": "weaknesses and propose a thing that they"
  },
  {
    "startMs": 559680,
    "text": "want to improve on the model. And then"
  },
  {
    "startMs": 562400,
    "text": "other people who are interested in that"
  },
  {
    "startMs": 564240,
    "text": "would, you know, propose to join the"
  },
  {
    "startMs": 565760,
    "text": "project and they will work on for a"
  },
  {
    "startMs": 567839,
    "text": "couple of weeks or even a couple of"
  },
  {
    "startMs": 569560,
    "text": "months. And when they work out, the"
  },
  {
    "startMs": 571800,
    "text": "final thing is shipped to our model. It"
  },
  {
    "startMs": 574480,
    "text": "it is, you you we use that in our final"
  },
  {
    "startMs": 576720,
    "text": "training and it's shipped out to the"
  },
  {
    "startMs": 578720,
    "text": "audience."
  },
  {
    "startMs": 579480,
    "text": ">> Interesting. So, you can have people"
  },
  {
    "startMs": 580560,
    "text": "working for a really long time on"
  },
  {
    "startMs": 582000,
    "text": "project. When you say a couple of"
  },
  {
    "startMs": 583240,
    "text": "months, it can be like really deep"
  },
  {
    "startMs": 585120,
    "text": "exploration of"
  },
  {
    "startMs": 586040,
    "text": ">> Yes."
  },
  {
    "startMs": 586840,
    "text": "Yes."
  },
  {
    "startMs": 588040,
    "text": "I would say, for example, architecture"
  },
  {
    "startMs": 590160,
    "text": "might require longer time of"
  },
  {
    "startMs": 591800,
    "text": "investigation, research, experiments,"
  },
  {
    "startMs": 594320,
    "text": "even redoing the evaluations for"
  },
  {
    "startMs": 596960,
    "text": "pre-training."
  },
  {
    "startMs": 598400,
    "text": "Yes, so it might require longer time."
  },
  {
    "startMs": 600600,
    "text": ">> Very nice, yeah. And I know you're also"
  },
  {
    "startMs": 602160,
    "text": "very big on evaluation. I agree. We"
  },
  {
    "startMs": 603920,
    "text": "could talk about that. I think what One"
  },
  {
    "startMs": 605400,
    "text": "thing probably related to that is this"
  },
  {
    "startMs": 607120,
    "text": "unique specificity that M3 and your team"
  },
  {
    "startMs": 609640,
    "text": "has um around multimodality. So, not"
  },
  {
    "startMs": 612640,
    "text": "just text, but the similar can also"
  },
  {
    "startMs": 614680,
    "text": "understand image and video. And as I"
  },
  {
    "startMs": 617080,
    "text": "understand, but but please explain"
  },
  {
    "startMs": 618760,
    "text": "explain better,"
  },
  {
    "startMs": 620120,
    "text": "when you read the model card on Hugging"
  },
  {
    "startMs": 621720,
    "text": "Face, it say the model was trained from"
  },
  {
    "startMs": 623280,
    "text": "the first step as a multimodal, not just"
  },
  {
    "startMs": 625440,
    "text": "have a like user one as after solved,"
  },
  {
    "startMs": 627800,
    "text": "right? Can you tell us a little bit more"
  },
  {
    "startMs": 629480,
    "text": "about that and why you think it's"
  },
  {
    "startMs": 631040,
    "text": "important and"
  },
  {
    "startMs": 632360,
    "text": "and and why starting from the first step"
  },
  {
    "startMs": 634400,
    "text": "on multimodal training and not just just"
  },
  {
    "startMs": 636720,
    "text": "training this."
  },
  {
    "startMs": 637400,
    "text": ">> Um so, we call it native multimodality."
  },
  {
    "startMs": 640839,
    "text": "Um and so, it is somehow typical for"
  },
  {
    "startMs": 645120,
    "text": "model labs to train the multimodal,"
  },
  {
    "startMs": 648200,
    "text": "let's say, vision understanding"
  },
  {
    "startMs": 649680,
    "text": "capabilities after the text pre-training"
  },
  {
    "startMs": 652920,
    "text": "is done. Um they put adapters and then"
  },
  {
    "startMs": 655880,
    "text": "train that part."
  },
  {
    "startMs": 657560,
    "text": "But what we found out was that that"
  },
  {
    "startMs": 659680,
    "text": "would actually harm the text"
  },
  {
    "startMs": 661520,
    "text": "performance."
  },
  {
    "startMs": 663240,
    "text": "And the vision vision understanding"
  },
  {
    "startMs": 665720,
    "text": "performance wouldn't converge that well"
  },
  {
    "startMs": 667520,
    "text": "because the model is kind of converges"
  },
  {
    "startMs": 669360,
    "text": "towards the model the text"
  },
  {
    "startMs": 670960,
    "text": "understanding. Um and it's just not the"
  },
  {
    "startMs": 674040,
    "text": "most optimal. And also not the most"
  },
  {
    "startMs": 677040,
    "text": "scalable, if you think about it. We want"
  },
  {
    "startMs": 678880,
    "text": "to scale the data, right?"
  },
  {
    "startMs": 680839,
    "text": "And also, we can also some labs um train"
  },
  {
    "startMs": 684520,
    "text": "this capability from halfway through the"
  },
  {
    "startMs": 687160,
    "text": "pre-training. For example, continued"
  },
  {
    "startMs": 689000,
    "text": "pre-training. But what we found that"
  },
  {
    "startMs": 691720,
    "text": "this would be very, you know, uh recipe"
  },
  {
    "startMs": 695040,
    "text": "sensitive."
  },
  {
    "startMs": 696560,
    "text": "It is different for the recipe would be"
  },
  {
    "startMs": 698640,
    "text": "different for different architectures,"
  },
  {
    "startMs": 700800,
    "text": "different, you know, data mixtures,"
  },
  {
    "startMs": 702840,
    "text": "different learning rates."
  },
  {
    "startMs": 704600,
    "text": "It's hard to control, hard to, you know,"
  },
  {
    "startMs": 706880,
    "text": "scale to"
  },
  {
    "startMs": 709640,
    "text": "you can't really scale your experiment"
  },
  {
    "startMs": 712240,
    "text": "results and conclusions to a larger"
  },
  {
    "startMs": 714320,
    "text": "model."
  },
  {
    "startMs": 715560,
    "text": "And so"
  },
  {
    "startMs": 717320,
    "text": "you know, what we thought was why not"
  },
  {
    "startMs": 719680,
    "text": "just training from the very first step?"
  },
  {
    "startMs": 721960,
    "text": "That comes to the most natural. We know"
  },
  {
    "startMs": 724640,
    "text": "that a lot of labs run into problems"
  },
  {
    "startMs": 726960,
    "text": "doing that."
  },
  {
    "startMs": 728400,
    "text": "The model would collapse after a couple"
  },
  {
    "startMs": 730920,
    "text": "of steps of training, you know, both"
  },
  {
    "startMs": 733600,
    "text": "text and vision understanding, but we"
  },
  {
    "startMs": 736280,
    "text": "managed to solve that problem."
  },
  {
    "startMs": 738880,
    "text": "We did a lot of work on"
  },
  {
    "startMs": 741360,
    "text": "VIT and we did a lot of work on the data"
  },
  {
    "startMs": 744360,
    "text": "that we actually training. For example,"
  },
  {
    "startMs": 747520,
    "text": "we do interleave the data, what we call"
  },
  {
    "startMs": 749520,
    "text": "interleave the data."
  },
  {
    "startMs": 751200,
    "text": "It's actually natural data, but we keep"
  },
  {
    "startMs": 753920,
    "text": "the"
  },
  {
    "startMs": 755400,
    "text": "images and videos in instead of masking"
  },
  {
    "startMs": 757920,
    "text": "it out and we do some pretty"
  },
  {
    "startMs": 760760,
    "text": "good cleaning and masking on the data"
  },
  {
    "startMs": 763040,
    "text": "and we do very good reward modeling so"
  },
  {
    "startMs": 765760,
    "text": "that we train it from the first step and"
  },
  {
    "startMs": 767960,
    "text": "scales up"
  },
  {
    "startMs": 770160,
    "text": "a lot. Yeah, it does does not collapse."
  },
  {
    "startMs": 773080,
    "text": ">> That's really impressive. Impressive."
  },
  {
    "startMs": 775240,
    "text": "Should we Should we expect much larger"
  },
  {
    "startMs": 777160,
    "text": "model in the future? So this one is"
  },
  {
    "startMs": 778880,
    "text": "still fairly small, right?"
  },
  {
    "startMs": 780160,
    "text": ">> It's It's 428 billion parameters, 23"
  },
  {
    "startMs": 783480,
    "text": "active billion."
  },
  {
    "startMs": 785640,
    "text": "Well, do you think you will go past the"
  },
  {
    "startMs": 787760,
    "text": "trillion?"
  },
  {
    "startMs": 789200,
    "text": ">> Definitely. Yeah, definitely in the"
  },
  {
    "startMs": 791560,
    "text": "future."
  },
  {
    "startMs": 793120,
    "text": "There are many tasks that wouldn't be"
  },
  {
    "startMs": 795440,
    "text": "able to the more model"
  },
  {
    "startMs": 797760,
    "text": "wouldn't be able to perform very good at"
  },
  {
    "startMs": 799760,
    "text": "with smaller parameters. We are"
  },
  {
    "startMs": 802520,
    "text": "definitely going more ambitious than"
  },
  {
    "startMs": 804880,
    "text": "this."
  },
  {
    "startMs": 805640,
    "text": ">> It's great. Looking forward. Um another"
  },
  {
    "startMs": 808720,
    "text": "interesting thing I"
  },
  {
    "startMs": 810120,
    "text": "I always been find fascinating about Min"
  },
  {
    "startMs": 812040,
    "text": "Max is how how you also have this whole"
  },
  {
    "startMs": 814040,
    "text": "range of of apps and product, right? So,"
  },
  {
    "startMs": 816640,
    "text": "I remember already So, so Min Max"
  },
  {
    "startMs": 819040,
    "text": "started to open source things on the on"
  },
  {
    "startMs": 821000,
    "text": "the hugging face platform in in January"
  },
  {
    "startMs": 823160,
    "text": "last year. So, that 18 month ago and and"
  },
  {
    "startMs": 825760,
    "text": "we were chatting a little bit about the"
  },
  {
    "startMs": 827040,
    "text": "team to understand what you were doing"
  },
  {
    "startMs": 828720,
    "text": "and I remember you So, you were already"
  },
  {
    "startMs": 830320,
    "text": "having a huge usage on some of these of"
  },
  {
    "startMs": 833520,
    "text": "some of these apps. Um can you tell us a"
  },
  {
    "startMs": 836320,
    "text": "little bit how how this started, right?"
  },
  {
    "startMs": 838000,
    "text": "So, was it basically you had a lot of"
  },
  {
    "startMs": 840280,
    "text": "apps and then you thought we have we"
  },
  {
    "startMs": 841720,
    "text": "have all this data, why not training a"
  },
  {
    "startMs": 843360,
    "text": "model and then they build up research"
  },
  {
    "startMs": 845120,
    "text": "team. How is how is the story there?"
  },
  {
    "startMs": 847800,
    "text": ">> Um our our story is modeled from the"
  },
  {
    "startMs": 851200,
    "text": "first day. So, um I believe that"
  },
  {
    "startMs": 854200,
    "text": "multi-modality model a model that can"
  },
  {
    "startMs": 856840,
    "text": "understand all visions and outputs all"
  },
  {
    "startMs": 859080,
    "text": "modalities was the first thing that our"
  },
  {
    "startMs": 862000,
    "text": "um CEO planned on the first day even"
  },
  {
    "startMs": 864200,
    "text": "before the company even started. So,"
  },
  {
    "startMs": 866280,
    "text": "that was the dream of AGI. I think that"
  },
  {
    "startMs": 868040,
    "text": "was very very early even before ChatGPT"
  },
  {
    "startMs": 870440,
    "text": "came out."
  },
  {
    "startMs": 871240,
    "text": ">> Wow."
  },
  {
    "startMs": 871720,
    "text": ">> Um yeah."
  },
  {
    "startMs": 873120,
    "text": "And then apps were something that comes"
  },
  {
    "startMs": 875400,
    "text": "along because you have some model"
  },
  {
    "startMs": 877040,
    "text": "capabilities you want people to"
  },
  {
    "startMs": 878800,
    "text": "experience it well."
  },
  {
    "startMs": 880960,
    "text": "Not many people can use it with API,"
  },
  {
    "startMs": 883280,
    "text": "right? We can't expect everyone to"
  },
  {
    "startMs": 884839,
    "text": "experience with API. So, we need good um"
  },
  {
    "startMs": 888200,
    "text": "user interaction,"
  },
  {
    "startMs": 890040,
    "text": "you know, interfaces, good apps, good"
  },
  {
    "startMs": 892640,
    "text": "scenarios that people can"
  },
  {
    "startMs": 895240,
    "text": "can you know, experience experience"
  },
  {
    "startMs": 897160,
    "text": "model with. I think actually those apps"
  },
  {
    "startMs": 900240,
    "text": "covered more than 300 million people"
  },
  {
    "startMs": 903400,
    "text": "around 200 countries globally."
  },
  {
    "startMs": 907160,
    "text": "And I think over a million companies as"
  },
  {
    "startMs": 909880,
    "text": "well."
  },
  {
    "startMs": 910200,
    "text": ">> Yeah, this was a mind-blowing when I"
  },
  {
    "startMs": 911920,
    "text": "heard about the the size and we we don't"
  },
  {
    "startMs": 913880,
    "text": "often realize the size of of of this"
  },
  {
    "startMs": 916400,
    "text": "type of usage already. And and that I"
  },
  {
    "startMs": 918880,
    "text": "kind of brings me to the question around"
  },
  {
    "startMs": 920960,
    "text": "um"
  },
  {
    "startMs": 922080,
    "text": "open source business model, and and all"
  },
  {
    "startMs": 924480,
    "text": "of that, which is the"
  },
  {
    "startMs": 926000,
    "text": "always existing question, which is right"
  },
  {
    "startMs": 928680,
    "text": "now it's nice to open source model, but"
  },
  {
    "startMs": 930360,
    "text": "you you also need to have some revenue"
  },
  {
    "startMs": 932880,
    "text": "stream, right?"
  },
  {
    "startMs": 934360,
    "text": "So, I guess M3 is something you you"
  },
  {
    "startMs": 936720,
    "text": "decided, for instance, to be for free,"
  },
  {
    "startMs": 938560,
    "text": "and I'm I think it's it's it's great for"
  },
  {
    "startMs": 940480,
    "text": "the world. Um how do you see this? Do"
  },
  {
    "startMs": 943200,
    "text": "you also have some specific models you"
  },
  {
    "startMs": 944840,
    "text": "use for the app? Do you think about Do"
  },
  {
    "startMs": 947160,
    "text": "you think in the future you'll keep It's"
  },
  {
    "startMs": 949200,
    "text": "probably hard to say for sure, but do"
  },
  {
    "startMs": 950760,
    "text": "you think you'll keep open sourcing"
  },
  {
    "startMs": 952040,
    "text": "models? How is the culture around open"
  },
  {
    "startMs": 954040,
    "text": "sourcing right now?"
  },
  {
    "startMs": 955680,
    "text": ">> Personally, and also for the model"
  },
  {
    "startMs": 957520,
    "text": "research team, we always hope to open"
  },
  {
    "startMs": 960160,
    "text": "source the models. Um that is our plan."
  },
  {
    "startMs": 963120,
    "text": "Because we"
  },
  {
    "startMs": 964720,
    "text": "really see how the open source community"
  },
  {
    "startMs": 966960,
    "text": "together can help the model build"
  },
  {
    "startMs": 969120,
    "text": "better. For example, we receive a lot of"
  },
  {
    "startMs": 972440,
    "text": "um feedbacks on the model performance"
  },
  {
    "startMs": 974080,
    "text": "from the great community, and we receive"
  },
  {
    "startMs": 976400,
    "text": "PRs on uh wh- whatever we open source."
  },
  {
    "startMs": 980520,
    "text": "And those are very very valuable and"
  },
  {
    "startMs": 982040,
    "text": "come comes to our later versions. So,"
  },
  {
    "startMs": 984640,
    "text": "definitely open sourcing is great."
  },
  {
    "startMs": 987760,
    "text": ">> That's great. And actually, do you have"
  },
  {
    "startMs": 989400,
    "text": "some ask for the audience, people who"
  },
  {
    "startMs": 991240,
    "text": "are using uh M3 or MiniMax? Is there"
  },
  {
    "startMs": 993960,
    "text": "something you would love them to send"
  },
  {
    "startMs": 996400,
    "text": "back to you as feedback? Do you Do you,"
  },
  {
    "startMs": 998960,
    "text": "for instance, do you read when people"
  },
  {
    "startMs": 1000480,
    "text": "try to modify the models or play around,"
  },
  {
    "startMs": 1002960,
    "text": "you know, tweaks? Or what is the best"
  },
  {
    "startMs": 1005280,
    "text": "thing you you think you can take from"
  },
  {
    "startMs": 1007000,
    "text": "the community uh for future models, for"
  },
  {
    "startMs": 1010120,
    "text": "instance?"
  },
  {
    "startMs": 1010880,
    "text": ">> Mhm."
  },
  {
    "startMs": 1011960,
    "text": "I would say whatever um issues that"
  },
  {
    "startMs": 1014800,
    "text": "people are running into, especially with"
  },
  {
    "startMs": 1016520,
    "text": "multimodality, right? This is the first"
  },
  {
    "startMs": 1018240,
    "text": "time that we're combining it together."
  },
  {
    "startMs": 1020120,
    "text": "We are definitely going more ambitious"
  },
  {
    "startMs": 1022160,
    "text": "on that in the future. It might have"
  },
  {
    "startMs": 1024040,
    "text": "some flaws right now, but we are"
  },
  {
    "startMs": 1025480,
    "text": "improving on that. So, whatever that's"
  },
  {
    "startMs": 1028199,
    "text": "uh feedback that model is not good doing"
  },
  {
    "startMs": 1030120,
    "text": "that great, we will definitely improve"
  },
  {
    "startMs": 1032000,
    "text": "that in future versions. And also,"
  },
  {
    "startMs": 1034520,
    "text": "whatever features that uh people want."
  },
  {
    "startMs": 1037680,
    "text": "Say,"
  },
  {
    "startMs": 1038880,
    "text": "you know, for example,"
  },
  {
    "startMs": 1040600,
    "text": "thinking effort. Right? Some people ask"
  },
  {
    "startMs": 1042439,
    "text": "for that. Um"
  },
  {
    "startMs": 1044400,
    "text": "like everyone can ask, and we will try"
  },
  {
    "startMs": 1046720,
    "text": "to accomplish that in the future models."
  },
  {
    "startMs": 1049240,
    "text": ">> Yeah. Do you see a lot of usage right"
  },
  {
    "startMs": 1050760,
    "text": "now already in multimodality in terms of"
  },
  {
    "startMs": 1053120,
    "text": "coding agents? I feel like it's it's a"
  },
  {
    "startMs": 1055240,
    "text": "little bit un- underexplored."
  },
  {
    "startMs": 1058640,
    "text": ">> It is It is, but um it can actually"
  },
  {
    "startMs": 1062320,
    "text": "unlocks a lot of capabilities and a lot"
  },
  {
    "startMs": 1065000,
    "text": "of"
  },
  {
    "startMs": 1065760,
    "text": "uh agent applications."
  },
  {
    "startMs": 1067800,
    "text": "Say that for example, you want a model"
  },
  {
    "startMs": 1069520,
    "text": "to read a PP- PowerPoint, uh or to read"
  },
  {
    "startMs": 1072360,
    "text": "some report that is not very structured."
  },
  {
    "startMs": 1075840,
    "text": "Um and you want it to understand a very"
  },
  {
    "startMs": 1078000,
    "text": "long video, say that you dump in a"
  },
  {
    "startMs": 1080440,
    "text": "long-playing video, and then you want"
  },
  {
    "startMs": 1082440,
    "text": "the model to act uh using some tools uh"
  },
  {
    "startMs": 1085920,
    "text": "after understanding it. And it unlocks a"
  },
  {
    "startMs": 1088760,
    "text": "wide variety of um agent use cases."
  },
  {
    "startMs": 1092640,
    "text": ">> So, like the agent could finally watch"
  },
  {
    "startMs": 1094400,
    "text": "my YouTube tutorial and understand how"
  },
  {
    "startMs": 1096320,
    "text": "to use my coding tools, how I described"
  },
  {
    "startMs": 1098680,
    "text": "it? Is it something like that?"
  },
  {
    "startMs": 1100520,
    "text": ">> Uh"
  },
  {
    "startMs": 1101200,
    "text": ">> Could the agent finally watch YouTube"
  },
  {
    "startMs": 1103080,
    "text": "tutorials and understand things from"
  },
  {
    "startMs": 1104840,
    "text": "them?"
  },
  {
    "startMs": 1105200,
    "text": ">> Yeah, yeah, yeah."
  },
  {
    "startMs": 1106520,
    "text": "I think so."
  },
  {
    "startMs": 1107400,
    "text": ">> Do you use a lot of uh agent coding"
  },
  {
    "startMs": 1109720,
    "text": "tools internally? Is it like I mean,"
  },
  {
    "startMs": 1111640,
    "text": "coding for sure, but like is it also"
  },
  {
    "startMs": 1113800,
    "text": "already in terms of research? Is it"
  },
  {
    "startMs": 1115920,
    "text": "automated part or not? How How does this"
  },
  {
    "startMs": 1118480,
    "text": "uh"
  },
  {
    "startMs": 1118720,
    "text": ">> Yes."
  },
  {
    "startMs": 1119280,
    "text": ">> work?"
  },
  {
    "startMs": 1119640,
    "text": ">> Um we have our own research harnesses."
  },
  {
    "startMs": 1122840,
    "text": "Um we build our own research harnesses"
  },
  {
    "startMs": 1124760,
    "text": "that automate our workflows. I would say"
  },
  {
    "startMs": 1127200,
    "text": "a lot of our workflows are automated."
  },
  {
    "startMs": 1129800,
    "text": "You can see how um the latest frontier"
  },
  {
    "startMs": 1132880,
    "text": "models all pursues capability less"
  },
  {
    "startMs": 1136040,
    "text": "kernel optimization, right? Like let the"
  },
  {
    "startMs": 1138880,
    "text": "model post string other models."
  },
  {
    "startMs": 1141400,
    "text": "Um let the model build data."
  },
  {
    "startMs": 1143720,
    "text": "Auto data, stuff like that. Um you can"
  },
  {
    "startMs": 1146280,
    "text": "see how more and more models are capable"
  },
  {
    "startMs": 1149160,
    "text": "of doing those, including M3. Actually,"
  },
  {
    "startMs": 1151400,
    "text": "we were very good at those cases, longer"
  },
  {
    "startMs": 1154000,
    "text": "horizons and coronal organizations."
  },
  {
    "startMs": 1156680,
    "text": "Um and so we can use that model"
  },
  {
    "startMs": 1159520,
    "text": "capability, harness it together, and"
  },
  {
    "startMs": 1162360,
    "text": "help with our um daily routine, and make"
  },
  {
    "startMs": 1165320,
    "text": "our iterations even faster."
  },
  {
    "startMs": 1167320,
    "text": ">> Is M3 building M4 already?"
  },
  {
    "startMs": 1169640,
    "text": ">> Um building M3.1."
  },
  {
    "startMs": 1172120,
    "text": ">> M3.1, okay."
  },
  {
    "startMs": 1173773,
    "text": ">> [laughter]"
  },
  {
    "startMs": 1174000,
    "text": ">> Let's hit the gym."
  },
  {
    "startMs": 1174880,
    "text": ">> Already."
  },
  {
    "startMs": 1176000,
    "text": ">> Um I I would love to finish on what what"
  },
  {
    "startMs": 1178440,
    "text": "you find exciting in the coming month,"
  },
  {
    "startMs": 1181160,
    "text": "what what do you think it it can be Asia"
  },
  {
    "startMs": 1183000,
    "text": "in terms of feature or or things you"
  },
  {
    "startMs": 1185840,
    "text": "want to see happening in in AI or more"
  },
  {
    "startMs": 1188880,
    "text": "generally in terms of whatever whatever"
  },
  {
    "startMs": 1191160,
    "text": "really is top of your mind I would say"
  },
  {
    "startMs": 1192800,
    "text": "and it's going to"
  },
  {
    "startMs": 1194200,
    "text": "happen."
  },
  {
    "startMs": 1195480,
    "text": ">> A lot of things are very exciting. Um"
  },
  {
    "startMs": 1198600,
    "text": "but what I recently find the most"
  },
  {
    "startMs": 1200800,
    "text": "exciting would be a multi-agents that I"
  },
  {
    "startMs": 1203840,
    "text": "think a lot of AI applications are"
  },
  {
    "startMs": 1205760,
    "text": "using, model routing, multi-agents um"
  },
  {
    "startMs": 1209200,
    "text": "that allows even more capabilities, even"
  },
  {
    "startMs": 1212560,
    "text": "more complex tasks, and also it tells us"
  },
  {
    "startMs": 1216440,
    "text": "what the models are capable and not"
  },
  {
    "startMs": 1218600,
    "text": "capable of, and you can, you know,"
  },
  {
    "startMs": 1221920,
    "text": "do a lot of things with that. It's"
  },
  {
    "startMs": 1223120,
    "text": "pretty exciting."
  },
  {
    "startMs": 1224760,
    "text": ">> Thanks a lot, Alif. Pleasure to have"
  },
  {
    "startMs": 1226640,
    "text": "you."
  },
  {
    "startMs": 1227400,
    "text": ">> Thanks for having me."
  },
  {
    "startMs": 1228840,
    "text": ">> Thanks, everyone."
  },
  {
    "startMs": 1230104,
    "text": ">> [applause]"
  }
]