Minimax M2

Talk transcript

1. 0:00

   [on-hold music] Hi.

2. 0:21

   Hi, everyone. Um, I'm Olive. It's my great honor here today to present on our new model, MiniMax M2. Um, I actually lived in New York City for six years, so it feels great to come back, um, but with a different role.

3. 0:35

   Um, I currently study reinforcement learning and model evaluation at MiniMax. Um, let me just get a quick sense of the room. Who here has heard or have tried of MiniMax before?

4. 0:47

   Oh, a couple of there. Yeah. Not everybody, but I, I guess, yeah. But here's the value, right? Of me standing here today. Um, so we are a global company that works on both foundation models and applications.

5. 1:02

   We develop multi mo- m- modality models, including text, um, vision-language models, our video generation model, Hailuo, and speech generation, music generation stuff. And we also have, um, many applications, including agents and stuff, um, in-house.

6. 1:20

   So that, that's the specific thing that's different from the other labs for other companies. So we both develop foundation models, um, and applications. So we have research and developers sitting, uh, sitting side by side working on things.

7. 1:36

   Um, so our difference would be that we have first-hand experience from, uh, uh, our, um, in-house developers into developing models that developers would really need in the community. And here, I want to introduce our MiniMax M2, um, which is an open-weight model, very small, with only 10 billion active parameters, um,

8. 2:01

   that was designed specifically for coding, uh, workplace agentic tasks. It's very cost efficient.

9. 2:09

   Um, let me just go over the benchmark performance 'cause people care about it. So, uh, we, uh, rank very top in both, um, intelligence benchmarks and also agentic benchmarks.

10. 2:23

    Uh, we-- I think we're on the top of the open source models. But then numbers don't tell everything because sometimes you get those super high number models, you plug into them, um, into your environment, and they suck, right?

11. 2:38

    So we really care about the dynamics in the community. And in our first week, we had the most downloads,

12. 2:46

    and also we climbed up to top three token usage on OpenRouter. So we're very glad that people in the community are really loving our model, um, into their development cycle.

13. 2:58

    So today, what I wanna share is how we actually shape these main model characteristics that made M2 so good in your coding experience. And I'm gonna present to you, um, the training be-behind it that supports each one of them, from coding experience, to long-horizon state tracking tasks, um, to robust generalization

14. 3:23

    to different scaffolds, to multi-agent sca- uh, scalability.

15. 3:28

    So first, let's talk about code experience, which we sca-- uh, which we supported with, um, scaled environments and scaled experts.

16. 3:38

    So, um, developers need a model that can actually work in the language they use and across the workflow that they deal with every day. So which means that we need to utilize the real data from, from the internet and then, um, scale the number of environments so that the model, when during training, for example, during reinforcement learning,

17. 4:00

    it can actually, um, reacts to the, uh, environment, it can actually target verifiable coding goals and to learn from it. So that's why we scaled both the number, uh, of environments and also our, um, infrastructure so that we can perform those training very efficiently.

18. 4:20

    So, um, with data construction and reinforcement learning, we were able to train the model so that it's very strong, um, it's full stack multilingual.

19. 4:33

    And what I wanna mention here is that besides scaling environment that everybody talks about, we actually s-scale something called expert developers, um, as reward models. So as I mentioned before, uh, we have a ton of, um, super expert developers in-house that could give us feedback to our model's performance.

20. 4:54

    So they participated closely into the model development and training cycle, including problem definition, for example, um, bugs, bug fixing, for example, um, repo refactoring and stuff like that. And also, they identify the model behaviors that developers enjoy, and they i-identify what's reliable and, uh, what developers would trust.

21. 5:18

    And they give precise reward and evaluation to the model's behaviors to the final, um, deliverables so that, um, it is a model that developers really want to work with and that can add efficiency to the developers.

22. 5:34

    So with that, we were able to lead in many, um, languages in real use.

23. 5:40

    And the second characteristic that MiniMax M2 has is it, it, it performs good in those long-horizon tasks, uh, those long tasks that require interacting with complex environments, that requiring, um, using multiple tools with reasoning.

24. 5:59

    And we supported that with the interleaved thinking pattern, um, and reinforcement learning.

25. 6:07

    So what is interleaved thinking? Um, so with a normal reasoning model that can use tools, it, it normally works like this. You have the tools information given to it, you have the system prompts, um, you have user prompts, and then the model would think, and then it calls tools.

26. 6:26

    It can be a couple of tools at the same time. And then they get the tool response from the environment, and then it performs a final thinking and deliver a final content.

27. 6:37

    But, but here's the truth, right? In real world, the environments are often noisy and dynamic. You can't really perform this one test just by once. You can get, um, tool errors, for example.

28. 6:50

    You can get, um, unexpected results from the environment and stuff like that. So, um, what we did is that we imagine how humans interact with the world. We, we, we look at something, we get feedbacks, and then we think about it.

29. 7:06

    We think if the feedback is good or not, and then we make other actions, make other decisions, and that's why we did the same thing with our M2 model.

30. 7:15

    So if we look at this, um, chart over a diagram on the right, so instead of just stopping, um, after one round of t- tool calling, it actually thinks again and reacts to the, uh, reacts to the environments to see if the information is enough for it to, uh, get what, what it wants.

31. 7:38

    So basically, we call the interleaved thinking, or people call it interleaved thinking because the interleaved thinking with tool calling, um, a couple of time, it can be, you know, uh, tens to 100, um, turns of tool calling within just one user interaction turn.

32. 7:57

    So it helps, um, adaptation to environment noise. For example, uh, just like what I mentioned, the environment is, is, is not stable all the time, and then something is suboptimal, and then it can choose to use other tools or do other decisions.

33. 8:13

    It can focus on long horizon tasks, um, it can automate your workflow, um, using, for example, Gmails, Notions, um, Terminal all at the same time. You just need to, uh, maybe make one model call without minim- with minimal, um, human intervention.

34. 8:31

    It can do it all by itself. A- and here's a cool illustration on the right. Because it's New York City, I feel the vibe of, you know, trading and marketing.

35. 8:41

    Um, so you can see that there was some, um, there was some perturbations in the stock market, uh, I think last week, and then our model was able to keep it stable.

36. 8:53

    So just like I said, there's, like, environment noise, there's n- new information, there's, like, yeah, news. It looks like there, there's, like, other trading policies and stuff like that, but our model was able to, uh, to perform pretty stably in these kind of environments.

37. 9:12

    And the third characteristic is our robust, um, generalization to many agent scaffolds, which was supported by our perturbations in the data pipeline.

38. 9:24

    So we want our agent to generalize, but what is agent generalization? At first, we thought it was just tool scaling. We train the model with enough tools, various tools, kind of new tools.

39. 9:37

    We invent tools. Um, and then it would just perform good on unseen tools. Well, that was kind of the truth. It worked at first, uh, but then we soon realized that if we perturb the environment a little bit, for example, we change another agent scaffold, then it doesn't generalize.

40. 9:55

    So what is agent generalization? Well, we conclude that, um, it's adaptation to perturbations across the model's entire, uh, operational space. If we, uh, think back, what's the model's, um, operational space that we talked about, it can be tool information, it can be system prompts, it can be user prompts.

41. 10:19

    They can all, all be different. They can be the chat template. They can be the environment. They can be the tool response. So what we did is that we designed and maintained perturbation pipelines of our data so that, um, our model can actually gen- generalize to a lot of agent scaffolds.

42. 10:40

    And the fourth characteristic that I want to mention is the multi-agent sca- scalability, um, which is very possible with M2 because it's very small and cost-effective.

43. 10:54

    I have a couple of videos here. Um, this is M2 powered by our own MiniMax agent, uh, app. Uh, we actually have a QR code down side, so i- if you want, you can just scan and try it.

44. 11:07

    So it's like an agent app we, we, we developed, and here we can see different copies of M2, right? It can do research. Um, it can write the, uh, write the research results and analyze it and put it in a re- report.

45. 11:22

    It can put it in some kind of front end i- illustration, and they can work in parallel. So because it is so small, um, and so cost-effective, it can really, um, support those long-run agentic tasks and tasks that maybe, um, require some kind of parallelism.

46. 11:42

    So what's next, right? For MiniMax M2, from what I've introduced, we gathered environments, um, algorithms, data, expert values, model architecture, inference, evaluation, all this stuff to build a model, um, that was, you know, f- fast, that was, uh, intelligent, that could use tools, that generalizes.

47. 12:06

    What's next? For, um, M 2.1 and M3, we're in the future. We think of better coding, maybe memory work, context management, proactive, AI for workplace, vertical experts, and because we have those great audio generation, video generation, uh, models, maybe we can integrate them.

48. 12:30

    But all our mission is that we're committed to bring all these resources, whatever is on the screen and maybe more, uh, yeah, and values, and put them all together to develop models for, uh, the community to use.

49. 12:45

    So, um, we really need feedback from the community if possible, because we want to build this together and, you know, this is kind of a race that e- everyone needs to participate.

50. 12:57

    And then, um, we com- we are committed to share it with the community. Yeah.

51. 13:05

    And that's all the insights for today. Um, we really hope, again, we really hope you to try the model 'cause it's pretty good, and then we can contact, contact us up there.

52. 13:17

    You can try the models by scanning the QR code. Yeah, basically, that's it. Thank you all for listening. [upbeat music]