Robotics: why now?

Talk transcript

1. 0:00

   [upbeat music] So good morning.

2. 0:16

   Uh, thanks for being here with us. My name is Quan. This is Tobi. Um, our mission is to make a model that can control any robot to do any task.

3. 0:26

   Now, this is not something that's ready today. Uh, we believe that there are multiple scientific, uh, breakthrough that needs to happen for us to get there. And so we're very open.

4. 0:35

   We publish our research, we open source our model, and we re- talk very publicly about what is it that we do.

5. 0:41

   And so if you think about robotics before, and not to say that it's not useful, you know, it's incredibly impactful on the world, um, but the scenario that you often see robotic in, it's either in a very constrained environment such as, you know, a factory, very repe- repetitive motion, very structure environment.

6. 0:58

   Um, and then when you try to bring them into the real world, just kind of like semi-structure. I think this is a pretty kind of well-known video of a full-body humanoid, you know, struggling to perform a somewhat simple task.

7. 1:12

   Um, it was from some time ago. Um, if you look at robotics today, what do you see? You see,

8. 1:25

   you see kind of humanoid dancing. I know-- I don't think I can do that dance move. Um, [laughing]

9. 1:32

   um, I'll try. And so, uh, you see, you see kind of very complex physical motion that robots are capable of. Um, and you also see, uh, this video on the right, which is what we released late last year of a robot operating with kind of somewhat semi-structure objects.

10. 1:48

    You know, this is clothing that just came out of a dryer that ran before, so it's very hard to control, you know, the exact initial scene of, you know, how the shirt should be.

11. 1:57

    You know, it takes out all of the shirts, manage to put them in a basket, and in the full video, you know, it goes on to, you know, bring it to a table and then complete to fold them.

12. 2:06

    So what really changed? You know, the obvious answer is that there is this AI wave that we're all riding on. Robotic benefits a lot from kind of general AI development.

13. 2:15

    Uh, but there is also this vision-language-action model that, uh, we're pioneering. And so what are they? I'll pass it off to Tobi.

14. 2:24

    Cool. Yeah. So vision-language-action models, what are they? As Quan asked. Um, well, you probably all know by now what a multimodal LLM, or we often refer to it as a vision-language model, or VLM, is by now.

15. 2:37

    Um, essentially, a VLM generally takes text and images as input, so you have some sort of prompt for the model. Um, and then it embeds them and pass them to a transformer model to autoregressively produce an answer in text, right?

16. 2:50

    So you interact with these models probably, uh, as I do every day now. Um, so and then a VLA is essentially an adaptation of a VLM for the purpose of robotics.

17. 3:00

    The model additionally gets inputs describing the robot state, so the-- such as its joints positions, and, um, instead of asking questions about what's going on in the scene, we ask the model to produce actions to control the robot directly, right?

18. 3:17

    So if VLMs and VLAs are so similar in kind of principle, then what are the additional engineering challenges that we face when we try to train such a VLA?

19. 3:27

    Um, to understand this, I wanna contrast a little bit pipelines for VLM training to what we have to do when we train a VLA for robotics. So when you as kind of like a downstream customer of these like big models that have been trained want to use a VLM, you generally kind of take, um, a model, and

20. 3:44

    you can source data from the web, and maybe you have a little bit of extra data that, that you have for your specific task that you're interested in that you supplement.

21. 3:52

    Um, and then you probably train and use an off-the-shelf model, and you fine-tune it on a large cluster somewhere in the cloud. And then finally, you can use well-established libraries for inference and deployment in the cloud, right?

22. 4:03

    So all of this is like probably bread and butter for, for a bunch of you.

23. 4:08

    Now, in contrast to that, um, if we wanna do VLA training,

24. 4:12

    um, you want to train a model to exhibit kind of dexterous frontier-level behavior, then it's an open question what the analog of data source for the web actually is, right?

25. 4:24

    And we believe that this is kind of a trillion-dollar question for the industry in some sense, and it's an entirely open research question, as Quan already, uh, said.

26. 4:33

    And then secondly, while we can use, um, VLM backbones, so we can reuse some pretrained models, we typically have to adapt them and adapt the model architectures in order to allow for models that can control robots at so- somehow high frequency controls that we need to actually make progress.

27. 4:50

    And then finally, we-- there's also no standard solution for deploying large robot policies in multiple locations on premise, on device for robots, right? So this just doesn't exist. I won't really have time to talk about the third thing in detail here, but I wanna dive into a little bit about data and model training for robotics that we

28. 5:09

    do at PAI. So first, how can we design a data engine that enables robust, highly dexterous policies for difficult tasks with robots? At PAI, we kind of believe that there currently is no standard solution for this at all that would enable us to do this, and so we're building essentially a data engine from the ground up from

29. 5:27

    zero. We are designing this pipeline to get us very quickly to some sort of impressive capability, and I hope at the end of the talk you agree that it looks somewhat impressive, but also to enable significant scaling in the next few years.

30. 5:39

    Um, so we've seen firsthand that operationalizing this pipeline is actually kind of one of the main ingredients. It's probably more than fifty percent of the work is getting the data pipeline right, getting the right data, getting it to be high quality.

31. 5:51

    So how does it work? Well, we typically start from a set of ever-expanding tasks that we pick to test what's possible in the moment, right? So these are tasks such as folding clothes, buying groceries, many other tasks that we're interested in.

32. 6:04

    And we then have human operators control our robots using a custom runtime and teleoperation system.

33. 6:10

    You can see the system in this video here. Uh, so what happens there is there's human operators, and they are controlling what we call leader arms, where they basically trace out motions with their arms, with, uh, robot arms kind of strapped to their arms, and the motion gets transferred via software to the actual robot end effectors, right?

34. 6:28

    And this way you can demonstrate fairly intricate, highly dexterous tasks and collect that data for training afterwards. Um, and then afterwards, we have a bunch of, uh, facilities to do kind of, uh, tracking of metrics of what's going on at the moment.

35. 6:47

    I think there's another slide here. Yep. Uh, so we basically schedule these data collection, uh, sessions all the time, and each dash in this, uh, dashboard here show us from this week, this is I think from Th- from Tuesday, an operator doing a specific episode for a specific task, right?

36. 7:03

    So we collect a lot of this data all around the clock, basically.

37. 7:07

    And then we, uh, annotate that data in the cloud. We, we, we serve it, uh, in, in big buckets there and can filter it back down based on annotations and use it for model training.

38. 7:19

    After we've trained, we then get policies that can actually solve the denim inserted tasks autonomously. So here in this video, you see a model that is Pi Zero. So this is the model that Quan referred to earlier already that we released, uh, late last year.

39. 7:32

    And as you can see, it can kind of do these like f- fairly impressive, highly dexterous tasks such as, as shirt folding.

40. 7:40

    Okay, so how far have we gotten by following this approach? Well, when we started, the biggest publicly available dataset was the Open Cross Embodiment dataset, which contained about three thousand eight hundred hours of data, largely from static scenes in different robot labs around the world.

41. 7:56

    After kind of running this data pattern that I've described for six months, we had collected about ten thousand hours of successful episodes as a successful data, uh, in tens of environments covering hundreds of different tasks, enabling, for example, the kind of shirt folding policies you, you saw before.

42. 8:11

    Oops. Um, cool. Is that video actually playing? Why is that video not playing?

43. 8:20

    Can I get it to play? It can't.

44. 8:26

    Let's see what we do with the next video. Um, and then after another six months, oh, this plays, uh, we have collected many more hours of data in static scenes, but crucially have also started to collect significant amounts of data using mobile manipulation setups such as the ones you, you see here.

45. 8:42

    So the data now spans many, many more tasks, and importantly has massively grown in diversity, covering hundreds of different scenes and, and environments. And as you can imagine, this scale and diversity enables new leaps, as you can kind of see here by this policy that is r- already running autonomously, and I'll go into detail a little bit

46. 8:59

    how this works, but also brings lots of additional engineering challenges.

47. 9:04

    So now that we have kind of described how we get the data, then the question is what kind of capabilities can we elicit with this data in VLAs, right?

48. 9:13

    And to understand where we are today, I think it's kind of useful to draw an analogy between industry trends for VLMs and VLAs, which have been kind of ongoing in the last three years.

49. 9:23

    So first, for multimodal LLMs or VLMs, we have seen a constant stream of improvements over the last three years, kind of starting from initial conversational agents that you've all interacted with, all the way to the RL-trained multimodal reasoning models and coding assistants that we all have today, right?

50. 9:40

    And we all use. Um, for VLAs, they follow a similar but time-lagged trajectory, basically. So initial VLAs such as, uh, RT-2, for example, done by some of my colleagues that are now at PAI, emerged in twenty twenty-three after LLMs had already been enhanced with, with vision encoders.

51. 9:58

    And in fact, actually some of the earliest multimodal LLMs were trained for robotics purposes by some of my colleagues, Danny and others, that are now working with us at PAI.

52. 10:09

    These were kind of impressive as first proofs of concept and showed some generalization capabilities, so you could kind of like ask them to pick different objects in the same kind of scene that, that kind of is in the training data.

53. 10:20

    But they are generally held back by a lack of, of available robot data. But, you know, nonetheless, they sparked this big explosion of interest in the field, which is probably why you're here today.

54. 10:30

    Um, then in the mid-twenty, so mid-twenty twenty-four towards end twenty twenty-four, the first kind of really dexterous multi-robot VLAs, uh, appeared, right? And the industry at large has now produced several of those.

55. 10:42

    Uh, there are models, for example, such as Gemini for robotics or NVIDIA's Groot models that I think you'll hear a little bit about later as well. And our entry in this category was Pi Zero, which we believe is kind of perhaps the most dexterous, uh, multi-robot model that you can, you can use, and in fact is open

56. 11:00

    source as well. So these models generally adjust architectures to produce actions via diffusion to enable kind of very fast generation at, uh, high frequencies you need for, for robot control.

57. 11:12

    So if that's where we were then, what's next? Where are we now, right? Um,

58. 11:18

    so for us, the next leap was to study just how exactly model capabilities change when we increase data collection diversity, and this led us to develop Pi05, which is basically a VLA with open world generalization.

59. 11:32

    And I wanna talk a little bit more about this. Um, so what does this look like? In general, we have expanded massively, as I kinda said, the data we take in during training.

60. 11:40

    It now consists on both static and mobile robot data on the right here, as well as an extended set of multimodal VLM data, such as data from the web, object detection data, and general language annotations for the robot data that we've collected.

61. 11:54

    So we have a huge annotation pipeline as well, and this is what's on the left here. We then feed this data into a specially designed VLM, which starts from a pretrained transformer model and is expanded with an action expert transformer to the right.

62. 12:08

    This VLM part, so the big backbone of it, is trained to give predictions for both general questions about the scene now, but also to subdivide high-level requests that a human might have, uh, for the model, such as, for example, clean my bedroom, right?

63. 12:22

    So it is trained to subdivide these tasks into, um, subtasks such as pick up a pillow in the case of cleaning the bedroom. And at the same time, the action expert transformer can attend to the internals of the large VLM and can run at much higher rate and produces the actual continuous output actions via diffusion flow matching

64. 12:40

    objective, basically. Cool. I hope this video plays.

65. 12:47

    Can we get it to play? Yes. So training this architecture on all our data then leads us to a VLA that can perform difficult long-horizon tasks of up to ten minutes in, in each episode.

66. 13:00

    And this is, I think, much, much longer than what we've seen before, right? And it can do this in entirely unseen homes, showcasing perhaps the first sign, like the first true sign that broad generalization can emerge from VLA training.

67. 13:13

    So in this video here, you see my, uh, colleague Chelsea prompting the model in, uh, in an, yeah, you know, uh, an entirely h- new home that is not in the training data to essentially do p-perform multiple cleaning tasks such as cleaning a surface here, uh, in a kitchen.

68. 13:33

    Cool. And to understand this a-ability to generalize a little bit further, we tested how it emerges by training PIO 5 on a fixed number of data, but varying the number of, uh, homes from which data was introduced during training.

69. 13:49

    Basically, then, uh, testing it in a held-out location, right? And as you can see, with increasing amounts of locations added, so this is the yellow curve here, performance generally increases in the test scene, as you would expect, until it surprisingly perhaps matches and even slightly surpasses training with the held-out scene specifically, right?

70. 14:07

    So this was a very cool result for us because we saw that we could kind of expand with more and more training data collections in different homes the capabilities of the model in new environments, which I think is, is pretty cool.

71. 14:23

    And then here we see the same model performing a bedroom cleanup task in a new home. And in this specific case, Laura, a colleague of mine, uh, is prompting a model only to basically clean the bedroom, right?

72. 14:35

    And you see here the power of this, these capabilities of subdividing into several tasks such as, you know, throwing trash in the bin and then, uh, cleaning, uh, making a bed basically.

73. 14:45

    And the-- as you can see at the bottom with the timer, you know, this policy is autonomously collecting, uh, uh, controlling this robot for multiple minutes at a time, basically.

74. 14:57

    Cool. And with that, uh, I give it back to Quan to talk to you a little bit about partnerships.

75. 15:03

    Thank you, Tobi. So what you're seeing is a robot that we have never seen in person. Um, we mean the team at PAI, uh, will never have access to it.

76. 15:16

    Um, this robot is running very, very far away from where our office is. Um, it's performing a somewhat interesting task of making a cup of coffee end to end.

77. 15:27

    Um, and it works pretty well. We didn't need to kind of iterate many times to, to, to be able to produce this video. Now, why is this important? Um, when you think about robotic, you know, oftentimes you would think about hardware, you would think about kind of real deployment challenges.

78. 15:45

    Now, those are important, uh, but it's our belief that actually one of the main bottleneck is software and just model intelligence. And if you think about what, if successful, would scale with maximum velocity in the sense that, you know, suddenly next year you have thousand and millions of robot deploy, uh, it-it's really about demonstrating the hypothesis that

79. 16:06

    our model can run across many different hardware platform out there, um, without us having to invest significant time and effort into trying to work with that hardware platform. So I think the demonstration that we've never touched this robot before, we don't know how it works internally, and yet our model can control that robot to perform a fairly

80. 16:26

    interesting task, um, is a piece of evidence in that direction. Uh, we have many more evidence here. Um, and so that's why it's important. Um, we are also very open when we work with other company because we believe that, you know, the problem is far from being solved.

81. 16:42

    So when we work with this company, for example, that you can ask how do we run inference, we literally sent them the model checkpoint for them to run like inference with.

82. 16:51

    Um, and you know, we, we have very low level technical discussion with them. This is to say that, you know, if you think there is a company that we should be talking to, um, please let us know.

83. 17:01

    You can let me and Tobi know in person, and also, you know, just shoot us an email. Um, yep.

84. 17:09

    And if you ask, you know, what is our biggest bottleneck right now? Uh, because we're after this mission of building a model that can work on any robot to perform any task, um, it's a scientific problem, engineering problem, operation problem that are far from being solved.

85. 17:26

    And so our biggest bottleneck really is we need the best people in the world in this area to help us accelerate progress. Um, and so for a research organization, you know, any role that you might think we might need, we're hiring for it.

86. 17:38

    Even if there is a role that you think you're exceptional at that, you know, we don't have on our website, please feel free to also let us know that, you know, you should really be hiring for this role and happy to have a conversation with you about it.

87. 17:49

    Um, again, you can talk to me and Tobi in person about, you know, what is our hiring needs right now, but you can also apply online and shoot us a DM on Twitter.

88. 17:58

    Uh, thank you for listening. [clapping] [upbeat music]