Codex and Subagents

Talk transcript

1. 0:00

   [upbeat music] Hi, everyone.

2. 0:16

   Thank you for being here. So today, we're gonna talk about Codex. My name is Katia, Katia Gil Guzman, and I'm with VB. Uh, we are both, uh, working in the Developer Experience team at OpenAI based in London, and so our role is really to help developers build and get the most out of our products, uh, including Codex.

3. 0:39

   And so today, we're gonna start with a quick Codex overview. Uh, just so we know, how many of you here are using Codex? Can you raise your hands?

4. 0:51

   Yay.

5. 0:51

   Okay, cool. So we-we're not gonna stay too long on that overview part. Uh, and then we're gonna, we're gonna do some demos. So we're gonna show you plugins and automations.

6. 1:05

   Uh, VB's gonna talk about subagents and then about the bleeding edge. So hopefully, for those who already know Codex and use it, uh, you'll learn something that, uh, you didn't know about.

7. 1:16

   And then we'll have some time at the end for Q&A. So, um, feel free to ask anything. Also, this is a, a workshop format so, you know, if you have, like, a, a pressing question, um, feel free to ask.

8. 1:29

   And I see you all have your laptops, so also feel free to kind of follow along with us. We're gonna show you how to do some things, and you can, like, try it at the same time.

9. 1:38

   And during the Q&A, uh, that's also, like, the perfect time as well to try things on, on your side. Okay. So, um, to start, just for those who don't know Codex, or even if you know it, maybe you don't know it that well, uh, Codex is our OpenAI's-- is OpenAI's software engineering agent.

10. 1:59

    So it's not just a coding agent. It's not just a, you know, uh, an agent that writes code. It can do much more than that. It can run commands.

11. 2:08

    It can, uh, run tests. It can explore codebases. It can really do everything that a software engineer would do. And so it's based on our models as a foundation.

12. 2:19

    So for example, GPT 5.3 Codex was, uh, our previous ones. We also have the Spark version, which is, like, the super fast, uh, model that, that we have. The, the state-of-the-art model right now is GPT 5.4, and we also have a mini version that, uh, came out last week.

13. 2:36

    And, you know, every time we make improvements, every time we have better models, Codex benefits from it. But it's not just the models. On top of that foundation, we have what we call a unified agent harness that will manage, uh, evaluates the agent's behavior, and that is a wrapper for tool execution, for environment setup, for

14. 3:01

    everything that, uh, can let the agent, uh, do its work and run smoothly. There's also safety, uh, safety embedded in that harness. So all of that is Codex, and then you can interact with it through different surfaces.

15. 3:17

    So you have the Codex app that we're gonna talk about in a, in a few minutes. Uh, you can also interact with it through your IDEs with the extension.

16. 3:26

    You can interact with it through the CLI and also through other surfaces like Slack. For example, at OpenAI, we all the time just, like, ping Codex in Slack and ask it to fix things or in GitHub as well.

17. 3:38

    Uh, and on top of all of that, you can also integrate it with your preferred tools so that it can really, uh, work with everything that you're already using.

18. 3:48

    So, you know, you can integrate it with Figma, with Linear, with Notion. All of that combined, uh, can let you really, like, do everyth-- can let Codex do everything that a software engineer colleague would do.

19. 4:03

    And so as I mentioned, this is based on, uh, our models, and so I'm gonna let VB tell you a little bit more about that.

20. 4:11

    All right. Um, good morning, everyone. Uh, [chuckles] so, um, as we've been talking about, um, the Codex app, the IDE, um, extension, the CLI, and so on and so forth, all of these, like, harnesses as well as all of these surfaces would not be nearly as good, uh, without the models powering them, right?

21. 4:28

    And just to sort of like, uh, take a step back, uh, back when I joined, joined OpenAI, which is not really as far, uh, back along, was in December, um, our leading model at that time was GPT 5.2.

22. 4:40

    And, um, and, um, from there, we went on to release GPT 5.2 Codex, which was a specialized, um, you know, um, Codex variant of GPT 5.2, um, wherein we, we sort of pushed how far you can, you can take the model and, um, you know, run it on long-running tasks, how far you can let it just continue

23. 5:01

    chug along. Um, and then shortly after, we followed up with GPT 5.3 Codex. Um, shortly after that, in partnership with Cerebras, we followed up with, uh, GPT 5.3 Codex Spark.

24. 5:14

    Um, and, and most recently, we had-- uh, we released GPT 5.4. And, um, um, you can already see how we're sort of, you know, pushing this whole sort of model and harness flywheel as fast as possible, trying to bring the next floor-- uh, next frontier as, um, as fast as possible to you all, right?

25. 5:33

    Um, something to note is, uh, and, and, and something that's not on the screen is, uh, at the same time, we also-- whilst we were pushing for larger models, which are, um, really good for, um, long-running tasks as well as really complex tasks and so on and so forth, um, we also released, uh, GPT 5.4 mini, uh,

26. 5:52

    and GPT 5.4 nano, which you can use for, um, short-running tasks and subagents, um, which we'll talk about in a bit. Um-

27. 6:01

    And, um, and something that we haven't really, um, emphasized on this, uh, over here is, um, is two things. One, that as we, as we sort of pushed on making these models better, we also worked quite a bit on making sure that these models, um, can be served to you as fast as possible.

28. 6:19

    What that means in principle is, um, we, um, we introduced something called WebSockets, which allow us to, um, sort of create a connection between your, um, your device as well as where the, where the API resides to be able to, uh, give you roughly about 1.75x, uh, faster tokens without, um, without really, like, paying the cost over

29. 6:41

    to you. Uh, at the same time, we also released, um, a fast mode which allows you to, on top of the 1.75x, get 2x more faster tokens. And, um, and this is something which the team is continuously sort of hammering on.

30. 6:55

    Um, there's the, there's lots more speed improvements coming, um, over there.

31. 7:01

    And so, um, to bring this all together, um, at the start of this year, we, we brought together, uh, the Codex App. How many of you have used the Codex App?

32. 7:11

    All right. That's a, that's a fair good chunk of, uh, people. Um, to be honest, back in December, um, and, and, and even before, I was a hardcore CLI user.

33. 7:21

    And, um, at some point, um, w- during the app launch, whilst I was beta testing it, dogfooding it, I, um... You know, the Codex App became really, like a really important part of my workflow.

34. 7:34

    And the reason for that is, uh, it, it brings together a really nice way to work across projects, number one, and number two, within, within the same project, work on multiple features at, at the same time.

35. 7:47

    The way you can do that is, um, you can have indivil- individual projects like you can see on the, on the left side. You can have the Codex project, ChatGPT, Sora, and so on and so forth.

36. 7:56

    But also within those, you can, you can use, um, worktrees to work on individual sort of feature requests or, or, you know, bug fixes or just that Q&A, uh, all at the same time, uh, without really interfering with individual tasks.

37. 8:12

    This is something which, um, we're quite proud of. Um, and, you know, providing a native worktree support helps you, um, do the same task and do multiple tasks at the same time without really having to context switch as much.

38. 8:25

    Um, at the same time, um, through the, through the launch, we've been trying to sort of increase, um, the net benefit you can get out of the Codex App.

39. 8:36

    Um, and some of these features have been, um, like, you know, having a better automation support. Uh, automations is, is also something we're gonna talk about just in a bit.

40. 8:46

    Um, but the short summary of automations is that you can essentially have a, um, have a rough process that you want, uh, Codex to run, let's say every day at 9:00 a.m., or, or let's say every evening.

41. 8:58

    Or let's say you want Codex to look through your calendar, uh, and, and, and create like a briefing for you. And that all is possible all within like the native, uh, Codex App setup with, um, with automations.

42. 9:11

    Um, and then of course with the, with the worktrees and, like, more native Git support, um, you, you can sort of work across projects and, um, uh, and just be able to, you know, push changes as you want with whatever Git persona you want to, uh, do it with.

43. 9:29

    Um, last but not the least, based on which surface you use Ma- uh, you use the Codex App on, um, at the start of the year, we released it just on macOS, but now we have, uh, native Windows support, which comes along with native Windows sandbox.

44. 9:43

    Is there anyone here who's using Windows today?

45. 9:47

    I'm cheering for you, man. [laughs] And, um, so, uh, for the, for the, for the one gentleman over there, [laughs] uh, we have native sandbox support, uh, in Windows. We're the first of kind.

46. 9:59

    Um, there is no other, um, you know, competing harness which supports like a native sandbox for, uh, Windows.

47. 10:07

    Cool. And so I've been talking on and on about, like, the Codex App, uh, been talking about, you know, all the models that we've been shipping, but what's, what's new in terms of all the features, um, that we've shipped?

48. 10:20

    Um, this is in, I think, if I'm not wrong, in the, in the sort of descending order. So most recently we launched plugins. Um, plugins is, um, is a way that you can bring together skills, uh, MCPs, as well as prompts and, you know, any other thing really, uh, together in one bundle and allow the model to

49. 10:43

    do more nuanced, uh, matching whilst it's building. Um, we also released recently mini-models, um, which tie in quite well with subagents, which allow you to parallelize, uh, a particular feature or bug request or Q&A, whatever it may be, um, at, um, at a faster rate, all whilst making sure that you don't pay as much cost for your,

50. 11:06

    uh, for your particular models. Um, and then we have, like, bunch of other stuff which we'll, which we're gonna talk about as we go through. Uh, some of this is, you know, how you can, uh, how Codex is so good at, like, code review, how Codex is really good at security, um, and so on.

51. 11:22

    Um, all of this, um, whilst we talk about all of this, um, I want to sort of emphasize on this fact that, um, we're at OpenAI quite lucky, uh, that the community has really embraced Codex.

52. 11:33

    Um, in fact, just, um, just last night we crossed a milestone of, uh, how, um, of crossing three million weekly active users. Uh, and this is a pretty big deal, um, uh, for us.

53. 11:43

    And we, we want to continue sort of supporting the developer community, the, the, you know, enterprises, startups building on Codex. So, um, throughout the session, if you have any questions, please feel free to throw it at both Katia and myself, um, or even afterwards or just, you know, ping us.

54. 11:59

    Um, with this, I'll pass it over to Katia.

55. 12:02

    Thank you

56. 12:04

    And yeah, the, the 3 million weekly active users thing is really, it- it's really cool to see, and it's crazy to think that it's also more than tripled since January.

57. 12:14

    So just in a few months we've seen, like, huge adoption, and, uh, yeah. And it's, it's, uh, it's really, really cool to see. Um, okay. So

58. 12:28

    plugins. Um, plugins, I don't know if you've heard about it. It's, it's quite new on Codex, uh, the, the native support for plugins. Uh, the idea of plugins, I'm gonna show you what it looks like in practice and how you can use them, is that they bundle a bunch of things together.

59. 12:45

    So like skills, apps, integrations, uh, MCP servers, and they, they bundle that into reusable workflows. And so what, uh, skills, apps, and MCP servers are, again, I'm gonna show you, but just to introduce that a little bit.

60. 13:02

    So skills are essentially, um, reusable instructions packaged for specific processes. So if you have something that you're doing quite a bit, uh, you can actually create a skill for it so that Codex knows about it.

61. 13:17

    You can give it instructions, you can give it scripts as well, resources, and all of that, uh, will save you from just repeating yourself over and over. So every time you have, like, a sort of a neat workflow that is always the same, you can package that into a skill.

62. 13:33

    You can actually ask Codex to create the skill for you as well. And then apps are connections to other services. So, uh, you know, again, uh, we- we'll see a quick demo, but the, the tools that you use every day, like Notion, Linear, all of that, uh, you can let Codex connect to it.

63. 13:53

    And MCP servers, uh, you might be familiar with this already, but, um, they, they basically expose tools for Codex, uh, to just extend its capabilities further, and it's tools from external systems.

64. 14:07

    And so all of these three things are already very useful on their own, and what plugins do is that they bundle that so that you don't have to, you know, set up everything manually.

65. 14:18

    You don't have to install multiple skills. You don't have to connect multiple apps. You don't have to connect multiple MCP servers. You can just add a plugin.

66. 14:27

    And, uh, another thing I wanted to talk about in the Codex app, and that we'll, we'll show a quick demo for, um, is automations. Personally, this is, like, one of my favorite things to do with Codex, uh, because you can set up automations that run in the background, so like a cron job, and you can connect apps.

67. 14:49

    You know, you can use, uh, plugins there too, and just set it to run on a scheduled, uh, time. So for example, you know, you can set an automation for, to run every day at a certain time.

68. 15:02

    And, uh, it's just an instruction that Codex will run in the background.

69. 15:08

    And the last thing I wanted to show you with the demo right after is, uh, specific skills for web app and game development. 'Cause we've, um, you know, we've heard a lot about developers who want to, to use Codex to build these things, to build apps, to build games, and every time, you know, they kind of repeat

70. 15:29

    themselves, every time they kind of use the, the same skills. So we actually package that into specific plugins. And, uh, there's two skills that I wanna highlight, uh, that are super useful, and hon- honestly, that are a game changer when you're developing something visual.

71. 15:44

    It's, um, Playwright Interactive. And so for those who don't know, Playwright is essentially like a, um, a headless browser, like a, um, you know, a sandbox browser, uh, that you can, that Codex can just run and use that to see what it's doing.

72. 16:02

    So you can open your app in a browser, and with the interactive version, you can actually click things and, uh, you know, just navigate your app, um, and, and take screenshots and see the, and analyze those screenshots.

73. 16:15

    And then ImageGen, uh, is a great way to just generate visual assets for your apps and games. So enough talking. I'll show you a demo. Um, I'm going to start by actually running this, uh, this one, because this one is pretty long.

74. 16:35

    Uh, when I ran it yesterday, it took, like, an hour [laughs] to build. So I also have, like, a, the final version, but I wanted to show you, like, this, this prompt, how Codex is going through it.

75. 16:47

    And so what I'm doing here is I'm using, uh, the games studio plugin, which is, uh, again, a bundle of a bunch of skills that are helpful for, uh, game development.

76. 16:59

    And I'm asking it to use ImageGen to create visual assets or sprites for the games and using, uh, Playwright Interactive to also debug the game and make sure that it works well.

77. 17:09

    So we're gonna let that run, and, uh, then we're gonna talk about plugins for a little bit. So let me switch to another project here. Uh, so this developers website one.

78. 17:28

    Okay, so this one is, uh, the repo for our developers website, which is

79. 17:37

    here. Sorry, I'm gonna put that in full screen. Um, and so on our developers website, we have this page with all of the Codex meetups we have. So, you know, there's a lot, and all of that is actually in our repo, like, in our code base, in YAML files.

80. 17:57

    And so I'm, what I'm gonna do is I actually added this Google Drive plugin here. Uh, you know, we have a, a lot of featured plugin, uh, built by us that you can choose from.

81. 18:08

    You can also, of course, add your own plugins. But I connected this Google Drive plugin that lets Codex access my Google Drive. And so what I did is that I prepared this, uh, this spreadsheet

82. 18:22

    Called Codex Events with the event name, date, and city. And I'm gonna ask, uh, Codex to just update this sheet with the current Codex meetups, uh, listed in the code base.

83. 18:36

    So I'm gonna start this. Again, it's gonna take a little while, uh, and so let's check in on... Okay, for the, the game task is still running. I'm gonna show you when it's doing a little bit, uh, uh, some more interesting things.

84. 18:51

    Uh, but the last thing that I mentioned is automations. And so automations is, again, something that you can just set up using apps. Do- you can just ask Codex anything, but instead of it being interactive, like you're actually using the Codex app, you can set it up to run in the background.

85. 19:09

    So for example, some automations that I've set up, um, that are honestly helping me a ton in my day-to-day lives, um, is one for Slack messages. So I connected Codex to Slack, and I'm asking, "Hey, Codex, can you check, uh, every day at 9:00 AM the messages that I should reply to, and flag if it's time sensitive

86. 19:32

    or waiting for an urgent response? Can you also do a summary of all the things, uh, that have happened since yesterday on Slack?" Uh, and I'm asking that to bucket it, to bucket, uh, per topic, and then, uh, important information to be aware of.

87. 19:50

    So we have, like, important channels where company information, uh, generally the, the things that you can, uh, that [laughs] that, that leaks in, like, [laughs] one day, but, uh, gen- so com- uh, important company information is in there.

88. 20:04

    And so I just wanna make sure that I don't, uh, miss anything here. So that's the kind of stuff that I ask Codex to just summarize for me. Another one, uh, that is, uh, pretty cool is the, is connecting Gmail, and same thing, like I receive honestly an ungodly amount of emails per day.

89. 20:24

    And so I'm just asking Codex to check if there are emails that I should actually reply to and, uh, you know, to check, you know, if it's time sensitive or if it looks legit or not, because I do get a lot of requests that's, um, not necessarily something that I would, uh, [laughs] that I would, uh, uh, reply

90. 20:42

    to, but this is, like, saving me hours per, per day. And so the way you can create automations is you can create it from here, or you can also just, you know, uh, say something like, uh,

91. 20:58

    "Hey Codex, can you, uh, create an automation that will, um, look at Slack and look for anything that mentions, uh, Codex use cases, and then, uh, list all of the important use cases that I should, um, that I should, uh, put on our website?"

92. 21:22

    So I'm gonna let Codex think about this for a second. I should have used Spark. [laughs]

93. 21:30

    And it's gonna come up with this, you know, it's gonna create the automation for me, basically, and I didn't specify, um, when I wanted to run it, but I can actually, like...

94. 21:42

    Oh, interesting. It's doing something different because this is a live demo, so obviously it wouldn't, uh... Okay, normally it will, it should, like, do a little pop-up, uh, so I can just, like, click on the...

95. 21:58

    Oh, it's doing it. Perfect. It was just very chatty this morning.

96. 22:04

    Okay. Interesting. Interesting. Okay. So [laughs] please create the automation. [keyboard clacking]

97. 22:15

    So this, it should show a little pop-up if everything goes well, but if not, you can still, like, create it manually. Uh, let's just see if it is doing it.

98. 22:29

    Okay. I don't know what's going on. But okay, let's just do it manually. So it will...

99. 22:37

    You can also create it from here, and basically all you have to do is just call the plugins you wanna, you wanna use, uh, you know, like use, uh, Slack, and then, uh, choose, you know, the frequency where the automation should run, which project it should run in, et cetera.

100. 22:56

     Okay, so let's check on our other tasks.

101. 23:01

     Uh, this one is still running. Okay. It generated some, some pretty cool sprites. We'll look at this after. Uh, let's check on our, uh, task to update the spreadsheet.

102. 23:09

     So here Codex took two minutes to actually analyze the code base. It found the source for all of the Codex events where we have our YAML files, and then, uh, it wrote the 57 event rows.

103. 23:23

     So we have 57 events, uh, currently listed on the website. And, uh, so let's check. Let's see our spreadsheet. And yeah, we can see that it was updated. Nice.

104. 23:36

     So this is something, you know, this is a simple example, but every time you have something that's very, you know, uh, time-consuming and, uh, anything that has anything to do with data, data review, for example, you can actually ask Codex to do it for you.

105. 23:51

     It has access to everything, uh, on your code base, and you can also feed it other inputs, you know, like other CSV files. And then you can just ask Codex to do that type of work for you.

106. 24:04

     Okay. Now last thing, let's check on our, uh,

107. 24:09

     game. So as you can see, Codex is actually using ImageGen to generate... I'm gonna, uh, de-zoom out a little bit. So, oh, nice. So it's generating, like, all the sprites, all the game assets that I asked it to do.

108. 24:27

     And this looks pretty nice. Uh, it's also... So it's gonna take a while. Uh, what I'm gonna do is I'm actually gonna show you, um, final results. But, uh, as you can see, like, Codex is just reading, um...

109. 24:47

     Sorry, it's just generating all of these assets, and then it's gonna use the Playwright skill to see how that looks like in the app. So unfortunately, we don't have an hour to wait for this final result, so let me just show you the one that it did yesterday.

110. 25:00

     So this is un- uh, [laughs] untouched. Like, I haven't touched it. It's literally just Codex, um, who built this, and all of that was-- Like, it-- I had-- I gave zero input.

111. 25:14

     I was just like, "Do a platformer game with platforms made of bricks." That's it. And, uh, yeah, it generated everything. So granted, the, the overall UI is not like, you know, uh...

112. 25:27

     I would probably iterate on that. But, um,

113. 25:31

     I think the, the platformer itself is pretty cool. And what is really cool here is that literally, like, all the sprites, like here, you know, I'm just like moving all around and, you know, that-that's at least like five different sprites of the little character, and I didn't have to do any of that.

114. 25:50

     You could also, you know, do a custom game with your face as input and, uh, have ImageGen just, like, create a, a 2D version of you. Um, so that's a way that you can, like, leverage the ImageGen skill, the Playwright interactive skills, and that game studio, uh, plugin.

115. 26:09

     And just to show you what's inside, like we have also the same thing for web apps, but it's a bundle of like all of these skills together. Um, so yeah, that's, uh, that's it for me.

116. 26:22

     Uh, I'm gonna pass it back to VB. Thank you.

117. 26:27

     Thank you, Katia. All right. Um, perfect. So just to do like a very quick, uh, checkpoint, uh, and like a recap on what we've spoken so far.

118. 26:42

     So we went through, like, all the, um, all the models that power, um, the Codex ecosystem. Then we went through all the surfaces you can consume Codex from. Um, and then we went through, uh, plugins, how to use them, and what are some of the plugins that you can use.

119. 26:59

     You can also create your own plugins, um, using Plugin Creator. Um, you... And, and then we went through, uh, to speak about, uh, automations, um, and ImageGen and, and so on and so forth.

120. 27:12

     Um, now something to note is, like, as we, as we continue sort of delegating more, you know, more and more work on these, um, agents, it could be any of your favorite agents, uh, Codex or not, um, one thing that, um, that you want to be sure of is whatever it is that your agent produces is of

121. 27:32

     the utmost quality. Which means that, um, as we, as we start sort of working on multiple features at the same time, multiple projects at the same time, it, it's going to be quite likely that it's impossible for you to, uh, go and look through each and every line of code.

122. 27:49

     Which means that at least for the first pass, you want to have a way, um, which you can rely on, um, to review your code. And this is where, um, code review, um, sort of comes in.

123. 28:01

     Um, it's, um... By no means, um, am I bragging about this [laughs], but, uh, in my own biased way, uh, Codex Code Review is one of the best in the industry right now.

124. 28:12

     This is, uh, something which, you know, uh, people on Twitter, on LinkedIn, um, on our own, uh, sort of, you know, platforms, Discord and so on and so forth, keep raving about, uh, that how is Codex Code Review so, so good?

125. 28:24

     Um, so I wanted to spend like a quick hot minute on, um, on what it does. So first of all, um, it is available on the surfaces that you work at, which means, number one, you are able to use Codex Code Review on GitHub.

126. 28:38

     Um, so you can connect your ChatGPT account with GitHub, and for each and every pull request that you create, um, you can set it up such that Codex can automatically review each and every pull request, and it would typically give you, um, you know, some sort of, uh, uh, some sort of, um, you know, um, what's this

127. 28:57

     called? A call-out [laughs] like this on the pull request itself saying that, "Hey, like this is something that is missing. Hey, m-maybe, you know, uh, uh, P zero, fix this.

128. 29:06

     P one, fix that. P two, you know, this is something that would be a good to have," and so on and so forth. Uh, at the same time, you can use a slash review on the, on the Codex, um, CLI or the Codex app, and Codex will spin up, you know, large, um, sort of review process and

129. 29:22

     so on and so forth. Um, and very recently last week, uh, with my colleague Dom, we shipped, um, a Cloud Code plugin for Codex, uh, which allows you to, um, you know, essentially invoke Codex within your Cloud Code sessions to be able to get the same sort of state-of-the-art code review, but in your Cloud Code sessions.

130. 29:43

     Um, so, um, something to sort of see here is, let's say that I am working on a project like this. By the way, this is my, this is my actual working setup at work.

131. 29:59

     Uh, I-- This is like all which I work on. Uh, I'm not... Like, everything that you see here is-- Like, all of these threads, all of these projects is something which I work on day to day.

132. 30:10

     So if you see something which you shouldn't, just close your eyes. Uh [laughs]. And so typically what I would do is I would, um, I would go through, you know, like a, like a feature request, or I would go through, um, you know, some sort of ask from, from someone.

133. 30:24

     Um, and, um, uh, let's say over here, I ask the, I ask Codex to do a bunch of things. So I'm just gonna ask it to review its changes.

134. 30:35

     Um- And so then you get an option to, you know, either choose from, uh, a base branch if you have multiple branches in, in the Git repo. You can choose it against a feature branch, against an eval branch, whatever it may be, and so on and so forth.

135. 30:50

     Uh, in this case, I'm just gonna ask you to review, um, uncommitted changes. Uh, and what it does is if you see, um, here,

136. 31:01

     what it does is it spins off a totally new thread. Um, and what that thread would do is, um, is it would essentially spin up a totally new Codex process, which has, uh, like our own, you know, review system prompt.

137. 31:16

     Um, and it would continue sort of looking through not just the diff or like the list of all the changes, but it would also contextualize it with everything that is there on the, uh, on the model repo itself, right?

138. 31:27

     And so a lot of the times, um, um, Codex code review will like find, find out changes which would have second-order effects, um, which is not limited to just the, you know, diff or whatever changes you've made, but also to some other like modules which you haven't even touched in the pull request itself or in the changes

139. 31:44

     itself. And this is, um, this is so effective that one hundred percent of pull requests across all OpenAI repos, um, made by all employees, um, including Greg [chuckles] are, uh, are reviewed by Codex code review by default.

140. 31:58

     Um, and that's when, uh, you know... That's the first pass that you take. Um,

141. 32:04

     cool. And so as you can see over here, um, Codex worked for a minute, and it came up with these, with these sort of, uh, you know, updates like P1, you know, localize whatever revenue detail, P2, uh, translate this to this, and, um, and so on and so forth.

142. 32:19

     And what you can do, like after this is, um, like essentially ask Codex to, uh, either like take a pass at fixing this or like open another sort of PR on the-- on whichever branch you're at, and then sort of go on from there.

143. 32:35

     Cool. Now we get to subagents, which is something which I'm personally quite excited about. So, uh, first and foremost, what is subagents? Subagents is the, um, is, is essentially the ability, uh, wherein you can spin off, um, a master task into decomposable, parallel and independent tasks, which you can hand off to agents, which can, uh, which can

144. 33:00

     allow these agents to sort of work independently and then at the end of their run, get back to you and, um, you know, give you a response. And, um, over here, like sky is literally the limit.

145. 33:11

     Like you can spin up as many agents as you want. Um, of course, as long as your API key or your, uh, you know, Cha- whatever ChatGPT Pro Plus Go subscription you're on, um, can, can, can take.

146. 33:24

     Um, you can do a lot of like, interesting things, uh, with subagents. Um, for example, what I'm doing, um, on the screenshot on the left is, um, I have a Codex agents repo, which we're gonna look at in a sec.

147. 33:38

     It's not public yet, but I hope that we'll be able to make it public very soon, uh, which has a lot of personas for subagents that you can use.

148. 33:45

     So it, it's kinda meta. Uh, [chuckles] it's, uh, it's essentially subagent personas like doc reviewers or, you know, um, test case creator or test case runner and, and so on and so forth.

149. 33:56

     And what I-- Um, every now and then, we would change the, change the spec. And this is from before we wanted to change the spec of how, um, how subagents work.

150. 34:07

     So what I wanted, um, it to do is to go through all of these forty, fifty different subagent personas, review them and, and, and make sure that they are up to spec.

151. 34:17

     And of course, doing it without subagents would have meant that, um, Codex would open each and every file and then review it, and then give me a summary, and continue doing it for like fifty different subagents.

152. 34:29

     In this case, um, it, it essentially created review slices, which means it created, say, you know, these are the two, uh, files that, um, that, you know, uh, subagent, uh, Polly or subagent Plato, uh, should, you know, uh, essentially review.

153. 34:46

     And then they would spin up a new Codex environment, they would review those, and then at the end, Codex will collate all of these and, um, you know, give me back a response.

154. 34:57

     So let's, let's give this a shot. Um,

155. 35:06

     so the repo in question is this. Um, it's, um, it's just a Codex ag- agents repo, which has a bunch of personas. Um, you can see that we have, um, we have quite a few sort of personas over here.

156. 35:20

     Um, we've got like an accessibility reviewer, architect, and so on and so forth. And this is like actually something which you can create yourself, and we're gonna touch on that in just a, uh, in just a minute, is, um, you can, you can define your own custom subagents, right?

157. 35:35

     Um, but think of this repo as like a collection of these subagents. And, um, this is typically what you would have for, for each and every subagent. You would have a name, you would have a description, you would have a different sort of like, you know, sandbox mode, whether you want it to be write-only, whether you want

158. 35:50

     it to be read-only. Um, you-- And then you would have some sort of like, you know, instructions, um, and so on. And so now what I'm gonna do is I'm gonna ask Codex to, um...

159. 36:03

     I'm gonna go over to my Codex agents. Um, I'm gonna switch to,

160. 36:10

     let's do medium over here. Let's close this.

161. 36:16

     Gonna make this full screen. All right. Um, so let's give it, give it a task. Um, spin up twenty subagents to review all the subagents.

162. 36:37

     So this is a very simple task. All I'm asking, uh, Codex is to do, uh, the same task which I was showing be-before, um- Wherein I wanted to review all the different subagent personas in this repo, and you can see that, um, uh, you know, there's...

163. 36:52

     It, it already figured out that there's, like, agents and skills, and it's looking into it. There are 45 curated persona files. And, uh, what, what it's gonna do is it's, it's gonna create 20 reviewers and, um, um, it, it's gonna give them all of those, um, um, TOML files, and then it's gonna review those.

164. 37:12

     And you can see that, um, there's two things which is quite interesting over here. Number one, Codex automatically decided that this is potentially, uh, a complex task, so it automatically kickstarted the plan mode, uh, which is what's active over here.

165. 37:25

     So you can see that, uh, it, um, it essentially came up with five tasks, uh, to solve this particular problem. Um, you can explicitly invoke plan mode as well, but, uh, in this case, it decided to do it on its own.

166. 37:38

     Um, it's, it's then partitioning all of these persona files, um, and then it's gonna spawn 20 subagents very soon. Um,

167. 37:48

     I swear it's faster. Uh [laughs] But, um, so now what it's doing is it's, um...

168. 37:57

     Oh. Uh, so for some reason, on my, on my particular setup, I have a cap on six num- like, six concurrent agent threads that can be run at the same time.

169. 38:13

     Um, we can fix that, um, but to go back up,

170. 38:20

     what we can see over here is that, uh, it actually spin up six agents, which is my limit, uh, for now, and I can see all of those agents, um, you know, working over here.

171. 38:29

     I can quickly see, like, what Jason, the agent over here, is doing, uh, or Hume, and so on and so forth. And you can see that, uh, something to note here is that the, the main Codex model over here- Hi. [laughs]

172. 38:43

     The main Codex model over here, um, essentially created a persona, right? Um, a- and, and not just that, it doubled down and it, it gave the exact files that this particular subagent should review, right?

173. 38:56

     Um, a- and, and, uh, additionally, it also gave it some, some insight on, um, there's, there's repo guidance in repo.md, in contributing.md, in skills, and so on and so forth.

174. 39:08

     And, um, it will sort of continue going down th- this, this route for all the different subagents, right? Um, and what it does towards the end stage is that, um, it will tear down all of these subagents when, when they have gone through, um, when they have gone through, uh, their whole process of looking through all the

175. 39:27

     TOML files and so on. And if I go back to, uh, my main thread, um, you can see that two of the agents are, are still working, um,

176. 39:40

     but eventually, like, it would collate all of this feedback that it ta- that it has gotten from, um, all of these individual subagents and, you know, proceed. Um, now you can, you can think of this, this is like a very simple sort of explorer use case, right?

177. 39:55

     But you can think of this from, for example, a cybersecurity perspective wherein you have, um, a Git commit, or you have a, a particular Git repo, and you want Codex to spin up and run multiple, uh, you know, vulnerability, um...

178. 40:13

     Sorry, one sec. You wanted to create multiple sort of, you know, vulnerability analysis from different points of views or from different hypotheses, and you wanted to sort of tackle the same diff or the same GitHub repo and try and come up with, like, a vulnerability map, right?

179. 40:27

     And this is something we actually use, um, um, quite a bit, or I personally use quite a bit when I'm brainstorming a particular feature. I would just spin up multiple, um, Codex subagents to sort of look through how I would approach a problem, right?

180. 40:42

     So let's say I want to add a feature. I would ask Codex to create a plan for what are, say, five or six or ten different ways that, uh, that a model, um, that a particular feature could be implemented, and then I would quickly double down on Cod- uh, like, and ask Codex to, um, then create multiple

181. 41:01

     subagents to get me what-- some sort of understanding for, um, for these tasks. Sorry, my watch was constantly vibrating [laughs]. Um, and, and so that's like a, that's like a quick high-level overview of how subagents work.

182. 41:18

     Uh, by default, we ship three subagents, um, three subagents' personas. Um,

183. 41:26

     let me quickly open... [keyboard clicking] So by default, we ship, um, three personas. One is like a default general purpose fallback agent.

184. 41:43

     Another is a worker, which is sort of execution-focused, so this is something that you would use for, um, you know, when you want Codex to write a particular feature request, uh, or work on a particular feature.

185. 41:55

     Then there's explorer, which is the same one which we used, uh, before. And, and then, uh, for, for each of these, you can double down and create your own Codex, um, subagent personas like we saw before, and we will create one right now.

186. 42:11

     Um, something to note is, um, is that these particular subagents, um,

187. 42:19

     they... Like, for each of these, you can define what model you want to use. You can define what reasoning effort do you want to use. You can define what sandbox mode do you want to use, and so on and so forth.

188. 42:31

     Um, the reason why this is important is f- for a review agent, you would almost always 100% want to use the review agent in read-only mode. You would never want your review agent to execute anything, right?

189. 42:42

     Um, for same reason for, like, a cybersecurity vulnerability, um, assignment, you would want your, um, your subagent to always be in read-only mode. But for a, for a, um, for, like, a docs writer or for something which, like, you know, creates, um, docs for a particular feature that you've created or a bug report and so on, you

190. 43:02

     do want to give it write access so that it can execute stuff and also create a, um- Create a bug report for it as well. Um, something to note is that you can also double down and give these, um, subagents, you know, more capabilities by giving them, uh, MCP access.

191. 43:22

     So y-you can just give, um, you know... Let's say you can give a subagent MCP access to Sentry so that it can look through all of your, um, um, all of your reports over there, or, like, one subagent access to your Linear, um, you know, backlog so that it can, um, it can interact with Linear, it can,

192. 43:41

     uh, read through all the, um, uh, all the issues, uh, added to you, triage them, and so on and so forth. You can also give them skills. Um, so really you can, um, um, if you really want to, you can quite heavily customize this entire setup for your own, um, for your own

193. 44:01

     use case. [chuckles] So let's open, um, our Codex app again. You can see that it went through all of these subagents. It created a bunch of, uh, other subagents just to go through all of these, and, uh, it came up with these findings.

194. 44:17

     Um, it's like based on README, based on contributing, uh, performance investigator, um, is over-privileged. Um, P1 has a sandbox mix, uh, sorry, verifier has a sandbox mismatch, same for writer, and so on and so forth.

195. 44:34

     And so you can see that this is already quite useful, um, and it saves you quite a bit, bit of time to be able to go through all of these, uh, individually or sequentially, and so on and so forth.

196. 44:44

     Um, now let's go back and see a bit more about custom subagents.

197. 44:52

     Um, so as I mentioned that we ship three, um, subagent personas, but at the same time, you can create your own custom subagents. In fact, we do recommend creating your own subagents, or just ask your, your Codex to look through your past sessions and create subagents for you.

198. 45:09

     Um, both of these scenarios work, and, um, work quite well. So in, in this particular case, uh, you can see that we have a PR Explorer subagent, which, um, reads your, um, your code base, uses GPD 5.3 Codex Spark, which is our, um, research preview model, text only, um, deployed on Cerebras, um, and is blazingly fast, is

199. 45:33

     quite fit for this particular use case. And we set sandbox to read-only, so we don't want the model to sort of execute, and we give it certain, um, you know, in, in- instructions.

200. 45:44

     So in this case we say, "Hey, stay in the exploration mode, uh, trace the execution path, you know, um, don't propose any fixes," and, and, and just like, you know, search through and, and, and figure out, like, what, what, what exactly do you want us to do?

201. 45:59

     Um, now let's quickly try and, try and, um, create a subagent.

202. 46:12

     So let's say we want to do, um, docs researcher. In this case, what I, what I typically do is to just go

203. 46:22

     and ask, um [keyboard clicking] , "Hey, Codex, can you create this subagent, um, for me? Uh, here's, here's its persona." Um, and then let's see.

204. 46:41

     And so what Codex is going to do, because Codex is aware about, um, about how it works and, you know, uh, what it's supposed to, um, do and where it's supposed to place, uh, all of these things, uh, what it's gonna do is it's gonna create a, create a TOML file for this docs reviewer.

205. 47:01

     And in this particular case, this is, this uses the docs MCP server, which we created, um, um, from the DX team, um, which packages all the API references, all the docs, all the guides, all the, you know, toolkits and so on and so forth.

206. 47:17

     And, uh, it will add that as an MCP server so that every time we ask it, um, ask it a question about, "Hey, like, what's the best way to use GPD 5.4 with WebSockets?"

207. 47:27

     Or, "What's the best way to use GPD real-time with, um, um, with..." I don't know, pick your favorite way of using GPD real-time, and, um, and, "Can you create a React plugin for this?"

208. 47:38

     And so on and so forth. Um, uh, it would be able to reference all of these things.

209. 47:45

     So I'm gonna let it do its thing, and in the meantime, um, head back over to the slides.

210. 47:56

     And so just to go back... Sorry, one second.

211. 48:00

     Um, what you can do just to sort of invoke, um, you know, a particular subagent is you can say, um [keyboard clicking], "Hey, can you

212. 48:14

     reviewer subagent and review each and every persona based on the

213. 48:28

     developer's docs?" So, uh, in this case, you can, you can essentially, like, use the same particular, um, subagent, uh, leverage it again, and then ask it to do the particular task that you want to do.

214. 48:40

     Now, what are some, like, interesting ways that you can use this is, um, imagine, like, you have, like, a long build process, or you have a test process. You can have a subagent which can run your test case locally.

215. 48:52

     You can have a subagent which can, uh, always make sure to, um... Oh, I'm, I'm, I'm being told that I don't have as much time. Uh, [chuckles] um, you can have a subagent which can, uh, pull the latest from, uh, from GitHub as soon as you do a pull.

216. 49:07

     You can have a subagent which can, uh, you know, quickly, um, pull all of the context from a linear issue, and so on and so forth. So really, like, you can, you can, you can do this for- You can leverage this for a lot of, um, things.

217. 49:19

     And the best thing that I like to do is to just ask Codex to look through my past sessions and recommend me certain automations, certain sub-agents, and so on and so forth that I can use.

218. 49:29

     Cool. So now we're at the, at the bleeding edge. Um, this is bunch of stuff which we have shipped in the past and, um, we haven't really made as much of a splash about.

219. 49:39

     Um, so, um, what we're gonna do is we're just gonna quickly go, uh, around and see, like, what each and every one of these, um, uh, do and, and how you can leverage them.

220. 49:50

     So first and foremost is guardian approvals. This is an experimental feature. You can activate it today, uh, by just going on slash, slash experimental. Um, so it would be something like,

221. 50:02

     um, Codex. Hopefully it works. [laughs] And then you can look at, um, experimental and you can, um... In, in my case, I already use guardian approvals, and you can activate it this way.

222. 50:22

     Um, what guardian approval does is, um, all of us, including myself, at some point were, um, guilty of using YOLO mode all the time, which means that you, by default, give unfettered, um, access to your coding agent to do literally whatever the hell it wants, right?

223. 50:40

     And this, by all means and measure, is not safe. Um, hence we came up with something called guardian approval, which for each and every time Codex needs, um, a privilege, needs to run a privileged task.

224. 50:52

     Let's say it is, uh, can I remove this particular directory? Can I run a, a server? Can I expose a particular file to, um, um, to the internet? Whenever all of these things sort of pop up, what Codex will do is it will spin up a new sub-agent, right, which will, based on a particular prompt, try and

225. 51:12

     verify whether or not this is something which needs my human interruption or not. Um, and in most cases it doesn't need, you know, human interruption, so it will just say, "Hey, go on, run this particular, you know, privileged tool or privileged task," and so on and so forth.

226. 51:28

     And, um, this way what we, what we hope to do is we hope to reduce the human fatigue, uh, that comes by just, you know, always sort of having to approve, you know, do this task, do this, run this particular bash script, or run this, and, and so on and so forth.

227. 51:44

     In, um, in principle, how would that look is,

228. 51:50

     um... Trying to see if there was, um...

229. 52:13

     Uh, uh, uh. Okay, it doesn't show, show it to me right now, but if I, just in the interest of time, I'm gonna ask, uh, "Hey, can you run the dev server?"

230. 52:30

     And I'm gonna, instead of full access mode, uh, which for some reason, again, I'm not able to,

231. 52:38

     uh, click on. Let's, let's try and see, um, if it, if it invokes, um, guardian approvals. Whilst this, this works, I'm gonna head over, uh, to the next step, which is hooks.

232. 52:54

     Hooks is also something which is experimental right now. We're, we're trying twenty-four/seven to try and make this, uh, a better experience. Uh, currently Codex supports three hooks. One is after each tool use, one is at the start of a session, and third is at the-- uh, when you stop a session.

233. 53:09

     What hooks allow you to do is it allows you to programmatically ask Codex to do a thing X, uh, based on a particular event. So let's say that when you start your, um, your Codex session, you want Codex to pull the latest from your GitHub repo.

234. 53:27

     So in that p- in, in that particular case, you would want to set up a start hook. Um, if you want Codex to do something after each tool use.

235. 53:34

     Let's say, um, for a lot of researchers who want to document each and every tool use, they might have, like, a per tool use hook wherein they document what Codex has done, uh, per session, and so on and so forth.

236. 53:46

     So you can do that with that. Um, and, um, last but not the least, something which I personally use is the stop hook, which is when I'm running long-running tasks, I would, um, at the end of each turn, uh, of Codex, I would ask it to keep going so that, [laughs] like, it just continues, uh, you know, continuously

237. 54:05

     keeps running a particular task. And, um, in, in theory, how this would look like is, um,

238. 54:16

     is... Where is it? Is... [laughs] Sorry, one second.

239. 54:29

     Wow, I was really prepared for having more time, um, I have to say. Uh, [laughs] but, um, in theory how this would look like is, um, is that you have some sort of a Python script, um, and you have-- you define, like, a hooks.json.

240. 54:42

     So in this particular case, you can see over here that you have a pre tool use. Um, you have some sort of a, you know, matcher. You say, like, on startup or resume, run this particular session dot-- session start.py and so on.

241. 54:55

     Uh, and you can define how you want to, uh, in this particular case. Um, so what I did, for, for example, uh, for the sales dashboard example that I've been showing you so far, is I created a hook for stop, which runs, um, this Python script, which is keepgoing.py.

242. 55:12

     Um, which is every time it encounters the stop, um- Um, hook, it would just ask Codex, um, to keep going, do one more pass, run one, one solid validating command, tighten one more thing, and then stop and give the result.

243. 55:28

     And so for really long-running tasks, you can just set it up and, like, ask it to continue doing its own thing. Um,

244. 55:36

     last but not the least, um, we have personality changes, which means that you can go on Codex, and you can ask it to, um,

245. 55:46

     quickly look at personalization. You can set up different personalities. You can set up a more, uh, friendly personality or a pragmatic personality based on whatever you want to do.

246. 55:56

     You can also add custom instructions, so you can ask it to always cite whatever it is it is doing, and so on. Um,

247. 56:07

     right. And then last two things, um, is we released something called Codex Security. This is our state-of-the-art, uh, model, which allows you to find and fix vulnerabilities in, in your GitHub projects.

248. 56:21

     And, um, you know, essentially what it does is it would go through commit by commit, and, um, it would create a vulnerability patch, uh, and then-- and it would use Codex to then sort of patch the said changes as well.

249. 56:35

     Um, lastly, um, as I mentioned before, we released a Cloud Code plugin, uh, which allows you to use Codex in, uh, in Cloud Code. Uh, this is something which was, uh, surprisingly used quite a bit by the community.

250. 56:47

     Um, and, uh, this is something which allows you to sort of ask Codex to review whatever it is that you've done so far, run an adversi-adversarial review, or just, like, ask Codex to rescue whatever changes you've done so far as well.

251. 57:02

     Um, that's it. Thank you so much for, for joining us, and feel free to ask any questions that you might have.

252. 57:10

     Hi. So we don't have a lot of time for Q&A. Unfortunately, we should [laughs] have started maybe a little bit earlier. But, uh, happy to take maybe a couple questions in the room, and then we'll stay here anyway, so if you have questions and you don't have anywhere to be, you can come to us.

253. 57:24

     Yeah.

254. 57:25

     Hey. Thank you so much. Um, I have a question. You, uh, said a couple of times that there's, like, a way to, uh, scan, uh, let-- for Codex scan the past sessions and basically give you a recommendation for that.

255. 57:36

     How exactly do I do that? Uh, so like a project with, like, twenty threads or something like that, how do you scan that?

256. 57:41

     Yeah. So what you typically do is, like, all of the sessions within Codex are put in, uh, .sessions within a particular-- within the same .codex folder.

257. 57:50

     Ah.

258. 57:50

     And Codex has the ability to just, like, scan through all your sessions and then, you know-

259. 57:54

     Okay

260. 57:55

     ... do things.

261. 57:55

     This, this you can do CLI, but not through-

262. 57:57

     Yeah, yeah.

263. 57:59

     I can do this using CLI but not using the Codex, uh, UI?

264. 58:03

     You can use it... Uh, you can use Codex app, you can use Codex CLI, anything. Um, you just have to ask it to look through the sessions and, yeah, do whatever you want to do.

265. 58:12

     Thank you.

266. 58:13

     Thanks. There's another... Oh, okay. Maybe a couple more. Yeah.

267. 58:19

     In the back here.

268. 58:23

     Hi.

269. 58:24

     Hi.

270. 58:25

     Is there a way to hand off a task to a cloud agent? So let's say I'm here working on a task, and I'm-- I have to close my laptop, so I-

271. 58:33

     Yes

272. 58:33

     ... hand it off to a cloud agent.

273. 58:35

     Yes, definitely. We didn't touch on that, but actually, uh, you can do that from the Codex app directly. Like, um, maybe you can, you can share your screen. But you can either work locally, and as you've mentioned, you can do it, uh, like we support Git worktrees as well.

274. 58:49

     Uh, but you can also just select cloud here, and you can select the number of, uh, times this task should run. Like you can parallelize. We call that, like, best of N.

275. 58:59

     So you can, like, run it four times in the cloud and then just pick the best output. Uh, so that's something that is, like, built in in the, the Codex app, in the IDE extension, and you can also, like, access it directly from the, the web interface.

276. 59:13

     And there's more cool stuff coming on that very soon. [laughs]

277. 59:17

     More what?

278. 59:17

     There's more cool stuff coming on that very, very soon.

279. 59:20

     All right. Okay.

280. 59:21

     I think there was one right here. Yeah.

281. 59:28

     Thank you so much. Um, my question was actually about the cloud, uh, UI as well, 'cause, um, today subagents aren't supported, if I'm not wrong.

282. 59:38

     Yeah.

283. 59:39

     And, uh, especially the thing that bothers me is, uh, it doesn't use the, the skills that are in the repo. Is that coming soon or...?

284. 59:49

     So, um, there's like a... At the risk of, uh, uh, you know, talking about the whole roadmap, uh, we, we, we, we definitely have a lot more changes coming up on that particular front.

285. 1:00:01

     Um, I'm not sure if skills within cloud is going to be as soon as I say that it [laughs] is going to be. But, uh, it's definitely at the top of the mind, and we do want to sort of add, uh, you know, give you the ability to sort of like have your own trusted MCP servers to be

286. 1:00:17

     able to run there or CLIs and so on. Um, and also the ability to just, like, have SSH agents, uh, that you can just spawn off, uh, a particular task to on a VM and so on.

287. 1:00:28

     So lots of work on that. Like-

288. 1:00:30

     Yeah. It can use skills in the repo, right? That it-- that is checked in. It's-

289. 1:00:34

     Not on cloud tasks.

290. 1:00:36

     Yeah.

291. 1:00:36

     I mean, but, like, if you... Like, it, it reads instructions and stuff, and it can, like, find it and, like, still see it since it's in the code base.

292. 1:00:45

     It's more like the, the skills that you have locally that-

293. 1:00:48

     Yeah, I think-

294. 1:00:48

     ... were, were the same

295. 1:00:49

     ... the reason why we don't allow it on, on cloud is because there's no way for, um, the sandbox to know whether or not a skill is trusted or not.

296. 1:00:58

     Yeah.

297. 1:00:58

     Right? And so that's why we, we, we don't... And, like, skill can package, like, a Python script or an-

298. 1:01:03

     Yeah

299. 1:01:03

     ... or an executable.

300. 1:01:04

     Yeah, of course.

301. 1:01:04

     It w- it won't execute things, but, like, if you have, you know, like,

302. 1:01:08

     things, like resources, it can access it technically because it is, like, in the repo. It's just, yeah, it's not as good as the-

303. 1:01:14

     So I have to request it.

304. 1:01:16

     Yeah. [laughs] Yeah.

305. 1:01:17

     Thank you.

306. 1:01:18

     Thank you.

307. 1:01:22

     Were there any other questions? Cool. Have a great day.

308. 1:01:28

     Thanks.

309. 1:01:28

     Enjoy the day. And, uh, if you have any other questions, we're gonna be around, um, today, tomorrow, and also maybe on Friday. Um, feel free to reach out or just, like, drop a DM and, um, enjoy.

310. 1:01:41

     Thank you. [applause] [outro jingle]