Cooking with Codex — Charlie Guo & Gabriel Chua, OpenAI

A Friday brainstorming conversation contains the demo idea: build a real-time translator. It also contains a suggested translation capability and the idea of using a plugin to supply macOS development practices. In this workshop, Charlie Guo and Gabriel Chua of OpenAI turn that conversation into a build request without first rewriting it as a specification.

The opening examples establish how far the work can range: a synth plugin for Ableton, Premiere editing, games with separate art and mechanics work, and a self-driving golf cart. The workshop then follows six steps: give Codex context, build something, let it take actions, collaborate on complex tasks, make the work repeatable over time, and embed Codex elsewhere. Context is the first dependency because the useful specification may already live in a conversation, document or application.

An app shot captures the Slack window. It resembles a screenshot, but includes embedded text as well, so Codex receives both the visible conversation and its words. The accompanying request is short: “Hey, can you just build this?” The capture supplies what “this” means—the product idea and the surrounding brainstorming—while the request tells Codex to act on it.

While the translator builds, another artifact demonstrates a different source of context. A repository has code, Git commits and a history of changes; Codex uses that material to produce a Sites page explaining what is new in the Agents SDK, with code snippets and changes such as sandbox agents, Codex as a tool, tool search and real-time agents. Sites provides a shareable reading format for knowledge that would otherwise require someone to inspect the repository.

Codex Remote separates the place where a request starts from the machine that executes it. Charlie sends a request from the ChatGPT mobile app to a connected laptop or remote server: explain how the Agents SDK repository's primitives fit together, using the style of an IKEA instruction manual. The laptop remains on while Codex works. The phone becomes a way to give direction and check for blockers without sitting beside the executing machine.

Back in the translator build, plugins supply several kinds of reusable context and capability:

• Skills: best practices, prompts and team-specific procedures that describe how to do the work.
• Apps: configuration and connectivity to external services.
• MCP servers: tools the agent can call to perform the work.

Bundling these pieces gives the agent both guidance and access. A development practice alone cannot retrieve information from an external service; a connection alone does not explain the team's preferred workflow.

The macOS plugin used for the translator includes practices for AppKit, Liquid Glass, signing and inspecting the built application. The OpenAI developers plugin can create an API key when needed, removing a manual trip to the developer platform from this workflow. These plugins address different dependencies: one supplies platform knowledge; the other handles service setup.

A successful conversation can itself become reusable context. After working through an unfamiliar workflow, Charlie asks Codex to “make this a skill” or package it as a plugin. That preserves the procedure for a later task, so the next run can invoke the learned workflow rather than repeat the whole discovery conversation.

The translator is reported built in four minutes and two seconds. Gabriel selects German and configures audio input and output, then tries a microphone test. The first attempts require more audio adjustments. “You know it's a real demo, live demo, when you gotta do IT on stage.” Charlie suggests using local audio amid congestion in the room.

The next exchange produces the observable result. Gabriel says, “Hello, are you there?” The app responds, “Hallo. Bist du da?” It then translates his statement that he is at the AI Engineer Conference and his explanation that the microphone can pick up the speakers. The example has now moved from a Slack idea to an application producing German speech. This is a short live demonstration rather than a translation-quality benchmark, and the microphone picking up speaker output remains a practical limitation of the setup.

The causal steps are worth keeping distinct. The app shot carried the product idea into the task. Plugins supplied macOS practices and API setup. Codex produced a build. Choosing devices and speaking through the application tested whether that build worked in the room. A fast build still needed an actual input-to-output test; audio routing became the immediate problem once the software existed.

Context can persist beyond an individual request. Memory, described here as off by default, can learn preferences such as using uv for Python package management. Chronicle is presented as a research preview for Pro subscribers that adds screen activity context: a question such as “Why is this failing?” can refer to the pipeline the person has been viewing. Custom instructions apply across conversations, much like instructions placed in a home-directory AGENTS.md.

Persistence has costs and controls. The personalization settings can skip memory generation in chats involving tools, because tool results may contain information the user does not want retained. Memory and Chronicle also consume additional token budget. The decision is whether the saved context is useful enough to carry into future work, rather than assuming every returned fact should become a lasting preference.

For context that has not yet been written down, dictation provides a low-friction route. Talking through an idea can both supply detail and clarify the idea before sending it. App shots serve the same purpose for material already on screen. Neither requires a polished prompt first; the important part is providing the information needed to identify the task.

Building also includes checking and distributing artifacts. Sites can package an event recap or a comparison of offsite locations so a team can make a decision. For the translator, /codereview starts a review against uncommitted changes. The repository visualization requested from the phone returns as an SVG, after which Gabriel asks to use the Image Gen skill. That exchange exposes a useful distinction: requesting a visual does not necessarily select a particular image-generation mechanism.

A dashboard export shows why an agent sometimes needs to operate an interface. The dashboard has date pickers and dataset controls; doing repeated exports by hand means pressing many buttons. Gabriel captures it in an app shot and requests two different outputs: commit counts for the last seven days as CSV, and pull-request data from the start of the month to the current day as JSON. Codex invokes computer use, inspects the application and begins clicking and downloading.

The two exports require separate choices of dataset, date range and format. Giving all three properties for each output makes the intended action clear. On macOS, the demonstrated computer-use interaction has its own cursor, so Gabriel can continue using the computer while Codex works in the relevant application.

The next request turns questions already drafted in Google Docs into a feedback form, with every question marked mandatory. The Chrome extension controls the live browser tab to create it. Having the question text is only part of the task: the agent must also enter the questions, configure their options and set the required state in the form interface. The form later reports completion while the workshop moves on to long-running projects.

Three control mechanisms serve different needs:

• In-app browser: deeper integration with the browser engine supports inspecting layout, rendering and interactions in local web applications. It is described here rather than demonstrated.
• Computer use: drives desktop or legacy applications through the interface a person would use, especially when there is no convenient programmatic interface.
• Chrome extension: operates in the user's signed-in browser, with permission, so a task can use an existing session rather than start by trying to authenticate in a separate headless browser.

The choice depends on whether the difficult part is inspecting an application under development, operating software without an API, or accessing an already authenticated session.

The overnight projects cover three distinct kinds of development:

• Greenfield application: a live Q&A site intended for a room of about 400 people, with admin, participant and onstage views, moderation, and a 16:9 stage presentation. It uses Convex for real-time interactions and is intended for deployment on Vercel. Its goal is still running after 15 hours, 30 minutes and 55 seconds when shown.
• Repository adaptation: an existing open-source form builder becomes an internal Sites application using ChatGPT authentication to restrict access to the team. The reported run lasts five hours and 57 seconds.
• Research implementation: papers become a Python package with a Rust backend, under constraints to use heuristics rather than dynamic programming. The reported run lasts seven hours, one minute and 40 seconds.

These are examples of work at different starting points, rather than interchangeable evidence that every large task is complete after a particular duration.

The Q&A specification names the users, views, presentation format and service choices. This is the useful part of prompt engineering in the demonstration: make the intended behavior clear. Gabriel initially talks through the idea on his phone, then asks Codex to polish the prompt for presentation. The information matters more than a magic phrase.

For progress that spans hours, Gabriel requests a series of goals in goals.markdown and a progressdashboard.html page. The filenames are his chosen artifacts, not a required format. They make the intended milestones and current work inspectable. Separate threads run code reviews and goal audits at milestones; an audit compares the work with the original goals and can tell the main thread to correct course or reconsider the plan.

Testing can require a machine the main task cannot access. The remote Q&A thread asks a local thread to inspect the application using the Chrome extension, then receives the testing results back. What crosses between threads is a task and its findings; browser access stays on the local machine. Slack updates provide another view of the work: what is done, what remains and which blockers need a person.

How does a remote build get browser feedback from a local machine? The diagram shows the handoff and return path. The remote thread can continue owning the project while the local thread supplies a capability unavailable on the remote host.

A side thread lets a person ask about a running task without turning the main conversation into a status interview. /side opens a place to ask what happened in the last hour, what comes next and what needs help. If the proposed next step is wrong, the side thread can send a correction to the main thread. Side threads are ephemeral, so any decision that should last needs to be communicated back.

The form-builder project also demonstrates communication between two main threads. The remote task tests whether it can spawn a thread in the local project and use that location for future Chrome testing. The local thread then opens Chrome and creates a form end to end. This gives the remote implementation a functional check through the actual browser workflow.

Feedback can update an existing verification task. In the Retrodex interface work, Gabriel sends a screenshot and the blunt assessment “Not very good.” The main agent forwards the screenshot and directive to an already running subagent. The unwanted behavior is specific: messages are pushed to the bottom. The verification task then sends a short message to check the behavior. The important transition is from general dissatisfaction to a concrete interaction that can be inspected.

The progress dashboard shows milestones alongside thread responsibilities, including orchestration, communication, visual UX and the App Server protocol. It also exposes a genuine blocker: the remote Q&A work lacks authentication for Convex and Vercel. Gabriel asks a side thread to explain how to unblock it. A CLI login flow provides Vercel access, while a shell-based setup places the Convex deployment key in the remote environment instead of pasting it into chat.

The internal form builder returns as a working product flow: create a draft, preview it, publish a shareable link and view responses. Hosting it on Sites limits access to people who can access the ChatGPT workspace. Adapting the existing repository therefore changes both how the team distributes forms and who can use them.

Compaction lets a long conversation continue across multiple context windows by carrying forward condensed context. Gabriel connects this capability to model training across multiple windows and offers user anecdotes about long threads. The practical benefit is continuity during hours of work; those anecdotes do not make compaction a guarantee that every important detail survives.

Goals answer a different question: when should the agent stop? At each turn, the task evaluates whether its success criteria have been met and whether it should continue. Those criteria should be specific and verifiable. Charlie's “monkey's paw” warning captures the failure mode: an agent can technically satisfy a request in a way the person did not intend. More time does not repair an ambiguous definition of success.

Delegation has two useful forms:

• Subagents: keep task context separate and return work primarily to the main model. Named roles, developer instructions and model choices can be configured in subagents.toml; a reviewer might prioritize speed or thoroughness.
• Visible threads: keep separate workstreams available for the person to inspect and direct. In Charlie's game project, art, music, animation and mechanics threads all check with a creative-director thread before considering their work complete.

The deciding factor is how much of the delegated work the person wants to follow. Separate context helps in both cases, but visible threads make ongoing coordination part of the user experience.

Hooks put deterministic operations at selected checkpoints. Configured through config.toml in the Codex home directory, they can send conversations to logging, scan inputs or outputs for secrets, inspect proposed tool calls, or run validation when a turn stops. These are distinct from asking the model in AGENTS.md to remember to run a linter: the hook places a programmed check at a defined moment.

The Agents SDK repository provides a simple example: a Python script tidies the repository at the end of each turn. Predictable housekeeping does not need another language-model decision. As tasks run longer with less direct supervision, hooks keep such checks attached to the workflow while goals define what completion means.

Provisioning a remote host introduces waiting that neither the person nor the agent needs to watch continuously. The DigitalOcean plugin starts creating a droplet, then a heartbeat automation resumes the same thread in about five minutes to check readiness. Once the droplet is ready, the thread takes the next step and provides a deep link into Codex settings that preconfigures the SSH connection, creating an SSH key if necessary.

This is a timer-driven continuation: check a condition, wait again if needed, then proceed when the condition holds. The same pattern can monitor a deployment pipeline at five- or ten-minute intervals. Automations that create new threads serve another purpose: a weekly recap or a one-time analytics review can have its own conversation and be archived afterward.

The translator returns as the input to a development loop. A Slack channel contains demonstration feedback, including a request for a sound check before the audience sees captions. Gabriel asks an automation to read and reply every half hour, separating compliments, bug reports and feature requests. Then he steers the task while it is running: bugs and feature requests should also create a new thread in a worktree, address the issue, open a PR and request review.

What must happen between a Slack comment and a reviewable change? The diagram separates intake from implementation and review. This is the proposed automation workflow, rather than a demonstration of a completed sound-check feature. Further steps could build a test version, send it to someone and return their feedback to the PR reviewer.

Automations can also maintain the context used by later work. One shown automation checks the presentation through Chrome every five minutes and estimates whether the workshop will overrun. Other proposed uses review conversations for skills to improve or remove, or compare drafted emails with the replies the person actually sends. In the email example, a second automation writes lessons into a Markdown file that the drafting automation can use next time.

Charlie's own notification workflow reads updates from Slack, email and Linear and maintains daily notes and workstream notes in a local Obsidian vault. Later requests can draw on those notes to explain what changed since the last check-in. The timer therefore does more than repeat an action: it keeps a useful account of ongoing work available for the next conversation.

The App Server protocol connects interfaces to the core Codex harness. The workshop describes it as underpinning the Codex app, CLI and VS Code extension, carrying capabilities such as tool calls, plugins, compaction and steering. An application built on App Server can let users work with their ChatGPT subscription and token budget. Capabilities that depend on the local computer, such as computer use, are not automatically supplied by embedding the core harness.

Retrodex is the fourth overnight project: a custom interface built on App Server. It offers model selection and reasoning-effort controls, with increasingly energetic visual effects as effort rises. Gabriel sends “Hello,” allows access and receives a reply. The attempted follow-up display of matching threads in the Codex app is inconclusive on stage; the demonstrated result is the request and response through the custom interface.

The interface is playful, but the integration point is practical. A team can put Codex inside its own product or internal agent platform while retaining the core harness capabilities. Gabriel suggests inspecting the open-source Codex repository and asking Codex to explain its App Server as a starting point for building such integrations.

The closing questions shift attention from a successful request to the process around it. What should a loop check at each turn? What stopped the agent: missing context, missing permission or an unavailable capability? Can the solution become a skill or plugin for the next run? How should work be divided across agents or visible threads? These questions connect the workshop's individual features to the recurring problems of development.

Human involvement also needs a reason. API-key creation once looked like an unavoidable manual step; the developers plugin can handle it in the demonstrated workflow. Authentication for the remote projects still needed Gabriel's intervention. The useful decision is to retain people where their participation is needed and remove routine handoffs that only interrupt the work.

The translator makes the larger ambition concrete. A conversation becomes a build; a live audio exchange checks its behavior; Slack feedback becomes input to a proposed implementation-and-review loop. Charlie calls this building “the machine that builds the machine”: a software factory that produces individual changes through a repeatable process. Its useful checks still have to be designed—success criteria, testing, review and the handling of blockers do not appear merely because the loop runs.

The recording ends by moving from the tour into a hands-on build session, with OpenAI staff available to answer questions and unblock work. That is the natural next experiment: pick a real task, provide its context, observe the result, and turn the parts that worked into a process worth repeating.