Burn Your Flags: Interactive CLIs for Agents — Mark Lummus & Navinkumar Patil, PayPal

An interactive command can stop making progress without returning an error. In this PayPal workshop, Mark Lummus, Head of Developer Tools, and Navinkumar Patil, the developer tools tech lead, start with that quiet failure: an agent shells out to a CLI, but its harness does not handle the interaction that follows. The command is running. The interface has become unusable to its new operator.

What does each operator receive? The opening example is an arrow-key prompt. A human sees a question and navigates its choices. The agent receives a redraw loop, and the harness loses the UI state. The important distinction is between receiving terminal output and retaining the question that output represents. Here, the latter fails silently.

Adding command flags is the obvious repair, and Lummus gives it a clear place: a single pass through a known workflow whose inputs are independent. Interactive workflows become harder to flatten when one answer determines the next question. A CLI might also call an API and ask a question based on the response. Enumerating options in advance then has to account for decisions that only become meaningful during execution. Guidance and guardrails already present in the interactive flow can disappear in that translation.

An agent can potentially follow that branching process: reason about a response, ask, retry or back up. A fixed automation script has its decisions arranged ahead of time. The design question is therefore how to expose the decision tree to an agent that cannot reliably read the human interface. Lummus borrows the model of accessibility: “the same discipline with a new reader.” Preserve the guidance while giving the new reader a form it can use.

Barista 9000 is the workshop’s deliberately troublesome coffee-ordering CLI. Participants receive the CLI, change it, run it through an agent and submit information to a server that updates a leaderboard. A Mac Mini supplies the incentive. The desired outcome is simple enough to recognize—brew a coffee—but the original interactive interface is expected to fail when operated through an agent.

Patil places the exercise in a longer development-tool story: Eclipse, IntelliJ, VS Code and then AI editors such as Cursor and Windsurf. CLIs persisted through those changes; agents have given them renewed importance. Participants download the bare CLI from PayPal Cafe, register a name and try to complete an order. The init command handles registration. This is a working exercise, with hints introduced after participants have had time to encounter the problem themselves.

The hints are deliberately called “good ideas,” rather than best practices. Lummus treats this as an early design space where the audience may discover better approaches. That framing matters: the workshop offers ways to make the interface accessible, without claiming a settled standard for every agent harness.

Model choice adds another constraint. Lummus expects a frontier model at its highest thinking setting could probably brute-force the challenge, but asks participants to try Haiku or Sonnet, or a comparable faster, cheaper model. The useful engineering question is whether a clearer interface can spare the token budget otherwise spent navigating a CLI. This is a proposed challenge, rather than a measured comparison of models.

At roughly 26 minutes, Patil introduces tmux as the lowest-effort starting point. Begin with the CLI’s help command so the agent can discover how to operate it. Then use a terminal session that remains available for subsequent input and inspection, rather than treating the interactive command as a single request that must immediately finish.

How does the agent act on that session and observe the result? The two useful operations are sending keys and capturing a pane. Sending keys supplies the input an interactive program expects; capturing the pane returns the terminal contents the agent can inspect before choosing another input. The example also introduces detached execution, allowing the CLI to keep running while the agent works through these separate operations.

Separate sessions can hold separate long-running tasks, including work associated with multiple agents. The immediate advantage is continuity: the running process and its terminal remain available to revisit. The remaining cost is interpretation. The agent still has to recover meaning from terminal presentation, which motivates the next hint.

An IPC side channel gives the agent a structured way to communicate with the CLI. Patil’s analogy is a website with a REST API: the visual interface serves humans, while the API exposes machine-readable information. The CLI can similarly serve both operators without asking the agent to infer everything from a terminal redraw.

The operating principle is compact: “Read state, send input as a data.” JSON makes the prompt’s meaning explicit. Instead of locating an option on screen and translating it into arrow presses, an agent can receive a description of the current question and submit an answer as data. Patil introduces barista brew --agent in this discussion as an agent-facing mode; the explanation does not specify the underlying IPC transport.

What information must survive that change in representation? The coffee-size example includes a field, the choices small, medium and large, and a default of large. Those details tell the agent what it is answering, which answers are available and what the default means. They are useful input data, rather than decoration around a human prompt. Exposing them in a machine-readable format removes a layer of screen interpretation while retaining the choices.

The distinction between the first two hints is practical:

• Terminal access: send keys and inspect a captured pane while keeping the interactive process running.
• Structured access: read explicit state and submit input as data, reducing dependence on the rendered terminal.

The second approach requires an interface for structured communication, but it gives the agent information closer to the decision it must make.

Structured choices alone do not explain every decision. Context injection supplies information the agent needs to interpret those choices and ask useful follow-up questions. Patil describes providing the information together in an input file, rather than relying exclusively on the sequence of options a human sees.

The example is low oat-milk stock. The agent should receive the constraint and guidance about what to confirm with the user: is a smaller amount acceptable, and should it choose a backup? Patil mentions two gallons remaining and a question about two ounces; these quantities illustrate inventory context and a possible serving confirmation, without defining a complete stock-allocation rule. The useful mechanism is making the shortage and the required confirmation available before the agent chooses how to proceed.

The three hints address different needs. tmux supplies continuing access to the terminal. A side channel supplies clearer feedback and structured interaction. Context supplies the scenario that makes a particular question necessary. They can complement one another: a communication path gets the agent to the decision, while contextual information helps it make that decision sensibly.

As scores approach 5,000, an audience question exposes the ranking rule. The intended order is score first, then response time: an equally accurate response should rank higher if it arrives sooner. But several entries appear to have completed three of three in zero seconds. Lummus flags that display as unexpected. The zero-second values do not establish instantaneous completion; the timing issue remains unresolved in the discussion.

When the floor opens, a participant identifies a more consequential limitation: the benchmark is easy to game by fulfilling one order with very low latency. A deterministic parser can handle that narrow task, and placing it close to the server can shorten the network trip. The participant says this is what they are building. Success can therefore come from specializing the solution and reducing network latency, without demonstrating better general navigation of an interactive workflow.

Lummus accepts the benchmark criticism while finding a useful design decision inside the workaround: keep the CLI deterministic. The LLM already supplies a variable decision-maker. The tool should remain a “small, sharp instrument,” focused on a predictable operation. Putting another agent inside that tool adds another source of uncertainty for the outer agent to manage.

That recommendation also qualifies what predictability means for the earlier API-driven examples. Live stock or API responses can change the available choices; a useful CLI still needs consistent handling of its current inputs and state. Interactive branching and predictable tool behavior can coexist. The reasoning agent chooses a path through the workflow, while the CLI carries out its defined operations.

The leading participant’s explanation makes the distinction between building and running especially clear. Their agent uses the Claude Agent SDK, with Slack as the control interface. An existing evaluation-bench skill creates an environment and optimizes the solution. For this challenge, the agent builds a deterministic CLI solver, then runs it in tmux against the APIs.

The participant reports one response at about 29 milliseconds and suspects the client and server are in the same region. That location explanation is their hypothesis. More importantly, when Lummus asks about model choice, the participant clarifies that the completed CLI is deterministic: the agent built it and now runs it through tmux. Model reasoning is used to construct the solution; it is no longer required to decide each order in the resulting solver.

This result answers a narrower question than the workshop’s original challenge. It demonstrates an agent producing a fast, specialized program and operating it in a persistent session. It does not establish that a cheaper model can now navigate arbitrary interactive prompts. The benchmark has made an alternative economical: use the agent to build away the repeated reasoning task.

Danny, another participant, reaches the exercise from a different starting point: no previous CLI-building experience and little coding knowledge. Using Claude Code in VS Code, Danny gives the agents the problem and asks them to read it, explain it and fix it. The interaction is goal-driven, with the agents handling the implementation details.

After the first working result, the request becomes concrete: make it faster and suggest improvements. Danny reports two scripts running, with the first scoring 4,998, then asks for a copy and suggestions to reach 4,999. The recording does not reveal the script changes, so this example supports a development pattern rather than a particular speed optimization: obtain a working solution, inspect the result, and ask the agent to pursue the next specific outcome.

Lummus recognizes that many participants have chosen to ask an agent to build the CLI itself: “Make it fast. Make it deterministic.” The workshop ends with that audience contribution alongside the presenters’ original recommendations—give the agent useful context, a side channel and access through tmux. Those remain “good ideas to start with.” The competition adds a practical distinction: sometimes the agent needs a better way to follow an interactive decision; sometimes it can build a predictable program that handles the narrow task directly.