ChatGPT is poorly designed. So I fixed it

Why should asking for an email by voice make it harder to work with the written draft? Ben Holmes opens with a broader frustration: an app he describes as one of history’s fastest-growing has become surprisingly confusing to use.

The first sign is two separate voice controls: one for voice-to-text, another for voice-to-voice. Holmes opens the conversational mode and exchanges greetings. The response sounds natural enough that he comments on its vocal fry. Then he asks for something more useful: a group email expressing excitement about the AI Engineer World’s Fair and suggesting a lunch meetup.

The assistant starts reading a suitably worded email aloud. But listening to a draft is different from collaborating on it. In the interface shown, Holmes has to end the call and find the transcript to get the formatted written response. The draft exists; the interaction makes him leave one mode to use it in another. This is the historical interface in the recording: OpenAI later announced integrated voice, text and visuals in its November 2025 release notes.

Voice and text ought to be complementary parts of one conversation. Instead, Holmes says, they feel like products built by different companies. He connects that feeling to Scott Hanselman’s phrase “shipping the org chart.” In the clip Holmes plays, Hanselman describes sitting in an electric vehicle and noticing different fonts on the map, climate controls and speedometer. Realizing that he is looking at three Android tablets chained together makes the manufacturer’s organizational divisions suddenly visible.

Model selection creates a similar problem. Holmes sketches a sequence in which a researcher ships a technical improvement, consumers unexpectedly want it, and marketing has never been consulted. This is his critique of how products reach users, rather than an established account of OpenAI’s internal process. The resulting menu becomes a science fair of names: o3 and GPT-4o, followed in his joke by Xbox 360, One Series S and Pro. The user inherits the work of interpreting the labels before accomplishing a task.

The redesign has two goals: allow voice and text at the same time, and choose a suitable model from the user’s request. Holmes proposes using GPT-4o Realtime for the live audio conversation, with tool calls providing the other capabilities. A text tool can deliver links or long drafts; a research tool can hand a question to a more capable model and return its answer. The Realtime API supplies the conversational foundation without requiring every task to be completed through speech.

The interface starts in a dormant state. Pressing the voice button opens a familiar floating orb, with mute and end-call controls. A new chat button opens an iMessage-like panel while the call continues. Holmes compares it to texting a friend during FaceTime: the call controls stay at the top, and the conversation remains available underneath.

The panel includes a reminder of the spoken request, followed by the detailed written response. An email draft can therefore be read while the voice conversation remains active. The selected frame shows a gray request labeled “From voice” above a blue email response beginning “Hey Cora team.” With the output problem addressed, Holmes turns to a second question: what should happen when the request needs more work? He introduces Warp Terminal, a developer tool for writing code across environments.

In Warp, Holmes asks to undo his last commit. The request goes to a coding agent that prepares a terminal command. The visible interface includes a command approval panel and a Run button: it shows the proposed action awaiting approval, rather than establishing that the undo completed.

He then raises the scope to refactoring the entire codebase to use Flutter. For this request, he describes Warp detecting the complexity and using a reasoning model to write a plan before proceeding. The distinction is between choosing an immediate action and first working out a sequence of changes; producing a plan does not establish that the full refactor succeeds.

Holmes suggests that explicit requests for detail or tradeoffs can serve as simple routing signals. While the reasoning model works, the interface can communicate how long it is thinking, then return the fuller response. The user expresses the depth of help they need without first having to select a model name.

The prototype demonstration brings the two changes together using off-the-shelf APIs. After a brief greeting, Holmes says he has just landed in California and asks for a link to a park he should visit. The assistant replies aloud that it will send a link to a popular park. A link is a useful test of the combined interface: it belongs in text, even when the request arrives through speech.

Holmes follows up by naming Yosemite and asking for its history: “Go deep.” The assistant gives a short spoken introduction, mentioning Native American heritage and Yosemite’s establishment as a national park, then directs him to the chat for more detail. Speech maintains the conversation while the written channel accommodates the longer answer.

The written output is handled through a send-chat-message tool. Its description tells the conversational model to send details that are easier to explain in text. Holmes reports that this description was sufficient to elicit the behavior in his demonstration; he did not need an additional system prompt for it. The tool makes text delivery an action available during the voice conversation.

A separate tool handles requests that need deeper reasoning. The visible definition names it ask_reasoning_model and requires a string field called details. A compact Dart definition expresses that contract:

final reasoningTool = <String, Object>{
  'type': 'function',
  'name': 'ask_reasoning_model',
  'description':
      'Ask a reasoning model for a deeper answer when the user '
      'wants to explore a topic in detail.',
  'parameters': <String, Object>{
    'type': 'object',
    'properties': <String, Object>{
      'details': <String, Object>{
        'type': 'string',
        'description':
            'The question and context needed for the deeper answer.',
      },
    },
    'required': <String>['details'],
  },
};

For the Yosemite follow-up, details needs to carry the subject and the request for an in-depth history. The tool description supplies the routing cue; the argument supplies the work to hand off.

Once the reasoning result arrives, the application has two delivery options:

• Return it to the conversational model: let the voice interaction continue with the information it needs to answer.
• Send it directly to the client: place the detailed result in the app’s written conversation.

This completes the design: live audio remains the conversational entry point, while tool calls deliver readable material and invoke deeper reasoning when requested. Holmes closes by pointing viewers to FixGPT, the source code on his GitHub.