AI and Human Whiteboarding Partnership

On January 1, 2020, Christopher Chedeau was working at Facebook and avoiding performance reviews. He needed to write feedback for colleagues and his manager, but knew he would leave it until the deadline. Instead of spending the intervening time scrolling on his phone, he turned to a blog post he had wanted to write. He called this active procrastination: if the required work was not going to happen yet, something useful could still happen.

The post needed handwritten illustrations. In Chedeau’s recollection, the drawing website Zwibbler was down, so the side project acquired a side project: how hard would it be to build a replacement? He opened CodeSandbox and used Rough.js to make a drawing prototype. Chedeau recalls about 200 likes on the first tweet about the demo, along with enthusiastic replies. That prototype became Excalidraw.

Today, he would begin that experiment with a coding agent. At an Anthropic keynote two weeks before the talk, a Claude agent had been demonstrated implementing an Excalidraw table layout—what Chedeau described as the most requested feature. A demonstration was not yet a contribution to the project, however: at talk time, he was still waiting for a pull request.

Three months after Excalidraw began, COVID sent people home. Teams that had collaborated around office whiteboards needed a replacement. Users started telling Chedeau that his tool worked well for online whiteboarding, which surprised him: he had built it for blog illustrations. Their use of the product prompted him to investigate what existing whiteboarding tools offered.

In his retrospective account, the tools he encountered reproduced physical whiteboard limitations too literally. These had been small, lightly used projects before the sudden demand for remote collaboration. Excalidraw’s ordinary digital editing operations removed several constraints at once:

The improvement was not simply putting a board on a screen. It was making the board behave like editable digital content.

A change of medium creates an opportunity to change the interaction. Chedeau sees AI integration at a similar stage: teams often take an existing workflow and add AI without reconsidering how the task should work. The difficulty is that established digital editing conventions already offered a guide for virtual whiteboards, while useful AI interaction patterns are still being discovered. Excalidraw’s experiments provide concrete examples of what that discovery entails, including capabilities that worked technically but did not belong in the product.

The first experiment came from Görkem at fal.ai. A sketch containing objects such as a palm tree and a sun drove live image generation: as the drawing changed, the generated picture reacted. The compelling part was the immediate connection between a rough sketch and a realistic image.

But when the team explored integrating it into Excalidraw, the intended use did not match what people came to do. Users generally were not drawing realistic pictures. A model could perform its task well and still add little value to this particular editor. Model capability is not product fit. Under pressure to add AI, it is easy to ship a feature because the model exists; Chedeau’s warning is that an irrelevant integration can make the overall experience worse.

People did draw diagrams, so generating diagrams looked like a better fit. The team first tried direct generation, with poor results. The house illustration shown in the talk was generous compared with the actual outputs, which Chedeau says did not even resemble a house. That was a finding from their experiments, not a permanent limit: the day before the talk, someone working on Gemini had shown him a demo producing Excalidraw files, giving him reason to revisit the approach.

The approach that had worked for the team used Mermaid, a textual language for describing diagrams. Instead of asking the model to produce the final drawing directly, they asked it to express the diagram’s structure. The small example connects AI to Engineer in a left-to-right flowchart:

flowchart LR
    A[AI] --> B[Engineer]

Here, LR specifies direction, A and B identify nodes, the bracketed text supplies their labels, and --> connects them. In the team’s experiments, LLMs could generate this kind of syntax successfully. The diagram’s meaning could therefore travel through a compact textual representation.

There was still an output mismatch. As Chedeau describes it, Mermaid produced SVG or Canvas output, while the editor needed Excalidraw objects. An Excalidraw backend for Mermaid supplied the missing conversion step. The resulting path was:

1. Accept a prompt describing the desired diagram.
2. Have the LLM generate its Mermaid representation.
3. Convert that representation into Excalidraw content.

The simple prompt starts with AI and an engineer, but Chedeau also reports people creating organization charts, TCP and network-request diagrams, and architecture diagrams involving classes and inheritance. The intermediate language let the model describe relationships while the conversion handled the application’s output format.

The useful result is an actual Excalidraw component that the user can manipulate. A generated image can look almost right yet leave the person stuck with details they cannot change directly. Native diagram content gives them a practical next step: use the editor. In the demonstration, that means moving elements to change the layout, then adding colors to make the result match the intended presentation.

Editable output makes an approximate draft useful. The interaction no longer depends on obtaining a perfect answer in one request. AI provides a starting point, and the person finishes it with familiar tools. At talk time, this was a one-way handoff from generation to manual editing. Chedeau’s proposed next step was to send the edited work back to the AI for further changes, alternating between the two; that repeated revision loop was not yet available.

Another integration addresses a less spectacular but familiar problem. Excalidraw’s SaaS offering manages multiple files and synchronizes them in the cloud. Without deliberate naming, the file list fills with documents such as Untitled one, Untitled two, and Untitled three. An LLM can use the document’s content to suggest a more useful name.

The timing is part of the design. The application waits until the user stops working, so there is content to name, and then presents candidate titles. The user clicks a suggestion to accept it. This removes much of the naming burden without requiring the person to interrupt drawing to write a prompt, and without treating a generated title as automatically approved.

The earlier realistic-image experiment did not rule out image generation altogether. Users wanted illustrations and infographics inside their diagrams and slides. Generating those supporting visuals served a different task from turning a canvas into a realistic picture. The illustration feature was in closed alpha at talk time, with plans to open source it and ship it within a few weeks. The displayed example shows a hand-drawn brain connected by curved cables to four server racks.

Logo preparation was another concrete source of friction. Chedeau regularly wanted to place logos in diagrams and slides but could not find a reliable way to remove their backgrounds. His example began with a screenshot of the AI Engineer World’s Fair logo from the event website. He challenged the audience to build reliable logo background removal that could run in the browser, offering to integrate and ship a working contribution in Excalidraw. This was an open request with a deployment condition, not a demonstration of a completed feature.

The transition from physical to virtual whiteboards succeeded by changing what users could do: type instead of trace, select instead of erase pixel by pixel, and copy instead of redraw. AI presents another opportunity to discover interactions suited to a new capability. But the criterion remains the customer’s task. A model belongs in the product when it helps people accomplish what they came to do, not merely because it can be integrated.

The technical lesson Chedeau closes with is to give the model a structured target. Find a domain-specific language that represents the problem, have the LLM generate that language, and use ordinary code to turn it into the application’s native output. In Excalidraw’s experiments, Mermaid supplied that bridge. The broader recommendation is an engineering strategy rather than a guarantee: let the model express the structure, let non-AI code perform the conversion, and deliver an artifact the person can continue working on.