Books reimagined: AI to create new experiences for things you know — Łukasz Gandecki, TheBrain.pro

Reading a book about Donald Trump’s re-election, Łukasz Gandecki kept encountering people he did not recognize. As a reader from outside the United States, he lacked some of the background the book assumed. He built a small AI companion to help him keep track of its cast.

The prototype looked terrible, but it supplied something useful: images of the people on the current page and short explanations of who they were in that page’s context. The page gave the companion a narrower job than answering arbitrary questions about the book. Within a month, this experiment had begun turning into Books Reimagined, a broader reading experience.

In the first experience, based on The Snow Queen, the sorcerer’s apprentices fly upward carrying a mirror that distorts reality. Narration and music accompany their ascent toward a distant heaven. When the mirror crashes and shatters, the distortion spreads through the surroundings. The audiovisual changes follow a particular event in the text, making the scene itself the organizing unit of the experience.

The audience cannot hear the intended narration and music in the recording’s demonstration, a limitation Gandecki acknowledges. This first experience is in Polish; he then switches to an English version of 1984 built for the conference. That moves the demonstration from an immersive scene to interactions with the book.

While reading 1984, the reader can send a voice note asking what is happening in the current scene. Venue audio again prevents Gandecki from demonstrating the audible exchange. He connects the design to familiar voice-assistant frustrations: delays, speech that starts at the wrong moment, and interruptions. He criticizes Siri and says the previous day’s Google demo was introduced as working 50% of the time, without identifying the task or evaluation conditions.

Press and hold makes the turn boundary explicit. The reader holds a control while speaking and releases it to request a response. The interface therefore gives the reader direct control over when their question is finished. Gandecki claims a response within 100 milliseconds of release; the failed audio demonstration and absence of measurement details leave that latency unverified.

The interaction also carries reading-position context across questions. After asking about one scene, the reader can scroll forward and ask for a summary of what happened between the previous question and the current position. This makes the earlier question a useful reference point for the next one. Gandecki points the audience to bookgenius.net to try the experience.

Readers often remember what happened without remembering the words used to describe it. In 1984, someone might want the scene where Winston met O’Brien. Exact search depends on matching the text; embeddings make it possible to retrieve a passage from a description of the remembered event. The reader can jump to the result, read around it, and then return to the original reading position.

A broader question requires a different operation. Asking about the ways Party propaganda works starts a deep-research task that, as Gandecki describes it, reads the book only up to the point the reader has reached.

The stopping point makes the research request bounded: the system should build its answer from material the reader has already encountered, avoiding later revelations.

Gandecki says the deep-research request will take a couple of minutes, then returns to the presentation without showing its answer. He next explains how the product emerged: the initial implementation was rapidly written vanilla JavaScript, confusing internally but useful for fast iteration.

Early in a product like this, much of the work is discovering what is worth building. Gandecki’s approach is to describe an idea to AI, generate something usable, and see where the experience disappoints. That feedback can arrive sooner than an exhaustive plan, especially when the requirements themselves are still uncertain.

Generated code also changes the emotional cost of experimentation. Throwing away code you carefully crafted can feel like abandoning an investment; Gandecki finds it much easier to discard AI-written code and try again. He describes development as waves of change: initially, the product may provoke sweeping rewrites, but their amplitude gradually decreases. Once whole-codebase changes stop happening day after day, he begins adding tests and refactoring. The transition to conventional engineering follows a more stable product direction.

The worst-looking code may still be a poor refactoring target. Gandecki’s OpenAI audio-processing code was hastily written JavaScript: untyped, confusing, and tempting to clean up. But he never needed to touch it. Its appearance created less practical trouble than code that repeatedly obstructed changes elsewhere.

His prioritization heuristic combines three factors: how bad the code is, how painful it is to work with, and how easy it would be to improve. Adding those factors can make ugly, easily changed code look attractive even when it causes no pain. Multiplication makes a negligible factor suppress the priority:

The same heuristic fits directly into JavaScript:

function refactoringPriority({ badness, pain, easeOfChange }) {
  return badness * pain * easeOfChange;
}

This is a decision aid, with no prescribed scoring scale. Its useful property is that zero ongoing pain produces zero priority, even when the code is bad and the cleanup is easy.

Gandecki contrasts the book experience with common AI product categories: ChatGPT wrappers, image generators, and unreliable voice assistants. For Books Reimagined, the production goal is to hide the AI inside a finished experience. AI makes the initial draft of an enriched book, and people complete the work.

That human contribution is primarily judgment. Gandecki argues that generated music still needs someone to decide whether it sounds good, graphics need assessment for visual appeal, and an avatar must fit the character it represents. Combining these simple building blocks requires attention to how they feel together. The reader should encounter a coherent book experience without having to manage the generation process.

Each ingredient has a familiar counterpart, but integrating it with the reader’s position changes its usefulness:

• Questions: A knowledgeable friend can explain a book, but may not be available whenever the reader needs help.
• Search: Natural-language, spoiler-free retrieval supports remembered scenes without requiring exact wording.
• Graphics: Images establish the book’s mood and help the reader recall its characters.
• Music: A soundtrack that follows the scene can shape emotion as film music does, while the reader continues through the text.

The experience comes from coordinating these capabilities around reading.

This suggests a product opportunity in small niches within familiar activities. Gandecki’s invitation is to look for experiences that were imaginable before AI but too expensive to produce at the necessary level of detail. He speculates that manually producing the graphics and music could cost hundreds of thousands of dollars per book; he supplies no itemized production budget. The economic premise is that generation makes a richly produced draft feasible, leaving people to shape its quality.

The production pipeline connects literary analysis to assets and player behavior:

1. Analyze scenes and detect characters. A combination of LLMs identifies the book’s scenes and the people in them.
2. Set an overall musical direction. For Sherlock Holmes, Gandecki’s example is Victorian London, with noir music and a somewhat sad tone.
3. Generate music from scene and mood. Scene analysis and mood detection guide music generation within that broader direction.
4. Produce structured XML metadata. The analysis also becomes information the player can use alongside the text.

The markup example makes that final step concrete. The displayed passage wraps BIG BROTHER in Big-Brother tags and Winston’s name in Winston-Smith tags. Character mentions become explicit identifiers associated with the text. The player can then use the mapping to display avatars for the characters in a scene. Structured metadata connects the generated analysis to visible reading behavior.

Annotating these details throughout an entire book would take substantial human effort. Automating the mapping gives the player the information it needs without requiring a person to tag every mention manually. Gandecki closes the technical demonstration by announcing that the team is open-sourcing the player that day so others can create Netflix-style experiences for books. The recording supplies no repository or license details; the announcement extends the idea from a single reading application to a player others could build experiences for.