AI Engineer World's Fair 2026
LLM Knowledge Bases: a practical guide
Read the talk
LLM Knowledge Bases: a practical guide
Start with a folder of rough Markdown notes, then use agent skills to add links, build focused wikis, schedule maintenance and generate views of what you know.
From a talk by Ben Holmes
Before you start: Basic familiarity with Markdown files and an AI coding agent is helpful; no existing Obsidian setup is required.
Start with the notes you already lose
How do you turn a disorganized Apple Notes folder into something you can actually navigate—and rediscover ideas you have forgotten? Ben Holmes, who introduces himself as Developer Relations Lead at Warp, starts with that familiar problem. Even adopting a dedicated notebook such as Obsidian does not guarantee that organizing it becomes a regular habit. The useful starting point is a place to capture thoughts, including material developed with LLMs, without first deciding where everything belongs.
The storage layer in this demonstration is simply a folder of Markdown files. Holmes writes and browses them in Hubble, his own notes app, but the same starting point works with Obsidian, Warp or another Markdown viewer. Enrichment adds category tags and links between related notes, making them easier for both people and agents to find. A wiki then organizes sources, people and concepts into browsable entries. Finally, an agent with access to the files can generate HTML visualizations of their relationships. Each layer builds on the notes underneath it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Capture enough raw material
The first improvement is to make capture easier than opening a note and typing. Voice dictation can be awkward around coworkers, but it removes much of the effort of getting a thought down. Holmes estimates dictation at about 200 words per minute and expects it to outpace typing for most people. Treat that as his estimate, rather than a measured average for composing notes.
Wispr Flow is the familiar subscription example, but a subscription is not essential. Holmes highlights Handy, an open-source dictation tool that runs a local model on the device. This makes speech capture a capability you can add to your existing writing environment.
His own choice is VoiceInk. Holmes tentatively recalls a roughly $20 lifetime fee for updates; the pricing page checked on August 28, 2026 lists $29 for one Mac with lifetime access and updates. He also describes using a mobile app. In the desktop demonstration, holding an Option or Function shortcut starts dictation, and the output arrives as punctuated sentences and paragraphs. The recording cuts off a little at the end of the dictated passage, but the formatting is visible.
The longer example is a reflection dictated after listening to Acquired's episode about the founding of Walt Disney. What appears as several paragraphs began as one continuous transcription. Holmes uses the same approach on his phone and computer to collect raw notes. The important input to later wiki generation is plentiful source material, not immaculate formatting: a rambling reflection, a meeting transcript or observations after reading a research passage can all give an agent something useful to work with. Capture the thought while it is available; organization can follow.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn enrichment into a repeatable skill
A second podcast note, this time about Ferrari's founding, shows what enrichment adds. The central text still reads like a dictated reflection. Around it are tags identifying the material as a podcast and connecting it to focus areas, a source URL found by the agent, an enrichment timestamp and links to related podcast notes. The note remains recognizable, but it now has routes into and out of the rest of the notebook.
Holmes packages the operation in an agent skill called enrich note. Its instructions make the recurring decisions explicit:
- Record the enrichment time. A later agent can recognize that an earlier pass has already happened.
- Choose tags from a shared reference folder. This supplies a concrete vocabulary instead of letting each pass invent its own categories. Holmes explicitly asks Claude to resist creating tags, while allowing the list to grow when a real pattern warrants it.
- Research the source. Use web tools to locate the original material; Holmes suggests Exa when a harness needs search capability.
- Find related notes. Use file operations and key-term searches, then append links to those notes at the bottom.
The shared vocabulary matters because tagging is useful only when related notes converge on the same categories.
A compact Markdown instruction file can express that contract:
markdown
# Enrich note
Enrich the supplied Markdown note while retaining its original text.
- Record when this enrichment pass occurred.
- Read the tag vocabulary in the reference folder.
- Prefer existing tags. Add a new tag only when a recurring pattern
justifies extending the shared vocabulary.
- Use web search to locate the original source and add its link.
- Search the notebook for related terms and ideas.
- Append links to relevant notes in a Related notes section.
This separates the reusable operation from whichever agent harness invokes it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Run it on a note, then browse the connections
To invoke the skill manually, Holmes uses Warp, the terminal and agent environment he works on. He describes its support for open-weight models and switches to GLM 5.2, which he recommends for this task. The invocation itself is straightforward: call enrich note, copy the target note's file path into the request and ask the agent to enrich that file. The skill does not depend on this particular harness.
The same operation can run across the notebook. A batch request looks for notes without an enrichment marker, applies the skill to those files and saves the modifications. The timestamp added during enrichment therefore serves a second purpose: it gives subsequent runs a way to select unfinished work. Holmes leaves the live single-note run working in the background while moving on to the next layer.
Even without a wiki, the links can be enough. Holmes opens notes taken while reading The Left Hand of Darkness: observations scattered across different chapters have become related entries he can click through. Instead of remembering where he saved something two weeks earlier, he can follow a topic from one note to another. This creates the experience of a Wikipedia rabbit hole through his own thoughts. A generated wiki is an optional step for making that material more browsable, rather than a prerequisite for getting value from it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build a wiki around an area of interest
The next demonstration is a wiki about developments in AI. Its raw inputs include podcast notes, research documents and model system cards. The generated index separates sources, concepts, people and organizations. A Ralph Loop rabbit hole appears among the topics. Jazz musician Adam Neely appears among the people because Holmes captured material about AI's effects on the music industry. A focus area can therefore pull in relevant connections from outside its obvious boundaries. Holmes says the organization was generated programmatically; his time went into collecting the inputs.
Holmes bases this layer on Andrej Karpathy's LLM Wiki idea. The starting relationship is a raw directory feeding a wiki organized around a chosen focus. The gist supplies an organizing approach, rather than a turnkey implementation of Holmes's notebook. His request combines three ingredients: the interests he wants covered, the location of the raw folder and the gist's instructions. He then asks the agent to generate the wiki, adapting the structure to his own interests.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Keep people connected to their source context
A second wiki applies the same structure to notes from the Bible in a Year podcast. Here the practical problem is remembering unfamiliar characters with similar or hard-to-pronounce names. Generated person entries describe who each character is and reference what they did, providing an entry point into the accumulated notes.
The workplace analogue is a people section built from meeting notes: colleagues, meeting participants or clients encountered in customer-success work. Holmes proposes linking each person to related meetings and source material. An entity page becomes a route back to the raw record, so browsing a synthesized description does not mean losing access to the notes that supplied it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Move maintenance into the background
Generating and updating these files takes time—the live enrichment is still an example of work that need not occupy the user's attention. Once the skills exist, note enrichment, wiki generation and wiki updates can run on a daily or weekly cadence. The execution location determines whether that schedule depends on the user's computer being available.
| Execution option | Where the work runs | Practical dependency |
|---|---|---|
| Codex app automation, as described in the talk | The user's machine | The laptop must be available to run it |
| Warp's Oz | A cloud environment | The environment needs the notes and agent instructions |
Holmes describes the local Codex setup as requiring his laptop to remain open when the task runs. He presents Oz as the cloud alternative, with schedules and other configurable triggers. Slack messages and iMessage are examples he suggests; iMessage should be understood as a proposed trigger to configure, not an established built-in integration. His demonstrated wiki job uses a schedule with the earlier skill encoded in its instructions.
The cloud workflow is sync, process, sync:
- Bring the Markdown folder into the execution environment. Holmes describes a cloud box, optionally powered by Docker, with access to the notebook.
- Provide synchronization. His setup uses the Obsidian headless CLI to bring the files down and send changes back. A Git clone from a GitHub repository is an alternative; he prefers background synchronization to managing note pushes and pulls himself.
- Run the agent operation. For enrichment, instruct the agent to process only notes that have not yet been enriched. It performs the searches, tool calls and file edits specified by the skill.
- Synchronize the changed files back. The local notebook can then receive the results.
The headless synchronization tool is the component Holmes names for his cloud setup; the demonstration does not supply installation or command syntax to reproduce it.
In Holmes's setup, Oz starts the cloud sandbox each day. The wiki job tells the agent that the Obsidian CLI is already installed, asks it to pull down and update the wiki, supplies instructions specific to his notebook, and requests a final sync. A separate schedule runs enrich note across the raw notes. That separation lets the same capture folder support both related-note navigation and a maintained synthesis layer.
The resulting run is inspectable in the browser. Holmes opens an overnight enrichment run that connected notes captured during the preceding day. The intended morning experience is a refreshed wiki ready to read, like a personal daily paper assembled from material he chose to capture.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Return to the live result
Returning to the earlier live run closes the loop. The rough podcast note now has a heading and a source link found through web search, so Holmes can return to the episode. Its related notes include the Ferrari podcast and other founder-podcast material. This is the completed result of the request launched earlier, rather than a proposed output: phone dictation supplied the raw text, and the agent added the source context and paths to nearby ideas.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build views beyond Markdown
Once the notes are accessible as files, the output need not remain a collection of Markdown pages. Hubble can also display HTML. Holmes asks an agent to take the Markdown files and build a graph using HTML and Tailwind, giving him a bird's-eye view instead of requiring him to follow wiki links one at a time. The graph is something the agent generated for his notebook, rather than a separate graph tool he installed.
The view groups stray thoughts, books, startup founding, AI and engineering, and faith and scripture. These clusters make areas of interest visible. Holmes also asks for clickable nodes: selecting one exposes the note and its connections, allowing a broad overview to lead directly into a particular idea. Walking the graph can reveal both recurring interests and gaps worth exploring.
Because the view is generated, its presentation is also changeable. Holmes asks to put the graph in space, and the next version presents the connected notes as a navigable star constellation. He then extends the idea from relationships to habits: a burndown-style tracker could follow note-taking activity, while the demonstrated GitHub-like activity chart shows how consistently he has been writing. The file collection supports multiple views, each answering a different question about the same material.
Oz is the scheduling platform Holmes points readers toward for background updates. Hubble is the free, open-source, agent-accessible notebook used throughout the demonstration. He is explicit that Hubble still has many missing features and invites contributions and issue reports. Its appeal here is concrete: a familiar place to write notes, with Markdown and HTML files that agents can organize and turn into something worth revisiting.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Free, open-source notebook backed by Markdown and HTML, with downloads and links to its source.
Karpathy's idea file for having an agent build and maintain an interconnected Markdown wiki from raw sources.
Instructions for configuring cloud environments, scheduling agents and inspecting their runs.
Install Handy, choose an on-device speech model and configure a shortcut for dictation.
Current one-time purchase plans, device limits and lifetime-update terms for VoiceInk on macOS.
Further reading
Laboratory research comparing speech recognition and touchscreen typing for short English and Mandarin messages.
Read the transcript and explore slides
Loading transcript and slides…