# Design like Karpathy is watching 😎

Zeke Sikelianos, Replicate

AI Engineer World's Fair 2025

[Talk page](https://ai.engineer/talks/huQPkrwVWwc-design-like-karpathy-is-watching) · [Video](https://www.youtube.com/watch?v=huQPkrwVWwc)

## Design like Karpathy is watching

MenuGen turned restaurant menus into food illustrations, but getting it deployed exposed a harder problem: making APIs, documentation, and onboarding work for developers using AI assistants.

Basic familiarity with HTTP APIs, JSON, and AI coding assistants will help; OpenAPI and MCP are explained as they appear.

### From a working demo to a working service

[0:47](https://www.youtube.com/watch?v=huQPkrwVWwc&t=47s)

What makes an app easy to build with an AI assistant—and what still gets in the way when it has to become a real service? Zeke Sikelianos approaches that question from Replicate, which provides API access to open models such as **FLUX**, proprietary models from providers including Anthropic, OpenAI, and Google, and developers’ own public or private models. His presentation also has a companion slides repository for following its references.

Andrej Karpathy provides the framing: an AI researcher and educator whose accessible explanations, vibe coding experiments, and description of English as a programming language make him an influential user of developer tools. Sikelianos connects this trajectory to Software 2.0, describing it as anticipating machines writing software. That is a broad connection: the original essay concerned programs encoded in learned neural-network weights, rather than specifically LLMs generating conventional source code.

Karpathy’s [MenuGen](https://karpathy.bearblog.dev/vibe-coding-menugen/) makes the problem concrete. Photograph a restaurant’s text menu, and the app generates illustrations of its dishes. That can help someone interpret unfamiliar food names, navigate a language barrier, or simply choose something that looks appetizing. The images illustrate categories of food; they are not verified photographs of what that restaurant will serve.

The local demo was enjoyable to build. Turning it into a deployed application was much harder. A program working on a developer’s machine still has to survive hosting, external services, and the operational details that appear when it goes online. MenuGen eventually became a functioning application that accepted payments, but reaching that point required much more than generating its initial code.

![Quote describing vibe coding MenuGen as an exhilarating and fun escapade locally, but a painful slog as a deployed, real app.](https://ai.engineer/api/talk-essay-media/51e7b7afe1c6191745bc44e3c99e1efd4c4f2ffbb1136f14c1ee976318468dfd)

MenuGen: a fun local demo, but a painful deployment experience.

Karpathy’s account exposed failures across several providers, including Replicate: the assistant’s knowledge was stale, documentation was outdated, API behavior had changed, and rate limits made a legitimate paid account difficult to use. For Sikelianos, the useful response is to treat those complaints as specific product defects. Better code generation alone cannot repair missing instructions or an onboarding system that blocks the person trying to use the code.

### Put current instructions into the assistant’s context

[5:19](https://www.youtube.com/watch?v=huQPkrwVWwc&t=319s)

The first repair is to make documentation easy to consume as text. Sikelianos points to [llms.txt](https://llmstxt.org/) and Markdown versions of documentation as ways to supply useful material without the surrounding HTML interface. The linked proposal has since evolved; the approach demonstrated here is straightforward text delivery. Karpathy’s preference, as quoted in the talk, is a single Markdown document and a copy button over elaborate documentation-page presentation.

Replicate responded by adding several routes from a model page into an assistant’s context:

- **Copy Markdown:** take the page’s instructions into the tool where the work is happening.
- **Open in Claude:** start a conversation with **Claude** about the model and its capabilities.
- **Open in ChatGPT:** use ChatGPT to explore the model interactively.

These routes turn documentation from a page the user must search into material an assistant can discuss with them.

The same Markdown also serves implementation. A developer using Cursor or Windsurf can put the model’s documentation into the editor so the assistant has current instructions for running it. This addresses the stale-knowledge problem directly: the relevant usage information enters the current conversation instead of depending entirely on what the model learned during training.

### Make the operational contract easy to read

[7:03](https://www.youtube.com/watch?v=huQPkrwVWwc&t=423s)

Text documentation becomes especially useful when it includes a complete request. Sikelianos quotes Karpathy: “LLMs don't like to click, they like to curl.” The appeal of `curl` is its familiarity and independence from a specialized client interface. A request may look syntactically busy, but it puts the details needed to call a service in one place.

| Request detail | What the example communicates |
| --- | --- |
| Endpoint and HTTP method | Where to send the request and which operation to invoke |
| JSON payload | Which inputs the operation expects |
| Credentials | How to authenticate |
| Response type | What representation to request |
| Blocking or asynchronous behavior | Whether to wait or handle completion separately |

Together, these details make an example a compact operational contract. An assistant can use it to construct a request without reconstructing the interface from scattered descriptions.

[Cog](https://cog.run/llms.txt) applies the same idea to model development. It packages machine-learning models in Docker containers and gives them standardized inputs and outputs described through OpenAPI. Replicate collected Cog’s documentation into a single `llms.txt` file so an editor could bring the relevant instructions into context.

The workflow starts with an unfamiliar project:

1. Clone an existing open-source Cog model.
2. Open the project in an AI-assisted editor.
3. Reference Cog’s `llms.txt` documentation.
4. Ask the editor to use that context while changing the model’s code.

The documentation supplies framework knowledge alongside the project being edited. This leads to Sikelianos’s central design thesis: **treat the LLM as a primary audience for your product, service, or library.** The human still has the goal, but an assistant increasingly reads the instructions and performs the integration work.

### Use one API description for documentation and tools

[9:34](https://www.youtube.com/watch?v=huQPkrwVWwc&t=574s)

To explain MCP, Sikelianos first establishes what [OpenAPI](https://spec.openapis.org/oas/v3.1.0.html) contributes. An OpenAPI document describes an HTTP API in JSON or YAML. It includes paths, operations, query parameters, request bodies, and responses; it is more than a JSON Schema for an individual payload. For Replicate, that description covers tasks such as creating predictions, retrieving predictions, and searching for models.

Replicate’s human-facing HTTP API reference is generated from that description. A template renders the structured definition into a page a developer can read. The public-model search example shows the result: documentation organized around an endpoint, its description, and an example request.

![Browser showing the Search public models documentation, with Endpoint, Description and Example cURL request sections on a pink background.](https://ai.engineer/api/talk-essay-media/4fdbd303483091b23b38441ce04d06ce7e8a7fdbefac3b669bdb68b9a34c86bc)

Replicate's API reference for searching public models.

The same investment supports assistant-facing tools. Sikelianos introduces [Model Context Protocol](https://modelcontextprotocol.io/specification/2025-03-26) through Replicate’s approach of turning its OpenAPI description into capabilities an assistant can use. That conversion is an implementation strategy, not the definition of MCP itself: MCP standardizes communication for tools and context through JSON-RPC. OpenAPI describes the HTTP service; the MCP integration makes service operations available to the assistant.

The demonstrated [Replicate MCP setup](https://replicate.com/docs/reference/mcp/) uses Claude Desktop’s developer settings, a JSON configuration, and a Replicate API token. Sikelianos describes the local server setup as handled through that client workflow. This is a historical desktop demonstration; current documentation distinguishes local installation from hosted remote connections, so it should not be read as a universal promise of installation-free setup. Once connected, Claude can issue authenticated Replicate API requests.

That connection supports several related activities:

- **Discovery:** ask what the service can do before knowing its interface.
- **Search:** have the assistant find relevant models.
- **Project scaffolding:** use those capabilities while assembling the beginning of an application.

The foundation is a well-documented, comprehensive OpenAPI description. Sikelianos emphasizes that this existing investment made exposing Replicate’s capabilities through MCP much easier. He names Claude, GitHub Copilot, Visual Studio Code, and the OpenAI Agents SDK as integration contexts in the expanding tool ecosystem.

### Let legitimate users get to work

[13:09](https://www.youtube.com/watch?v=huQPkrwVWwc&t=789s)

Documentation and tool access do not help if the service blocks a legitimate customer. Returning to MenuGen, Sikelianos describes Karpathy registering for an API key, entering credit-card details, and then generating many images of French toast. In Sikelianos’s account, an approximately one-hour-old account sending 1,000 requests triggered an abuse mechanism and was blocked. Those numbers describe the incident as he recounts it, not a published rate-limit threshold.

Replicate’s CEO saw the public criticism and intervened to unblock the account, according to Sikelianos. That resolved one user’s problem, but most users cannot rely on a widely read blog post to get access restored. A product needs an ordinary path for legitimate users who arrive ready to do substantial work.

Sikelianos proposes prepaid credit as one such path: a user should be able to commit money up front and then use the service intensively. His $500 example illustrates that intent; it is not an announced price or a guarantee of unrestricted usage. He describes the change as work in progress. From there, he turns to another completion criterion: shipping a feature cannot end at merging its pull request.

### Document features and keep responses useful

[14:55](https://www.youtube.com/watch?v=huQPkrwVWwc&t=895s)

**Documentation is part of shipping.** A feature needs published instructions that users can discover and an assistant can consume and put to work. Producing those instructions in accessible text formats is part of the product, not an optional presentation layer added after implementation.

Technology choices affect this experience too. Sikelianos uses SQL as the example of a mature language with a large body of existing usage. His rationale for choosing established, stable technology is that models have encountered many examples and have fewer changing conventions to navigate. This is a design heuristic about familiarity, not a measured guarantee of correct generated code.

Finally, an API response may become input to a model with a limited context window. Returning every available field about every model can bury the information needed for the task. Sikelianos recommends smaller, information-dense JSON responses: retain useful details while avoiding an exhaustive metadata dump. The response’s usefulness to its next reader matters as much as its completeness as a database export.

### From generated documentation to informed decisions

[17:03](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1023s)

An audience question asks how to get started generating API documentation. Sikelianos’s answer begins before the choice of documentation site: write an OpenAPI description in YAML or JSON that captures the API’s behavior. He names Docusaurus, Read the Docs, and ReadMe as examples in the documentation ecosystem. Schema-driven tooling can also produce SDKs and clients in different languages, though those capabilities are not necessarily native to every named product. The reusable asset is the API description.

The final question extends the discussion from integration to discovery and distribution. If assistants increasingly inform purchasing decisions—and might eventually make some of them—what does a service need to expose? Sikelianos starts with strong search and API access to the information a user needs to make an informed choice.

Pricing is his concrete example. At the time of the talk, Replicate users had to visit pricing pages or individual model pages to find it. Exposing that information as public, structured JSON would let an assistant retrieve costs and compare options inside the conversation. This is a proposed capability in the discussion, not a demonstrated pricing endpoint.

The imagined request is to compare video models such as **Veo**, **Kling**, and **MiniMax** by expense, speed, and output quality. Pricing data would support the cost comparison; speed and quality would require corresponding evidence of their own. No model ranking is demonstrated. The substantive requirement is that the facts needed for a decision be available in a form the assistant can retrieve and reason over. Search can find the options, but structured decision data makes those options possible to evaluate.

## Resources

- [Zeke's companion page](https://zeke.sikelianos.com/karpathy/) — The speaker's presentation page with official and rough-cut recordings and a transcript.
- [Vibe coding MenuGen](https://karpathy.bearblog.dev/vibe-coding-menugen/) — Karpathy's account of building, deploying, and adding payments to a menu-illustration app.
- [Cog's consolidated documentation](https://cog.run/llms.txt) — Text documentation for packaging models in containers and exposing typed inference APIs.
- [MCP specification from March 2025](https://modelcontextprotocol.io/specification/2025-03-26) — A historical specification for connecting applications to model-facing context and tools.
- [OpenAPI 3.1 specification](https://spec.openapis.org/oas/v3.1.0.html) — The formal structure for describing HTTP operations, parameters, request bodies, and responses.
- [Current Replicate MCP setup](https://replicate.com/docs/reference/mcp/) — Instructions for connecting supported assistants to Replicate through local or remote MCP servers.
- [Replicate's September 2025 search API](https://replicate.com/blog/new-search-api) — A later search endpoint covering models, collections, and documentation, with MCP response filtering.

## Slides

### Event sponsors

AWS is the innovation partner; six companies are listed as platinum sponsors.

[0:10](https://www.youtube.com/watch?v=huQPkrwVWwc&t=10s) [AWS innovation partner and six platinum sponsor logos on a black background.](https://assets.ai.engineer/sites/aiecode2025/sha256/7014dca4343b4bf40b422a18f9dc33012b4df43ef39e3a58fa399bca1b3e374b.webp)

INNOVATION PARTNER

aws

PLATINUM SPONSORS

Graphite
Windsurf
MongoDB®
daily™
augment code
WorkOS

### Zeke

Zeke introduces himself as @zeke on GitHub and X and says he works at Replicate.

[1:02](https://www.youtube.com/watch?v=huQPkrwVWwc&t=62s) [The handle @zeke centered on a yellow slide.](https://assets.ai.engineer/sites/aiecode2025/sha256/7ce478bd9a1fb975e658c5890e1c722c1d33b3e237b1da5ef2177106f7fee67d.webp)

@zeke

AIE

### Logo variants

Zeke introduces Replicate, a cloud platform for running AI models through an API.

[1:07](https://www.youtube.com/watch?v=huQPkrwVWwc&t=67s) [Colorful and monochrome versions of a geometric logo on a yellow slide.](https://assets.ai.engineer/sites/aiecode2025/sha256/7cb8aa286cbf2a7923290972179912b8eb85360a9dba28d9f710b337b9a5280b.webp)

Interface text:
AIE
Microsoft
smol ai

### Run AI with an API

Replicate uses the same run call to access open-source and proprietary models, as well as custom public and private models.

[1:10](https://www.youtube.com/watch?v=huQPkrwVWwc&t=70s) [Code examples call several AI models and a custom model using the Replicate API.](https://assets.ai.engineer/sites/aiecode2025/sha256/58b0d2583e6bf8218323de7ca84beae000a6972e63414cb7e4b2032d6251e6f7.webp)

// Run AI with an API
replicate.run("black-forest-labs/flux-kontext-pro")
replicate.run("anthropic/claude-4-sonnet")
replicate.run("openai/gpt-4o")
replicate.run("google/imagen-4")
replicate.run("kwaivgi/kling-v2.0")
replicate.run("your/custom-model")

Interface text:
AIE
Microsoft
smol ai

[1:19](https://www.youtube.com/watch?v=huQPkrwVWwc&t=79s) [Code examples show Replicate running several AI models and a custom model.](https://assets.ai.engineer/sites/aiecode2025/sha256/5d84925d7d6eb23c63f93fed372c1cf52122857efd3b54edac6173033c7a1afe.webp)

// Run AI with an API
replicate.run("black-forest-labs/flux-kontext-pro")
replicate.run("anthropic/claude-4-sonnet")
replicate.run("openai/gpt-4o")
replicate.run("google/imagen-4")
replicate.run("kwaivgi/kling-v2.0")
replicate.run("your/custom-model")

Interface text:
AIE
Microsoft
smol ai

### Andrej Karpathy

The speaker highlights Karpathy’s accessible AI education, his term “vibe coding,” and his Software 2.0 manifesto, which he says predicted models writing code.

[1:44](https://www.youtube.com/watch?v=huQPkrwVWwc&t=104s) [Bullets describe Karpathy's AI research, teaching, videos, vibe coding and Software 2.0 manifesto.](https://assets.ai.engineer/sites/aiecode2025/sha256/dfa4b03ed3f66ca2fd3622475c0463b2c51290023a429da20a3c2adbb4b4af06.webp)

• AI researcher at Google / OpenAI / Tesla / OpenAI / Eureka Labs
• Stanford professor
• YouTube educator
• Coined the term “vibe coding”
• Thinks the hottest new programming language is English
• Wrote the Software 2.0 manifesto (7 years ago)

Interface text:
AIE
aws

### What is MenuGen?

MenuGen generates images of menu items from photos of a text-only restaurant menu.

[2:48](https://www.youtube.com/watch?v=huQPkrwVWwc&t=168s) [Title slide asks what MenuGen is.](https://assets.ai.engineer/sites/aiecode2025/sha256/351dca4c735bca6bb14ee8282bdddd31dad07c8f5fbf7549d13091abc9679e38.webp)

What is MenuGen?

Interface text:
AIE
aws

### MenuGen demo

MenuGen turns a photo of a text menu into images of dishes. The demo shows an uploaded menu, generation at 0/17, and an illustrated laksa card.

[2:52](https://www.youtube.com/watch?v=huQPkrwVWwc&t=172s) [A mobile MenuGen demo displays illustrated restaurant dishes with names and descriptions.](https://assets.ai.engineer/sites/aiecode2025/sha256/8de1ca35e29c8d8ea10f1f20daf4e2e60b28eb1e651daf83cfc8efe8a1e19330.webp)

Soy, Chili Oil, Black Vinegar, Coriander, Garlic

SPAGO BREAKFAST SIGNA[illegible]RES
Spago's Laksa Noodle
Little Neck Clams, Tiger Prawns, Calamari, Onion Sambal, Calamansi, Laksa Leaf

Interface text:
MenuGen demo
9:50
MenuGen
$4.10 credits
Watch later
Share
Play
Watch on YouTube
menugen.app
AIE

[3:05](https://www.youtube.com/watch?v=huQPkrwVWwc&t=185s) [MenuGen displays an uploaded restaurant menu and an image-generation progress bar.](https://assets.ai.engineer/sites/aiecode2025/sha256/c7017859ecc65c44773d5f2bda5775aca9de2c576bb2f0753522a95e4b5c29aa.webp)

Click to upload or drag and drop
PNG, JPG, GIF up to 10MB

Uploaded Menu Image
Spago
WOLFGANG PUCK
SPAGO BREAKFAST SIGNATURES
BREAKFAST CLASSICS
ON THE SIDE
BOWLS AND SALADS
COFFEE SELECTION
TEA SELECTION
FRESH JUICES & SMOOTHIES
COCKTAILS
[Other menu text illegible]

Generating images (0/17)

Interface text:
9:50
MenuGen
$4.10 credits
menugen.app
AIE
Microsoft
smol ai

### From local demo to deployed app

MenuGen’s creator found building the local demo exhilarating and fun, but called the deployed, real app a bit of a painful slog.

[3:28](https://www.youtube.com/watch?v=huQPkrwVWwc&t=208s) [A quote describes building the local MenuGen demo as fun and deploying the real app as painful.](https://assets.ai.engineer/sites/aiecode2025/sha256/f06e7aeafe9d5af9b60f6c6cc4a32883f71852ee70c25e027e7f80cf2b334fcf.webp)

“Vibe coding menugen was [an] exhilarating and fun escapade as a local demo, but a bit of a painful slog as a deployed, real app.”

AIE
Microsoft
smol ai

### Troubles with services

Andrej’s MenuGen post names five services where he encountered trouble. The speaker says their developer experience needs work, even though MenuGen became a functioning paid app.

[3:57](https://www.youtube.com/watch?v=huQPkrwVWwc&t=237s) [A quote names OpenAI, Replicate, Vercel, Clerk and Stripe as services involved in the difficulties.](https://assets.ai.engineer/sites/aiecode2025/sha256/d7d4bfa5da650b14a31be668fa9c21c0856ad76174c2f3037b2c601ba8e8ea6b.webp)

“Here is where some of the troubles started [with] OpenAI, Replicate, Vercel, Clerk, Stripe...”

AIE
aws

### Replicate API difficulties

A new Replicate user encountered outdated model knowledge and docs, a changed API response, and rate limits that made debugging difficult. The speaker treats the criticism as an opportunity to improve the product.

[4:49](https://www.youtube.com/watch?v=huQPkrwVWwc&t=289s) [A quote describes outdated model knowledge and documentation, API changes and rate limits that complicated development.](https://assets.ai.engineer/sites/aiecode2025/sha256/ae8a2f4de9641ff29a7752e4d5ca65239e37a30d908e587cbdc417212a5d68f8.webp)

“I needed to generate images [...] I signed up for a new Replicate API key and ran into issues relatively quickly. My queries didn’t work because LLM knowledge was deprecated, but in addition, this time even the official docs were a little bit out of date due to recent changes in the API, which now don’t return the JSON directly but instead some kind of a Streaming object that neither I or Claude understood. I then faced rate limiting on the API so it was difficult to debug the app. I was told later that these are common protection measures by these services to mitigate fraud, but they also make it harder to get started with new, legitimate accounts.”

AIE
Microsoft
smol ai

[5:01](https://www.youtube.com/watch?v=huQPkrwVWwc&t=301s) [A quote describes outdated model knowledge and documentation, API changes, and rate limits that made a new account difficult to use.](https://assets.ai.engineer/sites/aiecode2025/sha256/212079a505ff6e9d49e04ee217c8a78a9411815e1e56fcb29c7ed687304aa40d.webp)

“I needed to generate images [...] I signed up for a new Replicate API key and ran into issues relatively quickly. My queries didn’t work because LLM knowledge was deprecated, but in addition, this time even the official docs were a little bit out of date due to recent changes in the API, which now don’t return the JSON directly but instead some kind of a Streaming object that neither I or Claude understood. I then faced rate limiting on the API so it was difficult to debug the app. I was told later that these are common protection measures by these services to mitigate fraud, but they also make it harder to get started with new, legitimate accounts.”

AIE
Microsoft
smol ai

### Embrace llms.txt

The speaker proposes llms.txt as a way for Replicate to offer text or Markdown documentation that language models can consume more easily than webpage HTML.

[5:19](https://www.youtube.com/watch?v=huQPkrwVWwc&t=319s) [A dark slide reads “Embrace llms.txt.”](https://assets.ai.engineer/sites/aiecode2025/sha256/9345ff5b9ae61980ffc91817ab7f01e41808eaa83150a405f070d5eb9b4b97e1.webp)

Embrace llms.txt

AIE
Microsoft
smol ai

### A single Markdown documentation file

Karpathy favors a single Markdown documentation file with a copy button. The speaker says text-based documentation is easier for language models to consume than webpage HTML.

[5:47](https://www.youtube.com/watch?v=huQPkrwVWwc&t=347s) [A quote attributed to Karpathy contrasts elaborate documentation pages with a single Markdown file and a copy-to-clipboard button.](https://assets.ai.engineer/sites/aiecode2025/sha256/067b68620b57c4978d4dbf5b541e2da6616fbb610f4c9a92cac1bee140172c33.webp)

Tired: elaborate docs pages with fancy color palettes, branding, animations, transitions, dark mode.

Wired: one single docs .md file and a ”copy to clipboard” button.

-- Karpathy

Interface text:
AIE
aws

### Asking Claude about a Replicate model

The demo sends a Replicate model page to Claude and asks how to run the model, using its llms.txt page as context.

[6:20](https://www.youtube.com/watch?v=huQPkrwVWwc&t=380s) [Claude begins responding to a request for help running a model on Replicate.](https://assets.ai.engineer/sites/aiecode2025/sha256/71846e8530256b6a24e6883b360c82d02095506f98364c9a817850a7428952d7.webp)

Show me how to run this model on Replicate
https://replicate.com/black-forest-labs/flux-kontext-pro/llms.txt

I'll help yo

Interface text:
Untitled
Reply to Claude...
Research BETA
Claude Sonnet 4
AIE
Microsoft
smol ai

### ChatGPT model setup guidance

ChatGPT responds to a request linking to the model’s Markdown URL, starting its setup guide with an API token and the Node.js client.

[6:37](https://www.youtube.com/watch?v=huQPkrwVWwc&t=397s) [ChatGPT displays the beginning of a guide to running FLUX.1 Kontext on Replicate.](https://assets.ai.engineer/sites/aiecode2025/sha256/548993066f1adc0e696b61179a4f2a211ea1f325378a133166f7140c115d1181.webp)

Show me how to run this model on Replicate
https://replicate.com/black-forest-labs/flux-kontext-pro/llms.txt

To run the FLUX.1 Kontext [pro] model on Replicate, follow these steps:

Step-by-Step Guide to Using FLUX.1 Kontext [pro] on Replicate

1. Set Up Your Environment
- Obtain an API Token: Sign up at Replicate and retrieve your API token.
- Install the Replicate Client: For Node.js, install the client using:

npm install replicate

- Set the API Token: Configure your environment with the API token:

Interface text:
ChatGPT 4o
Share
bash
Copy
Edit
bash
Copy
Edit
Search the web
Search
ChatGPT can make mistakes. OpenAI doesn't use Replicate workspace data to train its models.
AIE
Microsoft
smol ai

### Model documentation in Markdown

Replicate exposes the model documentation as Markdown. The speaker suggests copying it into Cursor or Windsurf so the editor knows how to run the model.

[6:52](https://www.youtube.com/watch?v=huQPkrwVWwc&t=412s) [A browser displays model information, input descriptions, and the beginning of an output schema.](https://assets.ai.engineer/sites/aiecode2025/sha256/af0986663d405b381598e2dcc5f53a876c762dfd3249f8d1d21f4fc065461cc1.webp)

## Basic model info

Model name: black-forest-labs/flux-kontext-pro
Model description: A state-of-the-art text-based image editing model that delivers high-quality outputs with excellent prompt following and consistent results for transforming images through natural language

## Model inputs

- prompt: Text description of what you want to generate, or the instruction on how to edit the given image. (string)
- input_image: Image to use as reference. Must be jpeg, png, gif, or webp. (string)
- aspect_ratio: Aspect ratio of the generated image. Use 'match_input_image' to match the aspect ratio of the input image. (string)
- seed: Random seed. Set for reproducible generation (integer)
- output_format: Output format for the generated image (string)
- safety_tolerance: Safety tolerance, 0 is most strict and 6 is most permissive. 2 is currently the maximum allowed when input images are used. (integer)

## Model output schema

{
  "type": "string",
  "title": "Output",
  "format": "uri"

Interface text:
replicate.com/black-forest-labs/flux-kontext-pro/llms.txt
AIE
aws

### LLMs like to curl

The speaker quotes Karpathy’s view that LLMs prefer curl to clicking, then calls curl a tool that is here to stay.

[7:01](https://www.youtube.com/watch?v=huQPkrwVWwc&t=421s) [A slide attributes the quote about LLMs preferring curl to clicking to Karpathy.](https://assets.ai.engineer/sites/aiecode2025/sha256/f9d5a5375fe059f8854def724548851ec4da84d5a2b55a46a020d2229705776a.webp)

“LLMs don’t like to click, they like to curl.”

- Karpathy

Interface text:
AIE
Microsoft
smol ai

### Making an API request with curl

The curl example puts the POST method, JSON input, authorization, headers, and endpoint in one command, giving an LLM the details needed to make an API request.

[7:37](https://www.youtube.com/watch?v=huQPkrwVWwc&t=457s) [A curl command shows a POST request with a JSON payload, authorization placeholder, headers and an API endpoint.](https://assets.ai.engineer/sites/aiecode2025/sha256/948df4b45ef588672bf1bafaf3ceb672cdb29159c67801719d88c8afdf7c43af.webp)

curl -s -X POST \
  -d '{"version": "replicate/hello-world:5c7d5dc6d", "input": {"text": "Alice"}}' \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H 'Content-Type: application/json' \
  -H 'Prefer: wait' \
  https://api.replicate.com/v1/predictions

Interface text:
curl.sh
AIE
aws

### Example classification output

The example ranks tiger_cat above tabby and Egyptian_cat. The speaker describes Cog’s standardized model API and documentation collected in an llms.txt file.

[8:30](https://www.youtube.com/watch?v=huQPkrwVWwc&t=510s) [A terminal displays example classification results for three cat categories with confidence scores.](https://assets.ai.engineer/sites/aiecode2025/sha256/1506ba2ab8bffa31120b4356cf088d67a7442131c4aef3178ab0bcb53ed8896a.webp)

```

If you see the following output

```
[
  [
    "n02123159",
    "tiger_cat",
    0.4874822497367859
  ],
  [
    "n02123045",
    "tabby",
    0.23169134557247162
  ],
  [
    "n02124075",
    "Egyptian_cat",
    0.09728282690048218
  ]

Interface text:
design-like-karpathy-is-watching — ~/git/zeke/design-like-karpathy-is-watching
AIE
Microsoft
smol ai

### Updating a Cog model to support multiple inputs

The speaker describes giving an editor Cog documentation as context, then asks it to update a model for multiple inputs. The demo shows the editor generating a loop over input objects in predict.py.

[8:49](https://www.youtube.com/watch?v=huQPkrwVWwc&t=529s) [An editor chat prompt asks to update a Cog model to support multiple inputs.](https://assets.ai.engineer/sites/aiecode2025/sha256/139a76e1be1894b00b5fca7b863496ec78fe28a2a05d0fe9fc00fb9749859803.webp)

Update this Cog model to support multiple inputs

Interface text:
New Chat
@ predict.py
Agent ⌘I
Auto
AIE
Microsoft
smol ai

[9:03](https://www.youtube.com/watch?v=huQPkrwVWwc&t=543s) [A code editor shows a model predictor being updated to accept multiple inputs.](https://assets.ai.engineer/sites/aiecode2025/sha256/5d581604461fb86639a0c87ce56a47c1ed59496010949fd2116f20574b0a5497.webp)

class SchnellPredictor(Predictor):

    def predict(
        self,
        inputs: list = Input(
            description="A list of input objects, each containing the parameters for [illegible]
            default=None,
        ),
    ) -> List[Path]:
        """
        Accepts a list of input dicts, each with keys matching the previous single in[illegible]
        Returns a list of output Paths, one for each input.
        """
        results = []
        for input_obj in inputs:
            prompt = input_obj.get("prompt", None) or ""
            aspect_ratio = input_obj.get("aspect_ratio", "1:1")
            num_outputs = input_obj.get("num_outputs", 1)
            num_inference_steps = input_obj.get("num_inference_steps", 4)
            seed = input_obj.get("seed", None)
            output_format = input_obj.get("output_format", "webp")
            output_quality = input_obj.get("output_quality", 80)
            disable_safety_checker = input_obj.get("disable_safety_checker", False)
            go_fast = input_obj.get("go_fast", True)
            megapixels = input_obj.get("megapixels", "1")

I'll edit the file:
predict.py
    model = self.fp8_model if go_f[illegible]
    width, height = self.size_fro[illegible]
    imgs, np_imgs = model.predict(
        prompt,
        num_output[illegible]
        num_inference_steps=num_in[illegible]
        seed=seed,
        width=width,
        height=height,
    )
    return self.postprocess(
        img

Interface text:
predict.py — cog-flux
predict.py
New Chat
Generating...
Stop
Plan, search, build anything
Agent ⌘I
Auto
AIE
aws

### An LLM as the primary audience

Karpathy frames the LLM as the primary audience for a product, service, or library. The speaker embraces that shift after showing an editor use documentation to write code.

[9:15](https://www.youtube.com/watch?v=huQPkrwVWwc&t=555s) [A quote attributed to Karpathy says the primary audience for a product, service, or library is now an LLM.](https://assets.ai.engineer/sites/aiecode2025/sha256/c219ff69ea7620e9bb2369650dc60ca093c07aaf175ef8526c043813b566277a.webp)

“The primary audience of your thing (product, service, library, ...) is now an LLM, not a human.”

-- Karpathy

Interface text:
AIE
Microsoft
smol ai

### Replicate HTTP API reference

Replicate generates this human-friendly API reference from its schema. The page lists operations for predictions, models, and collections, including model search.

[10:31](https://www.youtube.com/watch?v=huQPkrwVWwc&t=631s) [A browser displays Replicate's HTTP API reference with a list of prediction and model endpoints.](https://assets.ai.engineer/sites/aiecode2025/sha256/354af69d69b76d07384313c0d9a1ac74b0249bc22c0eb8f351f9809eea11942c.webp)

Authentication
predictions.create - Create a prediction
predictions.get - Get a prediction
predictions.list - List predictions
predictions.cancel - Cancel a prediction
models.create - Create a model
models.get - [illegible] a model
models.list - List public models
models.search - Search public models
models.delete - Delete a model
models.examples.list - List examples for a model
models.predictions.create - Create a prediction using an official model
models.readme.get - Get a model’s README
models.versions.get - Get a model version
models.versions.list - List model versions
models.versions.delete - Delete a model version
collections.get - Get a collection of models
collections.list - List collections of models

Interface text:
HTTP API - Replicate docs
replicate.com/docs/reference/http
zeke
Dashboard
Explore
Pricing
Docs
Blog
Changelog
Run a model
Create a model
Publish a model
Model versions
Model hardware
Private and public models
Training destinations
Delete a model
Predictions
Deployments
Webhooks
Organizations
Billing
Site policy
Reference
How does Replicate work?
Client libraries
HTTP API
OpenAPI schema
MCP server
Open source
AIE
aws

### Replicate MCP documentation

Replicate’s MCP documentation shows how OpenAPI schemas describe tools to apps such as Claude Desktop, allowing a language model to discover and run APIs from natural-language instructions.

[10:56](https://www.youtube.com/watch?v=huQPkrwVWwc&t=656s) [A browser displays an introduction to MCP and highlights a paragraph about connecting applications to tools.](https://assets.ai.engineer/sites/aiecode2025/sha256/bef91c640c384cef6d00975adb42f7cba3091c3fd2ae2fe696b71ab8f219cde5.webp)

What is MCP?
Replicate’s MCP server
Running the MCP server
Using Replicate MCP with Claude Desktop
Using Replicate MCP with Cursor
Using Replicate MCP with GitHub Copilot in VS Code

What is MCP?

The Model Context Protocol (MCP) is an open standard developed by Anthropic that defines how applications share context with large language models (LLMs).

MCP extends the capabilities of apps like Claude Desktop, Cursor, or GitHub Copilot by feeding them OpenAPI schemas that describe tools or services, like Replicate’s HTTP API.

MCP lets you give natural language instructions to a language model, and it can discover and run APIs automatically on your behalf.

Here are some examples of the kinds of prompts you can use:

“Search Replicate for upscaler models and compare them”

Interface text:
AIE
MCP (Model Context Protoc[illegible]
replicate.com/docs/reference/mcp
zeke
Dashboard
Explore
Pricing
Docs
Blog
Changelog
Run a model
Create a model
Publish a model
Model versions
Model hardware
Private and public models
Training destinations
Delete a model
Predictions
Deployments
Webhooks
Organizations
Billing
Site policy
Reference
How does Replicate work?
Client libraries
HTTP API
OpenAPI schema
MCP server
Open source

## Full transcript

[0:00](https://www.youtube.com/watch?v=huQPkrwVWwc&t=0s) [upbeat music] How many of you know who Andrej Karpathy is?

[0:18](https://www.youtube.com/watch?v=huQPkrwVWwc&t=18s) Raise your hand. Okay, maybe half of you. Raise your hand if you are not Andrej Karpathy.

[0:26](https://www.youtube.com/watch?v=huQPkrwVWwc&t=26s) Just trying to gauge audience participation here. Okay, so I got 80% there, something like that. Got a lot of Andrejs in the room right now. Um, raise your hand if you work at Replicate.

[0:37](https://www.youtube.com/watch?v=huQPkrwVWwc&t=37s) All right, so if you wanna talk to any Replicate folks, there's, there's your group right there. All right, so, um, for those who don't know who Andrej Karpathy is, I will jump into that and explain that.

[0:47](https://www.youtube.com/watch?v=huQPkrwVWwc&t=47s) Um, these are my, uh-- There's a GitHub repo that corresponds to this, um, these slides, so if you wanna grab that, this will... I'll put this slide up at the end too, so you can, um, track down any URLs or anything that I mention in the talk.

[1:03](https://www.youtube.com/watch?v=huQPkrwVWwc&t=63s) Uh, my name is Zeke. I am Zeke on GitHub, um, Zeke on X as well, um, and I work for Replicate. So, uh, Replicate is a cloud platform that lets you run AI models with an A-API.

[1:17](https://www.youtube.com/watch?v=huQPkrwVWwc&t=77s) So we have, um, you know, open source models like all the great Fla- FLUX models from Black Forest Labs, but we also have, you know, proprietary models from Anthropic, OpenAI, Google, et cetera.

[1:32](https://www.youtube.com/watch?v=huQPkrwVWwc&t=92s) Um, and of course, you can also run your own custom public and private models on Replicate as well. So let's get to the point. Who is Andrej Karpathy? Well, he's an AI res-- he's an AI researcher who's worked at all these big, uh, companies and organizations, Google, OpenAI, Tesla, OpenAI, Eureka Labs.

[1:53](https://www.youtube.com/watch?v=huQPkrwVWwc&t=113s) Um, Eureka Labs is his new thing, uh, an educational, uh, platform. Uh, but most importantly to me, he is a YouTube educator and does some really amazing talks that are highly accessible that explain how AI and machine learning works for general audiences.

[2:11](https://www.youtube.com/watch?v=huQPkrwVWwc&t=131s) Um, he coined the term vibe coding a few months ago, and of course, that's taken the world by storm. We're all really interested in that now, and subscribes to the idea that the hottest new programming language is English.

[2:23](https://www.youtube.com/watch?v=huQPkrwVWwc&t=143s) Um, kind of a hot take. Um, he also wrote something called the Software 2.0 Manifesto, which was, um, now seven years ago, kind of a eternity in machine learning time.

[2:35](https://www.youtube.com/watch?v=huQPkrwVWwc&t=155s) Uh, basically predicting this world in which, uh, machine learning models would write code for us, um, and that it would be-- they would be better at it than, than humans.

[2:46](https://www.youtube.com/watch?v=huQPkrwVWwc&t=166s) And so of course, here we are. Um, so today I wanna talk about MenuGen. So MenuGen is, um, an app that Andrej created recently at a... I think he was at a hackathon doing like a, a vibe coding experiment.

[3:00](https://www.youtube.com/watch?v=huQPkrwVWwc&t=180s) So MenuGen is basically this, uh, web app where you take photos of a menu at a restaurant that's all in a text format, and it generates image representations of the contents of the menu for you.

[3:15](https://www.youtube.com/watch?v=huQPkrwVWwc&t=195s) So if you don't know what the words mean, or English isn't your first language, or you just like to see tantalizing photos of food that may be good, um, that was the idea behind it.

[3:28](https://www.youtube.com/watch?v=huQPkrwVWwc&t=208s) So he was actually able to build this app, which he described as an exhilarating and fun escapade as a local demo, but a bit of a painful slog as a deployed real app.

[3:39](https://www.youtube.com/watch?v=huQPkrwVWwc&t=219s) So you've probably-- many of you have probably experienced this, where you are working on something locally, you have it running on your machine. "Oh, cool. It really works. It's amazing."

[3:48](https://www.youtube.com/watch?v=huQPkrwVWwc&t=228s) And then you try to deploy it to, you know, Vercel or Cloudflare or something like that, and that's where a lot of the, the pain begins. Um, so we're gonna talk about that.

[3:58](https://www.youtube.com/watch?v=huQPkrwVWwc&t=238s) So, um, Andrej kind of wrote this blog post about the experience of creating MenuGen, um, and saying, "You know, I was able to make this thing, publish it, get it online, uh, add payments for it, and it's a working, functioning app that people can pay for, and it was super fun to build."

[4:19](https://www.youtube.com/watch?v=huQPkrwVWwc&t=259s) However, he kind of rakes all these different companies over the coals because of the sort of developer experience challenges of working with all of them. So for me, it was cool because it was like, okay, Replicate is mentioned among all these big hotshot companies like OpenAI and Vercel, um, but we also all have work to do to

[4:37](https://www.youtube.com/watch?v=huQPkrwVWwc&t=277s) improve our products to make them better. So here's a blurb about kind of what he, what he experienced when he started using Replicate API. So the LLM's knowledge of Replicate was outdated.

[4:51](https://www.youtube.com/watch?v=huQPkrwVWwc&t=291s) The docs on Replicate were out of date. Um, there were changes in the API. He experienced rate limiting, and it was harder to get started with a new legitimate paid account.

[5:04](https://www.youtube.com/watch?v=huQPkrwVWwc&t=304s) So this is kind of embarrassing, but it's also kind of like an opportunity to fix our product and make it better and really listen to, you know, the kind of voices that are loud and correct about the problems with our products.

[5:19](https://www.youtube.com/watch?v=huQPkrwVWwc&t=319s) So what can Replicate do better? Um, one of them is embracing llms.txt. llms.txt is this thing where you can, uh, basically, uh, modify your website or your API or existing services to, um, render text-based or Markdown-based versions of your documentation in a format that is friendly for language models to

[5:43](https://www.youtube.com/watch?v=huQPkrwVWwc&t=343s) consume. Um, more friendly than like the HTML contents of a webpage. So he said, "Tired: elaborate docs pages with fancy color palettes, branding, animations, transitions, dark mode. Wired: one single docs Markdown file and a copy to clipboard button."

[6:02](https://www.youtube.com/watch?v=huQPkrwVWwc&t=362s) So it sounds simple, um- And maybe not the most glamorous thing, but it is actually the thing that your language models want to consume. So in response to this, we added a new feature on the Replicate website where you're viewing any model page.

[6:16](https://www.youtube.com/watch?v=huQPkrwVWwc&t=376s) You have a button to copy the contents of that page, uh, as Markdown for a language model, or to send the page directly to Claude to have an interaction with the contents of the model page to learn more about what the model can do.

[6:33](https://www.youtube.com/watch?v=huQPkrwVWwc&t=393s) Similarly, we added that support for linking to ChatGPT. You basically just say, "I'm on a model page," you jump into ChatGPT, and you start having a conversation about the model.

[6:43](https://www.youtube.com/watch?v=huQPkrwVWwc&t=403s) So it's a lot more interactive than just going to a webpage and reading and trying to find the most relevant content.

[6:52](https://www.youtube.com/watch?v=huQPkrwVWwc&t=412s) Of course, we also just dump the Markdown here too. So if you're using a pro-- a tool like Cursor or Windsurf, grab this content, put it into your editor, and it knows how to run this model.

[7:03](https://www.youtube.com/watch?v=huQPkrwVWwc&t=423s) So next thing. This was not necessarily from the blog post, but this is from s-- I'm grabbing some quotes from recent, uh, tweets from Andrej Karpathy. So LLMs don't like to click, they like to curl.

[7:17](https://www.youtube.com/watch?v=huQPkrwVWwc&t=437s) So love it or l- love it or hate it, curl is, um, a tool that is here to stay. It's developed. It's been around for, I don't know, since the '90s maybe.

[7:29](https://www.youtube.com/watch?v=huQPkrwVWwc&t=449s) Um, it's installed on everyone's machine, and it is b- basically a standardized way to be able to make API calls without any specialized tooling. So let's look at this curl command.

[7:41](https://www.youtube.com/watch?v=huQPkrwVWwc&t=461s) Maybe it looks ugly, right? It's-- There's a lot of syntax. It's not, it's not glamorous, but it covers everything that you or that a language model needs to know about how to make an I- API request.

[7:53](https://www.youtube.com/watch?v=huQPkrwVWwc&t=473s) What is the HTTP method? What is the JSON payload? How do you send your credentials? What kind of response type do you want? Do you wanna make a blocking request or a, an asynchronous request?

[8:06](https://www.youtube.com/watch?v=huQPkrwVWwc&t=486s) What is the API endpoint? That's all covered in this one little line of code, and this is exactly the kind of thing that LLMs wanna consume. If you give this content to an LLM, it now knows how to make API requests to your service.

[8:19](https://www.youtube.com/watch?v=huQPkrwVWwc&t=499s) So it's really powerful. So we have a tool called Cog at cog.run, which is an open source tool that you can use to package machine learning models in production-ready Docker containers.

[8:32](https://www.youtube.com/watch?v=huQPkrwVWwc&t=512s) It creates a standardized API around your model, um, with standard inputs and outputs using OpenAPI. So we took all of Cog's documentation and stuffed it into a single llms.txt file at cog.run.

[8:47](https://www.youtube.com/watch?v=huQPkrwVWwc&t=527s) And what you can do with that is drop it into your editor

[8:52](https://www.youtube.com/watch?v=huQPkrwVWwc&t=532s) on an existing project. Let's say you've cloned some open source Cog model, and you're like, "I don't even really know how this code works, but I wanna change it."

[9:00](https://www.youtube.com/watch?v=huQPkrwVWwc&t=540s) You open up the model, you drop a reference to that llms.txt, and

[9:06](https://www.youtube.com/watch?v=huQPkrwVWwc&t=546s) your editor knows how to consume that content, bring it into context, and use it to write code.

[9:14](https://www.youtube.com/watch?v=huQPkrwVWwc&t=554s) So pretty powerful stuff. All right. So the primary audience of your thing, your product, service, library, et cetera, is now an LLM, not a human.

[9:28](https://www.youtube.com/watch?v=huQPkrwVWwc&t=568s) This might be, like, a tough pill to swallow, but I think it's the world that we're in right now.

[9:34](https://www.youtube.com/watch?v=huQPkrwVWwc&t=574s) Um, so if you've been at this conference for a couple days, you've probably heard everybody talking about MCP, right? It's, like, such a big deal. But what even is it?

[9:43](https://www.youtube.com/watch?v=huQPkrwVWwc&t=583s) Like, how many of you actually feel like you really know what MCP is?

[9:49](https://www.youtube.com/watch?v=huQPkrwVWwc&t=589s) Okay. I like the honesty here. Like, there's, like, eight hands going up. Okay. So I'm gonna explain this for you, hopefully. So OpenAPI is this thing where you write a JSON schema that defines the behavior of your HTTP API.

[10:06](https://www.youtube.com/watch?v=huQPkrwVWwc&t=606s) It's basically just a giant JSON file that says, "Here are the paths, here are the endpoints, here are the query parameters, here's the payload for the body. Here's how you run this thing.

[10:17](https://www.youtube.com/watch?v=huQPkrwVWwc&t=617s) Here's how you create a prediction. Here's how you get your predictions. Here's how you search," all that sort of stuff. And it's just one big, giant JSON file that describes your whole behavior of your API.

[10:29](https://www.youtube.com/watch?v=huQPkrwVWwc&t=629s) So we have that on Replicate, and when you go to our HT- HTTP API page, all the content on this page is generated from that schema. So we just have a template that renders it all out as a human-friendly representation of how to use our API.

[10:45](https://www.youtube.com/watch?v=huQPkrwVWwc&t=645s) Here's an example where you can search for models. So here's where the M- MCP part comes in. So MCP is basically a way of taking an OpenAPI schema

[10:58](https://www.youtube.com/watch?v=huQPkrwVWwc&t=658s) and stuffing it into a format where a language model knows what to do with it.

[11:04](https://www.youtube.com/watch?v=huQPkrwVWwc&t=664s) So we now have an MCP server for Replicate, which you can install very easily. You basically open up Claude Code, for example. Claude Desktop, not the web app.

[11:17](https://www.youtube.com/watch?v=huQPkrwVWwc&t=677s) Um, go into your developer settings, add this tiny little line of JSON,

[11:22](https://www.youtube.com/watch?v=huQPkrwVWwc&t=682s) and all of a sudden, Claude now knows how to do everything that the Replicate API can do, and it has an API token. So you didn't have to install any software.

[11:33](https://www.youtube.com/watch?v=huQPkrwVWwc&t=693s) All you had to do is go get a token from the Replicate website, and Claude takes care of the installation of the MCP server locally. And now you can see on this page,

[11:46](https://www.youtube.com/watch?v=huQPkrwVWwc&t=706s) you can actually have an interaction with Claude

[11:50](https://www.youtube.com/watch?v=huQPkrwVWwc&t=710s) where it's able to run API requests on Replicate for you. So there's a few factors here. There's-- You can use this for discovery, so you don't know how to use the product yet, and you wanna know what it's capable of.

[12:04](https://www.youtube.com/watch?v=huQPkrwVWwc&t=724s) Or you wanna use a language model to do searches for you. Or you wanna start, um, kind of scaffolding out the beginning of a project

[12:14](https://www.youtube.com/watch?v=huQPkrwVWwc&t=734s) And you want your language model to help you with that. So that's exactly what MCP is for. It's a way of connecting

[12:21](https://www.youtube.com/watch?v=huQPkrwVWwc&t=741s) tools to your language model so that it can do all sorts of powerful things. And I wanna emphasize here that

[12:29](https://www.youtube.com/watch?v=huQPkrwVWwc&t=749s) at Replicate, all we really had to do to make this possible was invest in having an OpenAPI schema that was very well written, very well documented, that, um, covered everything that our API is capable of doing, and the process of turning that into an MCP server that can then connect with tools like Claude, uh, GitHub Copilot, and

[12:52](https://www.youtube.com/watch?v=huQPkrwVWwc&t=772s) Visual Studio Code. Um, and now, actually, I think OpenAI added MCP p- support to their agents SDK earlier this week. So MCP is just gonna be all over the map, and it's a way to really accommodate language models helping you do things.

[13:09](https://www.youtube.com/watch?v=huQPkrwVWwc&t=789s) So, um, this is sort, sort of a note to self, uh, for the things that we got wrong for Andrej and the things that we wanna fix. Some of them we've already addressed, as I showed in this talk.

[13:21](https://www.youtube.com/watch?v=huQPkrwVWwc&t=801s) Some of them we still need to get right. So maybe kind of a no-brainer, accept payments. Okay, so Andrej went on the website, he signed up for an API key, he entered his credit card info, uh, in Replicate, you know, basically legitimate user.

[13:39](https://www.youtube.com/watch?v=huQPkrwVWwc&t=819s) And then he started hammering Replicate with API requests to generate images of French toast. And

[13:45](https://www.youtube.com/watch?v=huQPkrwVWwc&t=825s) whatever-- for whatever reason, the way he was doing it, he was making a ton of API requests, and he triggered some kind of abuse mechanism in our website that said, "Oh, well, this user's only existed for one hour, and they've already sent us 1,000 requests.

[13:57](https://www.youtube.com/watch?v=huQPkrwVWwc&t=837s) Something must be wrong." So we blocked him. And this isn't something you wanna do, right? You wanna let your power users come to your product, dive right in. They know what they're doing, they know what they want, and don't get in their way.

[14:11](https://www.youtube.com/watch?v=huQPkrwVWwc&t=851s) Luckily, uh, our CEO saw this, you know, blog post from Andrej and immediately contacted him and, you know, unblocked his account. But not everyone has the power of being able to write a blog post and have everybody in the world see it and know about it.

[14:26](https://www.youtube.com/watch?v=huQPkrwVWwc&t=866s) So the lesson here for us is Replicate should accept, um, payments for credit. So if I go on a website, I should be able to say, "Here's 500 bucks.

[14:38](https://www.youtube.com/watch?v=huQPkrwVWwc&t=878s) Let me go nuts, do whatever I want, and don't ban me." So we're working on that. We're gonna fix that. Uh, next, document your shit. Literally just when you ship features on your product, don't just merge the pull request and walk away.

[14:55](https://www.youtube.com/watch?v=huQPkrwVWwc&t=895s) It's not done until it's documented and the world knows about it, and an LLM can consume the content and put it to use. So always document everything, especially now that LLMs are in charge.

[15:10](https://www.youtube.com/watch?v=huQPkrwVWwc&t=910s) We're still in charge, but you know what I mean. Um, okay, so feed the machines. Uh, basically, it's just a matter of, um, producing content in forms that language models can understand and consume more easily than traditional HTML web pages.

[15:27](https://www.youtube.com/watch?v=huQPkrwVWwc&t=927s) Use boring technology. So this means, um, if a technology has been around for a long time, SQL. SQL statements have been around since,

[15:37](https://www.youtube.com/watch?v=huQPkrwVWwc&t=937s) hmm, I don't know, longer than some of us have been alive. That means that the language models know how to, how to write SQL because they've encountered so much of it.

[15:47](https://www.youtube.com/watch?v=huQPkrwVWwc&t=947s) So when you're building products, be sure to keep in mind that your language models are gonna have a better chance of writing these-- this software and using it if it's a well-established technology that doesn't change a lot.

[16:03](https://www.youtube.com/watch?v=huQPkrwVWwc&t=963s) And lastly, practice good API hygiene. This means when you're writing your HTTP service and you're designing what the JSON response should look like, keep in mind that it's probably going to be going into the context window of a language model now that has limitations.

[16:21](https://www.youtube.com/watch?v=huQPkrwVWwc&t=981s) So instead of dumping a JSON payload response that has everything about all the models under the sun, consider making it a more small, slimmed down,

[16:34](https://www.youtube.com/watch?v=huQPkrwVWwc&t=994s) information-dense version of what an LLM wants to see.

[16:41](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1001s) That's all I got. [audience applauding] Thank you. It looks like I've got

[16:48](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1008s) two minutes if anybody has questions. Maybe. No questions. Okay, I answered everything. Here we go.

[17:03](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1023s) Do you have any recommendations that were good at getting a jump start on, uh, generating your docs for your API? I know that the scalar.com or, um, you're helping build through that.

[17:13](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1033s) Just like quick hot tips on that.

[17:15](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1035s) Yeah, the question is, what are some recommendations for generating docs? So first thing to do, just start by generating your own AP- OpenAPI schema. Write schemas in YAML or JSON that describe the behavior of your API.

[17:29](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1049s) There's a ton of tools out there. Um, there's Doxasaur- Docusaurus, there's, uh, Read the Docs, there's ReadMe dot... What is it? Readme.com. There's a whole bunch of these services that know how to take an OpenAPI schema and turn it into not only documentation, but also, you know, SDKs, um, clients in different programming languages, all that stuff.

[17:53](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1073s) Yeah.

[17:54](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1074s) Are some of the main ways that you're thinking about, uh, discovery or distribution in light of what you mentioned, like the LLMs being in charge and making the purchase decision, uh, in the future potentially?

[18:04](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1084s) Uh, yeah. So the question was, are we thinking about discovery as the LLMs start to make purchasing decisions? Was that it?

[18:09](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1089s) For like, uh, inform us on purchasing decisions and eventually, yeah, maybe even the whole sort of-

[18:15](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1095s) I think the key to that is making sure that our API, um, has really good search capabilities and that a lot of the information that users need to make informed decisions is actually available via API.

[18:26](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1106s) So for example, with Replicate models right now, um, the pricing is currently something that you have to go to the webpage to look at, either on the pricing page or on the individual model pages.

[18:37](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1117s) If we expose pricing, um, you know, as a JSON structure that our API can consume, that a public user can consume, then it becomes a lot easier for you to do something like jump into a session with Claude and say, "Oh, look, I'm ev- of-- evaluating all the video models.

[18:52](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1132s) I'm looking at, you know, Imagen and, or, uh, you know, Veo and Kling and MiniMax and all the other things that are, that are out there. Show me a comparison of which models are the most expensive, which ones are the fastest, which ones can produce the highest quality output, et cetera."

[19:08](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1148s) And if the language model has access to the structured data to answer those questions, then it's gonna be a lot easier to make those decisions.

[19:18](https://www.youtube.com/watch?v=huQPkrwVWwc&t=1158s) All right. Thanks, y'all. [outro music]
