# GPU Cloud Deployment Without Leaving Your IDE — Audry Hsu, RunPod

Audry Hsu

AI Engineer Europe 2026

[Talk page](https://ai.engineer/talks/zDGHt0LB-dA-gpu-cloud-deployment-without-leaving-your-ide) · [Video](https://www.youtube.com/watch?v=zDGHt0LB-dA)

## GPU Cloud Deployment Without Leaving Your IDE

Audry Hsu demonstrates how RunPod Flash moves Python inference functions onto cloud GPUs, then extends image generation into a pipeline with prompt expansion and photo composition.

Familiarity with async Python functions, HTTP requests and the distinction between local code and remote GPU execution will help you follow the demonstration.

### From model training to infrastructure work

[0:35](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=35s)

Yunus, an Oxford attendee, already has a practical reason to use RunPod: his university provides credits, and he spends them on LLM training. That is the starting requirement—access to compute so he can work on a model.

RunPod supplies the GPUs and compute; developers bring their code and models. Audry Hsu describes the work that gets between those two things: aligning CUDA and PyTorch versions, testing new GPU SKUs, resolving hardware-related bugs, and configuring deployment and scaling. **Infrastructure configuration competes directly with model development.** The platform’s purpose is to move that work out of the developer’s path.

![Slide titled “Why Runpod Exists” with three cards about developer time, slow and opaque GPU access, and focusing on building.](https://ai.engineer/api/talk-essay-media/b84c7f82193323c5951adcb2e8ce7719acd908a67a973dd0c504daee12ccffe8)

Why Runpod exists: infrastructure work and slow GPU access distract developers from building.

### From spare GPUs to deployment options

[2:17](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=137s)

Hsu traces RunPod’s origins to Zhen and Pardeep, who founded the company in 2022 with basement GPUs left over from a failed crypto-mining venture. They built a prototype and offered free GPU access on Reddit in exchange for feedback. That became a pattern of building in public with the community; Hsu says the company generated revenue from the beginning.

Hsu reports around 500 developers on the platform, more than 30 data centers across 10 countries, and $120 million in annual recurring revenue. She names France, Romania and Iceland within the European footprint, and also mentions Asia Pacific. RunPod’s [company announcement](https://www.runpod.io/press/runpod-ai-cloud-surpasses-120m-in-arr) separately supports the ARR milestone as a company-reported figure. The customers shown span AI-native companies and large enterprises, connected by a need for flexible, reliable GPU infrastructure.

The product choice depends on how the workload uses that infrastructure:

| Product | Workload fit | Operating model |
| --- | --- | --- |
| Pods | Persistent development or compute environment | Rent on demand, pay by the second, tear down when finished |
| Serverless | Variable request frequency and load | Autoscale workers up and down |
| Clusters | Multi-node training | Coordinate compute across nodes |
| Hub | Quick exploration | Deploy pre-vetted open-source repositories |

A running Pod keeps its GPU reserved for that Pod. Serverless instead adjusts worker capacity to demand; Hsu introduces scale-down as the way to avoid leaving unused capacity running. Hub offers starting points such as ComfyUI, **Stable Diffusion** and vLLM. The demonstration follows the Serverless route.

### Keep the development loop local

[5:43](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=343s)

During inference development, testing a small code change can require a much larger deployment procedure:

1. Commit the change and push it to GitHub.
2. Build a Docker image.
3. Pull the image from a container registry onto a server.
4. Allocate a GPU.
5. Test the result, then repeat.

[RunPod Flash](https://github.com/runpod/flash), a Python SDK, is designed to remove those manual steps from the iteration loop so a function can run on a cloud GPU directly from the local development environment.

The execution boundary is an ordinary async Python function with a Flash endpoint decorator. Flash packages and deploys that function to the GPU cloud, while the surrounding main function and helper functions remain local. File reload repackages and pushes changes so the developer can test again without manually rebuilding and uploading a container. **The GPU function moves; local orchestration stays local.**

The recording uses the then-demonstrated endpoint decorator and `flash run`. The current repository documents `@Endpoint` and commands including `flash dev` and `flash deploy`; those should not be read as the exact API shown onstage.

![Runpod Flash documentation with a Python code block, explanatory paragraph, and Quickstart, Create endpoints, and Examples cards.](https://ai.engineer/api/talk-essay-media/335e816519f3a11bf1b29463640a8a55f524521124c688aa064ec23b948ac9b8)

Flash documentation shows a Python example and describes remote execution alongside local code.

### Send an image request through the local server

[7:42](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=462s)

The first GPU function generates an image with PyTorch and pretrained [**Stable Diffusion XL Turbo**](https://huggingface.co/stabilityai/sdxl-turbo), selected for fast image generation. It saves the image and returns a Base64-encoded representation. The project and dependencies are already installed when Hsu begins running it.

Hsu invokes `flash run`, which starts a local FastAPI development server. She refers first to `image_generation.py` and later to `image_generation_async` while finding the script to run. A local helper sends a POST request to the development endpoint, then decodes the returned image so it can be displayed. The serialization boundary can be expressed with two small Python helpers:

```python
import base64
from pathlib import Path


def encode_image(path: Path) -> str:
    return base64.b64encode(path.read_bytes()).decode("ascii")


def save_image(encoded_image: str, destination: Path) -> None:
    destination.write_bytes(base64.b64decode(encoded_image))
```

The GPU side returns text representing the image bytes; the local side reconstructs a file from those bytes.

The audience supplies the subject: cats flying through a cloudy London sky. Before that prompt reaches the model, the request needs live troubleshooting around its URL, localhost and HTTP. Once the request gets through, the local server reports that the job has started and entered the queue. This separates two questions that are easy to conflate during a demo: whether the request reached the service, and whether it carried the intended model input.

### Configure workers, correct the prompt, swap the model

[11:14](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=674s)

While the job runs, Hsu returns to the endpoint decorator. It carries the endpoint name, GPU-family selection and worker settings. Her spoken GPU-pool name corresponds closely to `ADA_80_PRO`, which the [GPU pool reference](https://docs.runpod.io/references/gpu-types) maps to H100 GPUs with 80 GB of memory; the pool identifier is not an architecture name. The demonstration sets a maximum of five workers and one active worker that stays running.

The first output is a dragon. The service has run, but the intended cats prompt was not passed as a flag. Hsu adds the prompt flag and submits again, then points out the idle-timeout setting, which controls how long a worker remains idle. The corrected request produces cats, but their abstract appearance is not the result she wanted.

The next edit changes the model rather than the request path. Hsu comments out the original implementation and substitutes [**DreamShaper**](https://huggingface.co/Lykon/DreamShaper), which she describes as a **Stable Diffusion 1.5** fine-tune. Her working comparison is XL Turbo for quick generation versus DreamShaper for the artistic and illustrative quality she wants here. She sets DreamShaper to 25 inference steps, reviews the size parameters, and resends the same cats request.

The change stays inside the IDE: edit the function, let Flash propagate it, and send the request again. There is no manual commit, Docker rebuild, upload or GPU-allocation sequence between those two model trials. Hsu and the audience prefer the new image. The useful result is a shorter development procedure and a preferred output in this session; no timed comparison or controlled image-quality evaluation accompanies it.

### Add prompt generation and photo composition

[14:34](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=874s)

A single remote model call is only part of an application. The next example uses prepared orchestration code to connect three stages:

1. Call [**Qwen 3**](https://qwenlm.github.io/blog/qwen3/) on an already-hosted public endpoint to generate richer image prompts.
2. Send those prompts to DreamShaper on the Flash-managed endpoint.
3. Pass the images through [**Nano Banana 2**](https://blog.google/innovation-and-ai/technology/ai/nano-banana-2/), Google’s **Gemini 3.1 Flash Image**, for photo composition.

Hsu’s intended subjects are RunPod’s founders. This extends the local orchestration pattern from calling one GPU function to coordinating prompt generation, image generation and an external composition model.

The audience shapes the new input into two men with glasses walking in London on a cloudy day, with a close-up of their faces. Hsu requests three generated images and starts the pipeline, leaving it running while she answers a pricing question.

### Understand what stays running

[16:55](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1015s)

Asked about pricing, Hsu initially describes a charge for the time each request runs. She then leaves the terminal to inspect the endpoint in the web console. She describes roughly five or six provisioned workers and three running for the three requested photos, before pointing to worker uptime as the billed quantity and discussing an H100 per-second rate.

![Runpod console for “image generator - SDXL turbo,” showing a worker table with location, GPU type, uptime, and version columns.](https://ai.engineer/api/talk-essay-media/3f7215348dab3e0f48250e4e4c15199c12d8d0c209e92b9d610716678fb55ddc)

The endpoint’s Workers tab lists GPU workers and their uptime.

**Worker lifetime, not just inference duration, is the useful cost boundary.** Current [Serverless billing documentation](https://docs.runpod.io/serverless/pricing) specifies charges from worker startup through full stop, including initialization, execution and idle timeout; active workers run continuously. That qualifies the request-only shorthand and matters for the always-on worker in this example. Hsu explains Serverless’s premium over Pods through the scaling it provides. Her recommendation is to start experimentation with a low worker count or a Pod when only one or two GPUs are needed, then use Serverless when the application needs hundreds of workers and distribution across data centers for availability.

### Inspect the generated and composed images

[18:40](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1120s)

The finished presentation separates the intermediate image from the final composition. On the left is DreamShaper’s output from Qwen 3’s expanded version of the original London prompt. The expansion adds thoughtful expressions, weathered faces, soft-focus background clouds, a muted urban palette of grays and deep blues, and overcast lighting. Qwen contributes visual direction before DreamShaper turns it into an image.

On the right is the composed photo of Pardeep and Zhen. Hsu jokes that Pardeep looks handsome while Zhen looks unexpectedly old, then scrolls to the reference photo supplied to the composition stage. Showing that reference makes the final operation easier to understand: the pipeline combines the generated scene with a separate photographic input.

The final visible result pair shows generated and composed images beneath golden trees in Hyde Park, London. The workflow still begins in the local development environment, even though the application coordinates remote inference and composition. Hsu closes by extending that starting point beyond the open-source models shown: developers can also bring their own private models.

![Two image cards labeled generated and composed show people beneath golden trees in Hyde Park, London.](https://ai.engineer/api/talk-essay-media/8793dcee9528293982934e9f3388e4352198f2c089b8392d9e82c74518dc4d5d)

Generated and composed Hyde Park images appear side by side.

## Resources

- [Runpod Flash](https://github.com/runpod/flash) — Python SDK for defining remote GPU functions, dependencies and deployment configuration. The repository documents the current API.
- [Serverless billing and worker costs](https://docs.runpod.io/serverless/pricing) — Explains active and flex workers and charges for startup, execution, idle timeout and storage.
- [Runpod GPU types and pools](https://docs.runpod.io/references/gpu-types) — Lists GPU identifiers, memory capacities and pool mappings, including ADA_80_PRO for H100 GPUs.
- [Stable Diffusion XL Turbo](https://huggingface.co/stabilityai/sdxl-turbo) — Stability AI's model card and downloadable files for SDXL Turbo image generation.
- [DreamShaper](https://huggingface.co/Lykon/DreamShaper) — Lykon's image-generation model repository, with model files and Diffusers usage examples.
- [Introducing Qwen3](https://qwenlm.github.io/blog/qwen3/) — The Qwen team's introduction to Qwen3 models, thinking modes and deployment options.
- [Introducing Nano Banana 2](https://blog.google/innovation-and-ai/technology/ai/nano-banana-2/) — Google's announcement of Gemini 3.1 Flash Image, also called Nano Banana 2, for image generation and editing.

## Slides

### AI Engineer Europe

[0:00](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=0s) [AI Engineer Europe logo in white on black.](https://assets.ai.engineer/sites/aiecode2025/sha256/24e440bf5dfd4ecccd4fffdc9ef9243be07fff10b16e87c6b80fbc594828e3ea.webp)

AI Engineer
EUROPE

### Presenting Sponsor

Google DeepMind is the presenting sponsor.

[0:05](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=5s) [Google DeepMind presenting sponsor screen.](https://assets.ai.engineer/sites/aiecode2025/sha256/9a43b90b83b6eca77e82f00387f025461fb2cd84d99ed7f55e0f7cc2bbdddc7d.webp)

PRESENTING SPONSOR

Google DeepMind

### Platinum Sponsors

[0:10](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=10s) [Platinum sponsors Braintrust, WorkOS and OpenAI.](https://assets.ai.engineer/sites/aiecode2025/sha256/0c718094ceabaf063b52992651f1bd938114cabd140b3cfee0a6769ea91a2d3e.webp)

PLATINUM SPONSORS

Braintrust
WorkOS
OpenAI

### Introducing Runpod

Runpod brings GPUs and compute, aiming to help developers deploy their code and models quickly while spending less time configuring infrastructure and thinking about scaling.

[1:03](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=63s) [Runpod introduction slide describing a platform for building, running and scaling custom AI systems, with a robot illustration.](https://assets.ai.engineer/sites/aiecode2025/sha256/e9f8357bfb35721a7bec6560d7cf350d6175ed1243d7473fa505211a8b685b39.webp)

runpod

Introducing Runpod

The foundational platform for building, running, and scaling custom AI systems.

AIE
AI Engineer EUROPE
Google DeepMind

Interface text:
docs.google.com / Intro to runpod - Runpod first call deck final version
Favorites
work
Chat

[1:32](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=92s) [Runpod introduction describes a platform for building, running, and scaling custom AI systems.](https://assets.ai.engineer/sites/aiecode2025/sha256/bb0a76b57a56599785faf5b4aeaacaf929b9f895ec87bfdb8845d8dc2af4bc09.webp)

runpod

Introducing Runpod

The foundational platform for building, running, and scaling custom AI systems.

AIE
Braintrust
WorkOS
OpenAI

### Why Runpod Exists

Hsu says teams she has spoken with spend more time on infrastructure than models, including aligning CUDA and PyTorch versions and testing GPU SKUs. The slide also flags slow, opaque GPU access.

[1:35](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=95s) [Three cards describe infrastructure consuming developer time, slow and opaque GPU access, and the need for builders to focus on building.](https://assets.ai.engineer/sites/aiecode2025/sha256/b6391e562fc5c9261bbc3a7c8311f71a8ff31e2f1082172482a620351cbf4020.webp)

Why Runpod Exists

Infrastructure eats developer time

GPU access is slow and opaque

Builder primary focus should be building

© 2026 Runpod Inc. All rights reserved. Confidential and proprietary.

AIE
Braintrust
WorkOS
OpenAI

Interface text:
docs.google.com / Intro to runpod - Runpod first call deck final version
Favorites
work
men1

### Runpod at a glance

Runpod reports 500K developers, 30+ data centers in 10 countries, and a $120M annualized run rate.

[3:05](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=185s) [Runpod reports 500K developers, more than 30 data centers in 10 countries, and a $120M annualized run rate.](https://assets.ai.engineer/sites/aiecode2025/sha256/baa30e3fd78f88b5af1a8e9f5f2b013eac96891ec34c5072bbcfa8c362ae960e.webp)

Runpod at a glance

500K
developers

30+
data centers in 10 countries

$120M
annualized run rate

© 2026 Runpod Inc. All rights reserved. Confidential and proprietary.

AIE
Engineering the future of AI

Interface text:
docs.google.com / Intro to runpod - Runpod first call deck final version
Favorites
work

### Running on Runpod

Runpod lists Anthropic, Perplexity, Toyota and Microsoft among its customers. Hsu says they span AI-native companies and large enterprises that need flexible, reliable GPU infrastructure.

[3:24](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=204s) [A grid of customer logos includes Anthropic, Databricks, Microsoft, Perplexity, Wix, Replit, Toyota, Civitai, and Zillow.](https://assets.ai.engineer/sites/aiecode2025/sha256/9feb5605948b1ebdfc146b61e1bee2d2def7ab02a0670d4dd2aaaae9811e050a.webp)

Running on Runpod

ANTHROPIC
Databricks
Microsoft
Perplexity
WIX
Replit
TOYOTA
CIVITAI
Zillow

© 2026 Runpod Inc. All rights reserved. Confidential and proprietary.

Interface text:
docs.google.com / Intro to runpod - Runpod first call deck final version
Favorites
work

AIE
Engineering the future of AI

### How teams build on Runpod

Pods offer persistent GPU VMs; Serverless scales workers with demand. Clusters support multi-node training, while Hub deploys pre-vetted open-source AI repositories.

[3:52](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=232s) [Four columns summarize Pods, Serverless, Clusters, and Hub.](https://assets.ai.engineer/sites/aiecode2025/sha256/257169b6634542870e03c52d39c22b90b71ab3bb45cbb3f1ad5cb4b9ed970264.webp)

How teams build on Runpod

Pods
Blink and it’s ready
Dozens of GPU options
Pay by the second

Serverless
Bring your container
Network storage
Scale globally

Clusters
Instant or reserved options
High-speed networking
Slurm and PyTorch

Hub
Deploy in seconds
One-click deployment
Autoscaling endpoints

© 2026 Runpod Inc. All rights reserved. Confidential and proprietary.

Interface text:
docs.google.com
AIE
Engineering the future of AI

### Serverless

Runpod positions serverless inference for real-time AI products and variable traffic, with automatic scaling, no pre-provisioning, and usage-based payment.

[5:18](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=318s) [A slide describes serverless inference use cases and benefits beside a Runpod console image.](https://assets.ai.engineer/sites/aiecode2025/sha256/c7d966a1229117fac139613be3c21b711abf2952a6d57e69a8cdcff8fc070501.webp)

Serverless

Autoscaling inference without infrastructure overhead

Best for
• Real-time inference
• Variable or spiky traffic
• User-facing AI products

Why teams use it
• No pre-provisioning
• Scales automatically
• Pay only for usage

© 2026 Runpod Inc. All rights reserved. Confidential and proprietary.

Interface text:
Embedded Runpod console: [illegible]
AIE
AI Engineer Europe
Engineering the future of AI


### Runpod Flash overview

Flash aims to shorten GPU development cycles: an endpoint decorator sends a Python function to Runpod while surrounding code runs locally. Flash handles provisioning and worker scaling on Runpod Serverless.

[5:41](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=341s) [Runpod Flash documentation shows an overview and a Python endpoint example.](https://assets.ai.engineer/sites/aiecode2025/sha256/88e97a3e0e8c4ce51d975b4bb40969549dbf35097b7abe3e5c51fda7375ded3e.webp)

Flash
Overview
Build autoscaling AI/ML apps using local code with Runpod Flash.

Flash is currently in beta. Join our Discord to provide feedback and get support.

Flash is a Python SDK for developing cloud-native AI apps where you define everything—hardware, remote functions, and dependencies—using local code.

import asyncio
from runpod_flash import Endpoint, GpuType

# Mark the function below for remote execution
@Endpoint(name="hello-gpu", gpu=GpuType.NVIDIA_GEFORCE_RTX_4090, dependencies=["torch"])
async def hello(): # This function runs on Runpod
    import torch
    gpu_name = torch.cuda.get_device_name(0)
    print(f"Hello from your GPU! ({gpu_name})")
    return {"gpu": gpu_name}

asyncio.run(hello())
print("Done!") # This runs locally

Write @Endpoint decorated Python functions on your local machine. Run them, and Flash automatically handles GPU/CPU provisioning and worker scaling on Runpod Serverless.

Get started
Quickstart
Write a Flash script for instant access to Runpod GPUs.
Create endpoints
Learn how to create endpoints of various types.
Examples
Browse example Flash scripts and apps on GitHub.

Interface text:
runpod
Search...
Docs
Examples
Community
CLI
API
Models
Release notes
Copy page
Get started
Welcome
Quickstart
Concepts
Manage API keys
Agent skills NEW
MCP servers NEW
Flash
Overview BETA
Quickstart
Pricing
Create endpoints
Custom containers
Configure resources
Build apps
Execution model
Troubleshooting
Serverless
Overview
Quickstart
Install SDKs
Pricing
Create handler functions
Deploy workers
Manage endpoints
Runpod console
runpod/docs
Ask AI
AIE
Engineering the future of AI

[6:00](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=360s) [Runpod Flash documentation showing an overview and a Python endpoint example.](https://assets.ai.engineer/sites/aiecode2025/sha256/11500deb402e7ce57cd1def1353e75ea02d19e6439ac90803d85df0786324105.webp)

Flash
Overview
Build autoscaling AI/ML apps using local code with Runpod Flash.

Flash is currently in beta. Join our Discord to provide feedback and get support.

Flash is a Python SDK for developing cloud-native AI apps where you define everything—hardware, remote functions, and dependencies—using local code.

import asyncio
from runpod_flash import Endpoint, GpuType

# Mark the function below for remote execution
@Endpoint(name="hello-gpu", gpu=GpuType.NVIDIA_GEFORCE_RTX_4090, dependencies=["torch"])
async def hello(): # This function runs on Runpod
    import torch
    gpu_name = torch.cuda.get_device_name(0)
    print(f"Hello from your GPU! ({gpu_name})")
    return {"gpu": gpu_name}

asyncio.run(hello())
print("Done!") # This runs locally

Write @Endpoint decorated Python functions on your local machine. Run them, and Flash automatically handles GPU/CPU provisioning and worker scaling on Runpod Serverless.

Get started
Quickstart
Write a Flash script for instant access to Runpod GPUs.
Create endpoints
Learn how to create endpoints of various types.
Examples
Browse example Flash scripts and apps on GitHub.

Interface text:
runpod
Search...
Docs
Examples
Community
CLI
API
Models
Release notes
Copy page
Get started
Welcome
Quickstart
Concepts
Manage API keys
Agent skills NEW
MCP servers NEW
Flash
Overview BETA
Quickstart
Pricing
Create endpoints
Custom containers
Configure resources
Build apps
Execution model
Troubleshooting
Serverless
Overview
Quickstart
Install SDKs
Pricing
Create handler functions
Deploy workers
Manage endpoints
Runpod console
runpod/docs
Ask AI
AIE
AI Engineer
EUROPE

### Flash endpoint example

A Flash decorator sends the async function to a cloud GPU, while the surrounding code runs locally. The example selects an RTX 4090, reads its GPU name, and returns it.

[6:47](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=407s) [Enlarged Runpod Flash documentation showing a Python function decorated as an endpoint.](https://assets.ai.engineer/sites/aiecode2025/sha256/aad699cc14eb64ece970e016251a1929e889bdac513148ffb4606385afea5f93.webp)

Flash is a Python SDK for developing cloud-native AI apps where you define everything—hardware, remote functions, and dependencies—using local code.

import asyncio
from runpod_flash import Endpoint, GpuType

# Mark the function below for remote execution
@Endpoint(name="hello-gpu", gpu=GpuType.NVIDIA_GEFORCE_RTX_4090, dependencies=["torch"])
async def hello(): # This function runs on Runpod
    import torch
    gpu_name = torch.cuda.get_device_name(0)
    print(f"Hello from your GPU! ({gpu_name})")
    return {"gpu": gpu_name}

asyncio.run(hello())
print("Done!") # This runs locally

Write @Endpoint decorated Python functions on your local machine. Run them, and Flash automatically handles GPU/CPU provisioning and worker scaling on Runpod Serverless.

Get started
Quickstart
Write a Flash script for instant access to Runpod GPUs.
Create endpoints
Learn how to create endpoints of various types.
Examples
Browse example Flash scripts and apps on GitHub.

Setup

Interface text:
runpod
Docs  Examples  Community  CLI  API  Models  Release notes
Search...
Get started
Welcome
Quickstart
Concepts
Manage API keys
Agent skills NEW
MCP servers NEW
Flash
Overview BETA
Quickstart
Pricing
Create endpoints
Custom containers
Configure resources
Build apps
Execution model
Troubleshooting
Serverless
Overview
Runpod console
runpod/docs
Ask AI
AIE
AI Engineer EUROPE

### Flash image-generation endpoint

Audry Hsu’s endpoint loads SDXL Turbo on a GPU, generates an image, and returns it as a Base64-encoded PNG with metadata. A helper decodes and saves the image.

[7:42](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=462s) [Code editor showing a Flash endpoint that loads an image-generation model on a GPU.](https://assets.ai.engineer/sites/aiecode2025/sha256/81c0e72f7461b07f6e58df9873b2b71155efae7902df9c4f345c2ec18813fa77.webp)

@Endpoint(
    name="image generator - SDXL turbo",
    gpu=[GpuGroup.ADA_80_PRO],
    workers=(1, 5),
    idle_timeout=900,
    scaler_value=1,
    scaler_type=ServerlessScalerType.QUEUE_DELAY,
    dependencies=["diffusers", "torch", "transformers", "accelerate"],
    env={"HF_TOKEN": os.environ.get("HF_TOKEN", "")},
    execution_timeout_ms=1800000,
)
def generate_image(
    prompt: str,
    negative_prompt: str = "blurry, very low quality, deformed, ugly, text, watermark, signature",
    num_inference_steps: int = 4,
    guidance_scale: float = 0.0,
    seed: int = 0,
    height: int = 512,
    width: int = 512,
):
    """Submit image generation requests to SD Turbo on a cloud GPU."""

    import torch
    import base64
    from io import BytesIO

    # — SDXL Turbo
    model_name = "SDXL Turbo"
    from diffusers import AutoPipelineForText2Image

    pipe = AutoPipelineForText2Image.from_pretrained(
        "stabilityai/sdxl-turbo",
        torch_dtype=torch.float16,
        variant="fp16",
    )
    pipe = pipe.to("cuda")
    image = pipe(
        prompt=prompt,
        negative_prompt=negative_prompt,
        num_inference_steps=num_inference_steps,
        guidance_scale=guidance_scale,
        height=height,
        width=width,
    ).images[0]

    # — DreamShaper 8
    # fine-tuned Stable Diffusion 1.5 text-to-image model for generating artwork.
    # model_name = "DreamShaper 8"
    # from diffusers import StableDiffusionPipeline
    # pipe = StableDiffusionPipeline.from_pretrained(
    #     "Lykon/dreamshaper-8", torch_dtype=torch.float16,
    # )
    # pipe = pipe.to("cuda")
    # image = pipe(
    #     prompt=prompt, negative_prompt="blurry, warped, low quality",
    #     num_inference_steps=35, guidance_scale=9,

Interface text:
Import "torch" could not be resolved
Import "diffusers" could not be resolved
pipeline.py
image_generation.py
Engineering the future of AI
AIE

[8:00](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=480s) [Code loads SDXL Turbo, generates an image, and encodes it as Base64.](https://assets.ai.engineer/sites/aiecode2025/sha256/5870d656a9b6b59e934bb8f71e88575593ce5d3509ecfa45cbb2e3c5ac9801a3.webp)

from io import BytesIO

# — SDXL Turbo
model_name = "SDXL Turbo"
from diffusers import AutoPipelineForText2Image

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/sdxl-turbo",
    torch_dtype=torch.float16,
    variant="fp16",
)
pipe = pipe.to("cuda")
image = pipe(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=num_inference_steps,
    guidance_scale=guidance_scale,
    height=height,
    width=width,
).images[0]

# — DreamShaper 8
# fine-tuned Stable Diffusion 1.5 text-to-image model for generating artwork.
# model_name = "DreamShaper 8"
# from diffusers import StableDiffusionPipeline
# pipe = StableDiffusionPipeline.from_pretrained(
#     "Lykon/dreamshaper-8", torch_dtype=torch.float16,
# )
# pipe = pipe.to("cuda")
# image = pipe(
#     prompt=prompt, negative_prompt="blurry, warped, low quality",
#     num_inference_steps=35, guidance_scale=9,
#     height=1024, width=1024,
# ).images[0]

buffered = BytesIO()
image.save(buffered, format="PNG")
img_str = base64.b64encode(buffered.getvalue()).decode()

return {
    "image_base64": img_str,
    "prompt": prompt,
    "num_steps": num_inference_steps,
    "gpu": torch.cuda.get_device_name(0),
    "model": model_name,
    "seed": seed,
}

def save_image(base64_string: str, filepath: str) -> Path:
    """Decode a base64 image string and save to disk."""
    img_data = base64.b64decode(base64_string)
    path = Path(filepath)
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_bytes(img_data)
    return path

Interface text:
Import "diffusers" could not be resolved
pipeline.py
image_generation.py
Google DeepMind
AIE

### Flash development server

Audry starts a local FastAPI server with uv run flash run, exposing a POST image-generation endpoint. She plans to send it a request and decode the returned image with a helper script.

[8:45](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=525s) [A terminal shows a running development server and an image-generation endpoint beneath the model code.](https://assets.ai.engineer/sites/aiecode2025/sha256/830a2e558ed9b9b1afcb3b702beb91ffd330ccd974cf7c4544db627fe74291e6.webp)

from io import BytesIO

# — SDXL Turbo —
model_name = "SDXL Turbo"
from diffusers import AutoPipelineForText2Image
Import "diffusers" could not be resolved

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/sdxl-turbo",
    torch_dtype=torch.float16,
    variant="fp16",
)
pipe = pipe.to("cuda")
image = pipe(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=num_inference_steps,
    guidance_scale=guidance_scale,
    height=height,
    width=width,
).images[0]

# — DreamShaper 8 —
# fine-tuned Stable Diffusion 1.5 text-to-image model for generating artwork.
# model_name = "DreamShaper 8"
# from diffusers import StableDiffusionPipeline
# pipe = StableDiffusionPipeline.from_pretrained(
#     "Lykon/dreamshaper-8", torch_dtype=torch.float16,
# )
# pipe = pipe.to("cuda")
# image = pipe(
#     prompt=prompt, negative_prompt="blurry, warped, low quality",
#     num_inference_steps=35, guidance_scale=9,

1: uv run flash run
[oh-my-zsh] plugin 'zsh-autosuggestions' not found
zsh: command not found: direnv hook zsh
zsh: command not found: pyenv init --path
zsh: command not found: pyenv init -
flash-get-started (main*) » uv run flash run

Flash Dev Server  localhost:8888

Local path | Description | Type
POST /image_generation/runsync | Submit image generation requests to SD Turbo on a cloud GPU. | QB

Visit http://localhost:8888/docs for Swagger UI
Press Ctrl+C to stop — provisioned endpoints are cleaned up automatically

Interface text:
iTerm2
pipeline.py
image_generation.py
AIE
AI Engineer EUROPE

### Image-generation endpoint request

Hsu tries the audience’s flying-cats prompt against a local endpoint, but the request fails because the URL lacks a scheme. She checks the command’s options and re-enters the request.

[9:57](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=597s) [A terminal shows image-generation parameters followed by an invalid URL schema error.](https://assets.ai.engineer/sites/aiecode2025/sha256/9a4caa105a1a109e3b3cd3523b5735e579ed96e3641555da916c48f6367f153b.webp)

uv run run_endpoint.py --url localhost:8888/image_generation/runsync "cats flying on a cloudy day in the sky somewhere in London"

Input params:
  prompt: a majestic steampunk dragon soaring through a cloudy sky, intricate clockwork de...
  negative_prompt: blurry, very low quality, deformed, ugly, text, watermark, signature
  width: 512
  height: 512
  num_inference_steps: 4
  guidance_scale: 0
  seed: 1337
  num_images: 1

Sending sync request to: localhost:8888/image_generation/runsync
Traceback (most recent call last):
  File "/Users/audryhsu/repos/flash-get-started/run_endpoint.py", line 247, in <module>
    main()
  File "/Users/audryhsu/repos/flash-get-started/run_endpoint.py", line 189, in main
    result = run_sync_local(args.url, payload)
  File "/Users/audryhsu/repos/flash-get-started/run_endpoint.py", line 106, in run_sync_local
    resp = requests.post(url, json=payload)
  File "/Users/audryhsu/repos/flash-get-started/.venv/lib/python3.12/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
  File "/Users/audryhsu/repos/flash-get-started/.venv/lib/python3.12/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/audryhsu/repos/flash-get-started/.venv/lib/python3.12/site-packages/requests/sessions.py", line 592, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/audryhsu/repos/flash-get-started/.venv/lib/python3.12/site-packages/requests/sessions.py", line 700, in send
    adapter = self.get_adapter(url=request.url)
  File "/Users/audryhsu/repos/flash-get-started/.venv/lib/python3.12/site-packages/requests/sessions.py", line 795, in get_adapter
    raise InvalidSchema(f"No connection adapters were found for {url!r}")
requests.exceptions.InvalidSchema: No connection adapters were found for 'localhost:8888/image_generation/runsync'

uv run

Interface text:
Last login: Thu Apr 9 09:11:05 on ttys003
[oh-my-zsh] plugin 'zsh-autosuggestions' not found
zsh: command not found: direnv hook zsh
zsh: command not found: pyenv init --path
zsh: command not found: pyenv init -
AIE
AI Engineer EUROPE
Engineering the future of AI

[10:18](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=618s) [Terminal showing an image-generation request and a URL error.](https://assets.ai.engineer/sites/aiecode2025/sha256/d951b98521b4f31d8e37528b55a67a3b9177702a66c157bb3c6219c87fb70ee8.webp)

uv run run_endpoint.py --url localhost:8888/image_generation/runsync "cats flying on a cloudy day in the sky somewhere in London"

Input params:
  prompt: a majestic steampunk dragon soaring through a cloudy sky, intricate clockwork de...
  negative_prompt: blurry, very low quality, deformed, ugly, text, watermark, signature
  width: 512
  height: 512
  num_inference_steps: 4
  guidance_scale: 0
  seed: 1337
  num_images: 1

Sending sync request to: localhost:8888/image_generation/runsync
Traceback (most recent call last):
  File "/Users/audryhsu/repos/flash-get-started/run_endpoint.py", line 247, in <module>
    main()
  File "/Users/audryhsu/repos/flash-get-started/run_endpoint.py", line 189, in main
    result = run_sync_local(args.url, payload)
  File "/Users/audryhsu/repos/flash-get-started/run_endpoint.py", line 106, in run_sync_local
    resp = requests.post(url, json=payload)
  File "/Users/audryhsu/repos/flash-get-started/.venv/lib/python3.12/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
  File "/Users/audryhsu/repos/flash-get-started/.venv/lib/python3.12/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/audryhsu/repos/flash-get-started/.venv/lib/python3.12/site-packages/requests/sessions.py", line 592, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/audryhsu/repos/flash-get-started/.venv/lib/python3.12/site-packages/requests/sessions.py", line 700, in send
    adapter = self.get_adapter(url=request.url)
  File "/Users/audryhsu/repos/flash-get-started/.venv/lib/python3.12/site-packages/requests/sessions.py", line 795, in get_adapter
    raise InvalidSchema(f"No connection adapters were found for {url!r}")
requests.exceptions.InvalidSchema: No connection adapters were found for 'localhost:8888/image_generation/runsync'

uv run run_endpoint.py --h
usage: run_endpoint.py [-h] [--url URL] [--api-key API_KEY] [--prompt PROMPT] [--negative-prompt NEGATIVE_PROMPT] [--size SIZE] [--width WIDTH] [--height HEIGHT] [--steps STEPS] [--guidance-scale GUIDANCE_SCALE] [--seed SEED]
                       [--num-images NUM_IMAGES] [--output-dir OUTPUT_DIR]
                       [endpoint_id]
run_endpoint.py: error: ambiguous option: --h could match --help, --height

uv run run_endpoint.py --url localhost:8888/image_generation/runsync "cats flying on a cloudy day in the sky somewhere in London"

Interface text:
Last login: Thu Apr 9 09:11:05 on ttys003
[oh-my-zsh] plugin 'zsh-autosuggestions' not found
zsh: command not found: direnv hook zsh
zsh: command not found: pyenv init --path
zsh: command not found: pyenv init -
AIE
AI Engineer EUROPE
Engineering the future of AI

### Image-generation model and server

The demo loads SDXL Turbo onto CUDA while the local dev server shows the image-generation job in queue. Hsu waits to see whether it finishes.

[11:07](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=667s) [Editor showing image-generation model code above server logs.](https://assets.ai.engineer/sites/aiecode2025/sha256/b5989444aa58316d5906f5eac5e6b08f7238a27aa2ce0183fc68b29a371cad15.webp)

# — SDXL Turbo
model_name = "SDXL Turbo"
from diffusers import AutoPipelineForText2Image

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/sdxl-turbo",
    torch_dtype=torch.float16,
    variant="fp16",
)
pipe = pipe.to("cuda")
image = pipe(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=num_inference_steps,
    guidance_scale=guidance_scale,
    height=height,
    width=width,
).images[0]

# — DreamShaper 8
# fine-tuned Stable Diffusion 1.5 text-to-image model for generating artwork.
# model_name = "DreamShaper 8"
# from diffusers import StableDiffusionPipeline
# pipe = StableDiffusionPipeline.from_pretrained(
#     "Lykon/dreamshaper-8", torch_dtype=torch.float16,
# )
# pipe = pipe.to("cuda")
# image = pipe(
#     prompt=prompt, negative_prompt="blurry, warped, low quality",
#     num_inference_steps=35, guidance_scale=9,

uv run flash run
zsh: command not found: pyenv init --path
zsh: command not found: pyenv init -
flash-get-started (main*) » uv run flash run

Flash Dev Server  localhost:8888
Local path | Description | Type
POST /image_generation/runsync | Submit image generation requests to SD Turbo on a cloud GPU. | QB

Visit http://localhost:8888/docs for Swagger UI
Press Ctrl+C to stop — provisioned endpoints are cleaned up automatically
2026-04-09 16:15:58,224 | INFO | LiveServerless:[illegible] | API /run
2026-04-09 16:15:58,913 | INFO | LiveServerless:[illegible] | Started Job:[illegible]
2026-04-09 16:15:58,938 | INFO | Job:[illegible] | Status: IN_QUEUE
2026-04-09 16:15:59,235 | INFO | Job:[illegible] | .
2026-04-09 16:16:00,508 | INFO | Job:[illegible] | ..
2026-04-09 16:16:06,176 | INFO | Job:[illegible] | ...

Interface text:
Import "diffusers" could not be resolved
Looks like you're trying to copy to the pasteboard, but mouse reporting has prevented making a selection. Disable mouse reporting?
Temporarily  More Actions...
AI Engineer EUROPE

### Image-generation endpoint configuration

While the image-generation job waits in the queue, Hsu points to the endpoint decorator: it selects a GPU family, sets a maximum of five workers, and declares the dependencies.

[11:18](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=678s) [Editor showing endpoint settings and image-generation parameters above server logs.](https://assets.ai.engineer/sites/aiecode2025/sha256/34db76971a662b067ef2a20bba56cd2d2a665ee1b31c46454db59710b5884d63.webp)

load_dotenv()

@Endpoint(
    name="image generator - SDXL turbo",
    gpu=[GpuGroup.ADA_80_PRO],
    workers=(1, 5),
    idle_timeout=900,
    scaler_value=1,
    scaler_type=ServerlessScalerType.QUEUE_DELAY,
    dependencies=["diffusers", "torch", "transformers", "accelerate"],
    env={"HF_TOKEN": os.environ.get("HF_TOKEN", "")},
    execution_timeout_ms=1800000,
)
def generate_image(
    prompt: str,
    negative_prompt: str = "blurry, very low quality, deformed, ugly, text, watermark, signature",
    num_inference_steps: int = 4,
    guidance_scale: float = 0.0,
    seed: int = 0,
    height: int = 512,
    width: int = 512,
):
    """Submit image generation requests to SD Turbo on a cloud GPU."""

    import torch
    import base64
    from io import BytesIO

    # — SDXL Turbo —
    model_name = "SDXL Turbo"

Import "torch" could not be resolved

1: uv run flash run
zsh: command not found: pyenv init -
flash-get-started (main*) » uv run flash run

Flash Dev Server  localhost:8888
Local path | Description | Type
POST /image_generation/runsync | Submit image generation requests to SD Turbo on a cloud GPU. | QB

Visit http://localhost:8888/docs for Swagger UI
Press Ctrl+C to stop — provisioned endpoints are cleaned up automatically

2026-04-09 16:15:58,224 | INFO | LiveServerless:[illegible] | API /run
2026-04-09 16:15:58,913 | INFO | LiveServerless:[illegible] | Started Job:[illegible]
2026-04-09 16:15:58,938 | INFO | Job:[illegible] | Status: IN_QUEUE
2026-04-09 16:15:59,235 | INFO | Job:[illegible] | .
2026-04-09 16:16:00,508 | INFO | Job:[illegible] | ..
2026-04-09 16:16:06,176 | INFO | Job:[illegible] | ...
2026-04-09 16:16:23,726 | INFO | Job:[illegible] | ....

Interface text:
iTerm2  Shell  Edit  View  Session  Scripts  Profiles  Window  Help
nvim pipeline.py
pipeline.py  image_generation.py
~/repos/flash-get-started
AIE
AI Engineer EUROPE

### Image model setup

Hsu works in the IDE with SDXL Turbo commented out and DreamShaper 8 active. The server reloads after file changes and reports a job in progress.

[14:13](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=853s) [An editor shows commented SDXL Turbo code above the DreamShaper model setup.](https://assets.ai.engineer/sites/aiecode2025/sha256/97710258991473a4f056b68ae04dc55215799ac4a4a44dedfdc3edde3b8b8cb5.webp)

from io import BytesIO

# — SDXL Turbo —
# model_name = "SDXL Turbo"
# from diffusers import AutoPipelineForText2Image
#
# pipe = AutoPipelineForText2Image.from_pretrained(
#     "stabilityai/sdxl-turbo",
#     torch_dtype=torch.float16,
#     variant="fp16",
# )
# pipe = pipe.to("cuda")
# image = pipe(
#     prompt=prompt,
#     negative_prompt=negative_prompt,
#     num_inference_steps=num_inference_steps,
#     guidance_scale=guidance_scale,
#     height=height,
#     width=width,
# ).images[0]

# — DreamShaper 8 —
# fine-tuned Stable Diffusion 1.5 text-to-image model for generating artwork.
model_name = "DreamShaper 8"
from diffusers import StableDiffusionPipeline
pipe = StableDiffusionPipeline.from_pretrained(
    "Lykon/dreamshaper-8", torch_dtype=torch.float16,
)
pipe = pipe.to("cuda")

Import "diffusers" could not be resolved

1: uv run flash run
Loading pipeline components...: 71% | 5/7 [00:00<00:00, 12.21it/s]
Loading weights: 100% | 196/196 [00:00<00:00, 11030.68it/s]
Loading pipeline components...: 100% | 7/7 [00:00<00:00, 12.86it/s]
0% | 0/4 [00:00<?, ?it/s]
75% | 3/4 [00:00<00:00, 21.69it/s]
100% | 4/4 [00:00<00:00, 21.95it/s]
/usr/local/lib/python3.12/dist-packages/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py:748: FutureWarning: `upcast_vae` is deprecated and will be removed in version 1.0.0. `upcast_vae` is deprecated. Please use `pipe.vae.to(torch.float32)`. For more details, please refer to: https://github.com/huggingface/diffusers/pull/12619#issue-3606633695.
  deprecate(
WARNING: WatchFiles detected changes in '.flash/server.py'. Reloading...
WARNING: WatchFiles detected changes in '.flash/server.py'. Reloading...
WARNING: WatchFiles detected changes in '.flash/server.py'. Reloading...
2026-04-09 16:19:09,798 | INFO | LiveServerless:[illegible] | API /run
2026-04-09 16:19:10,095 | INFO | LiveServerless:[illegible] | Started Job:[illegible]
2026-04-09 16:19:10,123 | INFO | Job:[illegible] | Status: IN_PROGRESS
2026-04-09 16:19:10,420 | INFO | Job:[illegible] | .
2026-04-09 16:19:11,725 | INFO | Job:[illegible] | ..
2026-04-09 16:19:16,290 | INFO | Job:[illegible] | ...

Interface text:
AIE
Engineering the future of AI
iTerm2
nvim pipeline.py
pipeline.py
image_generation.py
generate_image
NORMAL
main

## Full transcript

[0:00](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=0s) [on hold music] Hey, everyone.

[0:16](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=16s) I'm Audry. Um, I work at RunPod. Have-- Were any of you in my earlier session?

[0:24](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=24s) Okay, good, 'cause then I'm gonna s- s- This intro is the same, but what I'm gonna show is l- a little bit different. Um, d- has anyone heard of RunPod or used RunPod before?

[0:35](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=35s) You have? Do, do you mind if I ask you, um, how you've used us or heard about us before? Yeah, as part of our university, we have some RunPod credits.

[0:42](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=42s) So I, I use this as, uh, yeah, for LLM training and- LLM training? Okay, at your university. And where do you go to uni? Uh, u- Oxford. Oxford. Yeah.

[0:49](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=49s) I did a study abroad there one summer. It's awesome there. Yeah. [laughs] Okay. Love it. Thank you. Okay. And your name is? Yunus. Yunus. Okay. So Yunus might know a little bit about this already, but I'll, I'll, I'll talk you guys through a little intro.

[1:02](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=62s) Um, what do we do? We're a AI cloud infrastructure company. Um, and our mission is to build the foundational platform for developers to scale their AI workloads. And basically, what that means is, um, we bring the hardware, we bring the GPUs, and the compute.

[1:23](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=83s) Um, we make it easy for you guys to bring your code, bring your models, and deploy as quickly as possible. We don't want you spending time, um, configuring infrastructure and thinking about things like, um, scaling.

[1:39](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=99s) W- why RunPod exists. A lot of teams that we've talked to, they are all wrestling with the same thing. That infrastructure-- They're spending more time with the infrastructure than, than they are with the models.

[1:49](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=109s) So things like CUDA version, um, alignment, like what versions of PyTorch run well together, um, which, which, uh, new of the GPU, uh, SKUs, like, have been tested, and, like, keep figuring out the bugs there.

[2:05](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=125s) So a lot of that is-- are things that we try to take that, um, configuration problem away from you guys, so you guys can just build, um, focus on training your model or building your apps.

[2:17](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=137s) And a little bit of a backstory about our company. So this is Zhen and Pardeep, our two founders. Um, they started RunPod in 2022. They had a failed crypto mining venture, so they had a bunch of spare GPUs in their basement.

[2:34](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=154s) Um, they built a prototype of what is the foundation of RunPod today, and they just posted on Reddit and said, "Does, does anyone want some free GPUs, um, in exchange for feedback?"

[2:47](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=167s) And that is literally how our company started, and ever since then, we've been building with, in public with the community. Um, so we've been revenue generating from the very beginning, um, which is very, very rare.

[3:00](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=180s) Um, and even today, we have around five hundred developers on our platform.

[3:06](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=186s) We're in thirty-plus data centers across ten countries. Um, in Europe, that includes France, Romania, Iceland, if that's part of Europe, uh, [laughs] Asia Pacific. Um, and we recently hit, um, a pretty big milestone of one hundred and twenty million in annual recurring revenue.

[3:27](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=207s) This is just a quick glance of some of the customers that we have. You might be surprised seeing that some of these are AI native companies, um, and some large enterprises as well.

[3:41](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=221s) And kind of the bottom line of what they have in common is that they need flexible and reliable GPU infrastructure. So, um, I definitely would say we're punching above our weight class.

[3:55](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=235s) Really quick, high level, um, there's different ways to build on RunPod, depending on what you're trying to do. Um, so if you need a more persistent, um, VM environment, then Pods is a great use case.

[4:07](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=247s) Um, if you, you can rent a pod on demand, um, pay by the second, and once you're done, you can tear it all down, um, and start again. Pods are if you need, um, reserved GPU, so as long as your pod is running, then the GPU is yours and no one can take it away from you.

[4:26](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=266s) Um, serverless, if you have-- if you're ready to deploy something and you, and, and you care more about scaling, so your workloads are more variable in terms of, um, frequency and load, um, we help you auto scale, um, your workers for you and scale them back down when you don't have any requests happening, so you don't pay

[4:45](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=285s) for, um, any idle time. Um, clusters, great use case for training, multi-node. And then Hub is also, um, a place where you can deploy already, uh, open source AI repos that have already been pre-vetted by us for popular models, um, like ComfyUI, um, Stable Diffusion, um, vLLM.

[5:11](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=311s) Um, and that's one way if you're just exploring, uh, to just click around and get started really quickly.

[5:18](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=318s) Uh, I'm gonna talk about serverless, um, today, and the product that we just...

[5:26](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=326s) And I'm gonna switch my displays again here really, really quick so we can mirror my screen.

[5:43](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=343s) So one of the things that is a huge pain for developers is if they're still in the iteration or the development phase. So normally, um,

[5:54](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=354s) when you are working on, let's say, some, some code around your, um, inference model and you're still testing things out, you have to make a commit- Push it to GitHub, um, build your Docker image, um, pull it down from the i- um, container registry, [smacks lips]

[6:13](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=373s) um, and then load it onto a server, and then allocate a GPU on it, and then you get to test it and see if it's working as you expected.

[6:21](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=381s) And then you do that all over again [chuckles] until, until you're ready. So the problem that Flash is trying to solve here, and Flash is our Python SDK, is that we want to eliminate all of that iteration cycle so that you can, um, deploy your function on a GPU right from your local development environment.

[6:45](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=405s) And I'll zoom in here really quick. So this is all you need to know about Flash in one little paragraph, is it's... You have a regular async Python function.

[6:58](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=418s) You add our Flash endpoint decorator, and it's gonna deploy and package everything inside your function onto a GPU cloud. Everything ar- around it, your, uh, main function, any helper functions that you have, those all run on your local development environment.

[7:15](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=435s) But if you need GPU compute, that, that can run, um, on the cloud, and you can... We have hot mod, uh, h- file reload, so if you change anything in your application anywhere, then it gets repackaged and pushed up immediately, and you can test and iterate super quickly.

[7:33](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=453s) And I'm just gonna show an example of this.

[7:42](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=462s) Okay. So I have a function here, generate image. Simply, I'm loading PyTorch. I'm loading a pre-trained Stable Diffusion model, Stable Diffusion XL Turbo.

[7:59](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=479s) Really great for fast, uh, generation of images. And [smacks lips]

[8:05](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=485s) I'm going to save the image down and that... gonna return it Base64 encoded. [smacks lips]

[8:12](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=492s) So I can run this right now here. I've already, um, installed all my dependencies. I already have a Flash project going. I'm going to flash run,

[8:27](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=507s) uh, image_generation.py. And what I'm going to actually do is I have a little [keyboard clacking]

[8:34](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=514s) flash, flash run. So flash run spins up a local development server here. Uh, it's just a FastAPI server, and I can send my request here

[8:52](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=532s) to this endpoint, and I'm gonna do that really quickly. Just get to, to my project. And [keyboard clacking]

[9:06](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=546s) this is just a little helper script that's gonna send a post request to it and then, um, decode that image so that you guys c- actually get to see what it looks like once it's generated.

[9:20](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=560s) And it was image_generation_async. Here we go. [keyboard clacking] And let's pass a prompt to it.

[9:38](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=578s) Can I get a help from the audience? [chuckles] What do we wanna generate today? [chuckles]

[9:43](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=583s) Literally anything, anything random. Cats flying in the sky. Okay. Cats flying in the sky. W- what does the sky look like? What time of day is it? Cloudy. Um- London.

[9:56](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=596s) London ... in London. Yeah. Flying in, flying on a cloudy day in the sky

[10:05](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=605s) s- somewhere in London, and I passed it correctly.

[10:17](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=617s) He's, he's looking at it so closely. He's helping me debug live. I love it. [laughs] Thank you. [laughs] It's super hard to see in dark mode, but, uh-

[10:26](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=626s) I passed URL, and that's true. Based after a localhost? Um.. [keyboard clacking]

[10:39](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=639s) I must have... Oh, you've got your... Oh, no, never mind Is it my HTTP?

[10:53](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=653s) Okay, there we go. Okay. Going back to the local dev server.

[11:05](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=665s) It sees the request. It's started the job. It's queued it.

[11:14](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=674s) And we're just gonna wait for a second to see if it finishes. And so while that's happening, let me bring your attention back to... [sighs]

[11:28](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=688s) Make it bigger for you guys. [smacks lips] Um, the endpoint decorator. So this is where all the magic happens.

[11:38](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=698s) I have passed a name for my endpoint. I specify a GPU family, um, so the Ada 80 Pros, these are different variations of NVIDIA H100 cards. Um, I can specify my max number of workers to be five, so I can have at max five of them running at once.

[11:57](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=717s) Um, I just put one active worker, so this is one that's always gonna be running and always on, [smacks lips] and that's definitely a dragon. [chuckles]

[12:06](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=726s) And it didn't take my prompt probably because I...

[12:11](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=731s) Did I not pass it as a... I didn't pass it as a flag.

[12:21](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=741s) Prompt, there we go. Okay, now it's definitely generating cats flying.

[12:30](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=750s) Okay, back to the endpoint decorator. Um, and then there's other different configurations for timeout, which is how long, um, a worker is idle. Here we go.

[12:41](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=761s) Okay, this looks terrible, guys. [laughs] They are cats. They're abstract cats. Um, and I'm not from London, but maybe someone can tell me if this looks like a London Jimny.

[12:56](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=776s) Maybe? Oh, yeah. [laughs]  Okay. So I don't, I don't, I don't like, I don't like what just happened, so what we're going to do instead is we're going to switch out our model.

[13:08](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=788s) So I'm just gonna comment out this code here,

[13:12](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=792s) and then down here let's swap in DreamShaper, which is a fine-tuned, um,

[13:24](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=804s) model based off of Stable Diffusion 1.5. Um, so this one is while... where Stable Diffusion XL Turbo is, um, more optimized for just quick generation, I think this one is gonna generate a more, uh, a better quality image for us, and it's specifically better for kind of like more art and illustrative styles.

[13:46](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=826s) So we've changed some of the parameters in it. It's gonna have a few more, um, inference steps to it. Just gonna set that to 25. Height and width, 10 by 24.

[13:57](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=837s) That's fine. And let's just send the same request again,

[14:03](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=843s) and let's see what happens, what's different. So again, what made this really fast is instead of making a code change, committing it, rebuilding my Docker, uploading it somewhere, and then allocating GPU infrastructure, all of this is happening right here from my IDE, and I never have to leave.

[14:28](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=868s) Oh, it's good. This is good, right, guys? Mm-hmm. We like this one?

[14:34](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=874s) Okay. So one more... one last thing that I'm just gonna show you guys to round things out is I think where, um, using a developer tool like Flash makes a big difference is when you're trying to, um, not just, like, make one single call to one model.

[14:52](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=892s) It's, it's about all the orchestration code around it, right? So I have here a pipeline that I've pre-prepared, um, and what it's going to do is instead of me generating and writing out every prompt, um, it's gonna send a request to Qwen that's already hosted on a public endpoint.

[15:13](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=913s) Um, and Qwen is g- Qwen 3 is gonna generate all the prompts for me. Um, and then after that, it's gonna send that to our, um, DreamShaper running on our endpoint, and then, um, after that, there's one more pipeline that it goes through.

[15:29](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=929s) It's gonna send the request to Nano Banana 2, which is a premium Google model that's really good at composing photos together. Um, and I'm hoping that I can compose some cool pictures of our founders, and I can send them to them after this demo is done.

[15:47](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=947s) Okay. Um, now let's run the whole pipeline here.

[15:56](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=956s) And check... Okay. Prompt, two men walking in London on a... It is cloudy today.

[16:10](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=970s) Cloudy day. Close up of their faces. Um, any other requests for these two men?

[16:21](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=981s) AI engineers? AI engineers? I don't know if it's- How do they look? Like, are they doing something?

[16:28](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=988s) Glasses? I don't know. Glasses? Yeah. Okay. Two men

[16:34](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=994s) with glasses. Close up of their faces. Okay, and let's generate,

[16:43](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1003s) let's generate three of those, and let's compose it together.

[16:55](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1015s) Okay. And so, so how does it work in terms of pricing? Pricing? Sure. Um, so every request that we send to it, you're only charged for how long that request is running.

[17:07](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1027s) So let's see. I'm gonna... I said this whole session was gonna be only in the terminal, but I'm gonna go back into the console just to show you what's, what's running.

[17:21](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1041s) Let's see. This is the endpoint that we created from the terminal. Um, here are the workers.

[17:32](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1052s) I think we said, like, five workers, so we have about five or six here that are provisioned. Um, three of them are running 'cause I asked for three photos.

[17:40](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1060s) Um, and so this is uptime. This is what you're being charged for, and let me see the cost of an H100 right now is .00116 cents per second.

[17:57](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1077s) Is it the same as for a pod, or is it, uh... or is this pricing a bit different? Pricing is a little bit different for serverless versus pods because pods, um, you don't get any of the scaling with it.

[18:08](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1088s) So there's a little bit of a premium for serverless. So what we usually recommend is if you're still, um, experimenting- Then either start with a very low worker count or start with pods, right?

[18:19](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1099s) 'Cause when you're experimenting, you might only need limited number of GPUs, one GPU at a time, two GPUs at a time. Um, serverless for when you need hundreds of workers running on hundreds of GPUs, and you want them distributed for a better availability across different data centers. [laughs]

[18:40](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1120s) Okay, guys. Here, [laughs] here's th- here's our final presentation. So this was our original prompt, two men with glasses walking in London on a cloudy day, close-up of their faces.

[18:49](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1129s) So on the left, this is what, um, DreamShaper generated based off of the, um, prompt engineering that Qwen 3 did for us. So it's a lot better of a prompt than what I sent in.

[19:02](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1142s) It has a lot better cues about notes on, like... I can read it out to you since I know it's hard to see. Um, "Thoughtful expressions and weathered faces.

[19:12](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1152s) Soft focus on background clouds. Muted urban palette with grays and deep blues. Overcast lighting."

[19:19](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1159s) Um, and then on the right is the final composed photo, um, of... This, this is a hand- this is a very handsome picture of Pardeep, and this is somehow a [laughs] very old photo of Zhen.

[19:32](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1172s) And I'm just gonna scroll down to show you that this is the reference photo that I sent it. [laughs] [laughs]

[19:41](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1181s) Um, but overall, um, overall, yeah, I just wanted to show you guys, like, this is how you can get started really quickly. Um, you can start in your local development environment.

[19:52](https://www.youtube.com/watch?v=zDGHt0LB-dA&t=1192s) You can use open sourced models. You can use bring your own model, um, private model. Um, and this was really fun to do. So thank you guys for hanging out with me. [clapping] [outro jingle]
