ComfyUI Workshop with ComfyAnonymous and Jedrick Kosinski

How do you move beyond typing a prompt when you need control over an image’s structure, a model’s behavior or an entire generation pipeline? ComfyUI exposes that pipeline as a graph of connected nodes. Jedrick Kosinski and its original creator, ComfyAnonymous, introduce an open-source canvas for image, video, audio, 3D and text workflows. The graph provides a common working surface for local models and models reached through remote APIs.

Kosinski describes rapid support for new models through implementations redesigned for ComfyUI, including a goal of supporting releases on day one. Local execution spans NVIDIA, AMD and Intel hardware; closed-source models can be accessed through APIs. Community-maintained custom node packs extend the system beyond what the core team can implement.

A generated artifact can also carry its recipe. When an image or video contains intact workflow metadata, dragging it onto the canvas can restore the graph and generation parameters. That makes sharing an output a way to share an editable experiment. Kosinski says most workflows circulating from the preceding year and a half remained compatible with the version shown in the workshop.

Kosinski reports that, after two years, ComfyUI had 78,000 GitHub stars and ranked among the top 150 repositories. He also reports 3–4 million active users, 20,000 daily downloads, and 22,000 custom nodes from 3,000 public developers. These are workshop-time figures; activity windows and counting methods are not specified. He names Amazon, Apple, Tencent and Netflix as adopters and describes widespread use in visual-AI startup backends, without quantifying that broader claim.

The practical attraction is access to controls that a prompt box cannot expose by itself: depth maps, line art, masks and model-specific operations. A creative user can explore combinations interactively; a developer can automate a graph or extend it with custom nodes. Those roles overlap, and the open-source ecosystem lets users contribute a missing capability instead of waiting for the core team.

ComfyAnonymous dates the personal project to January 2023. Six months later, he joined Stability AI, where ComfyUI served as a tool for internal model experimentation. After a year there, he left in June 2024 and joined Jolin and Robin to form the company behind Comfy. The sequence helps explain the software’s emphasis on exposing model internals: it developed alongside hands-on experimentation with those models.

The team photograph comes from a climb up Mount Fuji, which ComfyAnonymous remembers as difficult enough not to recommend. Kosinski missed it after his flight to Japan was rerouted to Alaska for a day; he reached Tokyo only five hours before the others were due to wake up for the climb. The team closes this introduction with an invitation to people interested in open-source generative AI to visit the Comfy careers page.

The first live example is ComfyUI’s familiar galaxy-bottle workflow. Kosinski changes the seed to produce a fresh result, with generation running locally. The initial model is Stable Diffusion 1.5, or SD 1.5. ComfyAnonymous distinguishes its relatively weak base model from the fine-tunes that originally inspired his work: the base model makes a convenient fast demonstration, but it does not represent the image quality of newer models.

At the workshop, the presenters describe SD 1.5 as nearly three years old. Its age does not obscure the graph’s useful structure. ComfyUI splits a Stable Diffusion pipeline into a diffusion model, a text encoder and a variational autoencoder, or VAE, then connects those components to sampling and output operations.

Following these connections is more useful than treating the canvas as a collection of unrelated boxes. The sampler’s output is not yet the final visible image: it passes through VAE decoding before saving.

The presenters then replace the checkpoint with SDXL. Loading it and running the graph takes time on the laptop; disk access and local execution are visible constraints on the demonstration. The resulting bottle looks better to them, although they describe SDXL itself as about two years old. The useful comparison is structural: the model changes while the same basic pipeline remains recognizable.

Opening a video template makes that continuity more obvious. Kosinski says the selected template cannot run because roughly 60 GB of model files are absent from the laptop. That is a description of the missing files, not a VRAM requirement. The template uses Wan2.1, which ComfyAnonymous calls probably the best open video model available at the time. Its graph still uses a sampling node and VAE decoding, with settings appropriate to video.

ComfyAnonymous next adds a node that he describes as a CFG trick. It patches the sampling code’s guidance calculations, illustrating a second kind of reuse: a graph can change how sampling behaves without replacing the whole pipeline. A user can insert an existing node or write a new one to perform that modification.

Kosinski introduces classifier-free guidance, or CFG, through positive and negative conditioning; the negative prompt in this example targets text and watermarks. His subtraction explanation is an intuition for combining the model’s predictions under those conditions. The operation is on predictions during sampling, rather than on finished images or literal prompt strings. The visible graph shows the connected patch above the prompt and sampler nodes, with a purple bottle image on the right.

An audience question about CLIP Text Encode leads into the division of labor inside the model. A diffusion model does not receive the prompt as raw language. A text encoder first turns it into embeddings, and the diffusion model learns to use those representations. In ComfyUI, CLIP is a historical name for the text-encoder interface: early Stable Diffusion models used CLIP, while later models can use other encoders under the same label. Depending on the model, conditioning commonly uses the final or penultimate hidden state.

Why are there two text-encoding nodes? They encode the positive and negative prompts used for CFG. In the setup being discussed, setting the guidance scale to 1 leaves positive conditioning alone. ComfyAnonymous tries that setting, but the first unexpectedly poor result is confounded by the patch node he had added earlier. He identifies that interference before continuing; the subsequent positive-only image is still described as chaotic and poorly defined.

The guidance calculation can be written in terms of a negative-conditioned prediction ε₋, a positive-conditioned prediction ε₊ and scale s:

For arrays or tensors of predictions, the same operation is compact in Python:

def cfg_prediction(positive_prediction, negative_prediction, scale):
    return negative_prediction + scale * (
        positive_prediction - negative_prediction
    )

Increasing the scale extends the prediction farther in the direction from negative to positive conditioning. That explains the presenters’ description of pushing toward what the prompt asks for and away from what the negative prompt specifies. It gives a more defined result in their example; increasing guidance indefinitely is not a general quality rule, because guidance also changes the quality–diversity tradeoff.

The next question concerns the VAE. ComfyAnonymous explains that an important source of Stable Diffusion’s efficiency is generating in a compressed latent representation. Earlier pixel-space diffusion models worked directly over the image’s pixels and were comparatively slow. With latent diffusion, the repeated sampling work happens on a smaller spatial grid.

For the Stable Diffusion architecture discussed here, the VAE downsamples each spatial axis by 8: a 512 × 512 × 3 image becomes a 64 × 64 × 4 latent representation. The fourth latent channel is a reminder that this is a learned representation, not simply a small RGB picture. During text-to-image generation, the graph samples in that latent space and then decodes to pixels. The dimensions explain the architectural reduction; they do not specify a measured runtime speedup.

Once generation is a workflow, it is natural to ask whether evaluation can become another step: can a node decide whether an output is a good image? ComfyAnonymous starts with the difficulty of defining good. A preference learned from one group of users can favor a style that another group does not want. In his account, adding user-preference data during training can make results worse for those other tastes.

The questioner suggests critique models or multimodal prompts that inspect an image. ComfyAnonymous recalls experiments at Stability AI that generated an image from a workflow and then asked another model to rate it. He says those experiments did not work particularly well. The exchange supplies a caution about the evaluation target, rather than a working automated judging recipe.

A separate question asks how to take a graph built in the UI, run it headlessly and scale it as an application. The canvas is an editor; the backend executes the workflow. ComfyAnonymous points to existing third-party inference and workflow-to-app services, without naming a particular provider. At the time of the workshop, an official hosted application-serving solution remained future work.

Kosinski returns to the canvas to show API nodes, introduced in the month preceding the workshop. These spend paid credits to run generation remotely. He opens a template for Black Forest Labs’ FLUX.1 Kontext image-editing model. In the recording, Kontext is available through APIs, and the presenters say support is ready for an anticipated local release. The laptop has no internet connection, so this demonstration cannot run. Remote execution avoids local model execution but introduces a network dependency.

The template browser then moves to Hunyuan 3D. ComfyAnonymous describes a model that produces a voxel-like representation, followed by an algorithm that converts it to a mesh. He finds the generation impressive but the resulting mesh quality limited. The gallery shows example scenes and figures; it is not a live mesh-quality comparison. The required models are absent from the laptop, so this workflow is also inspected rather than executed.

Support for a modality can exist at different layers of the ecosystem:

• Local LLMs: available through community custom nodes, rather than core ComfyUI in the workshop’s version.
• Audio generation: supported, though ComfyAnonymous describes the available model as weaker than some proprietary alternatives.
• Text-to-speech: recently released models already have custom-node support; inclusion in core remains a separate decision.

For ComfyAnonymous, core integration should expose useful additional control—extra parameters or ways to manipulate the model. If it merely wraps an operation already handled well by a custom node, the benefit is less clear.

An audience member asks about virtual try-on: provide a person or mannequin image and an image of a garment, then generate the person wearing it. ComfyAnonymous points to both local and API-based workflows and suggests trying Kontext first. His recommendation is tentative, and he warns that searchable workflows can age quickly as models improve. During the exchange, he clarifies the name as Kontext, with a K.

At this point in the recording, local Kontext execution is still contingent on the expected release. That historical distinction matters when following the workshop today: BFL subsequently released Kontext dev weights on June 26, 2025 under a non-commercial open-weight license, rather than an unrestricted open-source license. The offline template shown here remains an API-era demonstration.

Kosinski contrasts this with workflows he encountered at a Japan meetup. Older try-on systems could require a few dozen nodes: one model found the region to change, and another inpainted the garment into that mask. Newer edit models accept an image and an instruction directly. Masks still have a role, because they can constrain the operation when the model changes parts of the image that should remain untouched.

The next question asks how to preserve a character across a sequence of story images. ComfyAnonymous offers two paths: train a character LoRA, or use an edit model such as Kontext. He places the recent edit-model trend in the wake of GPT-4o image editing and recommends Kontext for this use at the time. He describes its character preservation as very strong, but the workshop does not run a consistency evaluation.

The proposed reference-image procedure is straightforward:

1. Generate a character image that you are happy with.
2. Supply that image as the reference for an edit model.
3. Request the character in a particular scene or activity.
4. Reuse the approved reference for the other scenes in the story.

The reference supplies identity information that would otherwise need to be reconstructed from text each time. This is the workflow ComfyAnonymous describes for the audience, rather than a sequence generated live on the disconnected laptop.

Kosinski expands the character example into a more controlled graph. A LoRA, or low-rank adaptation, modifies a model’s behavior. He describes assigning a character A adaptation to one image region and a character B adaptation to another, with separate strengths and potentially a schedule. A pair of prompt branches can thus grow into a larger graph that specifies where particular concepts should act. The workshop describes this regional-LoRA capability without identifying the node implementation; it should not be confused with simply loading two LoRAs globally.

The two approaches trade preparation for control:

Kosinski recommends Kontext to beginners because the instruction does much of the work. For subject LoRA training, Kosinski suggests roughly 10–30 images, compared with one reference image for the newer edit-model approach. That is his practical guidance for this discussion, not a fixed requirement for every LoRA training task.

ComfyAnonymous then explains the weight-level mechanism: instead of training the full model, train a small patch to its weights. That patch can represent a style, a character or multiple concepts. In the graph, the LoRA is chained into the model path so that downstream sampling uses the adapted model.

The available example is an anime-character LoRA prepared for a Japan presentation. It belongs to the SD 1.5 model family, and the laptop has no corresponding SDXL LoRA. The presenters consider using an anime checkpoint, while warning that the other available models could give poor results. They ultimately decide not to press Run. The canvas shows how to connect the adaptation, but no generated LoRA result is demonstrated.

The presenters browse ControlNet examples next. The template system provides example inputs and generated images in place of a fresh input demonstration. ControlNet is another way to condition generation more explicitly, but ComfyAnonymous speculates that improving edit models may reduce the need for some of these specialized control workflows. His point is that the usefulness of a technique can change even when the capability remains available.

They then open a regional-prompting workflow that applies different prompts to different parts of an image. Kosinski selects the default SD 1.5 model and fixes the seed. They expect it to work if it does not depend on missing input images, though ComfyAnonymous initially qualifies that at least half the workflow should work. The resulting discussion identifies the intended spatial division: night at the top and daytime at the bottom.

Regional conditioning turns location into part of the instruction. Instead of asking one global prompt to express both times of day, the graph associates each condition with an area. ComfyAnonymous believes this old workflow can still work with recent models because the underlying technique belongs to the diffusion pipeline. Once a new diffusion model is implemented, many older sampling and conditioning techniques remain usable. That compatibility is not demonstrated across newer models here, and some techniques become unnecessary or make larger models slower on constrained local hardware.

Kosinski returns to the earlier hesitation about running the anime example. In the early image-generation ecosystem, community fine-tunes could substantially improve a base model, but their datasets could also produce content unsuitable for a conference screen. ComfyAnonymous offers a possible failure path: a slightly broken model might begin with a skin-colored blob and converge toward nudity. That is his explanation of the risk, not a diagnosed trace of the unexecuted example.

The practical rule is to review generated outputs before presenting them, especially when using unfamiliar community checkpoints. Local execution gives users freedom over the models and outputs they run, but it also leaves that review with them. Kosinski attributes some improvement in newer models to more constrained training sets; ComfyAnonymous emphasizes that better models make fewer random mistakes. Both connect that progress to more dependable prompt adherence.

The team reiterates hiring needs across backend, frontend, core model inference and cloud deployment. For users getting started, the two practical entry paths are a local GPU or API nodes. Kosinski reports that some early models have run on eight-year-old GPUs. ComfyAnonymous says models will usually run on a broad range of NVIDIA hardware, while warning that execution can be extremely slow. Being able to load and run a model is a different threshold from making it comfortable to use.

Asked why someone should choose ComfyUI over alternatives, ComfyAnonymous argues for its depth of control, community support and extensions. He also identifies the main drawback: it can be difficult to get into. Kosinski agrees that an unfamiliar node canvas can be intimidating. He describes roughly 3,000 node packs in the ecosystem, alongside the previously cited custom-node and developer counts. The breadth that makes the system powerful also increases what a new user has to understand.

For an enterprise, Kosinski presents local operation and inspectable source code as data-control advantages. A team can examine the implementation and contribute fixes rather than depending entirely on an opaque remote service. He also claims leading inference performance and rapid adoption of new optimizations, though the workshop supplies no comparative performance measurements. These are the presenters’ reasons for choosing the platform, rather than a tested ranking of competing products.

For support, Kosinski points to Comfy Org’s Discord and the large body of YouTube tutorials. He also notes the emergence of paid tutoring. He treats that market as feedback for the product team: if users need to pay someone to make the software approachable, onboarding needs improvement.

The final questions turn from individual graphs to product architecture. Asked whether there is a published roadmap, ComfyAnonymous does not identify one. He describes official cloud workflow execution as work that has not really started yet. Before it can succeed, custom-node installation and management need to become smoother. The node interface will remain, but he expects an additional layer that lets users build a more traditional interface from a workflow graph. New model releases can change those priorities, so he offers a direction without delivery promises.

An audience member sketches the team product that could sit on top: one power user builds a template that preserves style, brand, character or design rules, while other users access it through an API or a restricted interface. Role-based permissions would expose approved parameters without allowing arbitrary graph edits. ComfyAnonymous endorses this direction, but places reliable cloud inference—with custom nodes working correctly—before those higher-level services. The proposed permissions and template controls are not presented as existing features.

Kosinski says a blog post about upcoming work is planned for the end of the event week. One proposed feature is subgraphs: combine several nodes into a single node, then double-click to enter the nested workflow. That would let a graph retain its internal detail while presenting a simpler outer surface.

The other foundation is dependency management. Custom node packs can request different Python packages, creating conflicts in the environment that executes the graph. Kosinski describes isolation or better compatibility reporting as ways to make those dependencies manageable. The workshop’s closing engineering priority is therefore local reliability: once workflows and their custom nodes become easier to run consistently on one machine, building a dependable cloud product becomes easier too.