Blog

Most ComfyUI users tune checkpoints, LoRAs, and samplers to make a workflow faster or fit in VRAM — but almost nobody touches the attention backend, even though it can be the single largest performance lever outside of the model itself. The attention computation is where transformer-based models spend a disproportionate share of their time and memory, and ComfyUI ships several interchangeable implementations of it. Choosing the wrong one for your GPU, or leaving a default that silently falls back to a slow path, is a common cause of “why is this suddenly slower” and “why did this OOM after I upgraded.”

This guide walks through the attention backends ComfyUI registers, what each one actually does, the command-line flags that control them, the fallback order, and how to verify which backend your workflow is really using. Everything below is grounded in the current ComfyUI source and the official documentation, not synthetic benchmarks.

What an Attention Backend Actually Does

Every transformer-based diffusion or video model applies scaled dot-product attention across its layers: the model computes query, key, and value tensors, then multiplies them to produce attention scores. Written naively in PyTorch, this materializes an N × N attention matrix that grows quadratically with sequence length — fine for a 77-token text prompt, catastrophic for a high-resolution image or a video with thousands of spatial-temporal tokens.

An attention backend is a specialized implementation of this operation. The differences between backends boil down to three practical concerns:

• Speed — does it use fused GPU kernels (Flash Attention, cuDNN, Triton) or fall back to generic operations?
• VRAM — does it materialize the full attention matrix, or compute it in tiled blocks that avoid the huge intermediate tensor?
• Precision and compatibility — does it force fp32, respect fp16/bf16, and does it work with masked attention (which video models like Wan heavily rely on)?

ComfyUI exposes these backends through its comfy/ldm/modules/attention.py module and selects one at startup based on command-line flags and what is installed in your environment. Understanding that selection logic is the key to troubleshooting “my workflow got slow” or “OOM after update” reports.

The Backends ComfyUI Registers

ComfyUI registers multiple named attention functions and picks a default. The most relevant ones today are:

• pytorch (SDPA) — PyTorch’s native scaled_dot_product_attention. This is the modern default on most installs.
• sage (SageAttention) — the quantized attention kernel backed by the sageattention package, enabled with --use-sage-attention.
• xformers — Meta’s memory_efficient_attention, used when xFormers is installed and enabled.
• sub_quad — ComfyUI’s sub-quadratic attention, a memory-saving fallback.
• split — split cross-attention, another memory-conservation option (--use-split-cross-attention).
• flash — a Flash Attention path with an automatic fallback to SDPA if it fails.
• basic — the naive PyTorch implementation, the slowest and most memory-hungry path.

There is also an experimental comfy_kitchen_int8 backend and, for Blackwell-generation GPUs, a sageattn3 backend (sageattn3_blackwell). Most users on consumer hardware will realistically choose between SDPA, SageAttention, and xFormers.

Within the SDPA path specifically, ComfyUI sets an explicit backend priority order via torch.nn.attention.sdpa_kernel: Flash Attention → cuDNN Attention → Efficient (memory-efficient) Attention → Math. If your PyTorch is too old to support set_priority, ComfyUI logs a warning and lets PyTorch pick. This means “SDPA” is itself a family of kernels, and the actual kernel can change depending on your PyTorch build and GPU.

Symptom: “OOM After Update” or “Suddenly Slower”

The classic sign that an attention backend changed underneath you is a workflow that worked fine one day and either runs out of memory or slows down dramatically the next. A ComfyUI or PyTorch update can shift which backend gets selected without you changing a single workflow setting.

Diagnose it by looking at the startup console. The relevant lines appear near the top when ComfyUI initializes the model management layer. You are looking for one of these messages:

• Using pytorch attention — SDPA is active
• Using xformers attention — xFormers path selected
• Using sub quadratic optimization for attention… — the sub-quadratic backend is active, which the message itself flags as a possible speed issue
• Using split optimization for attention — split cross-attention is active

If you see the sub-quadratic or split line and you did not intentionally enable it for a low-VRAM card, that is your answer: the backend fell back to a slower, memory-focused path. Conversely, if a recent update pulled in a newer PyTorch that lost Flash Attention support (for example, a driver/torch mismatch), generation can drop to the Math kernel and become several times slower with no error at all — it just works badly.

Another diagnostic is the xFormers bug guard in ComfyUI. Versions 0.0.21 through 0.0.26 of xFormers are treated as broken (a CUDA error triggers when the batch dimension of the query exceeds 65535) and are automatically disabled by ComfyUI. If you pinned an old xFormers version for a legacy workflow and it silently stops being used, this guard is why.

Choosing a Backend for Your GPU and VRAM

There is no single “best” backend — the right choice depends on your GPU generation, driver, PyTorch build, and whether you are VRAM-constrained. The following decision flow reflects how the backends behave in practice:

• Modern NVIDIA GPU (Ampere or newer) with a current PyTorch: leave SDPA as the default. Flash Attention and cuDNN attention are fused kernels that are both fast and memory-efficient, and the automatic priority order handles the rest.
• Want more speed at the same VRAM, on a Turing/Ampere/Ada card: try SageAttention. It uses int8 quantization of the attention computation and can measurably accelerate generation for diffusion and video models, especially on higher-end cards. Install it with python -m pip install sageattention in your ComfyUI environment, then relaunch with --use-sage-attention.
• Very low VRAM (8 GB or less): consider --use-split-cross-attention (or let it fall back to sub-quadratic), which trades speed for reduced peak memory by processing attention in slices.
• Blackwell (RTX 50-series): use the sageattn3 package (sageattn3_blackwell) rather than the original sageattention, which targets older architectures.

A practical caution on SageAttention: enabling it requires the package to be present, and if it is not, ComfyUI logs an explicit error telling you to run the pip install command. SageAttention also has a mask-support caveat — ComfyUI checks whether the installed sageattn signature accepts attn_mask (stored as SAGE_ATTENTION_SUPPORTS_MASK) because many video and editing models pass a mask. If your version predates mask support, masked attention will not behave correctly even though the backend loads.

For memory-troubleshooting specifically, the relevant knob is not the backend alone but how it is combined with model offloading. The split and sub-quadratic backends reduce peak attention memory but do not magically make a 16 GB model fit into 8 GB — they only help at the margin. If you are trying to run a 14B-class video model on a 16 GB card, the effective tools are offloading (moving weights off the GPU between blocks) and quantization, which we cover in a separate guide on running Wan2.2 Animate 14B on 16GB VRAM with BlockSwap.

Reproducible Steps to Verify Your Active Backend

Do not guess which backend you are on — verify it. The reliable sequence is:

1. Launch ComfyUI from a terminal (not a desktop shortcut) so the startup log is visible. Capture the full console output.
2. Search the first hundred lines for the attention selection messages listed above (Using pytorch attention, Using xformers attention, etc.).
3. Confirm your environment: run python -c "import torch; print(torch.__version__)" and, if relevant, python -c "import xformers; print(xformers.__version__)" inside the ComfyUI Python environment.
4. If you changed a flag (for example, added --use-sage-attention), relaunch and confirm the log now reflects the change. Re-run one queue prompt and compare wall-clock time against your baseline for the same seed and workflow.
5. If you see a fallback you did not expect, check the ComfyUI console for the specific warning — it usually names the reason (missing package, broken xFormers version, too-old PyTorch).

Because the SDPA path silently ranks Flash Attention over cuDNN over Efficient over Math, a good sanity check for “am I actually getting the fast kernel” is to watch the first inference after a cold start: if the fast fused kernel is active, there is no giant single-tensor allocation spike and generation completes in line with your expectations. A sudden multi-second pause with high CPU usage right at the first denoise step often indicates a fallback to a non-fused path.

Common Pitfalls to Avoid

• Installing SageAttention but not passing the flag. The package alone does nothing — it must be enabled via --use-sage-attention at launch. If the flag is present but the package is missing, ComfyUI errors out rather than silently ignoring it.
• Pinning an old xFormers. Versions 0.0.21–0.0.26 are auto-disabled, so your “working” config may already have reverted to SDPA without you noticing.
• Assuming attention is the bottleneck. Before chasing backend speedups, rule out the real culprits: model loading off disk, VRAM swapping (watch for CUDA out of memory retries that degrade to CPU), and VAE decode, which is a separate memory spike entirely. We covered the VAE-decode OOM case specifically in our open-source video generation stack overview.
• Ignoring the mask-support caveat on SageAttention. Video and inpainting models that rely on masked attention can produce corrupt output on older sageattn versions, even when everything else looks healthy.
• Mixing environments. The pip install must target the exact Python interpreter that runs ComfyUI. A system-level pip install sageattention against the wrong environment is the most common reason the flag appears to “do nothing” while ComfyUI logs the missing-package error.

Primary Documentation

For the authoritative reference on attention behavior and the full list of command-line options, consult the official ComfyUI documentation and the ComfyUI source repository, particularly the attention.py, ops.py, and model_management modules. Command-line flags and backend selection change between releases, so verify against the version you actually have installed rather than relying on second-hand summaries.

Understanding which attention backend is active — and why — turns a frustrating “why is this slow/OOM” investigation into a five-minute check of your startup log. It is one of the highest-leverage fixes available to local ComfyUI users, and it costs nothing but a careful read of your own console output.

You sized your checkpoint to fit comfortably in VRAM. The sampler step completes without a hitch. Then, at the very last moment — the moment you actually want to see the image — ComfyUI throws torch.cuda.OutOfMemoryError at the VAE Decode node. The workflow was 95% done, and it died at the finish line.

This is one of the most confusing OOMs in local Stable Diffusion because it happens after the part everyone thinks is memory-heavy. The diffusion UNet is no longer allocating; the model is idle. So why did VRAM spike at decode time? The answer is that the VAE is a separate, independently-allocated model with its own precision and its own memory profile — and ComfyUI’s launch flags let you control each of these independently. This guide walks through why the decode step OOMs, how to confirm it, and which exact flags and settings fix it without touching your diffusion model.

Why the VAE Decode Step Is a Separate Memory Spike

The misunderstanding starts with treating your workload as “one model.” It is not. A Stable Diffusion pipeline is several models loaded and unloaded in sequence:

• Text encoder / CLIP — runs first, produces conditioning.
• UNet (diffusion model) — the big memory consumer during sampling.
• VAE decoder — runs last, converts the latent tensor into pixel-space RGB.

ComfyUI’s default behavior loads each model when needed and offloads it afterward to conserve VRAM. By the time sampling finishes, the UNet has been released. When the VAE runs, it is allocated fresh. If the VAE itself — plus the latent, plus intermediate buffers — exceeds what is left after the operating system, the GUI, and any lingering model weights are accounted for, you hit OOM. The failure has nothing to do with your diffusion model being “too big” for your card.

A second, subtler cause is precision. VAE decoders are notoriously sensitive to reduced precision, and running them in fp16 can produce black images — an entirely different failure mode with its own flag (more below). The point for now is that the VAE’s memory and precision are controlled separately from the UNet’s, and that separation is exactly what makes the decode step behave like a hidden second model.

Confirming the OOM Is Actually the VAE

Before changing any flag, confirm you are looking at the decode step and not something else. Three diagnostics will pin it down.

1. Read the node title in the error. ComfyUI wraps node execution failures with the node’s ID and type. If the traceback names VAEDecode (or VAEEncode on the img2img side), you know the VAE node is the boundary. If it names the KSampler or a UNETLoader, your problem is upstream and this guide is not the right fit.

2. Watch the VRAM curve, not the peak. The timing of the failure matters. A decode OOM surfaces after the sampler progress bar reaches 100% and the UI has already shown you the latent preview. If you can reproduce the crash reliably at that exact step across multiple seeds, you have a decode-stage allocation problem, not a random sampling spike.

3. Try a tiny latent first. Temporarily set the empty latent to something like 256×256 (or the smallest multiple-of-8 your workflow tolerates) and re-run. If the OOM disappears at small resolution and returns only as you scale the latent up, the VAE decode allocation is tracking the latent size — direct evidence that decode is the memory cliff.

None of these steps require a GPU monitoring tool or a benchmark. They are all observable in the standard console output and UI.

Fix 1: Move the VAE Off the GPU with –cpu-vae

The single most direct fix, and the one that requires no model changes, is to run the VAE decoder on the CPU. ComfyUI exposes this as a launch flag:

--cpu-vae

The flag is documented in ComfyUI’s comfy/cli_args.py as “Run the VAE on the CPU.” It trades decode speed for stability: the CPU has access to far more addressable memory than VRAM, so a decode that OOMs the GPU will still complete on system RAM. On a 16 GB card that was sitting near its ceiling after sampling, offloading the VAE is frequently the difference between a finished image and a killed job.

There is a real cost. CPU decoding of large latents is slow — a 1024×1024 or larger image can take several seconds to decode rather than a fraction of a second. For batch work or interactive iteration this is painful; for one-off large generations it is often perfectly acceptable. Treat it as a reliable fallback, not a default.

The VAE-related precision flags live alongside it and are worth knowing even if you stay on the GPU. Running the VAE in fp16 is a documented cause of black images; the flag --fp16-vae explicitly warns “might cause black images,” and --fp32-vae forces full precision to avoid artifacts. If you move the VAE to CPU and suddenly get solid black output, the precision flags are the second dial to check.

Fix 2: Decode in Tiles to Cap the Memory Footprint

If you need to keep the VAE on the GPU for speed, the alternative to offloading is to bound its memory use with tiled decoding. Tiling splits the latent into overlapping patches, decodes each independently, and blends the seams. The peak allocation is capped at roughly the size of a single tile rather than the full latent.

ComfyUI ships tile controls directly in the VAEDecode node: a tile_size parameter (often 512 by default in tiled variants) and a temporal_size / tiling toggle depending on the node version. The mechanics are straightforward: smaller tiles lower the peak memory but increase seam blending time. Start at 256 and work upward if you are still OOMing at 512.

A practical caveat worth stating clearly: tiling is not a free lunch for quality. Because the VAE is effectively a Gaussian window that overlaps patch boundaries, aggressive tiling can introduce faint seams or slight color shifts along tile edges. For most photographic or illustrative outputs the artifacts are imperceptible, but for flat-color or very high-contrast images they occasionally show. Test a representative image before committing tiling as your permanent setting.

Fix 3: Tell ComfyUI How Much VRAM to Hold Back

The third lever is tuning how aggressively ComfyUI holds models in VRAM, so the VAE has room to be allocated in the first place. Three related flags govern this, all documented in the same CLI argument file:

• --vram-headroom <GB> — reserves a margin of free VRAM that ComfyUI tries to keep genuinely unused. Give the VAE (and the OS) breathing room by setting a small headroom rather than running at the red line.
• --reserve-vram <GB> — reserves an amount for the OS and other software, so the card is not treated as fully available to ComfyUI.
• --disable-smart-memory — forces ComfyUI to aggressively offload to system RAM instead of keeping models in VRAM when it can. This is the opposite of the default “smart” behavior, and it can free enough VRAM at decode time on constrained cards.

The default caching mode (--cache-ram) already offloads model weights by RAM-pressure thresholds. If you are on a 16 GB card and the decode OOM persists despite tiling and headroom, combine a small --vram-headroom with --cpu-vae for the decode step itself. The flags compose; they are independent dials.

A common trap is reaching for --lowvram or --novram and expecting them to fix a decode problem. Those flags govern the text encoder and model-weight placement when dynamic VRAM is not in use — they are not a VAE control. Using them to chase a decode OOM usually just slows down the whole pipeline without addressing the allocation that is actually failing.

Verifying the Fix Really Worked

After applying any of these changes, verify deliberately rather than assuming the crash is gone because one image rendered.

1. Re-run at the resolution that failed. Do not test at a smaller size. The exact latent that OOM’d before is the only meaningful proof. Run it two or three times across different seeds to rule out a lucky low-water moment.

2. Watch the log line, not just the image. ComfyUI logs model load/unload. After the fix, confirm the VAE is actually being offloaded to CPU (or tiled) in the output — a silent success with no change in behavior means the flag was not applied, often because the launch command was edited in one place but the server was started from another.

3. Check for the precision failure mode you may have swapped in. If you enabled --fp16-vae or changed precision, inspect the output for pure-black frames, not just for the memory error being gone. Solving an OOM by introducing black images is not a fix.

The complete symptom-to-resolution path, from fastest to most thorough, is: confirm the failing node is VAEDecode → tile the decode → add VRAM headroom → fall back to --cpu-vae. Most 16 GB cards will resolve at one of the first two steps; the CPU fallback is the guaranteed escape hatch when nothing else leaves room.

For the broader picture of VRAM management across a whole open-source pipeline — not just the VAE — see our walkthrough of running Wan2.2, CogVideoX, and Mochi on consumer hardware, and for the specific case of a large motion model squeezed onto 16 GB, the Wan2.2 Animate 14B BlockSwap guide.

Conclusion

A VAE decode OOM is not evidence that your GPU is too small for your diffusion model. It is evidence that two independent models — the UNet and the VAE — were being asked to share a memory budget that was never balanced between them. The tools to fix it are already in ComfyUI’s launch flags and node parameters: --cpu-vae to offload the decoder, tiled decoding to cap its allocation, and --vram-headroom / --reserve-vram / --disable-smart-memory to leave it room. Diagnose the failing node first, change one dial at a time, and verify at the exact resolution that failed — and the mystery ‘OOM at 95%’ becomes a straightforward memory-budgeting problem.

For the authoritative list of launch flags and their descriptions, consult the ComfyUI cli_args.py source, which documents every flag referenced in this guide.

What the Error Actually Means

If you run ComfyUI with the ComfyUI-GGUF custom node and see a line like this in your terminal or node console, you are not alone:

Failed to find the quantized version of the model
 or
AssertionError: Torch not compiled with CUDA enabled

The first message is the GGUF loader failing to locate a model file it can load. The second is a related class of failure that surfaces when the quantized weights cannot be handed to an available CUDA backend. Both share a common root cause chain: the custom node expects a specific file type, a specific dependency version, and a specific loader contract, and when any one of those is off, the node quits before any generation begins.

ComfyUI-GGUF is the community-standard bridge that lets low-VRAM cards run large diffusion and large-language models that would otherwise not fit. It works by loading quantized .gguf weight files instead of the full-precision .safetensors checkpoints. That substitution is exactly where the failure modes concentrate: the loader has to find a .gguf file where it expects one, parse its metadata, and map it onto the right model class. When it cannot, you get a terse message that tells you almost nothing about which of those steps broke.

Symptom vs. Cause: A Quick Diagnosis Table

Before changing anything, match your exact symptom to the most likely cause. This is the fastest way to avoid shotgun uninstall-and-reinstall cycles that reset your working setup for no benefit.

• Symptom: Loader says “Failed to find the quantized version” even though you can see a .gguf file on disk. Likely cause: The file is in a directory the node does not scan, or you selected the wrong loader type (for example, a Unet-GGUF loader pointing at a text-encoder GGUF file).
• Symptom: The custom node will not even import at startup; you see ImportError: No module named 'gguf'. Likely cause: The Python gguf package is missing or too old for the node version.
• Symptom: Import succeeds but loading a valid .gguf throws a version or attribute error. Likely cause: A gguf package version mismatch between what produced the file and what your node reads.
• Symptom: Quantized weights load but the run dies with a CUDA error only on certain models. Likely cause: The quantization type or backend does not match your PyTorch CUDA build, or the model requires an attention or offload flag your launch command does not set.

The order of these symptoms mirrors the loader’s internal sequence: import, locate file, parse metadata, allocate tensors. Working through them in that order is the reliable path.

Step 1 — Verify the Dependency Chain First

Most “quantized version not found” reports that are not simply a wrong file path trace back to the gguf Python package. ComfyUI-GGUF reads GGUF files with this library, and the project’s own requirements list a floor version (for example, gguf>=0.13.0 in recent releases). An older copy can fail to parse newer files and produce a misleading “not found” rather than a clear “version too old”.

Check what you have installed and compare it against the node’s requirements.txt:

python -m pip show gguf
# and inside the ComfyUI-GGUF folder:
cat custom_nodes/ComfyUI-GGUF/requirements.txt

If the package is absent, install it inside the same Python environment that runs ComfyUI (not a different venv):

python -m pip install -r custom_nodes/ComfyUI-GGUF/requirements.txt

A very common trap is running the install in a base environment while ComfyUI runs from a venv or a portable (embedded) Python bundle. The node only sees libraries in the interpreter that actually launches main.py. When in doubt, invoke ComfyUI’s own pip rather than the system one.

Step 2 — Confirm the File Exists Where the Loader Looks

The “quantized version” message is, at its core, a lookup failure. The node checks specific model directories for a .gguf file matching the name you picked in the dropdown. Three things commonly break this lookup:

1. The file is a .safetensors checkpoint, not a .gguf, and you assumed the node would convert it automatically. It does not. You need a pre-quantized download (for example the FLUX.1 or SD3.5 GGUF variants hosted on Hugging Face).
2. The file is a GGUF but for a different component type. A text-encoder GGUF selected in a Unet loader (or vice versa) will not match the expected architecture and the node rejects it.
3. The file sits in a custom path not declared in extra_model_paths.yaml, so the node never scans it.

Verify the expected location and type explicitly. Confirm the file extension, the directory, and that the loader node’s type matches the model component you are actually loading (checkpoint vs. clip/text-encoder vs. VAE). This single alignment check resolves a large fraction of first-time failures.

Step 3 — Correct the Quantization and Backend Mismatches

Once the file is found, the next failure class is mismatched quantization types. Different GGUF quantizations (Q2_K through Q8_0, plus the newer IQ variants) have different memory footprints, quality trade-offs, and — critically — different compatibility with certain attention implementations. A file that quantized fine on the uploader’s machine can fail on yours if your PyTorch build or launch flags differ.

Check your launch command for flags that affect where tensors live and how attention runs. For low-VRAM setups, ComfyUI’s --lowvram and --normalvram flags change offload behavior, and --use-split-cross-attention or --use-pytorch-cross-attention change the attention backend. If a GGUF model loads into system RAM only or OOMs when attention spikes, adjusting these flags — not reinstalling the node — is usually the fix.

Also confirm your PyTorch build actually has CUDA. The AssertionError: Torch not compiled with CUDA enabled message appears when the quantized loader tries to move weights to a GPU that the installed PyTorch cannot see. Verify with:

python -c "import torch; print(torch.__version__); print(torch.cuda.is_available())"

If that prints False on a machine with an NVIDIA GPU, the PyTorch install (not the GGUF node) is the problem, and the fix is reinstalling the CUDA-enabled wheel from the official PyTorch index rather than the CPU-only default.

Common Pitfalls That Mask the Real Error

Several habits make these failures harder to diagnose than they need to be:

• Editing the node’s files by hand to “fix” a path. This works until the node updates and silently reverts, at which point the error returns with no obvious cause. Prefer extra_model_paths.yaml for path configuration.
• Cloning the node into a different folder name. ComfyUI registers custom nodes by folder, and renaming the folder can break the loader’s package-relative imports.
• Mixing GGUF versions across a workflow. A workflow saved against one node version can send parameters the newer loader no longer accepts, producing an error that looks like a file problem but is actually a schema drift.
• Ignoring the startup log. ComfyUI prints import warnings and node registration failures at launch. A node that fails to import often leaves a stale or broken entry that later manifests as “not found” errors on load.

Whenever the error is ambiguous, restart ComfyUI and read the first 20-30 lines of the console before loading anything. The import-stage messages contain the specific reason far more often than the in-workflow popup does.

How to Verify the Fix

A fix is only real if you can demonstrate the loader now reaches the parse-and-allocate stage. Use a deliberately small test rather than your full production workflow:

1. Load the GGUF model that previously failed, with an empty prompt and the smallest resolution your model supports.
2. Watch the console: you should now see metadata parsing (file size, quantization type, tensor count) instead of the “quantized version” message.
3. Run one generation to completion. A successful single image or token proves the dependency chain, file location, and backend are all aligned.
4. Only then re-import your real workflow, checking that the node types and parameter schema still match.

If the small test passes but the real workflow still fails, the problem is workflow-specific (a mismatched loader type or stale schema), not the node installation, and you can narrow the search accordingly.

Conclusion

The “Failed to find the quantized version” error in ComfyUI-GGUF is almost never a single fault. It is a symptom of a three-stage chain — dependency, file lookup, and backend alignment — where any one broken link produces the same terse message. Diagnosing in that order (verify the gguf package, confirm the file type and location, then correct quantization and CUDA mismatches) turns a frustrating black-box failure into a short, deterministic checklist. For deeper background on the surrounding tooling, see our piece on running open-source video models on consumer hardware and the specifics of motion-model memory tuning.

For current requirements and the exact list of supported quantizations, always consult the ComfyUI-GGUF repository, since version floors and supported file variants change with each release.

Few ComfyUI errors are as frustrating as the one that interrupts a KSampler mid-run with a single terse line:

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

The message looks like a deep framework bug, but it is almost always a configuration mismatch: one part of your graph is running on the CPU while another expects the GPU. This guide walks through the symptom, the most common causes, how to isolate the offending node or flag, the traps that keep people stuck, and how to verify the fix without guessing.

Because device placement in ComfyUI is determined by a mix of launch flags, per-node dtype choices, and custom-node code, the fix is never a single button. It is a short, methodical diagnostic. I will point to the exact command-line arguments as defined in the ComfyUI source so you are reasoning from authoritative behavior rather than folklore.

What the Error Actually Means

The error is raised by PyTorch when an operation receives two tensors that live on different devices. In a diffusion sampler, the most common trigger is the matrix multiplication inside a linear layer: the model weights sit on cuda:0 while a latent, text-embedding, or conditioning tensor has been placed on cpu (or vice versa). PyTorch will not silently move tensors around for you; it fails fast and prints the two devices it found.

The key clue is the last part of the message — when checking argument for argument mat1 in method wrapper_CUDA_addmm. The addmm reveals the failure is inside a fully-connected layer, not in VAE decode or sampling noise. That already narrows your search to anything that feeds a Linear layer: the UNet/diffusion model itself, text encoders (CLIP/T5/umt5), or a LoRA adapter that attaches to those.

When you see cpu and cuda:0 together, ask one question: what is forcing a component onto the CPU while the rest of the graph stays on the GPU? There are only a handful of answers, and they are all addressable.

Cause 1: VRAM Offload Flags Splitting the Graph

ComfyUI has a family of command-line flags that deliberately run parts of the pipeline on the CPU to save VRAM. When these are mismatched — or when a custom node does not cooperate with them — you get exactly this device split.

The flags are defined in the ComfyUI source (comfy/cli_args.py), and their help text is the clearest reference for what each one does:

• --gpu-only — “Store and run everything (text encoders/CLIP models, etc.) on the GPU.”
• --lowvram — runs the text encoders on the CPU when dynamic VRAM is not enabled.
• --novram — “when lowvram isn’t enough,” offloading even more aggressively.
• --cpu — run everything on the CPU (slow, and a common source of mismatches if you have a GPU too).
• --cpu-vae — “Run the VAE on the CPU.”
• --async-offload and --disable-async-offload — control how weights move between RAM and VRAM.

A frequent story: a user adds --lowvram or --cpu-vae to squeeze a large model onto an 8 GB or 16 GB card, then loads a custom node that assumes every input is a CUDA tensor. The built-in nodes handle offloading correctly, but a third-party sampler or a custom KSampler wrapper can blindly call .to(device) on only some of its inputs, producing the split. The fix is not to remove VRAM saving entirely — it is to identify which node is not offload-aware and either replace it or stop offloading the component it depends on.

Start by reading your launch command. If you are using any of the offload flags, reproduce the error with --gpu-only (or simply without the offload flag, if your card can fit the model) and see whether the error disappears. If it does, you have isolated an offload incompatibility rather than a broken workflow.

Cause 2: Mixed-Precision and VAE Dtype Flags

A second, subtler source of device mismatch is precision flags that change where a component executes. The relevant flags are also in cli_args.py:

• --cpu-vae (again) — forces VAE decode onto the CPU.
• --fp16-vae and --fp32-vae / --bf16-vae — select VAE precision. The --fp16-vae help text even warns it “might cause black images.”
• --force-fp16 / --force-fp32 — global precision forcing.
• --fp8_e4m3fn-text-enc and friends — store text encoder weights in fp8.

Text encoders are the classic offender here. Wan2.2, Flux, and other new pipelines load a large T5/umt5 text encoder that users routinely quantize to fp8 or offload to CPU to fit in VRAM. If the encoder weights are stored with --fp8_e4m3fn-text-enc but a downstream conditioning node expects them in bf16 on the GPU, PyTorch can surface the device error while it is really a dtype-and-placement disagreement.

The diagnostic is the same shape as before: temporarily remove the precision flag, run once, and watch whether the error location moves or clears. If the failure moves from the text encoder into the sampler, you have confirmed the encoder precision was the trigger. (For a closely related, name-specific failure — the umt5 text encoder refusing to load — see our earlier note on fixing the umt5-xxl fp8 filename; the two issues often appear together when a filename and a dtype both go wrong.)

Cause 3: Custom Nodes That Assume a Device

The third cause is the most common on a heavily extended install: a custom node that hardcodes or mis-infers a device. Any node that calls torch.Tensor.to(), torch.zeros_like, or torch.empty with an explicit or implicit CPU device can introduce the split, even when every launch flag is correct.

To tell whether the node or the flag is at fault, use ComfyUI’s built-in node-by-node execution. Open the workflow, disable --async-offload if it is on, and run. Then read the full traceback where the error is raised — the line that fails references the node’s class_def or its def input_types. Note the node type, then search its source in custom_nodes/ for the word .to( or device=. This is a text search, not guesswork, and it almost always pinpoints the offending line.

Common culprits include:

• Video-pipeline wrappers (WanVideo, AnimateDiff) that build frame tensors with torch.zeros but forget to move them to the model’s device.
• Upscaler or face-restoration nodes that run a sub-model on CPU by design but forward GPU latents into it.
• LoRA loaders that patch a model in memory but leave the base weights on a different device.

When you find the offending node, the short-term fix is either to update it, replace it with a maintained equivalent, or stop combining it with the offload/precision flag that the traceback implicates.

A Repeatable Diagnostic Sequence

Rather than toggling settings at random, work through this fixed order. It separates “flag problem” from “node problem” in minutes:

1. Capture the launch flags. Note every --lowvram, --cpu-vae, --gpu-only, --force-fp16, and --fp8/--bf16 flag you are using, straight from your startup script.
2. Read the devices in the error. The pair at the end (“cpu and cuda:0”) tells you the two locations; the operation (addmm, add, copy_) tells you where in the pipeline.
3. Bisect with a clean run. Drop all offload and precision flags (if VRAM permits) and run the same workflow. Error clears → it is a flag/node interaction. Error persists → it is the workflow or a node.
4. Re-add flags one at a time. When the error returns, the last flag you re-added is the culprit for that node.
5. Locate the node. Read the full traceback to get the node class, then grep its source for device or precision calls.

This sequence respects one important reality: the correct reference for every flag’s exact meaning is the source help text linked below, not a forum summary. Flags drift between versions, and a flag that meant one thing in 2025 can change by 2026.

Common Traps That Keep You Stuck

• Reinstalling PyTorch or ComfyUI first. The error is a placement bug, not a broken install. Reinstalling delays the real fix and can change your CUDA version, introducing new problems.
• Assuming it is always the VAE. The addmm signature points at a linear layer, which is the diffusion model or text encoder — not the VAE decode. Chasing the VAE wastes time when the traceback says otherwise.
• Stacking every VRAM flag at once. --lowvram + --cpu-vae + --novram together create multiple CPU/GPU boundaries that multiply the places a mismatch can appear. Use the minimum offload that fits your card.
• Ignoring the traceback in favor of the forum fix. The exact argument name in the error is your best clue. Read it before you search.

Verifying the Fix

After you make a change, verify it with a deliberate method, not a single lucky render:

• Run the workflow three times in a row. A device mismatch is deterministic on a given graph — if it clears once it clears consistently, but a flaky appearance means an offload timing issue (--async-offload) rather than a hard placement bug.
• Watch startup and run logs for device-placement lines. ComfyUI logs which component is being loaded where; confirm the text encoder and VAE are on the device you intend.
• If you changed a flag, record the working command so the fix survives a restart and a tool upgrade.

For the broader picture on how ComfyUI’s VRAM flags interact — --lowvram, --normalvram, --cpu, and the smarter dynamic offloading — see our earlier deep-dive on ComfyUI VRAM launch flags, and for the CUDA-out-of-memory side of the same coin on a 16 GB card, our note on llama.cpp CUDA OOM applies the same offload reasoning to local LLM inference.

Conclusion

“Expected all tensors to be on the same device” is a routing problem, not a mystery. It happens because ComfyUI legitimately splits work across CPU and GPU to fit large models — and a mismatch between a launch flag, a precision setting, or a custom node’s assumptions breaks that split. Work systematically: capture your flags, read the devices and operation in the traceback, bisect with a clean run, and locate the exact node before changing anything. Once you have found the boundary that is being crossed, the fix is usually a single flag or a single node update — but only the diagnostic sequence will tell you which one.

References

• ComfyUI command-line arguments as defined in source: comfy/cli_args.py (authoritative flags for --lowvram, --gpu-only, --cpu-vae, --async-offload, and the fp8/bf16 precision flags).
• PyTorch device semantics for the .to() method and tensor placement: torch.Tensor.to.

If you are wiring ComfyUI into an automation pipeline, the first thing most people reach for is a webhook: submit the prompt, give it a callback URL, and get notified when the image is ready. That assumption is the source of a lot of wasted debugging, because ComfyUI does not expose a webhook endpoint. There is no POST /prompt callback URL, no notify_url field, and no built-in outbound HTTP hook in the current server.

This guide is based on reading the ComfyUI server and execution source directly, not on a particular UI version. The automation path that actually works has three reliable pieces: the WebSocket event stream at /ws, the per-prompt history endpoint at /history/{prompt_id}, and the queue state endpoint at /queue. Understand all three and you can build notifications that are at least as reliable as a webhook, without waiting for one to be added.

Why There Is No Webhook (And What To Use Instead)

ComfyUI’s HTTP server (server.py) exposes a fixed set of routes. The prompt submission route is POST /prompt, and it returns immediately with a prompt_id plus a number (the queue position). It does not accept a callback URL, and nothing in the request body causes the server to make an outbound request later. Job completion is communicated only two ways: over the in-band WebSocket channel, or by polling the history endpoints.

This is a deliberate design. A webhook would require ComfyUI to hold outbound network state, handle retries, and trust caller-supplied URLs — all friction for a local-first tool where the client is usually on the same machine. The result is that you own the notification. The good news: everything you need to build one is already there.

Note that the newer /api/jobs endpoints (visible in the route table) expose job objects with lifecycle state, but for image-generation automation the three endpoints above remain the practical, stable surface. Different ComfyUI frontends and third-party runners agree on them precisely because they are the lowest common denominator.

The WebSocket Stream: The Event-Based Replacement for a Webhook

The closest thing to a push-based callback is the WebSocket at GET /ws. It accepts an optional clientId query parameter; if you omit it, the server assigns one and returns it in the initial status message. Reuse that clientId across reconnects so the server keeps you bound to the same logical session.

When you submit a prompt, you can pass extra_data.client_id in the JSON body. The server folds that value into the queued item, and the execution engine uses the socket bound to that id to stream progress. The message types you care about are:

• executing — sent as each node starts, with the node id
• executed — sent per node as it finishes, including the node’s output
• execution_success — sent once when the whole prompt finishes, carrying the prompt_id
• execution_error / execution_interrupted — the failure signals
• progress — per-step sampler progress when a progress handler is active

A minimal polling-free listener looks like this (Python, using the websockets library):

import json, asyncio, websockets

async def listen(client_id):
    uri = f"ws://127.0.0.1:8188/ws?clientId={client_id}"
    async with websockets.connect(uri) as ws:
        while True:
            msg = json.loads(await ws.recv())
            if msg["type"] == "execution_success":
                print("DONE", msg["data"]["prompt_id"])
                return
            if msg["type"] in ("execution_error", "execution_interrupted"):
                print("FAILED", msg)
                return

The key operational detail: the event is tied to your client_id. If you open the socket without passing the id you submitted with, or if you submit without extra_data.client_id and listen on a fresh socket, you will not reliably receive the completion signal. Align the two ids and this behaves like a push notification.

Polling /history/{prompt_id}: The Durable, Restart-Safe Fallback

WebSocket messages are ephemeral. If your listener process dies, sleeps, or reconnects at the wrong moment, the execution_success frame is gone and you have no record. For long-running or unattended pipelines, poll GET /history/{prompt_id} instead. It returns a durable record keyed by prompt id:

{
  "1234": {
    "prompt": [...],
    "outputs": {
      "9": {"images": [{"filename": "ComfyUI_00042_.png", ...}]}
    },
    "status": {"status_str": "success", "completed": true}
  }
}

Poll it on an interval (a few seconds is plenty for image generation), and treat status.status_str as the source of truth: success means the outputs in that same object are final. This pattern survives a client restart, a network blip, and even a ComfyUI restart (history is kept in memory, so a full server restart clears it — see the caveats below).

Inside the output object, the images array gives you filenames under the ComfyUI output directory. You can fetch them over GET /view?filename=... or read them straight off disk when running locally.

Reading /queue To Decide When To Submit

Blindly submitting prompts is how you end up with a backlog. GET /queue returns queue_running and queue_pending, so you can gate submissions:

import requests
q = requests.get("http://127.0.0.1:8188/queue").json()
ahead = len(q["queue_running"]) + len(q["queue_pending"])
print(f"{ahead} items ahead of a new submission")

This matters most when multiple workers or a web frontend share one ComfyUI instance. queue_running is a single item (the currently executing prompt), while queue_pending is the waiting list. If a job is already in flight, your new prompt will simply queue behind it — the number returned by POST /prompt tells you exactly how far back you are.

Common Pitfalls When Automating

Several failures come up repeatedly and are almost never ComfyUI bugs:

• client_id mismatch. The WebSocket received execution_success for nothing because the listener’s id differs from the submitter’s id. Pass extra_data.client_id explicitly, and reuse it.
• Assuming JSON output is a URL. History gives you a filename, not a public URL. You must resolve it via /view or the filesystem; a webhook-style consumer that expects a URL will silently break.
• Reliance on in-memory history. /history is not persisted to disk. Restart the ComfyUI process and prior records vanish. If you need durable job tracking, persist prompt ids and outputs yourself.
• Ignoring execution_cached. When a node’s inputs are unchanged, ComfyUI may return cached outputs and emit execution_cached rather than re-running it. Your listener should treat cached completion the same as fresh completion or you will stall waiting for an executed frame that never comes.
• Polling too aggressively. History-polling every 100 ms just adds load to a local server. A 2–5 second interval is enough and far friendlier to the aiohttp event loop.

Verifying Your Automation End To End

Build a tiny harness that strings the three pieces together and confirm each layer independently before trusting a big pipeline:

1. Submit a trivial no-op or tiny workflow and capture the returned prompt_id.
2. Queue check: confirm /queue shows queue_running populated while it executes.
3. History check: poll /history/{prompt_id} until status.status_str == "success", then read the output filenames.
4. Socket check: with the same client_id, capture execution_success and confirm it carries the matching prompt_id.

Only after all four pass should you scale up to real workflows. The WebSocket path is what you want for low-latency, interactive-looking notifications; the history-polling path is what you want for unattended batch jobs that must survive a listener restart. Used together — WebSocket as the fast signal, history as the durable record — they cover everything a webhook would, without depending on a feature that does not exist.

Conclusion

Stop looking for a ComfyUI webhook: there is not one, and the maintainers’ local-first design makes one unlikely to appear in the form people expect. Instead, treat the WebSocket /ws event stream as your push channel, /history/{prompt_id} as your durable source of truth, and /queue as your backpressure gauge. Align the client_id across submit and listen, resolve filenames yourself, and persist prompt ids if you need records to survive a restart. That combination gives you reliable, auditable automation on top of the API ComfyUI actually ships.

For the broader API automation harness that handles submission, history, and output download in one place, see our guide on building a reliable Python harness for ComfyUI. If you are running video models on a local GPU and want the resource picture before automating, the Wan2.2 Animate 14B BlockSwap walkthrough walks through VRAM constraints that affect queue timing. For the authoritative API surface, refer to the official ComfyUI endpoints documentation and the ComfyUI server source, which is the ground truth when UI behavior and documentation disagree.

You load a quantized model, fire up llama-server or llama-cli, and the first few turns feel snappy. Then something strange happens: by the time the conversation reaches a few thousand tokens, generation crawls to a fraction of its original speed, memory usage climbs, and every prompt takes a painfully long “prefill” burst before the first token appears.

This is not a broken model, and it is not your GPU degrading. In the overwhelming majority of cases it is the KV cache—the growing attention state that stores every token already generated—combined with how you have (or have not) told llama.cpp to offload it and accelerate its computation. This article walks through the symptoms, the diagnosis, the exact flags to change, and how to verify the fix without guessing.

The Symptom: Fast Start, Then a Slow Crawl

The classic signature is easy to recognize once you know what to look for:

• Short prompts are fast. A single-turn query returns quickly because the KV cache is small and cheap to process.
• Long chats degrade. As the history grows, prompt processing (“prefill”) takes longer before the first generated token, and per-token generation drops.
• RAM usage grows. The KV cache scales linearly with context length, and if it has spilled off the GPU it is being read from system RAM on every decode step.
• The model eventually “forgets” or truncates. When the cache reaches the configured context size, llama.cpp triggers context shift: it drops the oldest tokens and reprocesses the remainder.

Any one of those is a signal, but the combination of growing memory plus longer prefill plus slower decode points squarely at KV cache management rather than at the model weights themselves.

Diagnosis: Separate Model VRAM from KV Cache

Before changing any flag, confirm where your memory is actually going. Run your server with the default settings and watch both GPU and system memory during a long session. On Linux use nvidia-smi for VRAM and free -h for RAM; if you are running the server under a manager such as Ollama, it exposes similar numbers in its own logs.

Two distinct regions matter here, because llama.cpp treats them differently:

• Model weights. The quantized tensors that get partially or fully offloaded to the GPU via --n-gpu-layers (short form -ngl). These are mostly static during inference.
• KV cache. The key and value tensors that grow with every token in the active context. Their size depends on context size, model dimension, and the KV data type you allow.

The single most common misconfiguration is loading the weights onto the GPU but leaving the KV cache — by default — unquantized and potentially split awkwardly across devices. By default the KV cache is stored at f16, which for a large context can consume more memory than the model weights themselves. Understanding this split is what turns the fix from trial-and-error into a deliberate configuration.

The Fix: Tune Offload, Flash Attention, and Context Shift

Here are the levers that matter, in order of impact.

1. Put the KV cache in the right device

-ngl / --n-gpu-layers controls how many of the model’s layers are offloaded to the GPU. The KV cache placement is tied to where the relevant layers live. A common “slow long-context” failure is loading only some layers to the GPU so that the KV cache ends up straddling GPU and CPU, forcing data movement on every decode step. If you want the KV cache to stay on the GPU, offload all layers (-ngl 99 or higher) so the K/V tensors are not split.

The inverse is also useful on memory-constrained cards: -ot / --no-offload-kqv keeps the K, Q, and V tensors on the CPU while the rest of the model runs on the GPU. This trades speed for lower VRAM and is a legitimate choice when you are trying to fit a large model into a small card with a long context.

2. Quantize the KV cache to shrink it

The single highest-leverage change for long-context degradation is quantizing the KV cache itself. llama.cpp exposes --cache-type-k and --cache-type-v (in some releases combined into a single --cache-type). Allowed values include f16, q8_0, and q4_0. Moving from f16 to q8_0 roughly halves the KV cache memory for a modest quality cost; q4_0 reduces it further with a larger quality trade-off.

This matters precisely because the KV cache, not the weights, is what balloons during a long chat. Quantizing it lets you hold a bigger context in the same VRAM, which postpones the moment context shift kicks in.

3. Enable Flash Attention for the attention math

-fa / --flash-attn switches attention to a fused Flash Attention kernel. On supported hardware this speeds up both prefill and decode and, importantly, lowers the memory footprint of the attention computation. If your GPU supports it, this is almost always a strict win for the long-context scenario that is causing the slowdown. If the build was not compiled with it, the flag will report as unavailable rather than silently failing.

4. Control context length and mmap behavior

• -c / --ctx-size sets the maximum tokens of history the KV cache can hold. Setting it too high wastes VRAM reserved up front; setting it too low causes premature truncation. Match it to your actual use (for example 4096 or 8192 for RAG, larger only if you genuinely need it).
• --no-mmap disables memory-mapping of the model file and instead loads it fully into RAM. This can improve stability on some systems at the cost of higher startup RAM and longer load time, but it does not affect the KV cache directly—apply it only if you observe page-cache thrashing or load-time issues.
• Context shift / --no-context-shift controls what happens when the cache fills. By default llama.cpp shifts the context (drops the oldest tokens and re-processes). This re-processing is one of the visible “stalls” users describe. Disabling context shift forces a full recompute of the entire history, which is almost always worse; instead, reduce the likelihood of hitting the limit by quantizing the cache and sizing the context correctly.

Common Pitfalls That Look Like This Bug but Are Not

Before you conclude it is the KV cache, rule out a few lookalikes:

• Thermal throttling. A card that boosts for the first minute and then throttles under sustained load produces the same “fast then slow” curve. Check nvidia-smi clocks and temperature before blaming the cache.
• Model weights partly on CPU. If you set -ngl too low, every layer left on the CPU serializes the pipeline. This is a weights-offload problem, not a KV cache problem, though the two are often confused because both are fixed by raising -ngl.
• Context size grossly oversized. Reserving a 32K context on an 8GB card leaves little room for anything else and can cause the GPU to spill or the OS to swap, both of which look like slow degradation.
• Concurrent workloads. If another process—say a ComfyUI image pipeline—is competing for the same 16 GB card, both will degrade. Always confirm GPU exclusivity when benchmarking.

Verifying the Fix

Change one flag at a time and measure, so you know which change actually helped:

1. Run a fixed multi-turn prompt of a known token length with your baseline flags and record the prefill time and tokens-per-second shown in the server log.
2. Apply --cache-type-k q8_0 --cache-type-v q8_0 and re-run the identical prompt. Note the change in decode speed and the lower VRAM reported by nvidia-smi.
3. Add -fa and re-run. Confirm the flag was accepted (the log states whether Flash Attention is active) and note the prefill improvement.
4. Finally, load a genuinely long context near your -c limit and confirm the server no longer stalls on context shift the same way.

If decode speed improves but prefill is still slow, you have fixed the cache but still need better attention (step 3). If memory drops but speed does not improve, your bottleneck is elsewhere—return to the pitfalls list and re-check clocks and layer offload.

Putting It Together: A Practical Starting Point

A reasonable baseline for a 16 GB card running a ~14B quantized model with a moderate context looks like this (flags shown as llama.cpp accepts them):

llama-server -m models/model-Q8_0.gguf \
  -ngl 99 \
  -c 8192 \
  -fa \
  --cache-type-k q8_0 \
  --cache-type-v q8_0

Adjust -ngl down (and consider --no-offload-kqv) only if you hit CUDA out of memory during load or when the context grows. Adjust the context size down if you still see stalls. The point is not any single magic value; it is understanding that the long-conversation slowdown is a cache problem, and that the cache is something you can deliberately size, quantize, and accelerate.

Conclusion

When a local model starts fast and ends slow, the weights are almost never the culprit—the KV cache is. By separating model memory from cache memory, quantizing the cache, enabling Flash Attention, and right-sizing your context, you can hold longer conversations without the telltale crawl. Make one change at a time, measure with real logs, and you will know exactly which lever fixed it.

For more on the lower-level mechanics of quantized models and GGUF loading, see our guide on llama.cpp CUDA out-of-memory and layer offload, and for tuning ComfyUI’s own GPU memory under the same constrained card, see our guide to ComfyUI VRAM launch flags.

Primary Documentation

For the current, authoritative definitions of these flags—including --cache-type-k/v, --flash-attn, --n-gpu-layers, and context-shift behavior—consult the official llama.cpp repository and documentation. Flag names and defaults change between releases, so verify them against the exact version you have installed.

One of the most common first-run failures in a Wan2.2 workflow has nothing to do with the 14B diffusion model at all. The queue starts, the model loads, and then the whole graph stops at the text-encoder node with a message that is easy to misread. This article explains why the Wan2.2 umt5-xxl text encoder so often fails to load, what the filename and directory rules actually are, and how to fix the mismatch without re-downloading a single 5GB checkpoint.

Why the Text Encoder Fails First

Wan2.2 is a latent video-diffusion family that depends on a large multilingual text encoder, commonly referred to as umt5-xxl, to convert prompts into conditioning for the diffusion model. Unlike the smaller CLIP-based encoders used by SD1.5 and SDXL, umt5-xxl is a big transformer on its own. In its fp8 form it still weighs roughly 5.7GB, and in many distributions it ships as a separate .safetensors file rather than being bundled into the diffusion checkpoint.

Because it is a separate file, the loader node has to resolve it by name, and the name has to resolve to a file inside a specific directory. When either the filename or the directory does not match what the node expects, the node fails before any sampling happens. The error usually looks like the model “is not there” even though it clearly exists on disk.

The Three Things That Must All Agree

A text-encoder load succeeds only when three independent values line up. When diagnosing a failure, check each one in order.

• The file actually exists on disk, in the directory ComfyUI scans for text encoders.
• The filename in the node matches the file on disk exactly, including case, underscores, and the .safetensors extension.
• The loader node type matches the encoder format — a CLIP loader, a WanVideo-specific text encoder loader, or a generic loader.

Almost every “text encoder not loading” report on a Wan2.2 workflow reduces to one of these three values drifting out of sync, most often after a model was downloaded under a slightly different name than the workflow template expected.

The fp8 Filename Contract

Wan2.2 community workflows and the Comfy-Org official releases use a consistent naming scheme for the text encoder. The fp8 quantized encoder is typically distributed as umt5_xxl_fp8_e4m3fn_scaled.safetensors, while the higher-precision variants drop the fp8 token or use a different precision suffix. The important detail is that the name is a contract, not a suggestion: a workflow authored against umt5_xxl_fp8_e4m3fn_scaled.safetensors will not silently accept a file named umt5-xxl-enc-fp8_e4m3fn.safetensors even though the two are the same underlying encoder.

The same consistency applies to the diffusion models. The Comfy-Org Wan2.2 release files follow a pattern like wan2.2_t2v_high_noise_14B_fp8_scaled.safetensors and wan2.2_t2v_low_noise_14B_fp8_scaled.safetensors, where the _high_noise_ and _low_noise_ tokens select different noise-level experts of the same 14B model. Understanding this naming discipline makes it much easier to spot when a migrated workflow still points at an old filename.

Diagnosing the Mismatch Step by Step

Start with the directory, because that is the failure that looks most like a “missing model.” ComfyUI scans specific folders under the models directory: text encoders live under models/text_encoders/, not under models/clip/ and not under models/diffusion_models/. A diffusion model placed in the text-encoder folder, or a text encoder placed in the diffusion-models folder, will not appear in the dropdown of the node that is supposed to load it.

Next, confirm the exact string the workflow is asking for. In the UI, click the loader node and read the value of its model or clip_name field. In API-driven flows, find the node with "class_type": "CLIPLoader" (or the WanVideo text-encoder loader) and read its clip_name input. Compare that string character-for-character against the filename on disk.

Finally, verify the loader matches the encoder. umt5-xxl is a T5-family encoder, so it belongs in a loader that expects a T5/umt5 format, not the clip_l/clip_g dual-loader used for SDXL. Mixing these up produces a state-dict or dtype error at load time rather than a clean “not found.”

The Most Common Fixes

Once you have located the mismatch, the fix is usually one of three small, reversible edits:

• Rename the file to match the workflow’s expected name (or update the node’s value to match the file you have). Renaming the file is the safer choice when you have multiple workflows sharing one encoder, because a single canonical name keeps all of them working.
• Move the file into models/text_encoders/ if it landed in the wrong directory during download.
• Switch the node to the correct loader type if the encoder was mapped through a CLIP-specific node.

None of these requires re-downloading the 5.7GB encoder. The checkpoints are identical bytes; only the name and location are wrong.

A Note on Precision and VRAM

The filename also encodes precision, which matters for people on tight VRAM budgets. The recommended order of preference for Wan models, from highest quality to smallest footprint, is generally fp16, then bf16, then fp8-scaled, then plain fp8. The scaled suffix indicates a per-tensor scaling calibration that recovers some of the quality that naive fp8 quantization loses. If you are moving a workflow from a cloud GPU to a 16GB local card, selecting the _fp8_scaled encoder rather than the fp16 encoder is often the difference between a graph that fits and one that exhausts VRAM before the diffusion model even loads.

For a deeper look at fitting large diffusion models onto consumer cards, see our guide to Wan2.2 Animate 14B on 16GB VRAM, and for the broader stack the model belongs to, read our overview of the open-source video generation stack.

Verifying the Fix

After correcting the name, directory, or loader, do not just wait for the image. The text encoder runs first and fails fast, so a successful fix shows up quickly: the loader node’s dropdown will now list the file, and the queue will advance past the encoding stage to the sampling stage. In the console or API response, watch for the text-encoder node to complete without a state-dict or “not found” error, then confirm the diffusion model begins sampling.

If the error still names the same file, re-read the exact string the node is asking for — the lingering problem is almost always a residual character difference (an underscore versus a hyphen, or a missing _scaled suffix) rather than a genuinely corrupt model.

Conclusion

The Wan2.2 text-encoder “not loading” error is a filename-and-directory problem, not a hardware or download problem. The fix is to make the file’s name, its directory, and the loader node agree — a rename, a move, or a node swap — and to keep in mind that the fp8_scaled vs fp16 choice encoded in the filename has real VRAM consequences on local hardware.

Primary Documentation

For the authoritative reference on Wan model files and precision variants, consult the WanVideoWrapper repository documentation and the official ComfyUI documentation. Verify the exact expected filenames against the version of the workflow and wrapper you have installed, because naming can shift between releases.

What the “Reconnecting” Overlay Actually Means

If you have used ComfyUI for more than a few days, you have almost certainly seen it: a small overlay in the corner of the canvas scrolling the word “Reconnecting” while the graph refuses to run. It can last a second, or it can sit there indefinitely. Either way, it means the same thing — the browser tab has lost its live connection to the ComfyUI server that is rendering your workflow.

The ComfyUI frontend is not a static web page. The moment the server starts, it exposes a WebSocket endpoint (by default ws://127.0.0.1:8188/ws) that the browser opens alongside the normal HTTP page. This socket is what pushes live progress updates, node execution status, preview images, and error messages back to the canvas in real time. When that socket drops, the frontend has no way to know that a generation is still running, so it shows the “Reconnecting” indicator while it repeatedly tries to re-establish the link.

The key thing to understand is that “Reconnecting” is a symptom, not a root cause. It tells you that the client lost the backend, but it does not tell you why. The distinction matters, because the fix for a crashed server is completely different from the fix for an overloaded browser tab, and mixing them up will send you in circles.

For a broader look at how the ComfyUI frontend and its HTTP API fit together, see our earlier breakdown of the open-source video generation stack on consumer hardware.

The Three Most Common Root Causes

When you strip away the noise, a persistent “Reconnecting” state almost always traces back to one of three things. Diagnosing which one you are dealing with should be your first step, before you change a single setting.

1. The Backend Process Actually Crashed

The most serious cause is the simplest to confirm. If the Python process running ComfyUI dies — from an out-of-memory kill, a custom node that raises an unhandled exception, or a hardware reset — the WebSocket closes instantly and the browser will spin “Reconnecting” forever, because there is nothing left to reconnect to.

Look at the terminal window where you launched ComfyUI. If it shows a traceback, a Killed message, or simply returned to the shell prompt, the server is down. This is common on 16GB-class cards when a video model like the Wan2.2 series pushes memory past the limit and the OS OOM killer steps in. The overlay is not the problem — the crash is.

2. A Long-Running or Blocking Operation on the Main Thread

ComfyUI executes prompt turns on the main event loop in many configurations. While a workflow is running — especially a heavy video generation or a node that does a lot of CPU-bound preprocessing — the server may not respond to the WebSocket heartbeat quickly enough. The browser interprets the silence as a dropped connection and flips to “Reconnecting,” even though the generation is still chugging along fine.

This is why the overlay often appears and disappears on its own without you touching anything. The socket reconnects, the frontend catches up on progress, and everything continues. It looks alarming but is frequently harmless.

3. Custom Nodes or a Slow Start Blocking Server Readiness

A related but distinct cause shows up at startup, not during generation. Every custom node package in your custom_nodes directory is imported when the server boots. Some of these packages — particularly larger ones or ones with heavy dependencies — take a long time to import. While imports are still running, the HTTP server may accept the page request but the WebSocket handler is not yet ready, so the first thing you see is “Reconnecting.”

Community reports on the ComfyUI desktop issue tracker describe exactly this: a desktop install that takes minutes to reach the UI, driven by a handful of slow custom node packs competing for import time. Disabling the offender in the Manager and restarting is usually enough to confirm it.

A Step-by-Step Diagnostic Path

Work through these in order. Each one either fixes the problem or narrows down where it lives, and none of them require guessing.

Step 1 — Confirm whether the server is still alive. Navigate to http://127.0.0.1:8188/ in a fresh tab. If the page loads but the graph never connects, check the terminal for a traceback. If the terminal shows a clean running state, the backend is up and the problem is on the connection side.

Step 2 — Watch the terminal during the “Reconnecting” episode. Trigger a generation and keep an eye on the console. If you see progress logs continuing while the overlay spins, you are looking at cause #2 (a blocking operation) and the connection will recover. If the logs stop dead, you are looking at cause #1.

Step 3 — Test the WebSocket in isolation. The browser’s own developer console (F12 → Network → WS) will show the WebSocket frames and any error. A clean 1006-style abnormal closure points at the server or an intermediary; a series of failed name resolutions points at the address being wrong (for example, accessing localhost from a different machine).

Step 4 — Isolate custom nodes. If the overlay appears mainly at startup or when a specific node runs, launch ComfyUI with custom nodes temporarily disabled and start re-enabling them one batch at a time. This is the fastest way to pin a single pathological package.

Step 5 — Check the transport, not the server. If you are reaching ComfyUI through a reverse proxy, running it in a container, or accessing it from another machine, the WebSocket is the first thing that breaks. Confirm that your proxy forwards the /ws upgrade correctly, and that you are using the same hostname the browser used to load the page.

Common Pitfalls That Keep the Error Around

A few failure modes look like the reconnect problem but are actually misdiagnosed, and they will keep throwing you off until you recognize them.

Accessing localhost from a remote browser. ComfyUI binds to 127.0.0.1 by default. If you open the UI from another machine using localhost:8188, you are pointing at your own machine, not the server. Use the server’s actual IP or hostname, and add --listen to the launch command if you need it to accept non-local connections.

VPNs, VPN kill-switches, and Tailscale. The WebSocket is a long-lived connection. If your VPN re-keys or a Tailscale interface flaps mid-generation, the socket drops and the overlay appears even though both the browser and server are healthy. This matches multiple community reports where the error only happens on certain network paths.

RAM/VRAM exhaustion rather than a real crash. On consumer hardware, a video model bumping into the VRAM ceiling can stall the process long enough to trigger the overlay without actually terminating it. If your “Reconnecting” correlates with heavy Wan2.2 or similar workloads, check your memory situation first — the fix may be a smaller resolution or a different set of VRAM launch flags rather than anything network-related.

Windows kernel timer resolution. Some Windows users report that the overlay is far more frequent when the system timer is at a coarse resolution, which delays WebSocket heartbeats. Running ComfyUI on a power plan that keeps the CPU at full clock, or disabling aggressive sleep, has resolved intermittent reconnects for a subset of reports on the official ComfyUI GitHub discussions.

How to Verify the Fix Actually Worked

You should not declare victory just because the overlay stopped flashing for a minute. A real fix survives a repeatable test.

1. Run a known-good workflow end to end. After applying any change, submit a simple generation and confirm the progress bar updates continuously from start to finish with no overlay flicker.

2. Stress the socket with a long task. The blocking-operation cause only shows up on long generations. Run a heavier workflow — a multi-step upscale or a short video clip — and watch for the overlay during the longest step. A healthy connection should stay silent the whole time.

3. Check the terminal for a clean close. When you stop ComfyUI, the process should exit without a traceback. A stack trace on shutdown is a sign that a custom node is still misbehaving and will likely bite you again.

4. Confirm network stability for remote setups. If you access ComfyUI remotely, run a couple of generations back to back and watch for the overlay appearing at regular intervals. Periodic drops on a fixed cadence almost always point to the transport (proxy, VPN, or Wi-Fi), not the server.

Conclusion

The “Reconnecting” overlay is ComfyUI’s way of telling you the live socket between browser and backend has gone quiet. On its own it is harmless — it often clears the moment the server catches up. But when it sticks, it is a signal that something specific is wrong: the backend crashed, the main thread is blocked, a custom node is delaying startup, or the network transport between you and the server is flapping.

The fastest path to a stable connection is to stop treating it as one generic error and instead pin down which of those categories you are in, using the diagnostic order above. Confirm the server is alive, isolate custom nodes, check the WebSocket transport, and only then worry about deeper memory or system-tuning issues. Do that, and the overlay goes from a recurring source of dread to a one-line signal you can read in seconds.

For the full context on how ComfyUI’s frontend, WebSocket, and HTTP API work together — including how to drive them programmatically — see our guide to automating ComfyUI over its API. For current, version-specific details on the interface and its configuration, refer to the official ComfyUI documentation.

Why ComfyUI “Randomly” Breaks After an Update

One of the most frustrating ComfyUI experiences goes like this: everything was working yesterday, you ran an update, and suddenly the same workflow that produced clean images now throws a confusing error — often something about float16 and float32 not matching, or an operation that “expected half” but “got float”, or a node silently returning zeros and producing black output.

The instinct is to blame ComfyUI core. In most cases, that instinct is wrong. ComfyUI itself is a stable, well-tested base. What breaks is almost always a custom node that sits on top of it — a plugin whose author has not yet caught up with a change in ComfyUI’s internals, or a plugin that patches model precision in a way that conflicts with newer inference paths.

This article walks through a concrete, repeatable diagnostic workflow for finding which custom node broke your install and neutralizing it, so you can stop reinstalling ComfyUI from scratch every time an update changes the ground under your feet.

Read the Startup Log Before Touching Anything

The single most valuable diagnostic artifact you already have is ComfyUI’s console output at startup. Custom nodes load during boot, and when one fails to import cleanly, ComfyUI usually prints an explicit (IMPORT FAILED) marker followed by the node package path and the underlying Python traceback.

The pattern looks similar to this:

### Loading: ComfyUI-Custom-Scripts (x.x.x)
### Loading: ComfyUI-FreeU (y.y.y)
(IMPORT FAILED): /path/to/ComfyUI/custom_nodes/ComfyUI-FreeU
Traceback (most recent call last):
  ...
AttributeError: module 'comfy.samplers' has no attribute 'FreeU2'

An (IMPORT FAILED) line is a smoking gun. The node package listed right after it could not be initialized at all, which means every node it provides is missing from your workflow — often causing “invalid node type”, “type undefined”, or the model-loading error you later see when a required node simply never registered.

If you are on a desktop install, launch ComfyUI from a terminal and scroll the boot output. If you run headless through a service, check wherever stdout is captured (your supervisor or systemd journal). Do not skip this step — many people spend hours re-downloading models when the answer was already printed on line 30 of a log they never read.

The dtype-mismatch Error Is Usually a Plugin, Not a Model

A specific class of error that surfaces after updates is a dtype mismatch: messages reporting that a float16 tensor was passed where float32 was expected, or that an operation “could not be run with the given precision.” Users frequently assume this means their checkpoint is corrupted or the wrong precision variant was downloaded.

In practice, this kind of error is often the signature of a custom node that hooks into the sampling or model-loading pipeline and assumes a fixed precision. A documented example is FreeU variants: the advanced FreeU nodes patch model weights during sampling, and if they were written against an older ComfyUI patch system, an update that changes how those patches are applied can surface as a confusing dtype error rather than a clean crash. The fix reported by users was not “downgrade your model” but “remove the stale plugin” — after which generation worked again. If you are new to local AI, our guide to running the ComfyUI VRAM launch flags explains how --lowvram and --normalvram interact with exactly these model-loading paths.

The general lesson: if a workflow ran fine on one model precision and now throws dtype errors after an update, suspect the nodes that touch the model or sampler. Bisect them before you touch the model files.

Bisect custom_nodes to Isolate the Culprit

ComfyUI loads every folder under custom_nodes/, so the fastest isolation technique is a binary search over those folders rather than guessing one by one. (The official ComfyUI docs on custom nodes confirm that any directory placed there is auto-imported at startup, which is exactly why a bad plugin can take down otherwise-healthy workflows.)

A reliable approach:

1. Snapshot a known-good baseline. Move every custom node folder into a temporary directory outside custom_nodes/ so ComfyUI boots with zero plugins. Confirm the core UI loads and a trivial built-in workflow (e.g. empty-latent-image to KSampler) runs. This establishes that core itself is healthy.
2. Re-add in halves. Move half the folders back, restart, and load your broken workflow. If it fails, the culprit is in that half; if not, move the other half back.
3. Recurse. Keep halving the responsible set until you reach a single folder.

You can also use ComfyUI-Manager rather than moving folders manually. ComfyUI-Manager (from the official Comfy-Org/ComfyUI-Manager repository) lists installed custom nodes and lets you disable or enable them from the UI, and it surfaces import errors and available Git updates for each package. Using its disable toggle for the same bisection is faster and avoids accidental folder damage, though the manual folder approach works even when Manager itself fails to load.

Key point: restart ComfyUI after each enable/disable change. Import-time failures and registration order are evaluated at boot, so toggling a toggle without restarting tells you nothing.

Fix It: Update, Pin, or Remove

Once you have identified the offending node, you have three realistic options, in order of preference:

• Update the node. Most actively maintained nodes ship a fix for a breaking ComfyUI change within days. Run the updater in ComfyUI-Manager, or git pull inside the node’s folder. This is the strongest fix because you keep the functionality.
• Pin to a compatible ComfyUI commit. If the node is abandonware but you genuinely need it, you can roll ComfyUI core back to the last commit that worked with it (using git checkout in the ComfyUI repo) and freeze custom_nodes updates. This buys time but leaves you vulnerable to security and feature lag.
• Remove or disable it. If the node is unused or has been superseded, disable it. A node you installed on a whim and never used is precisely the kind of plugin that quietly breaks the whole install later.

Resist the reflex to reinstall ComfyUI from scratch. A clean reinstall may temporarily remove the broken plugin, but if your workflows depended on it (or on any of the other nodes you lose), you will either reintroduce the bug or spend hours rebuilding state you could have preserved with a five-minute bisection.

Prevent It From Happening Again

A few habits make the next update painless:

• Keep a startup-log baseline. After a known-good boot, save the console output. The next time something fails, diff the new log against the baseline and look at what changed — usually an IMPORT FAILED or a new warning is the first clue.
• Update deliberately, not eagerly. Do not blindly click “update all” in ComfyUI-Manager. Update core and nodes separately, test after each, and keep a mental note of which nodes you actually use so you can disable the rest.
• Prefer maintained nodes. Before installing a node, glance at its repository for recent commits and open issues mentioning the ComfyUI version you run. A node whose last commit is two years old is a time bomb on the next core update.
• Version-pin a working environment. For production or repeatable pipelines, snapshot the Git commit of both ComfyUI core and each custom node you depend on, so you can reproduce a working stack on demand.

ComfyUI’s flexibility comes from the custom-node ecosystem — and that ecosystem is also the most common source of “mysterious” post-update failures. Learning to read the boot log and bisect custom_nodes/ turns a day-long panic into a ten-minute cleanup.

Conclusion

When ComfyUI breaks after an update, the problem is rarely the model and almost never the base application. It is a custom node that no longer matches the version of core it was written for. Trace the (IMPORT FAILED) markers in your startup log, bisect the custom_nodes/ folder to isolate the culprit, then update, pin, or remove it. Treat a dtype-mismatch error as a sign to inspect the nodes touching your sampler and model loader — not as a reason to redownload checkpoints. Do this before you reach for the nuclear “reinstall everything” option, and you will recover faster and keep your working pipelines intact.