If you have ever watched your NVIDIA card sit at a lazy 30% VRAM while your Linux box grinds to a halt — mouse cursor stuttering, malloc failures, the whole machine frozen — you have hit one of the most confusing failures in local AI: ComfyUI exhausting system RAM while the GPU still looks half empty.
The error is almost always reported as “CUDA out of memory” or a vanilla std::bad_alloc, which sends people straight to the wrong fix. They lower the image resolution, shrink the batch, or add --lowvram — and nothing improves, because the problem was never on the GPU in the first place.
This article breaks down the symptom, shows you how to prove which tier of memory is actually dying, and gives you a concrete fix for each case. I write this from a first-hand snapshot of my own T2 machine: a 30GB-RAM box running ComfyUI 0.39.0 with an RTX 5060 Ti (16GB VRAM), which was sitting at 284MB of free RAM and 22GB of swap used while the GPU reported 10.1GB of VRAM free. That is the exact state this article exists to explain.
The Symptom: VRAM Free, Machine Frozen
The tell-tale signature is a mismatch between two readings taken at the same moment. Your GPU is fine, and your system is not. Here is what it looked like on my box mid-incident:
nvidia-smi//system_stats: 10.1GB of 16.6GB VRAM free.free -h: total 30Gi, used 30Gi, available 625Mi, swap 22Gi used of 43Gi.ps aux --sort=-%mem: theComfyUI/main.pyprocess holding ~20.1GB RSS, 64% of all system memory.
When you see that, stop tuning the sampler. The GPU was never the constraint — ComfyUI is holding model weights in ordinary CPU RAM, and it has run the machine into the swap file.
The reason this is so common is that ComfyUI’s memory manager is aggressive about offloading. When a model (or a LoRA, a VAE, a text encoder like umt5-xxl, or a full video diffusion checkpoint) does not fit in VRAM at load time, ComfyUI keeps it in system RAM and streams pieces onto the GPU as needed. That is by design and usually correct. The subtle problem is that offloading and leaking produce the same free -h output — a huge RSS number — so you cannot tell which one you are looking at without one more step.
How to Prove Which Tier of Memory Is Dying
The single most useful diagnostic is to stop conflating VRAM and RAM, and read both at the same time. Three commands, run on the machine that is actually running ComfyUI:
1. Read the GPU from ComfyUI’s own API, not from nvidia-smi.
curl -s http://127.0.0.1:18188/system_statsLook at devices[0].vram_total vs vram_free. On my box this returned 16,610,820,096 total and 10,102,938,348 free. The API also reports torch_vram_total/torch_vram_free, which is the pooled-allocator view — the number to trust when a workflow is actively running under cudaMallocAsync.
2. Read system RAM and swap.
free -hIf available is in the hundreds of MB while swap used is climbing into the gigabytes, your system is thrashing.
3. Find the process responsible.
ps aux --sort=-%mem | headOn my box the top line was unambiguous: the venv Python running main.py --listen 0.0.0.0 --port 18188 at ~20GB RSS. If that same process is also the only thing consuming memory, you have a case of offloaded-weights residency; if the RSS climbs after every single queue run and never comes back down, you are looking at a leak.
The distinction matters because the fixes are opposite. Offloading that simply never releases is a configuration problem (it is holding too much, or it is holding it after you are done). A leak that grows per-run requires either a restart cadence or a bug fix upstream.
Rule Out a Real Leak Before Trying to “Fix” Offloading
The healthiest instinct when you see 20GB of RSS is to assume it is a leak, but ComfyUI offloading is specifically designed to look exactly like that. A reliable test is to watch the RSS trend across two identical queue runs with no model changes between them:
- Run the workflow once, note the RSS after it finishes.
- Wait 30 seconds, run it again, note the RSS again.
If RSS returns to roughly the same baseline after each run, you have offload residency — weights are parked in RAM and reused, which is expected. If RSS ratchets upward by the size of the model on every run and never returns, you have a leak, and the correct response is different: check whether a specific custom node is the cause (isolate it in a minimal graph), pin and update that node, and add a periodic restart while you wait for a fix. ComfyUI has an open, long-running issue about memory not being released after generation (Comfy-Org/ComfyUI#8298) that documents exactly this “RAM keeps growing after each run” behavior on high-RAM systems.
One common trap: people read a full RAM line and assume offloading is not working, then disable offloading entirely. That is backwards. Full RAM with weights parked there is offloading working; the failure only begins when RAM plus swap is exhausted and the kernel starts the OOM killer. The goal is not zero RAM usage — it is staying under the ceiling where the machine remains responsive.
Fixing the Offload Bloat Without Disabling It
If you have confirmed offload residency (not a leak), the goal is to shrink the system RAM footprint and give ComfyUI a cleaner release path. In rough order of impact:
Turn on Dynamic VRAM. ComfyUI’s dynamic VRAM mode improves how the manager moves tensors between VRAM and system RAM, which directly lowers the amount of RAM stuck holding offloaded weights and reduces OOM errors on tight machines. You enable it in the Settings → Memory → Dynamic VRAM toggle, or pass it at launch depending on your version.
Understand that --lowvram is not the same lever. The launch flags --lowvram, --normalvram, --highvram, and --novram control how aggressively weights are moved onto the GPU, which changes VRAM pressure — but they do not, by themselves, cap how much system RAM an offloaded or partially-loaded model holds. If you are OOM-ing in CPU RAM, changing these flags will not save you and can even make the machine worse by forcing more residency in RAM. I cover the exact behavior of each flag in ComfyUI VRAM Launch Flags Explained.
Close previews and stale generations. A huge and under-appreciated system-RAM sink is the accumulated preview / history backing store. If your browser tab has been open for hours and every queue prompt has rendered a live preview, that image data is sitting in RAM. Closing the browser tab (or the API client’s websocket that streams previews) and clearing the queue frees a surprising amount.
Use an explicit release / free-memory node. If you are scripting or chaining many video segments, ComfyUI does not always return offloaded weights to a clean state between segments. Scheduling an explicit memory release — either through your driver script hitting the appropriate endpoint, or by restarting the ComfyUI process on a cadence — is the pragmatic solution most production pipelines settle on. My own Wan2.2 Animate setup on the same hardware leans on BlockSwap and force-offload precisely to keep a 16GB-class video model from thrashing; the full settings are in ComfyUI Wan2.2 Animate 14B on 16GB VRAM.
Add swap deliberately — or remove the pressure first. A box that is sitting in 22GB of swap is past the point of tuning; it is thrashing. Give it headroom (close Chrome if it is running, stop the local LLM server you forgot about), then come back and re-measure free -h. Swap is a safety net, not a working tier — once you are depending on it for daily generation, every prompt becomes 10× slower.
Verify You Actually Fixed It
After applying whatever combination applies to your case, prove it with the same three commands you used to diagnose, and confirm the trend — not just a single lucky reading:
free -hshowsavailablecomfortably above the model footprint, andswap usedis flat or falling, not growing.- Re-run the same workflow twice and confirm RSS returns to a stable baseline (offload) instead of ratcheting (leak).
/system_statsshows the VRAM pattern you expect, and the UI/API no longer throws “CUDA out of memory” orbad_allocon a workload that previously died.
The decisive test is responsiveness. A machine that is genuinely healthy will let you open a terminal and type while a generation is running. If the mouse still crawls during a prompt, you have not freed the system-RAM tier — you have only moved the problem around.
Conclusion
“CUDA out of memory” with a half-empty GPU is almost never a GPU problem. It is a system-RAM problem wearing a CUDA costume. The fix is to measure the right tier (VRAM via /system_stats, RAM via free -h, and the culprit via ps), distinguish offload residency from a genuine leak, and then apply the matching lever — Dynamic VRAM for the former, an isolated-node hunt and restart cadence for the latter. Get that right and a 30GB box with 16GB of VRAM goes back to being a perfectly capable local-AI workstation instead of a swap-bound slideshow.
The memory management behavior is moving fast, so treat everything here as a snapshot to verify against the version you actually run. For current settings and the authoritative explanation of the memory manager, start with the official ComfyUI troubleshooting guide and the project’s memory-not-released issue thread.
Leave a Reply