There are two completely different ways a ComfyUI box can run out of memory, and treating them as the same problem wastes hours. One produces a tidy in-app error. The other silently freezes your desktop, spins your fan to 100%, and then kills the process without so much as a log line. This article is about the second one.
If ComfyUI prints CUDA out of memory and the workflow just stops, that is a VRAM problem — the GPU ran out of dedicated video memory. We have already covered that failure mode in the VAE-decode OOM article. What we are diagnosing here is different: your system RAM (the DDR DIMMs near the CPU) is exhausted, Linux starts swapping to disk, and the kernel’s OOM killer decides ComfyUI is the fattest process on the machine and shoots it. To the user it looks identical to a crash — but the fix is completely different.
The Symptom Pattern That Points at System RAM, Not VRAM
VRAM exhaustion almost always announces itself clearly. You get a torch error mentioning CUDA out of memory, a failed allocation, or a node that reports “Memory” in red. System RAM exhaustion is nastier because it rarely prints anything useful. The telltale signs to watch for:
- The GPU is idle while the machine is unresponsive. Run
nvidia-smiin another terminal: if the GPU shows 0% utilization but the box feels frozen, the bottleneck is the CPU/RAM/disk, not the card. - The process just vanishes. No error dialog, no traceback — the terminal shows
Killedor the service reports an exit code of 137 (128 + SIGKILL 9). That is the OOM killer’s signature. dmesgshows the smoking gun. Look for a line likeOut of memory: Killed process <pid> (python)oroom-kill. This is proof, not guesswork.- Swap is being hammered.
free -hshows the swap column nearly full while the machine crawls.
If you see the first two signs together — idle GPU plus a vanished process — stop chasing VRAM flags and start looking at system memory.
Why ComfyUI Eats So Much System RAM
People assume a GPU-bound tool only cares about VRAM. In practice ComfyUI’s peak system-RAM footprint can run 2–3× the size of the checkpoint you loaded, because of a few compounding effects:
- Model loading is a RAM staging area. A diffusion checkpoint is typically read from disk into system RAM first, then copied (or progressively streamed) into VRAM. During load, both copies can coexist briefly. A 14 GB video model can transiently need 28 GB+ of CPU RAM just to get onto the card.
- Custom nodes accumulate without releasing. Python objects referenced from a node’s output cache stay alive until the queue is cleared. A workflow that runs many generations, or one with a misbehaving custom node that never frees its intermediates, will grow the resident set over time.
- Latent and image buffers live in RAM too. Decoded images, upscaler intermediates, and video-frame latents are often staged in system memory before transfer, adding gigabytes on top of the model.
- The text encoder and auxiliary models may run on CPU. In
--lowvramor hybrid-offload setups, the text encoder can be held in RAM instead of VRAM, and a T5-XXL or umt5 encoder is several GB on its own.
The official ComfyUI troubleshooting documentation groups “system freezing” and “out of memory errors” under the same slow-performance banner and recommends lowering resolution, using the memory-optimization flags, and closing other applications — all of which reduce total pressure but do not directly address a RAM ceiling. See the ComfyUI troubleshooting guide for the baseline advice.
Diagnose Before You Fix: Prove It Is RAM
Run these exact commands during a generation that you suspect triggers the crash. Do not guess — measure.
# 1. Snapshot memory while it runs
free -h
# 2. Watch the highest-RAM processes live
top -o %MEM
# 3. Check whether the kernel killed anything
dmesg -T | grep -iE 'oom|killed process|out of memory' | tail -20
# 4. Confirm the death-by-137 signature on a service
systemctl status comfyui --no-pager
What you are looking for:
free -hshowingavailabledropping to a few hundred MB whileswapfills up.dmesgprinting anoom-killentry namingpythonorpython3as the victim, possibly with a score likeoom_score_adjor a memory figure.- An exit code of 137 — the definitive “killed by SIGKILL” marker.
If dmesg instead shows a CUDA error or the torch traceback, you are back in VRAM territory and the VRAM article applies.
Fix the RAM Ceiling, Not the Symptom
Once you have confirmed system memory is the ceiling, these are the levers that actually help, in rough order of effectiveness:
1. Give ComfyUI a hard memory limit instead of letting it free-run
An unbounded process can balloon until it trips the OOM killer and takes other services with it. On a systemd Linux host, wrap the daemon in a cgroup with an explicit MemoryMax. For example, a unit override that caps a generation daemon at 14 GB:
[Service]
MemoryMax=14G
MemorySwapMax=2G
ManagedOOMSwap=kill
ManagedOOMMemoryPressure=kill
The key detail is ManagedOOMSwap=kill and ManagedOOMMemoryPressure=kill: these let systemd-oomd kill only the cgroup that exceeds its limit, leaving the rest of the machine (your DE, your other services) alive. This is the single highest-leverage change — it converts a system-wide freeze into a contained, diagnosable failure. See the systemd resource-control documentation for the full option set.
2. Restore a small swap as a buffer, but do not rely on it
Two opposite mistakes cause most swap problems. Running with swap disabled means the OOM killer fires the instant RAM is exhausted. Running with a huge swap on a slow disk means the box thrashes for minutes before dying. A few gigabytes of swap on an NVMe gives the kernel room to evict cold pages and lets dmesg log a clean event instead of an abrupt kill. Think of it as a tripwire, not a reservoir.
3. Cut the transient load spike
The load-phase double-allocation mentioned above is often the real culprit — the machine only OOMs for the few seconds a big model is being staged into VRAM. Mitigations:
- Use
--lowvram/ progressive offload so the model streams in chunks rather than staging the whole file at once. - Avoid running simultaneous generations on the same host when a large checkpoint is involved; the peak RAM spike of two overlapping loads can be additive.
- Clear the queue and restart ComfyUI between heavy batches so cached intermediates from custom nodes get garbage-collected.
4. Check for a genuine memory leak in a custom node
If resident memory grows monotonically across generations even after the queue is cleared, you may have a leaky custom node. The official custom-node troubleshooting guide recommends --disable-all-custom-nodes followed by a binary search to isolate the offender. This is tedious but is the only reliable way to pin a leak on one node rather than ComfyUI core.
Common Pitfalls That Make It Worse
- Lowering
batch_sizeto “fix” a RAM OOM. Batch size primarily changes VRAM, not system RAM. Ifdmesgsays OOM-killed, shrinking the batch will not save you. - Adding RAM without checking the actual peak. If the leak is in a custom node, a bigger DIMM just delays the crash. Diagnose the slope of memory growth first.
- Disabling swap entirely “for performance.” This turns a recoverable slow-down into a hard kill. Keep a small, fast swap.
- Confusing
--lowvramwith--cpu.--cpuruns everything on the CPU and is a last resort;--lowvramonly offloads the parts that do not need to stay resident.
Verify the Fix
After applying a change, prove it worked with the same measurements you took during diagnosis:
- Re-run the workload that previously crashed while watching
free -handtop -o %MEM. Peak system RAM should stay below your ceiling. - Confirm
dmesgno longer logs anoom-killfor the ComfyUI process. - If you added a cgroup limit, deliberately exceed it once (e.g. a large batch) and confirm that only the cgroup is killed and the rest of the system stays responsive.
The goal is not “no errors ever” — it is a predictable failure boundary. A box that degrades gracefully and kills only the runaway process is infinitely easier to operate than one that hard-freezes the whole desktop while the GPU idles.
Note on testing: the commands and configuration in this article describe reproducible diagnostic steps and standard Linux mechanisms rather than benchmarked figures. Run them on your own hardware to get numbers specific to your checkpoint and workflow.
Leave a Reply