llama.cpp “error while loading shared libraries”: Fix libllama-common.so.0 and libllama-cli-impl.so Missing

I ran llama-server --version on a Linux box this morning and got exactly two lines back: a bright red error while loading shared libraries: libllama-common.so.0: cannot open shared object file: No such file or directory, followed by nothing. The llama-cli binary right next to it failed the same way, only it complained about a different file, libllama-cli-impl.so.

This is not a model problem. It is not a CUDA problem. It has nothing to do with your GGUF file or your GPU. It is a library installation problem, and it is one of the most common ways a working llama.cpp setup breaks after an upgrade, a partial make install, or a cleanup that removes files it should not have touched. It is the same category of silent environment breakage I have seen flip ComfyUI workflows from working to dead overnight, which I covered in this guide to llama.cpp CUDA out-of-memory on a 16GB card.

In this article I walk through the exact failure on a real machine, the three commands that pinpoint the root cause, the difference between a missing library and a misconfigured search path, and the fix that actually works. Everything below is reproduced from a live Raspberry-Pi-free Ubuntu host running an NVIDIA RTX 5060 Ti.

The symptom: an error that names a file you have never heard of

The first thing to notice is that both binaries fail, but with different filenames. On the machine I was debugging, the two failures looked like this:

$ /usr/local/bin/llama-server --version
error while loading shared libraries: libllama-common.so.0: cannot open shared object file: No such file or directory

$ /usr/local/bin/llama-cli --version
error while loading shared libraries: libllama-cli-impl.so: cannot open shared object file: No such file or directory

Two different missing files, two different binaries. That is the key clue that this is a partial install rather than a global LD_LIBRARY_PATH problem. If the loader simply could not find a directory, both binaries would fail on the same library. When each binary names its own private library, the libraries themselves are gone from disk.

This splitting behavior is relatively new. Modern llama.cpp is built as a set of shared libraries — libllama-server-impl.so, libllama-cli-impl.so, and a shared libllama-common — that the thin executables link against. Older builds produced a mostly self-contained llama-cli and llama-server. If you installed a recent version and any of these impl libraries did not make it into your library path, you get exactly this class of error.

Step 1: confirm it is really the library and not the environment)

Do not start editing LD_LIBRARY_PATH yet. First, determine whether the loader cannot find the library or the file genuinely does not exist. These two tools answer that in seconds.

ldd — what is the binary actually asking for?

$ ldd /usr/local/bin/llama-cli
        libllama-cli-impl.so => not found
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6

The => not found next to libllama-cli-impl.so is the smoking gun. Every other dependency resolved to a concrete path; only this one failed. That points at the specific missing file.

readelf — the declared dependency, not the guess

ldd runs the loader and can be influenced by your current LD_LIBRARY_PATH. To read the dependency that was baked in at link time, check the ELF metadata directly:

$ readelf -d /usr/local/bin/llama-server | grep NEEDED
 0x0000000000000001 (NEEDED)  Shared library: [libllama-server-impl.so]
 0x0000000000000001 (NEEDED)  Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)  Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)  Shared library: [libc.so.6]

Here is the subtle part that trips people up. llama-server declares a dependency on libllama-server-impl.so, and that file does exist on disk. But loading that impl library then fails on its own transitive dependency, libllama-common.so.0. The top-level readelf output looks clean; the missing library is one level deeper. That is why the error for the server names libllama-common.so.0 while the CLI names libllama-cli-impl.so — they are failing at different points in the dependency chain.

Step 2: search the whole filesystem for the missing library

Once you know the exact filename, search everywhere, not just /usr/local/lib:

$ find / -name "libllama-common*" 2>/dev/null
# (no output — the file does not exist anywhere on disk)

$ find / -name "libllama-cli-impl*" 2>/dev/null
# (no output — also absent)

An empty result is the decisive answer. If find turns up nothing, no amount of LD_LIBRARY_PATH or ldconfig configuration will save you — the bytes simply are not on the machine. The install is incomplete or was partially removed.

Contrast this with the libraries that are present. On the same host, /usr/local/lib contained libggml.so, libggml-base.so, libggml-cpu.so, libggml-cuda.so, libllama.so, and libllama-server-impl.so — but not libllama-common.so.0 and not libllama-cli-impl.so. That pattern is the signature of a build where the shared-library split was introduced, and the libllama-common target either failed to build or failed to install.

Step 3: check the library path is even registered

Even after you restore the missing files, the loader still needs to know where to look. A very common second failure — separate from the missing-file problem — is that /usr/local/lib is not in the linker’s search path. Verify it:

$ cat /etc/ld.so.conf.d/*.conf | grep local
/usr/local/lib
/usr/local/lib/x86_64-linux-gnu

If /usr/local/lib is not listed, that is a different fix: create a drop-in config and rerun the cache. It is worth ruling out even when you have already confirmed files are missing — the two problems frequently occur together after a botched upgrade.

There is a third, quieter failure mode worth knowing about too. On this host, running sudo ldconfig emitted a stream of warnings like ldconfig: /usr/local/lib/libggml-base.so.0 is not a symbolic link. That happens when real files were copied over the place where versioned symlinks should live. It does not stop the loader, but it is a strong signal that a manual file copy — rather than a clean cmake --install — created the /usr/local/lib contents.

The fix: repair the partial install, then rebuild the linker cache

Because the root cause here is missing files, not misconfiguration, the correct fix is to rebuild llama.cpp from a clean tree and reinstall it fully. A quick workaround — copying a libllama-common.so.0 from another machine or a downloaded release bundle into /usr/local/lib — can restore the binaries, but it leaves out the build’s own notion of which symbols belong to which library and tends to break again on the next upgrade.

1. Clone or update a fresh checkout and build the shared-library targets:

$ git clone https://github.com/ggml-org/llama.cpp
$ cd llama.cpp
$ cmake -B build -DGGML_CUDA=ON
$ cmake --build build --config Release -j
$ sudo cmake --install build

The cmake --install step is the important one — it places all produced libraries (libllama-common, libllama-server-impl, libllama-cli-impl, the libggml-* family) into the right location together, which is exactly what a manual copy misses.

2. Refresh the dynamic-linker cache so the new libraries are registered:

$ sudo ldconfig

3. Verify both binaries now load, and check that no NEEDED entry remains unresolved:

$ /usr/local/bin/llama-server --version
$ ldd /usr/local/bin/llama-cli | grep -i "not found"
# (empty output means everything resolved)

The cleanup advice is to prefer release packages over sudo make install for production boxes. A release tarball or a distro package bundles every library the binaries need; a manual make install relies on the target list being complete, and it is precisely where the libllama-common split has bitten people.

Common traps that are not the problem

There are a handful of things people assume caused this error that almost never do, and chasing them wastes an afternoon.

• Your GGUF model is corrupt. No — the binary fails before it ever opens the model file. Loader errors happen at process start, before llama-server parses any weights.
• Missing CUDA driver. A bad CUDA setup produces a different message (CUDA error: no kernel image is available or a libcuda.so failure), not a missing libllama-common.
• Wrong --n-gpu-layers. That only affects runtime VRAM allocation, never dynamic loading.
• You need to re-download the model. The model is fine. Deleting a 16 GB GGUF to “fix” a loader error only wastes bandwidth.

The one genuinely confusing case is a wrong library version: when libllama-common.so.0 exists but is from an incompatible llama.cpp build, the error changes to something like undefined symbol or a GLIBC version mismatch rather than “not found”. If you see undefined symbol, treat it as a version-mismatch, not a missing file, and rebuild the entire library set together instead of copying one file.

Verification checklist

After the fix, work through these four checks in order and only move on when each is clean:

1. llama-server --version prints a version instead of a loader error.
2. llama-cli --version prints a version.
3. ldd /usr/local/bin/llama-cli shows no => not found lines.
4. A minimal generation (llama-cli -m model.gguf -p "hi" -n 1) produces tokens rather than a shared-library error.

Conclusion

An error while loading shared libraries from llama.cpp is a library-installation failure, not a model or GPU failure, and it is usually one of exactly two things: a library file that is missing from disk, or a library path the loader has not been told about. The three commands that separate the two are ldd (what is unresolved), readelf -d (what the binary declares it needs), and find (does the file exist anywhere). If find comes up empty, rebuild and reinstall — do not paste LD_LIBRARY_PATH exports over a genuinely absent file and expect it to work.

If you run llama.cpp on a single-GPU Linux box, this class of error will eventually find you after an upgrade. Knowing it is a loader problem from the first line of the message — and not a rabbit hole into CUDA, models, or VRAM flags — is the part that saves the most time.