Skip to content

llama.cpp Mlock Failed on Raspberry Pi: Limits and Load Modes

If llama.cpp reports a failed memory lock on Raspberry Pi, check both the loading option supported by your binary and the locked-memory limit of the running process. Increasing a limit does not create RAM. A model that exhausts physical memory needs a smaller model, context, or workload before memory locking is useful.

This page covers lock failures and option migration. For building llama.cpp, choosing a GGUF model, and measuring inference speed, start with the Raspberry Pi llama.cpp setup guide.

Before you start

Run commands in Bash on the Pi, using your actual llama.cpp path, model and service name. Begin with inspection; model loading and service restarts are separate, deliberate steps. The optional one-page example also works on another Linux computer and needs no model download.

llama.cpp mlock error quick reference

Observation Check first Next step
error: invalid argument: --mlock Binary version and --help Fix the argument first; changing limits cannot repair an unknown option.
warning: failed to mlock Requested bytes, errno and process limits Read the lock warning; it is not automatically an OOM kill.
Process exits with Killed or status 137 Kernel and service logs Look for OOM evidence, not just an exit code.
Terminal launch works, service fails Service process limits Inspect the service PID, not the shell's ulimit.
Locking succeeds but speed is unchanged Comparable benchmark runs Keep locking only if it helps the workload you measured.

If --mlock is an invalid argument

This is an argument-parser failure, before a model can be loaded by that invocation. Do not run the server as root, change swap or increase a service limit to fix it. Inspect the exact binary you launch below: a package, container or service may use a different executable from your checkout.

If its help lists --load-mode, use that interface. If it lists the older --mlock flag instead, follow that version's help; a new command copied into an old executable can fail in the opposite direction. A wrapper such as llama-cpp-python may expose different options.

Check the current loading options

From your llama.cpp checkout, inspect the actual executable:

1
2
3
./build/bin/llama-server --version
./build/bin/llama-server --help 2>&1 | grep -E -- 'load-mode|mlock|mmap'
git rev-parse HEAD

The upstream parser reviewed on 6 November 2026, revision 6c01da6278ab, accepts --load-mode. Use the installed binary's help as the final authority. git rev-parse HEAD identifies the checkout, not necessarily the binary or library running in your service. No matching help output can also mean the binary failed to start; read its full error before assuming an option was removed.

Loading choice Purpose
--load-mode auto Let the runtime choose its normal loading path.
--load-mode mmap Memory-map the model without requesting locking.
--load-mode mmap+mlock Memory-map and request that model memory remain resident.
--load-mode mlock Use the non-mmap loading path with memory locking.
--load-mode none Use the non-mmap path without locking; it still loads the model and consumes memory.

For this parser, mmap+mlock requests mapped-and-locked loading; mlock requests the non-mmap locked path. It also offers dio, a separate Direct I/O path, not a general lock-error repair. See the model loader at the same revision.

Check service ExecStart and any LLAMA_ARG_LOAD_MODE environment setting too. Do not keep a stale --mlock argument while adding a new environment setting: the unsupported argument still fails parsing.

Establish an unlocked baseline first

Choose a model that fits with the OS, context/KV cache, compute buffers, and other services. Start with a short context and a single workload:

1
2
3
4
5
6
free -h
./build/bin/llama-server \
  -m /srv/models/model.gguf \
  -c 2048 -t 4 \
  --load-mode mmap \
  --host 127.0.0.1 --port 8080

Replace the model path with your own. Stop that server before starting the comparison run with --load-mode mmap+mlock. Keep the model, context, requests, and other options identical. The server remains local to the Pi in this example; you can check readiness using curl http://127.0.0.1:8080/health from another terminal on the Pi.

This command starts a long-running inference server and loads a model. It is not a read-only diagnostic. Use only a model and workload already known to fit, and stop a foreground server with Ctrl+C when finished. If the server does not support these flags, return to its local help first.

Memory locking can help when pages would otherwise be evicted or swapped under pressure. It does not increase CPU compute throughput. Compare startup time, request latency, and memory use rather than assuming a higher tokens/s result.

Read a failed-lock warning

Match this message shape in the startup log; the byte counts and final error vary:

warning: failed to mlock N-byte buffer (after previously locking M bytes): ERROR

This source-defined warning means a lock request failed. M is the amount already locked by that lock object, not the Pi's total RAM. A nonzero value can indicate partial locking; a server that stays up is not proof the whole model is locked.

  • Cannot allocate memory / ENOMEM: compare the requested bytes with the process's soft memlock limit. Linux can return this error for a limit violation even when free RAM remains; other mapping/resource failures are possible too.
  • Operation not permitted / EPERM: inspect the zero/low limit and the process's permission or sandbox policy. Do not grant root or CAP_IPC_LOCK as the first fix.
  • No lock warning, but the process vanishes: investigate its termination separately. Neither an option-parser error nor this warning establishes an OOM kill.

Inspect the running process's memory limit

For a shell-launched process, inspect the shell's soft and hard limits before launching:

ulimit -S -l
ulimit -H -l

In Bash, these values are in KiB. An unprivileged shell cannot raise its soft limit above its hard limit. Changing the limit in another terminal does not change an already-running server.

For an existing process, use its real PID. For a system service named llama-server.service:

1
2
3
4
5
6
7
server_pid=$(systemctl show llama-server.service --property=MainPID --value)
if [[ "$server_pid" =~ ^[1-9][0-9]*$ ]]; then
    grep 'Max locked memory' "/proc/$server_pid/limits"
    grep -E 'VmLck|VmRSS' "/proc/$server_pid/status"
else
    printf 'No running service PID; inspect its status and journal.\n'
fi

/proc/PID/limits reports the process limits in bytes for locked memory; VmLck reports locked memory in kB. A successful model load is not proof that every requested lock succeeded. Read llama.cpp's startup warnings as well as the process state. The Linux mlock manual explains locked-page behaviour, limit checks, and error conditions.

Use your actual unit name. If /proc access is denied, use an appropriately authorised administrator account; do not relax the machine's process-visibility policy. A restart changes the PID, so obtain it again. On a manually launched server, inspect its actual PID instead of the systemd example.

For example, ulimit -S -l reporting 8192 means 8 MiB, while /proc/PID/limits represents the same allowance as 8388608 bytes. These are illustrative values, not defaults for every Pi. VmLck: 0 kB means no pages are currently recorded as locked for that process, not that the model uses no RAM.

Optional: observe a refusal without loading a model

Run this on Linux as an ordinary user with Python 3, without sudo. It allocates one page (at most 64 KiB) and lowers only this short-lived Python process's memlock limit to zero. It does not change your shell, service, swap or system limits, and does not try to exhaust RAM:

python3 - <<'PY'
import ctypes
import errno
import mmap
import os
import resource

if os.geteuid() == 0 or mmap.PAGESIZE > 65536:
    raise SystemExit("Use an ordinary user and a page size no larger than 64 KiB.")
resource.setrlimit(resource.RLIMIT_MEMLOCK, (0, 0))
libc = ctypes.CDLL(None, use_errno=True)
for name in ("mlock", "munlock"):
    function = getattr(libc, name)
    function.argtypes = [ctypes.c_void_p, ctypes.c_size_t]
    function.restype = ctypes.c_int
with mmap.mmap(-1, mmap.PAGESIZE) as page:
    address = ctypes.addressof(ctypes.c_char.from_buffer(page))
    print(f"page_bytes={mmap.PAGESIZE}, soft_memlock_bytes=0")
    if libc.mlock(address, mmap.PAGESIZE) == -1:
        code = ctypes.get_errno()
        print(f"mlock refused: {errno.errorcode.get(code, code)}")
    else:
        libc.munlock(address, mmap.PAGESIZE)
        print("Lock succeeded: this process may bypass the limit; stop here.")
PY

Example output on an unprivileged Linux process with 4 KiB pages:

page_bytes=4096, soft_memlock_bytes=0
mlock refused: EPERM

This demonstrates a kernel lock refusal, not llama.cpp inference or an OOM event. Page size and security policy can differ. If locking succeeds, a capability may bypass the limit; the example unlocks the page and exits rather than increasing its allocation.

Set a bounded systemd limit when required

A system service gets its limits from the service manager. Your SSH shell's ulimit is not its configuration. Inspect the current unit and effective settings:

1
2
3
systemctl cat llama-server.service
systemctl show llama-server.service \
  --property=LimitMEMLOCK --property=LimitMEMLOCKSoft

Only if the measured workload fits RAM and needs locking, plan a system-service override. Read the existing unit first and record its effective soft/hard limits. Use an unused drop-in name such as 50-mlock.conf; if that file already exists, preserve its contents and do not overwrite unrelated settings.

These commands create a configuration directory and open an editor; they are not inspection commands:

sudo mkdir -p /etc/systemd/system/llama-server.service.d
sudo nano /etc/systemd/system/llama-server.service.d/50-mlock.conf

For that new file only, an example is:

1
2
3
4
[Service]
# Example only: choose a value from the actual model/workload budget.
LimitMEMLOCK=2G
Environment=LLAMA_ARG_LOAD_MODE=mmap+mlock

2G is an illustrative limit, not a model-size recommendation. The environment setting is supported by the checked upstream parser; if ExecStart already sets a loading mode explicitly, update that deliberate choice rather than maintaining conflicting settings. Keep the service's unprivileged user.

The systemd execution reference documents LimitMEMLOCK and the distinction between system and user-service limits. A user service cannot exceed the limits inherited by its user manager merely by adding a larger unit value.

Apply the override during a suitable restart window, then inspect the new PID:

1
2
3
sudo systemctl daemon-reload
sudo systemctl restart llama-server.service
journalctl -u llama-server.service -n 100 --no-pager

Setting a limit permits locking up to that amount; it does not preallocate that amount or guarantee a successful lock. Avoid granting every process an unlimited lock allowance to fix one service.

Verify and undo: obtain the new MainPID, repeat the /proc checks and read startup warnings. A unit property alone is not proof the restarted process inherited it. If this experiment should be undone, replace only the newly created 50-mlock.conf contents with:

[Service]
# No memory-lock overrides; inherit the previous service configuration.

Then reload, restart during the same planned window and verify the restored limits and loading choice. If you edited a pre-existing file or command line, restore its recorded original contents too. Do not use a blanket systemctl revert that could remove unrelated service customisations.

Distinguish a lock warning from an OOM kill

Check memory and kernel logs:

1
2
3
4
5
free -h
journalctl -k -b --no-pager | grep -Ei 'out of memory|oom|killed process'
systemctl show llama-server.service \
  --property=Result --property=ExecMainCode --property=ExecMainStatus \
  --property=MemoryCurrent --property=MemoryMax

A service can hit its cgroup memory limit even when the whole Pi has free memory. Increasing LimitMEMLOCK does not increase MemoryMax or the board's physical RAM. If the model only works unlocked and starts thrashing under load, reduce the model size, context, or concurrent workload before enabling locks. See the memory-pressure and OOM guide.

Killed or exit status 137 can mean SIGKILL, but not why the signal was sent. Correlate the timestamp and victim PID with kernel logs and the unit's result; an administrator or another policy can also terminate a process. A Result=oom-kill result or matching kernel OOM report is stronger evidence than an exit code alone. Missing journal access or absent records do not prove that no OOM occurred. Do not deliberately trigger OOM on the working Pi to test this page.

Record whether the change helped

Compare the same requests with unlocked and locked runs. Record the llama.cpp revision, loading mode, model checksum, context, concurrency, startup time, request latency, VmLck, peak memory, and thermal/power state. Separate a cold first load from a cached repeat. Retain the setting only when it improves the deployment's measured behaviour without starving other services.

Checkpoint: why can a lock request fail while RAM remains available? Why does a shell's ulimit not describe a separately managed service?

Donate