Ollama Errors: Model Not Found, Port 11434, GPU and OOM Fixes

Ollama is a client-server pair. The ollama CLI, the Python and JavaScript SDKs, and any OpenAI-compatible client all talk HTTP to a daemon on port 11434. "Connection refused" always means no daemon is listening at the address the client used, not that a model is missing.

The mirror-image error is "address already in use" when you run ollama serve manually. On macOS and Windows the desktop app already runs the server; on Linux the systemd unit does. That error usually means everything is fine and you should just use the running instance.

By default the daemon binds to 127.0.0.1, so it is unreachable from another machine, from a container, or from WSL against a Windows host. Bind it wider with OLLAMA_HOST=0.0.0.0:11434, and note that this exposes an unauthenticated model API, so keep it behind a firewall or reverse proxy rather than on an open network.

Ollama model references are name:tag, and the tag is not optional in practice, omitting it resolves to :latest, which for many models is a different quantization or size than you expect. ollama run llama3.1 and ollama run llama3.1:70b are different downloads with very different hardware requirements.

"model 'x' not found, try pulling it first" means the name resolved but nothing is stored locally: run ollama pull first, or let ollama run pull it. A 404 during pull means the name or tag does not exist in the registry at all: check the exact string, including the quantization suffix, on the model page.

Third-party GGUF models are addressed differently: prefix the Hugging Face repo path, as in ollama pull hf.co/<user>/<repo>:<quant-tag>. Pull failures that are neither of these are usually network: a corporate TLS-inspecting proxy, or a partial download. ollama rm the model and pull again to clear a truncated blob.

Ollama does not fail when it cannot find a GPU; it runs on CPU. The symptom is therefore not an error but a speed collapse: single-digit tokens per second on a machine that should manage many times that. ollama ps names the culprit directly in its PROCESSOR column: "100% GPU", "100% CPU", or a split like "45%/55% CPU/GPU" when only some layers fit.

On NVIDIA, confirm the driver is visible to the *service* user, not just your shell: the systemd unit runs as the ollama user and does not inherit your environment. Missing or mismatched CUDA driver libraries, a container started without --gpus all, and an unset CUDA_VISIBLE_DEVICES in the unit file are the usual causes. On AMD, ROCm support is version- and card-specific, and unsupported GPUs are often forced to work by overriding HSA_OVERRIDE_GFX_VERSION. On Apple Silicon, Metal is used automatically and there is nothing to configure; the equivalent failure is unified memory pressure, not a missing device.

The server log states which runner was selected and why. Restart with OLLAMA_DEBUG=1 and read the inference-compute lines near startup: they name the detected library, the device, and the available VRAM.

Ollama serves quantized GGUF weights, and the quantization tag is the main lever on memory. As a rough guide, a Q4_K_M model occupies a little over half a gigabyte per billion parameters, Q5_K_M somewhat more, and Q8_0 about one gigabyte per billion. An 8B model at Q4_K_M is roughly 4.7 GB of weights; add one to two gigabytes of runtime overhead and the KV cache for your context window on top.

Quality degrades gently down to about 4 bits and sharply below it, so Q4_K_M is the usual sweet spot and Q2 variants are a last resort. If a model does not fit, dropping a size class (a 8B at Q5 instead of a 14B at Q3) generally beats crushing the quantization.

When only part of the model fits, Ollama offloads the remaining layers to CPU rather than failing. That is why performance sometimes falls off a cliff after you raise the context window: a larger num_ctx enlarges the KV cache, which pushes layers off the GPU. ollama ps will show the split.

Ollama applies a modest default context window (4096 tokens in current releases) regardless of what the model supports, which is why a 128K-context model can appear to forget the middle of a long document. Raising it is a per-request option, a per-model Modelfile parameter, or a server-wide environment variable, and the request-level option wins.

Raise it deliberately: the KV cache grows linearly with num_ctx, so a large window on a card that was only just fitting the weights will push layers to CPU or fail the load outright. Increase it in steps and re-check ollama ps after each change.

Note that num_ctx is not the same as the model's trained context length. Setting it above what the model was trained for produces degraded output rather than an error; ollama show reports the real context length in the model's metadata.

Ollama unloads a model from memory five minutes after its last request by default. If the first call after a pause takes twenty seconds and later calls are instant, you are paying reload time, not inference time. OLLAMA_KEEP_ALIVE changes that window: a duration such as 30m, -1 to keep models resident indefinitely, or 0 to unload immediately after each request. The same value can be sent per request as the keep_alive field.

The mirror problem is too much resident at once. OLLAMA_MAX_LOADED_MODELS caps how many models stay in memory and OLLAMA_NUM_PARALLEL caps concurrent requests per model, each parallel slot needs its own KV cache, so raising it multiplies memory use.

When the error text is not enough, read the server log. On macOS it is ~/.ollama/logs/server.log; on Linux under systemd use journalctl -u ollama; on Windows it is in %LOCALAPPDATA%\Ollama\server.log; in Docker use docker logs. Set OLLAMA_DEBUG=1 before reproducing to get the layer-offload and device-detection detail.