> ## Content Index
> Fetch the complete content index at: https://computefit.dev/llms.txt
> Use this file to discover other available public pages before exploring further.

# Ollama’s Keep-Alive and Predictable Model Loading: Deciding When Your Local Model Stays in VRAM
- URL: https://computefit.dev/ollama-keep-alive-local-model-lifecycle-memory/
- Published: 2026-08-24T11:12:12.000Z
- Updated: 2026-08-24T14:39:38.000Z
- Description: Ollama can unload your model five minutes after it stops doing anything, and your next request pays the reload. How long to keep it resident depends on more than intuition: eviction priority is literally your keep-alive number.
- Author: Alex Vale
- Tags: Local AI, Ollama

You install a model into Ollama once with `ollama pull`. But what actually occupies your VRAM is not installed at pull time — it is instantiated on demand, and then quietly torn down when you stop using it. Between any two of your requests an idle loaded model disappears from memory without announcement, and your next request pays to bring the weights back from disk before a single token can come out.

Ollama’s internal scheduler makes these load and unload decisions for every request: how long a loaded session survives after its last activity, when an option change forces a full re-initialization instead of reusing what is already resident, and which model gets evicted when not everything fits on the card at once. If you run agents against your local server, benchmark first-token latency, or share one GPU between several workloads, that scheduler does more per-request bookkeeping than most people realize. This post covers what Ollama keeps in memory and for how long, which settings control it, how to tell a cold start from a warm response, and the concrete failure modes on VRAM-tight hardware.

## What stays in memory after a request finishes

The running state is not just “the weights”. When you ask `GET /api/ps`, each resident entry reports its name and tag, its digest, an expiration timestamp `expires_at` that the API documentation describes as the time when the model will be unloaded, the VRAM footprint in bytes (`size_vram`), the context length for that running instance, and parameter details such as the quantization level. (The same CLI surfaces this with `ollama ps`, and stops a resident model on demand with `ollama stop`.)

The entry does not disappear while a request is being served — Ollama tracks how many requests a loaded instance is currently handling, and only when that count drops back to zero and there is no pending activity does the idle clock start. From your point of view, then: a model is “loaded” from the moment its runner exists in VRAM until it expires or gets evicted, and any request landing inside that window skips straight to inference.

One caveat before going further: what fits depends on more than weight bytes. The KV cache grows with every token you add to a conversation, so a long session adds real memory on top of the base model — we computed that cost per architecture in [our article on the hidden VRAM cost of context length](https://computefit.dev/context-length-hidden-vram-cost-local-llm/). If you are choosing how long to keep sessions alive, size against weights plus your realistic peak context at once.

## The five-minute default

By default Ollama keeps a loaded model resident for 5 minutes of idle time. The variable that overrides this is `OLLAMA_KEEP_ALIVE`, documented by the project itself as “the duration that models stay loaded in memory (default "5m")”. It accepts either a duration string such as `2h30m`, or a bare number parsed as seconds — an integer like `7200` keeps the model warm for two hours. A negative value is treated as “keep alive forever”, i.e., pinning that model in memory until the server stops (or VRAM pressure forces it out, which we cover below).

Ollama evaluates this when deciding how long to hold a newly loaded runner, so set the variable on whichever unit or container owns your Ollama process and restart it; from then on you can watch each entry’s `expires_at` in `ollama ps` confirm what is actually in effect.

### The per-request `keep_alive`

A smaller, request-level variant exists on the chat and generate endpoints (and embeddings). Its documented description is brief: “model keep-alive duration (for example 5m or 0 to unload immediately)”. It accepts a string such as `"2h"` or a number of seconds. Two behaviors separate it from the environment variable:

- **It is request-scoped.** The duration travels with that specific call, so one client can extend (or shorten) residency without reconfiguring the whole server for everyone else on the network.
- **A value of zero means unload when this call finishes.** A documented curl example in Ollama’s API reference posts to `/api/generate` with a body of `{ "model": ..., "keep_alive": 0 }`. That is the same request shape behind the CLI’s `ollama stop`: from your side it simply tells Ollama this client considers its interaction done.

## How you tell a cold start from a warm request

You do not need external observability tooling. Two signals come back inside normal operations:

- **The per-call timing fields.** Every chat and every generate response carries `total_duration`, `load_duration`, `prompt_eval_count`, `prompt_eval_duration`, `eval_count`, and `eval_duration`. The `*_duration` fields are nanoseconds. On a warm call the load component is near zero; when it dominates, you know this call absorbed what would otherwise be an invisible stall.
- **The resident list.** If your client checks `/api/ps` (or `ollama ps`) right before and right after a request, the appearance/disappearance of an entry tells you whether the model was already there. This is cheap enough to do from a shell script or the same agent log file you keep for debugging.

If your benchmarking harness has ever reported one “token” as slower than five, this is usually why: the measurement folded in an entire load-time tail, not generation. The same trap applies to latency dashboards that sample at low frequency and are happy to silently include cold starts.

## Why your agent got a cold model

This is the most common way ordinary users first hit all of this. The pattern: your harness calls Ollama through its OpenAI-compatible surface — [the endpoint we documented for agent frameworks](https://computefit.dev/ollama-openai-compatible-endpoint/) — and between turns the agent is blocked on external work, reading a file or waiting for a shell command to finish or nothing in particular. If that gap exceeds whatever keep-alive value applies, Ollama will have unloaded the model by policy before you ask again, and your next call re-triggers a load with no error surfacing anywhere; it just looks like “a slow turn”.

The interaction gets worse when the same server is shared: if *any* client on that box issues an unload (or simply stops sending traffic), your agent’s session is no longer guaranteed to survive a pause of any length. On single-user hardware running one model, the practical fix is usually just “turn keep-alive up or pin it to -1.” On multi-client machines you will see this same problem expressed as eviction (next section), which points at shared capacity rather than time.

## Two models, one GPU

Ollama’s scheduler keeps track of every loaded runner and its memory footprint per request. When a new model arrives that would not fit alongside the already-resident ones without pushing over available VRAM, Ollama tries to make room instead of outright refusing: it looks for something already in an idle state (reference count effectively zero, not currently mid-turn) to drop first. This is *not* a fixed LRU — the scheduler’s actual eviction order puts lower keep-alive sessions in front of higher ones.

In other words: a model loaded with the default five-minute idle is more likely than one pinned at `-1` to be dropped out and reloaded. Keep-alive and “who will actually stay resident” are not separate problems in Ollama; they are literally two views of the same number.

This matters beyond curiosity: it explains why running a big model pinned for days and a small daily driver side by side regularly ends with one of them evicting the other under load. Keep in mind that pinning a model does not make it immunized — eviction happens whenever a newcomer cannot fit, and on cards where two twenty-GB-class residents simply do not coexist, pinned models can still be chosen when nothing has shorter remaining time to fall back on.

There is also an explicit upper-bounds variable people forget about: `OLLAMA_MAX_LOADED_MODELS`, documented by Ollama as “the maximum number of loaded models per GPU.” If this cap sits below the number of distinct models your workloads actually want concurrently, you will see load/unload churn no matter how high keep-alive goes — raise it (within real VRAM capacity) rather than pinning more runners than fit.

## When an option change forces a full reload

Not everything about unloading is time-based. If you send two consecutive requests against the same model name with different launch options, Ollama treats them as incompatible and re-initializes before responding to the second one rather than stretching the first instance across both. The most common trigger users hit: **changing context length mid-session without a Modelfile change**. If request A ran at context C and request B tries a different C, that is a full reload, because Ollama has already built the KV cache to match what was requested.

The same applies to numeric options like `num_gpu` or any parameter that requires rebuilding the runtime rather than just swapping in new weights. This is one reason context-length tuning in Ollama feels noticeably “heavier” than people who have come from a single fixed-size server expect: switching effective model shapes pays the load cost each time.

If you run several variants of the same base architecture on one card and your agents tend to switch between them (say, a base prompt model for routing plus an explicitly bigger-context variant for long files), treat those as *separate Ollama models with different names* rather than two sets of runtime options against one name. Named-per-variant lets keep-alive apply independently; option-toggles do not.

## Three failure modes that look like bugs but aren't

1. **The health-check starvation loop.** A monitoring script sends a small request every few minutes and never sets `keep_alive: 0`. That keeps the model resident — from outside it looks perfectly healthy. The problem is that your real interactive session, which expected to have the card to itself between uses, now shares VRAM with something that refuses to go away no matter what you do at its own `ollama stop`, because nothing has told Ollama “this specific check’s job is done.”
2. **The silent OOM-swap.** Two big models both pinned at infinity arrive with no idle gap to separate their demand, and available VRAM cannot hold both simultaneously. One of them must unload or the request fails; in most configurations you will see a long stall (Ollama is waiting for its scheduler to decide) followed by what looks like normal execution rather than an explicit “no room” error message.
3. **The context-swap reload tax described above, surfaced as latency swings.** If your workflow alternates between a long-context task and short prompts against the same model name at different C values — and does so repeatedly across turns — you are paying load-time for nothing but switching. The fix is structural: pre-decide which context value represents “the normal mode” of each workload and put it once in a Modelfile, rather than fighting the runtime ad hoc.

All three share one signature: latency spike or surprising memory state with no obvious error, where checking `/api/ps` timing data turns out to reveal exactly what actually happened. The fields described in the “cold start” section above are your evidence source for all of them.

## What to set (in practice)

Start with the defaults, not opinions of blog authors. There is no universal keep-alive that fits every machine — only numbers that fit *yours*, and you can measure your own answer in two steps:

1. Note what Ollama reports for a model already loaded after one full turn: its entry’s VRAM usage, plus how long `/api/ps` keeps showing it after all clients stop. That second part is your effective idle threshold (five minutes unless you already changed it).
2. Pick an upper bound below that number if you know the real users of this box will walk away for tens of minutes at a time, and above it only when “staying loaded” genuinely costs nothing on your hardware. If the GPU is shared between two different clients (for example, one interactive user at the same time as an agent loop), err shorter than the human case below.

### If you are running …

- **A single local model for yourself on a laptop or small desktop:** set `OLLAMA_KEEP_ALIVE=24h` (or pin to -1) if your GPU has room to spare, because most of the discomfort people report in this exact setup is “why did my fast feel slow” after a two-minute pause. The cost is bounded: one model on one card cannot really compete with itself.
- **An agent harness that calls Ollama repeatedly (including Claude Code's OpenAI-compat path):** do not rely only on the environment default, and consider passing `keep_alive` explicitly per call — a large model used for many turns in a row, pinned by the same client that is driving it, will not evict itself. The exact number you pick depends on how much other concurrent demand your machine has.
- **A shared box where two or more people/workloads alternate:** use named variants of each workload as separate `ollama pull`\-able models so their resident state is independent from one another, and size available memory before you even try to pin anything — run `/api/ps` for each candidate at the same time (you can do this by hand, sending dummy requests for a few seconds) and add them up. If they overshoot, that is an architectural problem keep-alive cannot save.
- **Benchmarking anything Ollama-backed:** log `load_duration` per call into the same line of output you use to record tokens/second or task completion. Without it your numbers are silently mixing warm and cold samples, which is a measurement bug rather than something about “model quality.”

One last practical note for people moving settings around: `OLLAMA_KEEP_ALIVE`, `OLLAMA_MAX_LOADED_MODELS`, and (less commonly) `OLLAMA_HOST` all apply at server start. If your Ollama runs under systemd, Docker, or the OS-installed app, changing a variable in an editor without actually causing that unit to restart will have zero effect — the live process is still holding its old env at boot time. A short `systemctl restart ollama`\-equivalent (or container re-create) is what makes a change real.

## Conclusion

The core model to carry out of this is not “Ollama keeps models loaded” or “Ollama unloads idle models” — it is that *Ollama’s scheduler owns a concrete, inspectable residency state per model instance*, where your keep-alive choice (global via the environment variable or request-scoped via `keep_alive: 0`) literally decides which sessions get evicted first when capacity gets tight. Once you treat that as the mechanism it is rather than a background mystery, “why did my response take ten seconds this time” stops being a riddle and becomes exactly the kind of thing your own logs already contain the answer to.