Ollama - ComputeFit

Local AI

Ollama’s Keep-Alive and Predictable Model Loading: Deciding When Your Local Model Stays in VRAM

Ollama can unload your model five minutes after it stops doing anything, and your next request pays the reload. How long to keep it resident depends on more than intuition: eviction priority is literally your keep-alive number.

Local AI

Ollama vs llama-server: Which Runtime Should You Actually Run Your Local Models Through?

Same engine, two different operating models. Ollama defaults to registry pulls and a VRAM-tiered context guess; llama-server asks you to set the window yourself and exposes layer split, KV-cache dtype, metrics, and API keys. A decision guide for both.

Local AI

The Hidden Cost of Context Length: Why Your Local Model Runs Out of Memory Before It Reads a Single Token

Raising context windows does not fit the same memory it used to. The KV cache is allocated in full before your first token, its size is computable from the model's config file, and hybrid-attention models change the number quietly.

Local AI

Ollama's OpenAI-Compatible Endpoint: What Your Agent Framework Actually Gets

Agent frameworks mostly assume OpenAI chat completions. What that contract looks like on a local machine: which /v1 routes Ollama v1 serves, which request fields are honored vs quietly dropped, context control by model aliasing, and where direct llama.cpp starts to pay.

Local AI

What Is llama.cpp? The Inference Engine Under Your Local Models, and When to Drive It Directly

GGUF came from llama.cpp, yet most of us meet it through a wrapper. What the engine actually exposes — server endpoints, slots, KV-cache math you can compute yourself, and the security defaults worth reading before exposing an endpoint.

Local AI

Ornith-1.5 for Local AI: What a New MoE Family Means When Your GPU Fits One Model but Not Two

The new Ornith-1.5 open-weight family (35B-A3B MoE, 9B, MIT) fits inside one consumer GPU. What you should actually run on a 24 GB card, why its hybrid attention makes long context unusually cheap from the math, and when it replaces what you already host.

AI Agents

Running Claude Code on a Local Model: How Ollama's New Compatibility Endpoint Works

Claude Code can run against a model on your own machine, no paid key involved. Current Ollama does it with ollama launch claude — what the endpoint actually covers, which models to try first, and why the context window is the setting that matters.

Local AI

Muse Glimmer on Your GPU: What the Official Tags and File Sizes Actually Require

Meta Superintelligence Labs' new open model for local deployment lands with a real footprint. We map Muse Glimmer 30B's official quantization ladder against your card tier and Ollama tags, using exact file sizes.

24GB Lab

Qwen3.8-27B on a 24 GB Card With Ollama: The Defaults You Inherit and the Settings That Actually Matter

Why different context surfaces can show 262K and 131K at the same time, which Ollama defaults qwen38-27b hands you after a pull, what thinking-on-by-default costs in real use, plus a day-one checklist from one measured RTX 5090 Laptop GPU session.