Alex Vale - ComputeFit

Local AI

Qwen3.8-27B Q4 vs Q5 on 24GB VRAM: What 32K, 64K, and 128K Context Actually Cost

In ComputeFit's measured 24GB test, UD-Q4_K_XL ran Qwen3.8-27B at ~31-32 tok/s across 32K, 64K, and 128K context with more observed VRAM headroom than UD-Q5_K_S, whose speed gap widened to ~45% (derived) at 128K. Quality was not measured.

Local AI

Running Local RAG with Embeddings in Ollama: Chunking, Dimensions, and the Failure Modes Nobody Warns You About

You do not need a vector database or extra VRAM to give your local LLM semantic search over its own documents. You need the right chunk sizes, the right embedding tag, and an index that fails loudly instead of silently cross-model-searching.

Local AI

Ollama’s Keep-Alive and Predictable Model Loading: Deciding When Your Local Model Stays in VRAM

Ollama can unload your model five minutes after it stops doing anything, and your next request pays the reload. How long to keep it resident depends on more than intuition: eviction priority is literally your keep-alive number.

Local AI

Ollama vs llama-server: Which Runtime Should You Actually Run Your Local Models Through?

Same engine, two different operating models. Ollama defaults to registry pulls and a VRAM-tiered context guess; llama-server asks you to set the window yourself and exposes layer split, KV-cache dtype, metrics, and API keys. A decision guide for both.

Local AI

The Hidden Cost of Context Length: Why Your Local Model Runs Out of Memory Before It Reads a Single Token

Raising context windows does not fit the same memory it used to. The KV cache is allocated in full before your first token, its size is computable from the model's config file, and hybrid-attention models change the number quietly.

Local AI

Ollama's OpenAI-Compatible Endpoint: What Your Agent Framework Actually Gets

Agent frameworks mostly assume OpenAI chat completions. What that contract looks like on a local machine: which /v1 routes Ollama v1 serves, which request fields are honored vs quietly dropped, context control by model aliasing, and where direct llama.cpp starts to pay.

Local AI

What Is llama.cpp? The Inference Engine Under Your Local Models, and When to Drive It Directly

GGUF came from llama.cpp, yet most of us meet it through a wrapper. What the engine actually exposes — server endpoints, slots, KV-cache math you can compute yourself, and the security defaults worth reading before exposing an endpoint.

Local AI

How Much Hardware Do You Need for Local AI Agents? A Practical VRAM, RAM, and CPU Guide

A practical sizing guide for running sustained local AI agent workloads: what 8 to 48 GB+ of VRAM really buys you, plus the system RAM and CPU decisions that come with each tier.

AI Agents

Cloud AI vs Local AI Agents: Choosing the Right Role

Cloud AI and local AI agents are not direct replacements for each other. Here is how I divide tasks between them and why a hybrid approach works better.

AI Agents

Why AI Agents Need Human Control: Lessons from Automation Failures

Building AI agents revealed an important lesson: automation alone is not enough. Learn why reliable AI systems need human control and clear boundaries.

AI Agents

Running Local AI Agents on a 24GB VRAM Machine: What I Learned

A practical look at running local AI agents on a 24GB VRAM machine. Learn about hardware limits, model selection, and what actually matters in daily use.

AI Agents

How I Connected OpenClaw with Slack to Build a Remote AI Assistant

A practical look at connecting OpenClaw with Slack to create a remote AI assistant. Learn why I chose Slack, how the workflow works, and what I learned.