Local LLM - ComputeFit

Local AI

What Is llama.cpp? The Inference Engine Under Your Local Models, and When to Drive It Directly

GGUF came from llama.cpp, yet most of us meet it through a wrapper. What the engine actually exposes — server endpoints, slots, KV-cache math you can compute yourself, and the security defaults worth reading before exposing an endpoint.

24GB Lab

RunPod vs Vast.ai for Local AI: What to Do When Your 24 GB GPU Is Not Enough

A practical comparison of RunPod and Vast.ai for local AI users who occasionally need more VRAM than a 24 GB GPU can provide — with the stay-local vs rent vs buy decision worked out in order.

AI Agents

Running Claude Code on a Local Model: How Ollama's New Compatibility Endpoint Works

Claude Code can run against a model on your own machine, no paid key involved. Current Ollama does it with ollama launch claude — what the endpoint actually covers, which models to try first, and why the context window is the setting that matters.

Local AI

Muse Glimmer on Your GPU: What the Official Tags and File Sizes Actually Require

Meta Superintelligence Labs' new open model for local deployment lands with a real footprint. We map Muse Glimmer 30B's official quantization ladder against your card tier and Ollama tags, using exact file sizes.

Frontier AI

Qwen3.8-Max Weights Are Public on Hugging Face — But They Won’t Fit a Typical Local PC

Qwen3.8-Max-class weights are now downloadable from Qwen on Hugging Face. Here's how large the A95B checkpoint really is, what the smallest GGUF costs, and why it is far beyond a typical local PC.

Local AI

Qwen3.8-27B Vision: What Images and Video Actually Cost When You Run It Locally

Images work out of the box in Ollama. Verified against the model's shipped configs: what each image costs in tokens, what the projector adds on GGUFs, and exactly where my verification stops at video.

24GB Lab

Qwen3.8-27B Quantization: Which File Actually Fits Your GPU

Exact Hugging Face file sizes for every Qwen3.8-27B GGUF, a blunt per-card recommendation (UD-Q4_K_M at ~24GB), when NVFP4 changes the maths on Blackwell, and two-minute checklist built from hard numbers.

24GB Lab

Qwen3.8-27B on a 24 GB Card With Ollama: The Defaults You Inherit and the Settings That Actually Matter

Why different context surfaces can show 262K and 131K at the same time, which Ollama defaults qwen38-27b hands you after a pull, what thinking-on-by-default costs in real use, plus a day-one checklist from one measured RTX 5090 Laptop GPU session.

Local AI

Should You Switch to Qwen3.8-27B? An Upgrade Decision Guide for Existing Local LLM Users

A decision guide for people already running a local LLM: what the vendor-published model card says about Qwen3.8-27B versus Qwen3.6-27B, where the reported differences actually matter agentically and multimodally, and how to fit it on your hardware.