Qwen3.8-27B Q4 vs Q5 on 24GB VRAM: What 32K, 64K, and 128K Context Actually Cost

If you run local AI on a 24GB-class GPU and you're choosing a quantization for a 27B-class model, the question is rarely "does it fit?" It's: how much does the extra precision cost me in speed, and how fast does that cost grow as I push the context window?

We ran that exact comparison for Qwen3.8-27B. Same model, same hardware, same runtime: the Unsloth UD-Q4_K_XL artifact against the Unsloth UD-Q5_K_S artifact, at 32K, 64K, and 128K context, measured for generation speed, observed VRAM, and observed headroom. This post reports what the measurements show — and, just as important, what they don't prove.

Why this comparison matters

Q4 and Q5 are often compared partly on the quality side of the quantization trade-off, with higher-precision quantization carrying a cost in file size and memory. The quality side is real and, as we'll note, not something this test can speak to.

The other side of the trade is concrete and machine-specific: a larger quantization takes more VRAM, and on a GPU with a hard 24GB ceiling, every extra GiB of model weights is a GiB less available for the KV cache and everything else the runtime needs. On a well-provisioned system you may never notice. On a 24GB laptop GPU running a 27B model at long context, the margin can be the difference between comfortable and cramped.

So the useful question is empirical: in a real 24GB configuration, what did Q5 actually cost versus Q4, and did that cost stay constant or grow with context?

Test setup

Everything below ran on one machine, one runtime, and two verified artifacts:

• Model: Qwen3.8-27B (canonical)
• Artifacts: Qwen3.8-27B / Unsloth / UD-Q4_K_XL and Qwen3.8-27B / Unsloth / UD-Q5_K_S — both with verified artifact identity (source GGUF provenance). The Q4 file is ~16.35 GiB, the Q5 file ~17.38 GiB on disk.
• GPU: NVIDIA GeForce RTX 5090 Laptop GPU, 24,463 MiB visible VRAM
• CPU: Intel Core Ultra 9 275HX
• Runtime: Ollama 0.33.1 (native mode), Windows 11 Pro, NVIDIA driver 610.78
• Method: for each context, 3 measured samples per artifact; reported values are medians. Every run (all 6 conditions) succeeded with 100% GPU residency and no unexpected CPU offload.

Two numbers from the method that shape how to read the results. First, Observed Peak VRAM is the peak VRAM observed during these runs (the median of the 3 samples) — not a "minimum required VRAM" for the model. Second, Observed Headroom is visible VRAM minus that observed peak: the margin left over in this specific test. Both describe what happened in this exact configuration, not universal requirements.

Results

Generation speed (median of 3 measured samples), Observed Peak VRAM, and Observed Headroom:

Derived comparisons (calculated from the measured medians above, not separately measured):

Two things jump out immediately. Q4 was faster at every context. And while Q5 used more VRAM at every context, the speed story is where the context length matters most.

32K: both fit, Q4 is faster with more margin

At 32K context this is the friendliest case for both artifacts. Q5's peak landed at 19.98 GiB — it fits comfortably, with 3.91 GiB of observed headroom. Q4 sits 1.10 GiB lower at 18.88 GiB and generates faster: 32.08 vs 28.21 tok/s, a derived ~13.7% advantage.

Nothing here is dramatic. If 32K is your ceiling, Q5 is a workable choice on this hardware; the cost is a modest speed hit and about a GiB less margin.

64K: the same pattern, less margin

At 64K, Q5's observed peak climbs to 21.18 GiB — headroom drops from 3.91 to 2.71 GiB — while Q4 moves to 20.12 GiB with 3.77 GiB to spare. Generation speeds: 31.48 tok/s for Q4, 27.71 for Q5, again a derived ~13.6% in Q4's favor.

The shape of the trade is unchanged from 32K: Q4 faster, Q5 slower, Q5 heavier. Longer context leaves less observed headroom on both sides, and Q5's thinner starting margin shows in its headroom (2.71 GiB vs Q4's 3.77 GiB).

128K: where the trade-off gets real

At 128K, Q5 hits 23.17 GiB observed peak. That leaves 0.72 GiB of observed headroom — the model nearly at the ceiling of this 24GB GPU in our measurement.

The cost in speed is the bigger story. Q5's generation speed falls to 22.21 tok/s, down from 28.21 at 32K — a ~21% decline as context grew fourfold (32K to 128K). Q4, by contrast, holds essentially flat: 32.15 tok/s, within a fraction of a token of its 32K and 64K rates. At 128K Q4's derived speed advantage over Q5 widens from ~13.7% to ~44.8%.

Notice the VRAM delta narrows at 128K (+0.65 GiB versus +1.10 at 32K) while the speed gap explodes. The extra bits cost roughly a GiB at 32K–64K, and the delta actually narrowed to +0.65 GiB at 128K — yet at long context, running Q5 leaves almost no room left in the memory budget, and that tightness coincides with the slowest generation of the whole test. If you intended to run long-context workloads on this GPU, 128K is where the Q4/Q5 decision stops being theoretical.

What this means for a 24GB local setup

Read strictly, the results say: in this exact tested configuration, UD-Q4_K_XL generated faster than UD-Q5_K_S at 32K, 64K, and 128K, stayed flat across the range, and left more observed VRAM headroom at every context — with the gap largest at 128K, where Q5 dropped to 22.21 tok/s with under a GiB of observed headroom.

For the practical decision, that translates roughly like this:

• Comfort-first, or you run long context (64K+, especially 128K) on 24GB: the measured evidence favors Q4. The margin Q4 leaves is what kept this full-GPU deployment comfortable in our test, and its generation speed didn't degrade as context grew.
• 32K is your ceiling and quality is your stated priority: Q5 is workable here in our test with ~3.9 GiB observed headroom, while Q4 generated ~14% (derived) faster. Whether Q5's higher precision is worth that cost depends entirely on the quality question — which this test did not answer.
• You plan to add other GPU load (browsers with hardware acceleration, a second model, image work): that observed headroom is the remaining margin those additional GPU workloads would have to share. 0.72 GiB at Q5/128K is thin; 1.37 GiB at Q4/128K is tighter but meaningfully more.

Two honest caveats to pair with any of these readings. First, "observed headroom" is what we measured, not a promise: a 0.72 GiB observed margin is an observation of how tight this configuration was in our runs, not a specification. Second, this is one runtime on one OS on one driver stack — Ollama 0.33.1 on Windows 11. Other runtimes may allocate more or less; none of these numbers is a minimum-VRAM requirement for Qwen3.8-27B on any hardware.

What this test does NOT tell us

It says nothing about answer quality. ComputeFit did not run a model-quality benchmark in this comparison. It was not measured whether Q5 produces better, meaningfully better, or equally good answers than Q4 on any task. None of the following is a supported conclusion: that Q5 is pointless, that Q5's extra bits provide no benefit, that Q4 matches Q5 in quality, or that Q4 should always be preferred. The measured profile above is a speed/VRAM profile. The quality side of the quantization decision is an open question this experiment does not settle.

It does not generalize to other GPUs. These numbers are from one machine: an RTX 5090 Laptop GPU with 24,463 MiB visible VRAM, an Intel Core Ultra 9 275HX, Ollama 0.33.1, NVIDIA driver 610.78. We cannot say they reproduce on an RTX 4090, an RTX 3090, another 24GB card, or a 16GB or 20GB card — in either direction. Use them as measured reference data for planning, not as hardware requirements.

It covers what it covers. Generation TPS at three context lengths, three samples each, generation phase only. Prompt-processing latency, accuracy under load, and behavior beyond 128K were not measured.

The ComputeFit takeaway

If your priority is fitting Qwen3.8-27B comfortably into a 24GB-class configuration like the one we tested, while preserving generation speed — especially at 64K and 128K context — UD-Q4_K_XL had the stronger measured performance/VRAM profile in our test: flat ~31–32 tok/s across 32K–128K and roughly a GiB more observed headroom than Q5, with the advantage growing to ~45% (derived) at 128K. Whether Q5's higher-precision quantization produces enough quality benefit to justify that measured cost was not measured here — and for many workloads, that's the part of the decision that actually matters.

We publish measurements like this one to move the decision from "pick by forum opinion" to "pick by what this tested hardware configuration actually measured." The live ComputeFit GPU Checker now exposes the measured context matrix behind this article, alongside per-hardware fit guidance — so you can compare a given GPU's hardware-fit guidance against this measured context matrix before you download anything.

Short final takeaway: On the tested 24GB RTX 5090 Laptop GPU configuration, Q4 generated ~14% (derived) faster than Q5 and used ~1.1 GiB less observed VRAM at 32K–64K — and at 128K that speed advantage widened to ~45% (derived), where Q5's observed headroom dropped to 0.72 GiB. Q4 stayed fast and left more margin at every context. Quality was not measured, so treat this as the performance/VRAM half of the quantization decision, not the whole decision.