Qwen3.8-27B Quantization: Which File Actually Fits Your GPU

Short answer first

If you run Qwen3.8-27B on a desktop or laptop card with around 24 GB of video memory, my conservative pick for a fixed 24 GB VRAM budget — working headroom over per-layer fidelity — is the UD-Q4_K_M GGUF (15.3 GiB download); Unsloth itself currently highlights UD-Q4_K_XL as its recommended dynamic 4-bit GGUF, so pull that one instead when you have verified your real context workload fits with its extra ~1 GiB. The plain Q8_0 file will not fit: at 27.1 GiB by itself — before a single context token arrives — it overshoots an entire 24 GB card.

The decision in this article is deliberately one number per card: your VRAM budget. For each realistic budget (16 GB / 24 GB), I give you exactly which quantization to pull and why, backed by hard file sizes from the official Unsloth repository on Hugging Face — not vendor claims about "quality".

The rule underneath all of it: download size ≠ runtime memory. Unsloth's own requirements table for this model is expressed in total system memory including VRAM, which already reflects KV cache and overhead. As a conservative planning rule in this article, I target roughly 4–6 GiB of headroom between loaded weights and your card ceiling for KV cache plus working set — that margin is my own planning choice, not something read off Unsloth's requirements table.

This complements the companion articles without repeating them: the settings guide tells you what defaults Ollama gives you; the field report shows the same model measured live on my own RTX 5090 Laptop (~23.9 GiB); this one answers which file to download.

• Should You Switch to Qwen3.8-27B? — upgrade decision, vendor benchmarks labeled separately.
• The Settings That Actually Matter — Ollama defaults, context surfaces, thinking-mode cost.

Measured numbers you may already have seen come from my own box (RTX 5090 Laptop GPU, 24,463 MiB) in the field report; everything else here is either official file metadata or an inference I flag as such.

What this model's files actually weigh

The table below is not an estimate and not a vendor marketing list — every byte count was read today from the file listing of unsloth/Qwen3.8-27B-GGUF on Hugging Face [official repo metadata]. Sizes are shown in GiB (divide by ~1.07 to get the decimal GB that Hugging Face displays next to each file).

A few things to keep straight, because they are where downloads go wrong:

• The 27B model is a hybrid: most of its layers use linear attention (Gated DeltaNet) with only one in four using full attention [official model card]. Practically, that means the model's per-token state growth with sequence length is far less aggressive than a naive full-attention transformer of the same total width [inference from architecture — I did not re-derive or measure KV cost myself]. It still grows as your session loads more content, so the headroom rule above stays in force either way.
• Vision capability is sold as a separate file: mmproj-F16.gguf is about 928 MB on disk (the figure this article uses, ~0.9 GiB); the actual runtime VRAM overhead was not measured here, so treat any number you plan around as a file-size starting point rather than a VRAM reservation; skip it for pure text work and your context budget does not need to account for it at all.
• The full BF16 weights total about 51 GiB across two files — listed here only so its absence from any single-card plan is a fact, not an oversight. Unsloth's own requirements table, read as total system memory (RAM + VRAM), maps 4-bit of this model to roughly the same band you'd compute yourself from the file sizes above: about 16–19 GB for usable working room [vendor documentation]. The two sources agree; that agreement is why the per-card choices in the next section are blunt rather than hedged.

The MTP draft deserves a note even though it is optional. Qwen3.8-27B ships with multi-token-prediction layers [official model card], and Unsloth publishes that head as its own 1.3 GiB Q4_0 file (see the table). Speculative-decode configs in open runtime files reference exactly this pairing [community config, observed]. The point for you: if your quant leaves roughly a gigabyte of slack, there is a small extra source of throughput available without changing quality much — but that is a speed option, not part of "does it fit", so keep the two budgets separate in your head.

Pick by your card, not by a score table

Ten different websites will tell you which quant is "better". For a fixed-VRAM machine that question almost always resolves to one integer: how much VRAM you refuse to spill. So here are the two budgets I would actually plan around.

A ~24 GB card (RTX 4090-class, RTX 5080/5090-class, 5090 Laptop)

• My conservative pick for this budget: UD-Q4_K_M (15.3 GiB download). After allowing for runtime overhead and the rest of the loaded working set, I treat roughly four to five GiB as practical working headroom at moderate context rather than simply subtracting the 15.3 GiB download size from nominal VRAM. With a hybrid-attention model like this one, "moderate context" goes further than it would on a pure dense transformer — [inference from my field report data rather than published figures].
• If you feel safe reserving more context room: UD-Q4_K_XL (16.4 GiB) is the same trade in the other direction — slightly better per-layer fidelity on the sensitive parts at a cost of about 1 GiB against your reserved space [inference]. It fits; it just squeezes your working set harder.
• Skip Q8_0 (27.1 GiB) unless you specifically want to offload layers to system RAM and are prepared for the speed change that implies. A card that cannot hold the full working set will require partial CPU/system-RAM offload or another runtime-specific split strategy, with the performance trade-offs that implies.
• Skip Q5/Q6 as "the answer". They can run on a 24 GB card with partial offload; they are simply no longer your default recommendation to someone asking which file fits. That is a different article, and it is not this one.

A ~16 GB card (RTX 4080-class down, many work laptops)

• Pull: UD-Q4_K_S or Q4_0 (~14–15 GiB). Both fit in the same envelope; K-quant formats give you more per-layer fidelity granularity at a given total size than plain Q4_0, so I'd default to UD-Q4_K_S and keep Q4_0 as the portable fallback [inference on format properties; sizes verified in the table above].
• If quality visibly drops: UD-IQ4_XS (13.3 GiB) buys back over a gigabyte of context headroom versus either 4-bit option above it, in exchange for more compression. It is the "does it still hold up" test point, not the comfortable point [inference].
• The 3-bit tier (UD-Q3_K_XL, 12.2 GiB) is where you start accepting real degradation for a meaningful context gain. I would only reach here if you have confirmed the 4-bit options are doing something visible your workload cannot tolerate [inference].

If these numbers are for a 16 GB card, one more boundary before the checklist: do not read a 14–15 GiB download size as proof that the full working set will remain comfortably GPU-resident on a 16 GB card. Depending on context size and runtime overhead, system-RAM offload or a smaller quant may still be required.

The one number to check against these recommendations: my own RTX 5090 Laptop (24,463 MiB) running a fully-resident Q4_K-class session ended up close enough to the ceiling that I was actively managing context length rather than forgetting about it — details and full measurements in the field report. Treat "it fits" as "it starts with room to spare", not "it fits at zero headroom". That distinction is why the table above lists download sizes rather than a single magic number per card.

Disclosure: This article contains a RunPod referral link. If you sign up through it and fund your account, RunPod may grant referral credits to both you and Local Frontier. This does not change the price you pay.

If the quant you actually want requires more VRAM than your local card has, renting a larger GPU for a few hours can be more sensible than stepping down several quantization levels just for one test. Try a larger GPU on RunPod is the option I'm currently evaluating for that workflow; it appears here as an evaluation of that workflow, not as a comparison between cloud providers.

If you have a Blackwell card: NVFP4 changes the trade

This is the one place where which file fits splits by GPU generation rather than VRAM amount. Unsloth ships a separate quant format family for Qwen3.8-27B called NVFP4. Their documentation states, in terms it prints for itself (vendor self-reported — not independently benchmarked here): roughly 2.5x faster, with FP8 KV-cache calibration for longer context lengths, and NVIDIA Blackwell hardware required — the RTX 50X series or DGX Spark (docs also list data-center B200/B300).

Three caveats from that same documentation page, all worth printing on your wall:

• It lives in the serving stacks that load it directly. Unsloth's current documentation carries both a vLLM and an SGLang path for this kind of quantization, so check which one (if either) is actually on your box before assuming availability. If your everyday path is Ollama or llama.cpp serving .gguf files, treat NVFP4 as outside that stack — same card or not [vendor documentation; unmeasured].
• "Comparable file sizes" to the GGUF tier per their description [vendor characterisation], so it does not solve a capacity problem; it solves a throughput + context-length problem given Blackwell silicon.
• If you are on anything older (RTX 40-series and below), the same documentation's advice is that GGUFs "work well" for those cards — i.e., there is no penalty for staying in this article's GGUF track. [Vendor recommendation.]

The honest short version: NVFP4 is a speed/context upgrade path, not a replacement recommendation. If you are on RTX 50-series and you already run or plan to run vLLM for local work, read their NVFP4 guide before choosing weights — but do it from the same file-size starting point this article built. That is where your budget math happens either way. Everything I can verify today is in the GGUF repository and Unsloth's model page; I am not going to dress up a vendor speed claim as an independent measurement.

One more distinction that quietly saves people from making the wrong call: "which quant" is not the same question as "should I upgrade to 3.8 in the first place". If you are still leaning between staying on Qwen3.6 or moving to this model, run through the decision checklist in the upgrade guide before spending download time here — this article assumes that question already got an answer.

The two-minute checklist

If you want the whole article in one screen, this is it. Run through top to bottom and stop at your card class:

1. Note your real VRAM budget. Total video memory minus anything else that will be resident (desktop compositing on consumer cards leaves a slice; subtract nothing dramatic for headless servers). For "a 24 GB card" in practice, plan around ~20–21.5 GB as spendable [inference from the field report's per-process occupancy].
2. Pick the quant:
   • ~24 GB spendable → UD-Q4_K_M (15.3 GiB). If you specifically need more context headroom, step down to UD-Q4_K_S or plain Q4_0. If you want the most fidelity this tier can buy and have verified your working set fits: UD-Q4_K_XL.
   • ~16 GB spendable → UD-Q4_K_S/Q4_0. Only reach the 3-bit tier after watching actual output quality on your real tasks.
   • Blackwell (RTX 50-series) and you run vLLM anyway → also evaluate NVFP4 against their official guide before committing to a GGUF, with the file-size column above as your shared yardstick. [Vendor documentation; unmeasured here.]
3. If you need vision input, mmproj-F16.gguf is about 928 MB on disk (~0.9 GiB); actual runtime VRAM overhead was not measured here, so treat that as a download-size starting point rather than a reserved amount; if you do not need it, ignore the file entirely rather than loading it "in case". [File size from repository metadata.]
4. Check your context plan after choosing a quant. The model's native window is 262K tokens natively, extensible to 1M [official model card], but how much of that you can actually hold at once depends on per-token KV cost times loaded length — which for this hybrid-attention architecture is smaller than a naive dense-model estimate [inference]. That is what the "leaves headroom" language in this article protects: don't let a downloaded file size be your only number on the table. The concrete, same-class-machine reading of how tight that gets at 24 GB is in the field report.
5. Load it and sanity-check once: confirm what you actually loaded matches intent via your runtime's own status view (in Ollama specifically, the /api/ps-style listing shows resolved quant and live context — the settings article walks through why this differs from shelf numbers). One check, not ceremony.
6. If the file you want exceeds your local VRAM budget, compare stepping down a quant against temporarily renting a larger GPU on RunPod.

Sources and how to audit every number above:

• File size table: hf.co/unsloth/Qwen3.8-27B-GGUF file listing, read on 2026-08-20 (bytes → GiB conversion at 1 GiB = 1,073,741,824 bytes).
• Hardware requirements table ("total memory: RAM + VRAM"), NVFP4 capability/compatibility claims: Unsloth's Qwen3.8-27B documentation, same date — treated as vendor self-reported, not independently reproduced.
• Architecture (hybrid Gated DeltaNet + Gated Attention), native 262K context / 1M extension path, MTP head: Qwen/Qwen3.8-27B [official model card] on Hugging Face.
• All local machine figures cited (24,463 MiB GPU occupancy and behavior while running a Q4_K-class file resident): my own field report linked above — treated as a single-machine reference point, not a universal constant.

Where this article intentionally did not go: it does not benchmark token quality differences across these quants against each other on a fixed harness. That's a real scientific question and I would need to actually run one to say anything defensible — treat any "X% quality loss" figure you see for these specific files as unverified here until that happens.