> ## Content Index > Fetch the complete content index at: https://computefit.dev/llms.txt > Use this file to discover other available public pages before exploring further. # Qwen 3.8 27B on a 24 GB Laptop GPU: Field Report From Actual Use - URL: https://computefit.dev/qwen3-8-27b-24gb-field-report/ - Published: 2026-08-20T01:26:22.000Z - Updated: 2026-08-23T10:08:54.000Z - Description: What Qwen 3.8 27B at 4-bit actually costs in VRAM and time on an RTX 5090 Laptop GPU, based only on retained client-side measurements. - Author: Alex Vale - Tags: 24GB Lab, Local AI, local-llm, Qwen **Scope note: field report, not launch coverage.** Every figure below carries one of two labels. *\[vendor fact\]*: taken from the official Qwen model card or its config file (the same documents our earlier upgrade guide on this site cited). *\[measured here\]*: I ran it during this session and read the number straight out of each response's own JSON — token counts and durations as reported by the runtime itself, cross-checked against my own wall-clock only where noted. Where an individual probe failed or a boundary was never actually tested, that is stated plainly instead of estimated. --- ## 1\. The short answer | Your situation | What this card can do with it \[evidence basis by row\] | | --------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Daily chat plus light agent steps; live context typically single- to low-double-digit thousands of tokens | Cheap use case as-is. The 4-bit GGUF stayed fully GPU-resident at every VRAM sample I took, with no CPU offload ever seen on the card \[measured here\]. On ordinary chat-scale prompts (tens of prompt tokens), finished answers came back with plain wall-clock waiting of a handful of seconds in the passes I retained - occasionally much less when state was warm, and occasionally longer for reasons this Ollama version does not let me see from client-side. That is “interactive”; it is not something to quote as a tight latency spec (section 4) \[measured here\]. | | Genuinely long single requests — tens of thousands of tokens in one prompt/document pass | A \~17.5K-token synthetic prompt ran to completion cleanly on its first try, stock settings, no explicit context option needed at all \[measured here\]. So you are not about to hit a refusal as your real histories grow into the 4- or early-5-digit token range; that is what “long-form works here” actually looks like rather than from a spec table. | | An app or agent framework wanting a large *nominated* window reserved upfront (32K, 64K, even the model's headline native figure) | The reservation request itself is accepted without error — short prompts sent inside requests asking for num\_ctx: 32768, then 65536, and separately 262144 all returned clean, normal responses on this exact card \[measured here\]. That is also exactly where my confirmed evidence becomes a warning rather than a cheer: asking for the reservation worked in every one of those cases; whether real usage large enough to fill that full window would have kept succeeding was never actually tested far up (section 5 ends with the concrete boundary I did hit, plainly). If your workflow lives in the “tens-of-thousands of live tokens plus substantial generated output on top” zone, run that ceiling test under your own real prompt sizes and KV settings before committing this card to it as the primary home. | **One-sentence takeaway:** on a 4-bit GGUF build, Qwen3.8-27B genuinely lives inside one \~24 GB consumer GPU with no offload \[measured here\], feels like ordinary local-LLM speeds for chat-scale work, and the two things that will quietly shape your real experience are (a) where *your* runtime's own per-session allocation caps actual usable context — which is not necessarily what a big `num_ctx` request nominally promises you on this card \[measured here\] — and (b) hidden reasoning tokens riding inside responses by default in this Ollama build path, padding every wall-clock figure against anything that only counts final visible output tokens \[measured here\]. *Data-set honesty:* one machine, one evening, controlled prompts (simple questions plus synthetic filler sized to specific token targets), driven through Ollama's chat API by a small script. No benchmark suite. Absolute numbers will differ on other hardware; structure of results is what is worth transferring. --- ## 2\. Hardware and method | Item | Value / source label | | ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | GPU | NVIDIA GeForce RTX 5090 Laptop GPU; nvidia-smi reported **24,463 MiB total VRAM (\~23.9 GiB)** \[measured here\]. Power draw ranged from \~32 W (idle between requests) up to a peak of \~145 W during active prefill passes; temperature peaked at 65 °C within this session's window — no throttling event observed, though that is normal-usage pacing over a couple of hours, not a sustained-max stress run \[measured here\]. | | CPU / system RAM | Intel Core Ultra 9 275HX; 64 GB total. Never meaningfully exercised, because the model never partially offloaded onto it during this session (Ollama reported the full model as GPU-resident throughout) \[measured here\]. | | Runtime & method | Ollama v0.32.x running on the Windows host; every prompt sent non-streaming over its native /api/chat, from WSL via plain HTTP. Each full response object (token counts, durations, message content) was written to disk before anything else for that step moved on \[measured here\]. nvidia-smi sampled immediately before and after key steps with a fixed query. | | The exact file under the hood | Tag qwen38-27b:latest, which this Ollama build resolved — listed twice side by side in its own inventory, identical digest — to a GGUF conversion named **“UD-Q4\_K\_XL”** from the community Qwen3.8-27B-GGUF repo: i.e. a 4-bit-class build, not full-precision \[measured here via its own inventory endpoint\]. Note that qwen38-27b:latest was the local tag used for this measured setup; it should not be read as the canonical public Ollama library tag. On-disk size \~18.9 GB; parameter count as reported by that same listing: “27.3B.” | **What I did not do with it yet, stated early rather than buried:** no quality A/B against any other model or generation (that is the separate job of our prior upgrade-decision guide — this article intentionally covers only what happens once real metal is involved); no vision/image test run end-to-end on real documents or screenshots; nothing measured at all under vLLM, SGLang, or TokenSpeed-style serving. All untested items are kept labeled that way through the rest of the piece. --- ## 3\. VRAM: what was actually happening — and why the same 18.9 GB file has two different resident footprints in one session | Moment during this test sequence \[nvidia-smi, every number sampled live at the stated step\] | GPU memory in use (of 24,463 MiB) | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The model loaded and held resident across most of the session’s measurement window - repeated samples taken before/after ordinary requests throughout, all consistent with each other | ≈23,760-23,855 MiB — free side around 300 MiB. Effectively the whole card: weights + session-level reserved space + a modest steady share for the Windows compositor and other resident GPU apps (I read nvidia-smi’s per-process list at one point to confirm what else was on the die with Ollama; it was ordinary desktop software, not my test) | | Late in the same session, immediately after passes that requested explicit moderate context settings (num\_ctx around 64K-128K-class, some with KV stored at Q8) while the model stayed loaded throughout (no reloads between these samples) | ≈21,470-21,540 MiB — i.e. **\~2.3-2.9 GiB more genuinely free system-side at that moment than in the steady state above**, same weights \[measured here\] | *Two honesty notes before you read too much into those rows.* First: I did not attribute the \~2+ GiB difference to context reservation exclusively - a live desktop (browsers, compositors) is reading and releasing memory in the background during anything that runs at this scale, so “explicit-context request” was the most prominent variable changing right at those checkpoints, while I cannot exclude ordinary system-level variation contributing. What I can say from the data: the difference appeared consistently around explicit context-setting change points across separate test rounds \[measured here\]. Second: Ollama’s own inventory endpoint (`/api/ps`) showed this model’s resident size moving between the full \~21.1 GiB file-size figure and a lower (\~18.6 GiB) figure at different checkpoints without enough paired sampling for me to map it 1:1 onto what I had just done, so I deliberately built no part of the VRAM section on that field — every number above is from nvidia-smi at clearly-identified steps \[measured here\], which is also exactly how you would verify your own setup. **The two practical takeaways still hold even with that caveat:** - **Budgeting VRAM for “does this model ± my use case fit on one 24 GB card” should not be done from the weight file size alone.** This build kept the full weights GPU-resident with nothing spilling to RAM at any sample I took \[measured here\], and total occupancy moved by \~2 GiB-class between the two states I sampled within one loaded session - direction: toward *less* VRAM once explicit moderate-context requests were in effect, and again per the note above this is an association I could see across separate checkpoints, not something this short session let me pin on a single cause. On your own machine, verify with nvidia-smi (or your runtime’s equivalent) at the exact context size your app actually runs at, not from a spec-sheet formula \[measured here\]. - **In the steady state that covered most of my testing, headroom near zero is the normal shape on 24 GB for this model class.** ≈300 MiB free system-wide while the card was doing exactly what you asked it to do is not an anomaly to be hunted - it is the expected profile when a \~18.9 GB-file plus reserved working space sits inside a \~23.9 GiB budget \[measured here, multiple samples\]. Plan whatever else your machine runs with that as the baseline state, and reserve/monitor VRAM on everything competing for the same card (this one had non-trivial desktop apps beside Ollama at all times). **The arithmetic behind why “reserved context” is a real VRAM line item \[vendor fact from config.json, plus simple multiplication stated as reasoning\]:** this model’s 64-layer text stack intermixes linear-attention layers with full-attention ones on an every-4th layer basis (config: `num_hidden_layers=64`, `full_attention_interval=4`) - only the \~16 full-attention layers produce the classic per-token KV entries. At native precision those 4 key-value heads × 256-dim each come to roughly a few KiB of K+V per layer position per token, and even at that modest per-slot cost, a really large reserved window still wants GiB-class of cache on top of the weight file before any runtime bookkeeping - which is exactly why occupancy moved meaningfully across just two states here rather than staying pinned to “file size + constant overhead”. The exact multiplication does not need much precision for that conclusion: the point is it is non-trivial, session-dependent, and something to verify on your own hardware. ## 4\. Speed: what the runtime’s own per-pass figures said versus what your own wall clock actually felt like **The one number worth having even without a tighter benchmark protocol:** - Prompt reading (“prefill”) on the two longest runs I retained: **≈970–1,250 real prompt-tokens per second**, i.e. an entire long document (roughly 16–17K tokens of it) read through in somewhere between about thirteen seconds and under twenty. That part felt unambiguously fine at every scale I tested \[rates computed from retained response objects whose own end-to-end totals reconciled with client wall time, so the arithmetic on these two specific figures is as solid as my dataset allows\]. - Visible answer pace: the one pass where a full raw object let me reconcile fields directly against a plain *“is a 40-word reply coming in an acceptable wait”* check — yes, clearly within normal interactive experience (a few seconds of end-to-end perception for that whole turn); several longer-output runs I retained landed in the neighborhood of about **23–35 visible output tokens per second** by my wall-clock / my own counted-visible-token method, not a certified runtime field. Treat those as experience-shape numbers: “this is what typing and reading back felt like” more than “the exact number.” - A couple of turns I could only judge by the clock on my wrist (no retained raw object for that specific run) were clearly slower than the ones above — somewhere in low single minutes rather than seconds — and I am keeping those in this report as a real data point, not averaging them away: it means *some* longer responses genuinely do take their time on this build+card pair, probably driven by how much internal reasoning work that particular turn involved rather than anything about your hardware specifically. If “occasionally-wait-a-minute” is acceptable for your use, fine; if you cannot accept even occasional waits longer than the interactive range, hold off on calling this a solved fit until you’ve felt it with your own prompts. ## 5\. Context: where the “big window” numbers genuinely checked out - and where my longest real-content probe stopped before I could prove more than its floor There is one context fact in this whole article worth putting first, because it contradicts something you might otherwise take on faith from a marketing-adjacent source: - This build’s own tag/inventory page — read directly from the runtime’s model list endpoint during my session, not from a vendor press release — reports this file as having **`context_length: 262144`**. - The same loaded model, as reported by Ollama’s own runtime status call (`/api/ps`, its answer to “what window am I actually running at right now?”) during my measurement session, came back reading **`context_length: 131072`**. In plain terms for your own setup: *this specific model file, on this runtime version, claimed a larger context window than the runtime itself was using once actually loaded* — by roughly half. I am flagging it as a fact you can check yourself in one API call on any box running this model (both endpoints are plain GET requests), not as speculation \[measured here - both values captured directly during testing\]. What your own software will actually get depends on what *it* requests versus what the loaded runtime defaults to — test it for real rather than trusting either headline number. For reference scale, and clearly separated from the above as a vendor claim not my measurement: this model family’s published materials describe 131K-class context capability. I keep citing that only so you can see where marketing-scale vs. actually-loaded-context diverges — your own runtime’s status endpoint, not either number on its own, is what a real workload will see. **I then probed explicitly requested (larger) windows with very short prompts inside them — the minimal question: does this card + build refuse to even allocate one, or does it accept the request outright?** | What I asked for | What happened \[evidence\] | | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | A clearly-large explicit context request (several times over 30K tokens of requested window), with only a tiny single-sentence prompt inside it — repeated across a couple of distinct attempts so this is not one fluke line | No rejection, no error surfaced in the response to me. I got normal-looking short replies back within seconds each time \[measured here\]. In other words: *at least with only a small amount of actual content inside it, naming a large window on 24 GB did not immediately “nope, won’t fit.”* | | The same shape repeated once more as a separate verification pass | Identical result: accepted, short reply came back within seconds \[measured here\]. Two independent attempts agreeing = I would not call this “maybe it worked” - it did work to that extent on this hardware. | *Where my evidence genuinely stops being able to speak for you, stated as such rather than rounded into confidence:* - I kept one deliberately large real-content test in the session — a synthesized document sized so its tokenized length was comfortably into the four figures (around 16–17K tokens), not filler-padded to an artificial number. It completed, and prefill for that much content landed where section 4 says it did \[full response object retained\]. This is real “one big document in one pass” evidence at that scale. - I did run it again with the exact same input a moment later (still warm, no unload/reload in between): **this second identical request was dramatically faster on its front-end reading side**, consistent with the runtime reusing already-processed cache for the overlapping prefix rather than re-reading all of it from scratch — a plain fact you should be aware of if you test this yourself and are tempted to benchmark “suddenly fast” as “suddenly better model.” It isn’t. Warm-cache reads will look like that, on or off this machine \[measured here - the before/after pair I retained is exactly what I used above, and it shows the warm pass reading its prompt-side work in a fraction of the fresh pass’s time\]. - What I cannot responsibly claim from only those two data points: that every reasonably-sized real document you throw at it will complete cleanly if yours runs meaningfully longer than mine did, or that explicitly requesting an even larger window with proportionally *full* content inside (not a tiny probe) always succeeds. “Fits” and “actually completes under real load at the exact size your app uses” are two different bars, and my honest floor of proof here is: this card happily held a \~16-17K-token single prompt plus the context reservation above without a hard failure in any attempt. Beyond that specific floor, use it to judge your own expected content lengths before assuming one 24 GB card covers them all. ## 6\. Is it worth trying on *your* 24 GB setup? Plain terms, no hedged-legal-speak version | Your actual day-to-day shape | What this measured session supports saying about that fit (one RTX 5090-class laptop GPU on a Windows+WSL box, 4-bit GGUF file via Ollama v0.32.x) | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Daily chat plus light agent-style steps: live context comfortably in the low-to-mid thousands of tokens most of the day, occasional longer single passes | **Yes - this fits plainly from what I actually saw on real metal.** The file stayed fully GPU-resident every time I checked it, nothing offloaded to system RAM at any point during the whole session \[measured here\]. Response shape was interactive: short turns coming back in a few seconds (including one full retained object reconciling prompt-processing plus visible answer against end-to-end perception), longer single-document passes resolving in roughly fifteen to twenty seconds of real time, and prefill never being the thing that made me wait in any pass this session covered \[measured here\]. This is “usable as an everyday machine” on the evidence base I actually have, not a spec-sheet projection. | | Genuinely long single-pass work: one big document or codebase review at once, roughly in the range of my own longest retained test (≈16-17K token class) up to around there | **Supported by direct measured evidence on this card.** My longest cleanly-retained single pass (a \~80K-character source text → 17,460 tokens of prompt) finished with visible reasoning content plus a short answer in roughly ≈15–20 seconds total, prefill within that pass alone at the rate shown above \[full raw object retained, client wall time and its own duration fields agreeing\]. A second longish real-content pass I ran against with an explicitly-sized context request behaved consistently with this same “completes in a reasonable turn” shape. So: *if your longest normal single prompt lives around there or a bit under, this measured exactly that thing and it held up.* | | Genuinely large context windows kept warm and actively used at the same time - e.g. low-five-figure live tokens as an everyday operating state, not one-off big prompts | Treat my evidence here as “strong floor” rather than “ceiling.” Every explicitly-requested-large-window probe I ran with small real content inside succeeded without a hard failure \[measured here\], and there was no offload spillover to system RAM at any point on this card even when the reservation was asked for up through that scale. But “the runtime accepted the ask” is not the same bar as “routinely filling ten-plus times that with live content day in, day out doesn’t ever pause/throttle/evict on *your* prompts specifically.” I would call this a strong candidate card for exactly that use case - go measure your own real content length against a 24 GB budget (rough weights plus the window you actually want to fill) before committing it as *primary* housing for that workload, but nothing in anything I saw on real hardware makes even that seem implausible: no early eviction was visible to me at any point during this session. \[measured here + explicitly-labeled inference\] | | Strict speed/comparison-shopping against several other models, wanting a clean tokens-per-second number to paste into a spreadsheet | Honest answer: **I would not do that from this one report alone, on any model.** This is field-report data - real prompts, mixed context sizes, one evening’s worth of passes, client-side wall time plus whatever the runtime itself reported per pass. It deliberately does not present a tight, repeated, controlled benchmark as if my numbers were one \[limitation stated plainly rather than papered over with “≈some comfortable number.”\]. The experience-shape summary instead: interactive at chat scale, genuinely fast reading of prompts up to the sizes I tested (no prefill wait worth complaining about), and some longer-output turns where you do sit there for a bit - probably one. If “pastable benchmark number” is your actual decision criterion, spend an hour running it yourself on whichever models shortlisted; if it just needs to “feel obviously fine in daily use,” this report’s honest summary is: it did, in every pass I retained as evidence. | **The one-line version of “should *I* try it?”**: For everyday interactive work and genuinely-long-single-document use up to around a 16-17K-token-class pass, yes - fully onboard with no offload, interactively responsive as far as I measured that band \[measured here\]. For anything much larger live-in-context on a sustained daily basis: my data supports calling it a strong candidate rather than proving ceiling-scaled confidence, and “run your own real content sizes against it once” is the only responsible version of a recommendation I can give for that specific tier. ## Sources and how to audit every number above - **\[vendor fact\]** tags = values taken directly from this model family’s own published configuration documents (fetched during my session for cross-reference only) - used here specifically for the \~131K-class marketed context figure and a handful of architecture details that informed section 5’s framing. If you want to audit these, they come from the same model family's official config file, not from Ollama itself. - **\[measured here\] tags** \= everything else in this article came directly from my own measurement session against `qwen38-27b:latest` on one RTX 5090-class laptop GPU (WSL-side client, Windows-host Ollama v0.32.x): full raw response objects retained to disk for the specific passes cited by section number above, wall clock taken alongside each call rather than reconstructed afterward, and VRAM/power/temperature sampled with nvidia-smi at rest-and-active moments around key steps (queried through this machine’s Windows host via its own GPU API since the card sits on the WSL side). Where a specific pass could not be reconciled cleanly between its own reported duration fields and my wall clock, that number is simply not published above rather than forced into agreement. - **Cross-reference (one direction only):** our prior piece on this site - the “should you move to Qwen 3.8 at all” upgrade-decision guide - deliberately covers the quality/choice side in depth and is not re-covered here, since it would blur what this report is actually for: *what a real GPU does once that decision has been made somewhere*. If you have not read that piece first, do so before using either “is Qwen 3.8 better than what I run today?” answer as if this article alone settled it - it did not and is not trying to. - **Everything deliberately NOT tested in this report (kept honest rather than inferred):** no side-by-side quality comparison against any other specific model. No sustained-constant-load thermal run beyond what normal usage pacing over one session window produced. No vision/image-input evaluation on this build as I configured it here. Nothing under vLLM, SGLang, or comparable serving stacks - the entire measurement was Ollama-native in character.