The “A3B” in Qwen 3.6 35B-A3B means three billion parameters activate per token, and that is what makes it fast — it generates at roughly the pace of a 3B model while answering like a 35B one. It is also the reason this model gets bought for the wrong hardware. Activation is a compute story, not a memory one. Routing is decided token by token at inference time, so all thirty-five billion parameters have to be resident, and qwen3.6:35b-a3b-q4_K_M is a 24GB download. This is a 24GB-card model. No 16GB card runs it, and neither does the 17GB dense 27B sibling.
NVIDIA GeForce RTX 3090
24GB GDDR6X24GB used at ~$820 is the least you can spend and still load Qwen 3.6 at Q4. Tight — plan on short context or the 23GB MTP build.
Affiliate links — we may earn a commission at no extra cost to you. Amazon ships globally; Shopee SG covers Singapore & ASEAN.
Qwen 3.6 at a glance
| Spec | Value |
|---|---|
| Total parameters | 35B |
| Active parameters | ~3B per token |
| Architecture | Mixture of Experts (MoE) |
| Default context | 262K tokens |
| Recommended context | 128K for consumer GPUs |
| VRAM at Q4_K_M | ~24GB (~23GB MTP build) |
| VRAM at Q8_0 | ~39GB |
| Ollama support | Yes (native) |
For exact VRAM numbers when running Qwen 3.6 through Ollama (including KV cache overhead), see our Ollama VRAM Requirements guide.
The 262K default context is impressive but aggressive for consumer hardware. On a 24GB card the weights alone consume the card, so context has to stay short; full 262K is a 32GB-plus proposition and the KV cache for it can balloon past 20GB on its own.
VRAM requirements
These are the builds Qwen 3.6 actually ships as, with their sizes taken from the Ollama tag listing rather than estimated from the parameter count:
| Build | Model weights | + KV cache (8K) | + KV cache (128K) |
|---|---|---|---|
| 27B dense, Q4_K_M | ~17GB | ~19GB | ~23GB |
| 35B-A3B, Q4_K_M (MTP) | ~23GB | ~25GB | ~29GB |
| 35B-A3B, Q4_K_M | ~24GB | ~26GB | ~30GB |
| 27B dense, Q8_0 | ~30GB | ~32GB | ~36GB |
| 35B-A3B, Q8_0 | ~39GB | ~41GB | ~45GB |
The planning decision is simpler than it looks, and it is not the one most guides describe. 24GB is the floor, not the comfortable option. A 3090 or 4090 loads the Q4 weights and has almost nothing left, so you run short context or you run the MTP build for the gigabyte it saves. Long context on this model means 32GB. And if 24GB is your ceiling but you want room to work, the dense 27B at ~17GB is the more sensible model to run on it.
VRAM capacity memory bandwidth Specs are manufacturer figures. Bar lengths are scaled independently per metric.
Performance benchmarks
Ollama at Q4_K_M with a 128K context window. Throughput is bandwidth-derived rather than benchmarked in a lab (methodology):
| GPU | tok/s | VRAM used | Fits? | Price |
|---|---|---|---|---|
| RTX 5090 (32GB) | ~120 tok/s | ~24GB | Yes, with context headroom | ~$4,900 |
| RTX 4090 (24GB) | ~85 tok/s | ~24GB | Only just — short context | ~$2,200 |
| RTX 3090 (24GB, used) | ~60 tok/s | ~24GB | Only just — short context | ~$820 |
| RTX 5080 (16GB) | — | — | No | ~$1,400 |
| RTX 5070 Ti (16GB) | — | — | No | ~$1,050 |
| RTX 4070 Ti Super (16GB) | — | — | No | ~$800 |
| RTX 4060 Ti 16GB | — | — | No | ~$425 |
| RTX 3060 12GB (used) | — | — | No | ~$250 |
The 16GB cards are not marked “tight” — they are marked “no”. A 24GB file does not load into 16GB at any context length, and there is no smaller official build to fall back on. The next section covers the one way around that.
Throughput is high relative to the model’s size for the reason the name advertises: only ~3B parameters are read per token, so a 35B MoE generates far faster than a dense 35B on the same card. Speed was never the problem with this model. Capacity is.
Check NVIDIA GeForce RTX 4090 on Amazon→Buy on Shopee SG→MoE offloading: the 12GB GPU trick
Qwen 3.6 supports --n-cpu-moe in llama.cpp, which offloads inactive expert layers to system RAM. This is the only way to run it below 24GB: the active ~3B parameters stay on the GPU while dormant experts live in CPU memory, so a 16GB or even 12GB card can load a model whose weights are nearly twice its capacity.
It also matters more than it used to, because the alternative is not “run it slower” — it is “do not run it at all”. Budget 32GB of system RAM on top of whatever the card has, and expect a significant drop (roughly 15-25 tok/s on an RTX 3060 12GB with DDR5). Good for evaluating whether the model is worth buying hardware for; not something to build a daily workflow on.
Which GPU should you buy?
Cheapest card that fits: RTX 3090 24GB used (~$820). The Q4 weights are 24GB and the card is 24GB, so this works with short context and no room to spare. Take the MTP build for the gigabyte it gives back. Still the best price-per-VRAM on the market.
Best for long context: RTX 5090 (~$4,900). 32GB is the first capacity where 128K context and the Q4 weights coexist without compromise. Expensive, and the only clean single-card answer.
If you already own a 24GB card: run the dense 27B at ~17GB instead. You give up some quality and gain seven gigabytes of working room, which on this model is the difference between usable and constantly out of memory.
16GB cards (5070 Ti, 5080, 4060 Ti 16GB): cannot load either variant. Your options are --n-cpu-moe offloading with the performance cost that carries, or a different model — Qwen 3 14B at ~9.3GB is the sensible 16GB pick.
12GB GPUs: offloading only, and slow enough that it is for evaluation rather than use.
Common mistakes
- Buying a 16GB card because “A3B” sounds small. The activation count describes compute, not memory. Ollama ships this model as a 24GB file and every byte of it stays resident. This is the most expensive mistake on this page.
- Comparing Qwen 3.6 to dense 35B models. The 3B active parameters mean it runs 3-4x faster than a dense 35B. Do not use dense model benchmarks to estimate Qwen 3.6 performance.
- Assuming the newer Qwen has the same hardware needs. Qwen 3.8 is dense rather than MoE and its default build is an 18GB download, so it wants 24GB where this one is comfortable on less — see best GPU for Qwen 3.8 before upgrading on version number alone.
- Skipping Q4_K_M for Q8. Q8 pushes the model to ~39GB — past every consumer card including the 5090. The quality difference between Q4 and Q8 on MoE models is smaller than on dense models because the active weights are a small fraction of total parameters, so this is a bad trade twice over.
Final verdict
| Your goal | Best GPU | Price |
|---|---|---|
| Cheapest card that loads it | RTX 3090 24GB (used) | ~$820 |
| Long context (128K) without compromise | RTX 5090 32GB | ~$4,900 |
| Already on 24GB, want working room | run the dense 27B instead | — |
| 16GB card, unwilling to offload | pick Qwen 3 14B instead | — |
NVIDIA GeForce RTX 5090
32GB GDDR732GB is the first capacity where Qwen 3.6 Q4 and a long context window fit together. The only single-card answer without compromise.
Affiliate links — we may earn a commission at no extra cost to you. Amazon ships globally; Shopee SG covers Singapore & ASEAN.
For the previous generation, see our best GPU for Qwen 3 guide. Running models through Ollama? The best GPU for Ollama article covers multi-model setups. For VRAM planning across all model sizes, how much VRAM for local LLM has the full reference. And if you want to understand quantization trade-offs in depth, read best quantization for local LLM.