Quick answer: 8GB runs Gemma 4 E2B, 12GB runs E4B or the 12B dense, and you need 24GB for the 26B-A4B or the 31B dense. Those are Q4-class downloads as Ollama ships them, and the QAT builds cut roughly 2-3GB off the smaller variants. Full per-quantization figures are below.
Google released Gemma 4 in five sizes, and two of the names actively mislead on hardware. The E-variants are named for effective parameters and the 26B-A4B for active ones, so both sound smaller than they download. This guide gives the sizes Ollama actually ships for every variant and quantization, so you can match a model to your card without being surprised at the end of a 20GB download.
NVIDIA GeForce RTX 3090
24GB GDDR6X24GB used at ~$820 is the cheapest card that holds the 26B-A4B's 18GB of Q4 weights with room for context. A 16GB card does not.
Affiliate links — we may earn a commission at no extra cost to you. Amazon ships globally; Shopee SG covers Singapore & ASEAN.
The short version
| Variant | Q4_K_M VRAM | GPU you need |
|---|---|---|
| E2B (~2B effective) | ~7.2GB (~4.3GB QAT) | 8GB+ GPU, or 6GB on the QAT build |
| E4B (~4B effective) | ~9.6GB (~6.1GB QAT) | 12GB+ GPU, or 8GB on the QAT build |
| 12B Dense | ~7.6GB | 12GB GPU (RTX 3060 12GB) |
| 26B-A4B (MoE) | ~18GB (~16GB QAT) | 24GB+ GPU (RTX 3090, RTX 4090) |
| 31B Dense | ~20GB | 24GB+ GPU (RTX 4090, RTX 5090) |
Two of those numbers routinely surprise people. The E-variants are named for their effective parameter count, not their file size — E2B ships as a 7.2GB Q4 download, not a 1.5GB one, because the naming describes the memory-efficient architecture rather than the weights on disk. And the 26B-A4B MoE is not a 16GB model: it activates ~4B parameters per token but stores all 26B, and Ollama ships it at 18GB. Only the quantization-aware-training build squeezes into 16GB, and then with nothing left for context.
If you want the best Gemma 4 that fits a 16GB card, it is the 12B Dense at ~7.6GB, with room left over. Full hardware recommendations are in our best GPU for Gemma 4 buyer’s guide.
Detailed VRAM by quantization
The Model Size column is the published download from Ollama’s Gemma 4 library, not a figure derived from parameter count — check any row yourself before buying. Spot-verified against that page on 2026-09-12: the 26B-A4B at QAT 16GB, q4_K_M 18GB, MTP q4_K_M 19GB, q8_0 28GB and bf16 52GB; the 12B Dense at q4_K_M 7.6GB; the 31B Dense at q4_K_M 20GB and q8_0 34GB. The Total VRAM columns add our own KV-cache allowance on top and are estimates, not published numbers.
Gemma 4 E2B (~2B parameters)
| Quantization | Model Size | Total VRAM (with KV cache) |
|---|---|---|
| QAT (4-bit) | ~4.3GB | ~5.5GB |
| Q4_K_M | ~7.2GB | ~8.5GB |
| Q8_0 | ~8.1GB | ~9.5GB |
| BF16 | ~10GB | ~11.5GB |
The “E2B” name is the thing to watch here. It denotes an effective parameter count of about 2B, which describes how much of the model computes per token — the file itself is a 7.2GB download at Q4_K_M. So this is an 8GB-card model, not a 4GB one, and the QAT build at ~4.3GB is what you want on a 6GB card or integrated graphics.
Gemma 4 E4B (~4B parameters)
| Quantization | Model Size | Total VRAM (with KV cache) |
|---|---|---|
| QAT (4-bit) | ~6.1GB | ~7.5GB |
| Q4_K_M | ~9.6GB | ~11GB |
| Q8_0 | ~12GB | ~13.5GB |
| BF16 | ~16GB | ~17.5GB |
Same naming caveat as E2B, one size up: Q4_K_M is a 9.6GB download, so 12GB is the practical minimum and the ~6.1GB QAT build is the one for an 8GB card. Worth noting that the 12B Dense is smaller than the E4B at Q4 (~7.6GB against ~9.6GB) and generally stronger, which makes it the better default on a 12GB card.
Gemma 4 26B-A4B (MoE — the important one)
| Quantization | Model Size | Total VRAM (4K ctx) | Total VRAM (8K ctx) |
|---|---|---|---|
| QAT (4-bit) | ~16GB | ~18GB | ~20GB |
| Q4_K_M | ~18GB | ~20GB | ~22GB |
| MTP Q4_K_M | ~19GB | ~21GB | ~23GB |
| Q8_0 | ~28GB | ~30GB | ~32GB |
| BF16 | ~52GB | ~54GB | ~56GB |
This is where VRAM planning matters, and where the model’s name misleads. The
26B MoE has 26 billion total parameters that all live in VRAM, even though only
~4B activate per token — routing is decided at inference time, so no expert can
be left out. At Q4_K_M the weights alone are ~18GB, which is what
gemma4:26b-a4b-it-q4_K_M actually downloads.
On a 16GB card (RTX 4060 Ti 16GB, RTX 5070 Ti, RTX 5080): Q4_K_M does not fit. The QAT build is exactly 16GB, which means it loads and then has no room for KV cache. Treat this variant as out of reach and run the 12B Dense instead.
On a 24GB card (RTX 4090, RTX 3090): Q4_K_M runs with roughly 6GB of headroom — enough for 8K context comfortably. This is the variant’s natural home.
Check NVIDIA GeForce RTX 4090 on Amazon→Buy on Shopee SG→Gemma 4 31B Dense
| Quantization | Model Size | Total VRAM (4K ctx) | Total VRAM (8K ctx) |
|---|---|---|---|
| Q3_K_M | ~16GB | ~18.5GB | ~20GB |
| Q4_K_M | ~20GB | ~22GB | ~24GB |
| Q5_K_M | ~24GB | ~26GB | ~28GB |
| Q6_K | ~28GB | ~30GB | ~32GB |
| Q8_0 | 34GB | ~36GB | ~38GB |
The 31B Dense is straightforward but demanding. At Q4_K_M, you need at least 22GB with any meaningful context. The RTX 4090 (24GB) barely fits it — long conversations or large context windows may cause out-of-memory errors. The RTX 5090 (32GB) is the comfortable choice, fitting Q4 and even Q5 with room to spare.
VRAM capacity memory bandwidth Specs are manufacturer figures. Bar lengths are scaled independently per metric.
KV cache: the hidden VRAM eater
Every table above includes estimated KV cache overhead, but actual usage depends on your conversation length. A rough guide:
- 2K context: +1-2GB over model weights
- 4K context: +2-3GB
- 8K context: +3-5GB
- 16K context: +5-8GB
For the 26B MoE this is what turns a 24GB card from “just fits” into “comfortable”: 18GB of weights plus 3-5GB of cache for a normal conversation lands around 21-23GB. Use nvtop or nvidia-smi to watch it during inference and shorten context if usage approaches 95%.
Which quantization should you use?
For Gemma 4 specifically:
- Q4_K_M is the standard recommendation. Minimal quality loss, good VRAM efficiency.
- QAT is the interesting one on this family — Google ships quantization-aware-trained 4-bit builds that are smaller than Q4_K_M at comparable quality. On a card that is one step too small, try QAT before you give up on a variant.
- Q8_0 is practical on the E-variants and the 12B Dense; on the 26B MoE it is 28GB, which is 32GB-card territory.
- Note that Gemma 4 has no Q3_K_M, Q5_K_M or Q6_K builds on Ollama. If a guide quotes you one, it is extrapolating rather than reporting.
For a broader guide to quantization trade-offs across all models, see best quantization for local LLM.
GPU recommendations by variant
| Variant | Budget pick | Best pick |
|---|---|---|
| E2B / E4B | RTX 3060 12GB used (~$250) | Any 12GB card; take the QAT builds on 8GB |
| 12B Dense | RTX 3060 12GB used (~$250) | RTX 4060 Ti 16GB (~$425) |
| 26B-A4B MoE | RTX 3090 used (~$820) | RTX 4090 (~$2,200) |
| 31B Dense | RTX 3090 used (~$820) | RTX 4090 (~$2,200) |
For full GPU benchmarks and buying recommendations, head to our best GPU for Gemma 4 guide, or our broader best GPU for Gemma overview spanning the 2B/7B/27B classics. Need general VRAM guidance across all model families? See how much VRAM for local LLM. And if you are running models through Ollama, our best GPU for Ollama article covers setup specifics. Budget shoppers should check best budget GPU for local LLM for affordable options.