Quick answer: Ollama automatically selects quantization based on your available VRAM. For 7B models, you need at least 8GB VRAM. For 13B models, 12-16GB. For 70B models, 48GB+ or accept heavy CPU offloading.
Check NVIDIA GeForce RTX 4060 Ti 16GB on Amazon→Buy on Shopee SG→Who this is for
You are running models locally through Ollama, or deciding which GPU to buy for it, and you want to know what actually fits rather than what a parameter count implies. Everything below is the published download size plus the cache it needs at ordinary context lengths — not an estimate derived from the model’s name, which stopped being reliable in 2026.
How Ollama uses VRAM
When you run ollama run llama3, Ollama loads the model weights into GPU memory. If the model does not fit entirely, Ollama offloads remaining layers to system RAM, which dramatically slows inference.
Key facts about Ollama’s VRAM usage in 2026:
- Ollama uses GGUF quantized models by default (Q4_K_M for most)
- The default
ollama pulldownloads a Q4_K_M variant unless you specify otherwise - KV cache for context uses additional VRAM beyond the model weights
- Running
ollama runwith a model already loaded reuses the same VRAM allocation
For a size not in these tables, the VRAM calculator works the same arithmetic — parameters, quantization, context length and the KV cache that comes with it.
VRAM capacity memory bandwidth Specs are manufacturer figures. Bar lengths are scaled independently per metric.
Want the short answer for your setup? The quick checker below covers the common combos:
What can you run on the GPU you already own? VRAM to model, 4GB to 80GB
Most of this page answers “how much VRAM does this model need”. If you already have a card — 4GB, 8GB, 12GB, 24GB or a pair of them — and want the question the other way round, start here. Everything below assumes Q4_K_M, Ollama’s default, with a few GB left for KV cache. Sizes are the published downloads, so a row telling you something does not fit means the file itself is larger than the card, not that it might be slow.
| Your VRAM | Runs comfortably | Tightest thing that still fits |
|---|---|---|
| 4GB | 1B-4B only — Qwen 3 4B (2.6GB), Phi-3 Mini (2.4GB), Llama 3.2 1B | A 7B at Q4 is 4.1-5.5GB, so it loads and then has nothing left for context |
| 6GB | The smaller 7Bs — CodeLlama 7B (4.1GB), Mistral 7B (4.4GB) | Llama 3.1 8B (4.9GB), with about a gigabyte for cache |
| 8GB | 7B-9B models — Llama 3.1 8B (4.9GB), Mistral 7B (4.4GB), Qwen 3 4B (2.6GB) | A 13B at Q4 (~8.5GB), with almost no room for context |
| 12GB | Everything above, plus 13B-14B — Qwen 3 14B (~10GB), Phi-3 Medium 14B (~9GB) | A 14B at Q8 (~15GB) will not fit; stay at Q4-Q5 |
| 16GB | 14B at Q8 (~16GB), Gemma 4 12B (~7.6GB), Qwen 3 14B with long context | Gemma 4’s 26B-A4B in its QAT build (~16GB), leaving nothing for cache |
| 24GB | 32B-34B at Q4 — Qwen 3 32B (~20GB), CodeLlama 34B (~20GB), Gemma 4 31B Dense (~20GB), Gemma 4 26B-A4B (~18GB) | Qwen 3.6 35B-A3B (~24GB), with no room for context |
| 32GB | Qwen 3.6 35B-A3B (~24GB) with real context, or a 34B at Q6 | A 70B at q3_K_S (31GB) — q3_K_M is 34GB and overflows |
| 48GB (2x 24GB) | 70B at Q4_K_M (43GB), Qwen 3 235B is still far out of reach | Qwen 2.5 72B (47GB), which leaves ~1GB for cache |
| 64GB | 70B at q5_K_M (50GB) with room, or q6_K (58GB) tight | Mixtral 8x22B (86GB) is still well beyond it |
| 80GB+ | 70B at q8_0 (75GB), Llama 4 Scout at Q4 (67GB) | Llama 4 Maverick (245GB) needs four such cards |
Two patterns are worth internalising. First, the 2026 MoE models do not land where their marketing names suggest — Qwen 3.6’s 35B-A3B ships as a 24GB file and Gemma 4’s 26B-A4B as an 18GB one, so both are 24GB-card models despite activating only 3-4B parameters per token. Second, every tier has a “tightest fit” that technically loads and then disappoints — a 70B at q3_K_S on 32GB is measurably worse than a 34B at Q6 on the same card. Fit is necessary, not sufficient.
VRAM requirements by model
Small models (1B-3B parameters)
| Model | Default Quant | VRAM Used | Min GPU |
|---|---|---|---|
| Llama 3.2 1B | Q8_0 | ~1.5GB | Any 4GB GPU |
| Llama 3.2 3B | Q4_K_M | ~2.5GB | Any 4GB GPU |
| Phi-3.5 Mini (3.8B) | Q4_K_M | ~3GB | Any 4GB GPU |
| Gemma 2 2B | Q4_K_M | ~2GB | Any 4GB GPU |
| Qwen 2.5 3B | Q4_K_M | ~2.5GB | Any 4GB GPU |
These models run on virtually any modern GPU. Even a GTX 1650 with 4GB handles them fine.
Medium models (7B-9B parameters)
| Model | Default Quant | VRAM Used | Min GPU | Recommended GPU |
|---|---|---|---|---|
| Llama 3.1 8B | Q4_K_M | 4.9GB | 8GB | RTX 4060 Ti 16GB |
| Mistral 7B v0.3 | Q4_K_M | 4.4GB | 8GB | RTX 3060 12GB |
| Gemma 2 9B | Q4_K_M | ~6GB | 8GB | RTX 3060 12GB |
| Qwen 2.5 7B | Q4_K_M | ~5GB | 8GB | RTX 3060 12GB |
| DeepSeek-R1 8B | Q4_K_M | ~5.5GB | 8GB | RTX 4060 Ti 16GB |
| Llama 3.1 8B | Q8_0 | ~9GB | 12GB | RTX 4060 Ti 16GB |
| Llama 3.1 8B | FP16 | ~16GB | 16GB | RTX 4060 Ti 16GB |
At Q4_K_M, all 7B-9B models fit on 8GB cards. However, 8GB leaves almost no room for context. A 12-16GB card gives much better real-world performance. For a model-specific deep dive, see how much VRAM does Llama 3 8B need?
Large models (13B-14B parameters)
| Model | Default Quant | VRAM Used | Min GPU | Recommended GPU |
|---|---|---|---|---|
| Llama 2 13B | Q4_K_M | 7.9GB | 12GB | RTX 4060 Ti 16GB |
| CodeLlama 13B | Q4_K_M | 7.9GB | 12GB | RTX 4060 Ti 16GB |
| Phi-3 Medium 14B | Q4_K_M | ~9GB | 12GB | RTX 4060 Ti 16GB |
| Qwen 2.5 14B | Q4_K_M | ~9GB | 12GB | RTX 4060 Ti 16GB |
| Llama 2 13B | Q8_0 | ~14.5GB | 16GB | RTX 5070 Ti |
The 16GB sweet spot: an RTX 4060 Ti 16GB or RTX 5070 Ti handles any 13B-14B model at Q4-Q8 with room for context.
XL models (30B-34B parameters)
| Model | Default Quant | VRAM Used | Min GPU | Recommended GPU |
|---|---|---|---|---|
| CodeLlama 34B | Q4_K_M | ~20GB | 24GB | RTX 3090 (used) |
| Yi 34B | Q4_K_M | ~20GB | 24GB | RTX 3090 (used) |
| Qwen 2.5 32B | Q4_K_M | ~19GB | 24GB | RTX 3090 (used) |
| DeepSeek-R1 32B | Q4_K_M | ~19GB | 24GB | RTX 3090 (used) |
| CodeLlama 34B | Q3_K_M | ~16GB | 24GB | RTX 3090 (used) |
24GB is the minimum for 34B models. A used RTX 3090 is the cheapest way there; an RTX 4090 has the same capacity and more speed for roughly two and a half times the price.
XXL models (70B+ parameters)
| Model | Quant | Download size | Min GPU | Recommended GPU |
|---|---|---|---|---|
| Llama 3.1 70B | q4_K_M | 43GB | 48GB | 2x RTX 3090 (used) |
| DeepSeek-R1 70B | q4_K_M | 43GB | 48GB | 2x RTX 3090 (used) |
| Qwen 2.5 72B | q4_K_M | 47GB | 48GB is not enough in practice | 2x RTX 5090, or offload |
| Llama 3.1 70B | q3_K_M | 34GB | 2x 24GB — does not fit one 32GB card | 2x RTX 3090 (used) |
| Llama 3.1 70B | q3_K_S | 31GB | 32GB | RTX 5090 (tight) |
| Llama 3.1 70B | q2_K | 26GB | 32GB | RTX 5090 |
Sizes are the published downloads for llama3.1, deepseek-r1 and qwen2.5 — check them yourself before buying.
Qwen 2.5 72B is the row to read twice. It is 47GB, not the 43GB the Llama-family 70Bs download as, so a 48GB build has about 1GB left over — which the KV cache eats immediately. Two used 3090s run Llama 3.1 70B comfortably and Qwen 2.5 72B barely or not at all. Do not assume the 70B-class models are interchangeable on size.
70B models do not fit on any single consumer GPU at good quantization levels. An RTX 5090 reaches q3_K_S (31GB) or q2_K (26GB) — not q3_K_M, which is 34GB against 32GB of VRAM — and quality suffers at both. For serious 70B usage, plan for dual GPUs or cloud.
Why the 2026 models break the parameter-count rule
Every table above maps parameter count to VRAM, which held reliably through the Llama 2 and Llama 3 generations. The 2026 releases broke it. Mixture-of-Experts models route each token through a small subset of their weights, so a model advertised at 35B parameters can be lighter on VRAM than a dense 20B — but only on compute. All expert weights still load into VRAM, because routing happens at inference time and Ollama cannot know in advance which experts a token will need. Judge these models by their file size, not their name.
| Model | Total params | Active per token | VRAM at Q4_K_M | Fits on |
|---|---|---|---|---|
| Gemma 4 E2B | ~2B effective | dense | ~7.2GB (~4.3GB QAT) | 8GB, or 6GB on QAT |
| Gemma 4 E4B | ~4B effective | dense | ~9.6GB (~6.1GB QAT) | 12GB, or 8GB on QAT |
| Qwen 3 14B | 14B | dense | ~9.3GB | RTX 3060 12GB |
| Qwen 3 30B-A3B | 30B | ~3B | ~19GB | RTX 3090 (used) |
| Qwen 3 32B | 32B | dense | ~20GB | RTX 3090 (used) |
| Gemma 4 31B Dense | 31B | dense | ~20GB | RTX 3090 (used) |
| Gemma 4 26B-A4B | 26B | ~4B | ~18GB (~16GB QAT) | RTX 3090 (used) |
| Qwen 3.6 35B-A3B | 35B | ~3B | ~24GB | RTX 3090 (used), no context room |
| Llama 4 Scout | 109B | 17B | ~67GB | Multi-GPU or cloud |
| Llama 4 Maverick | 400B | 17B | ~245GB | Cloud only |
Every figure in that table is the size Ollama actually ships, taken from the
per-tag listings on the Ollama model library —
not an estimate derived from the parameter count in the model’s name. It is
worth checking a model’s tag page before you buy hardware for it, because the
gap runs the wrong way: Llama 4 Scout activates 17B parameters per token and
sounds like a 24GB model, but llama4:17b-scout-16e-instruct-q4_K_M is a 67GB
download and every byte of it has to be resident.
The practical consequence for 2026 is the opposite of what the naming implies. MoE buys you speed, not capacity: Qwen 3.6 35B-A3B runs at roughly the pace of a 3B model while occupying the memory of a 35B one. A 16GB card is still a 14B-class card; the MoE tier starts at 24GB.
For the full quantization ladder on each, see our deep dives on Llama 4 VRAM requirements, Qwen 3 VRAM requirements, Gemma 4 VRAM requirements, and the best GPU for Qwen 3.6.
GPU recommendation summary
| Your Target | Best GPU | Price | Why |
|---|---|---|---|
| 7B models | RTX 3060 12GB | ~$250 used | Cheap, 12GB is plenty |
| 7B-13B models | RTX 4060 Ti 16GB | ~$425 | 16GB handles everything up to 14B |
| 13B-34B models | RTX 3090 (used) | ~$820 | 24GB for 34B at Q4, cheapest route to it |
| 34B, faster | RTX 4090 | ~$2,200 | Same 24GB, roughly a third quicker |
| 34B with long context | RTX 5090 | ~$4,900 | The only 32GB consumer card |
| 70B models | 2x RTX 3090 (used) | ~$1,640 | 48GB combined, Q4_K_M without offloading |
Which GPU should you buy for Ollama?
The short version is below; for the decision walked through properly — VRAM first, then bandwidth, then budget — see how to choose a GPU for Ollama.
If you run small models (1B-3B) for lightweight tasks, any 4GB+ GPU works. No need to upgrade.
If you run 7B-13B models for chat, coding, or writing, a 16GB card is the sweet spot. The RTX 4060 Ti 16GB ($425) handles every model in this range at Q4-Q8 with room for context. Upgrade to the RTX 4070 Ti Super ($800) if you want faster token generation.
If you run 34B models like CodeLlama 34B or DeepSeek-R1 32B, you need 24GB, and in 2026 the cheapest way there is a used RTX 3090 (~$820). The RTX 4090 has the same 24GB and generates roughly a third faster, but it now sells near $2,200 after production ended and GDDR7 supply moved to AI accelerators. Both load the same models; you are paying for speed, not capability.
If you want 70B models, no single consumer GPU holds one at good quantization. Two used RTX 3090s reach 48GB for around $1,640 and run 70B at Q4_K_M without offloading to system RAM, which is the configuration that actually works at this budget. Cloud rental stays the alternative if you only need it occasionally.
Common mistakes with Ollama VRAM
Not accounting for KV cache — Your model fits in VRAM, but crashes mid-conversation. The KV cache for context grows as you chat. Always leave 2-4GB of headroom beyond the model’s base size.
Running multiple models simultaneously — Ollama keeps models loaded in VRAM by default. If you pull and run a second model without stopping the first, both compete for VRAM. Use ollama stop to unload unused models.
Choosing Q2_K to squeeze a larger model — Dropping to Q2_K quantization to fit a 70B model on 32GB sounds clever, but the quality loss is severe. You are better off running a 34B model at Q6_K than a 70B at Q2_K.
Ignoring CPU offloading speed — Ollama silently offloads layers to RAM when VRAM runs out. The model “works” but runs 5-10x slower on offloaded layers. Run ollama ps and confirm the PROCESSOR column reads 100% GPU.
How do you check whether a model actually fits?
The number that matters is not what ollama pull downloaded, it is how much of
the model ended up on the GPU. ollama ps answers that directly — the PROCESSOR
column reports the split:
$ ollama ps
NAME ID SIZE PROCESSOR UNTIL
qwen3:14b a1b2c3d4e5f6 11 GB 100% GPU 4 minutes from now
llama3.1:70b f6e5d4c3b2a1 44 GB 38%/62% CPU/GPU 4 minutes from nowAnything other than 100% GPU means layers are running on the CPU, and that is
where the 5-10x slowdown comes from. The SIZE column already includes KV cache,
so it will read higher than the model’s download size.
For the GPU’s own view, including memory used by your desktop:
nvidia-smi --query-gpu=memory.used,memory.total --format=csvThree things reclaim VRAM when a model does not fit:
# 1. Unload a model you are done with (Ollama keeps them resident by default)
ollama stop llama3.1:70b
# 2. Shrink the context window — KV cache scales linearly with it
ollama run qwen3:14b
>>> /set parameter num_ctx 4096
# 3. Quantize the KV cache itself, which needs flash attention enabled
OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serveThe third option is the one people miss. On long-context work the KV cache can
rival the model weights, and q8_0 roughly halves it for very little quality
cost. Ollama documents the full set of environment variables in its
FAQ, and the
GPU support docs list which cards and drivers
are recognised.
For a deeper dive on VRAM planning, see our VRAM requirements guide. For GPU-specific Ollama performance, check our best GPU for Ollama article. If you have outgrown Ollama and are moving to a multi-user serving stack, our best GPU for vLLM guide covers the additional VRAM headroom PagedAttention requires.
When in doubt, buy more VRAM than you think you need. Models are growing faster than GPU memory, and Ollama makes it too easy to try the next size up.
Frequently asked questions
How much VRAM does Ollama need?
It depends entirely on the model, not on Ollama itself. An 8GB card comfortably runs 7B and 8B models at Q4. A 12GB card reaches the 13B-14B class. Getting into the 30B class wants 24GB, and 70B models want 48GB or more before you are running them without heavy CPU offloading.
What happens if a model does not fit in VRAM?
Ollama does not refuse to run it. It offloads the layers that do not fit to system RAM and runs them on the CPU, which works but is dramatically slower — often by an order of magnitude, because system memory bandwidth is a fraction of a GPU’s. A model that half fits will feel broken rather than merely slow.
How do I tell whether a model will fit before downloading it?
Compare the download size shown on the model’s Ollama page against your free VRAM, then leave one to two gigabytes on top for the KV cache at ordinary context lengths. If the weights alone are close to your card’s capacity, the answer is no — the cache and the desktop both need room.
Why do some newer models download smaller than older, smaller ones?
Because parameter count stopped predicting file size in 2026. Mixture-of-experts models and quantization-aware-training builds both break the old rule, so a model advertised with a larger parameter count can ship a smaller download than one with fewer. Read the published size rather than inferring it from the name.
Can I cut VRAM use without switching to a smaller model?
Yes, three ways, in rough order of payoff. Shorten the context window, since the KV cache grows roughly linearly with it. Quantize the KV cache itself with flash attention enabled, which roughly halves it for little quality cost. And unload models you have finished with, because Ollama keeps them resident by default.