LLM VRAM Calculator
Estimate exactly how much GPU VRAM you need to run any LLM locally. Covers quantization levels, context length, KV cache overhead, and runtime memory. Then maps the result to real GPUs from our comparison data.
Figures reviewed September 15, 2026.
Configure your workload
Parameter count. For MoE models, we use the total parameter count (VRAM must hold all weights).
Lower quantization = smaller VRAM footprint but quality tradeoff. Q4_K_M is the most common default.
KV cache grows linearly with context. Doubling context roughly doubles the cache size.
Q8 KV cache halves memory with minor quality impact. Supported in llama.cpp and newer Ollama.
CUDA context + framework overhead. Ollama and llama.cpp are leanest.
Total VRAM needed
16.4 GB
Recommended VRAM tier
16GB card
A 16GB GPU will handle this workload with ~4GB headroom for longer context or batching.
GPUs that fit this workload
How the math works
The total VRAM needed for running an LLM is the sum of three components: model weights, KV cache, and runtime overhead. Each scales differently based on your configuration.
1. Model weights
Weight memory scales with parameter count and quantization bits per parameter. For a 7B model at Q4_K_M (~4.5 bits/param), the weights occupy roughly 4 GB. At FP16 the same model takes ~14 GB. See our how much VRAM for local LLM guide for a full breakdown from 7B to 70B.
weights_GB = (params_in_billions × bits_per_param) / 8
2. KV cache
Every token of context adds to the KV cache, and its width is set by the model's key/value heads — not by its hidden size, which is the assumption that makes most VRAM estimates wrong. Grouped-query attention shares one set of keys and values across several query heads, so Llama 3 70B holds 8 KV heads of 128 dimensions against 64 query heads: 0.31 MB per token at FP16, about 10 GB at 32K context. Multi-head models from before GQA are far heavier per parameter — Llama 2 13B keeps 40 KV heads and costs 0.78 MB per token, two and a half times a 70B.
kv_GB = 2 × num_layers × (kv_heads × head_dim) × context × precision_bytes / 1024³
3. Runtime overhead
CUDA context, framework buffers, activation memory during inference. Ollama and llama.cpp are leanest (~1-1.5 GB). vLLM with heavy batching can consume 2-3 GB more. Text Generation WebUI with extensions loaded is heaviest.
Why "recommended" differs from "fits"
A workload that needs 15.8 GB will technically fit in a 16GB card, but there's no headroom. Any extra context, a second user, or a driver update can push it over and cause OOM errors. We recommend the next VRAM tier up when headroom is under 2 GB. This is why our Ollama GPU guide and used RTX 3090 guide often point readers toward 24 GB cards once models push past 16 GB.
This calculator gives representative estimates based on typical Transformer architectures. Actual VRAM usage varies by model family (Llama, Mistral, Qwen have slightly different layer counts and hidden sizes), driver, framework, and batch size. Treat results as ±10% accurate. See our methodology for how we derive these numbers.