Quick answer: Start with the model you want to run, check its VRAM requirement, then buy the cheapest GPU with enough VRAM and decent bandwidth. For most Ollama users, that means a 16GB card like the RTX 4060 Ti 16GB ($425) or RTX 4070 Ti Super ($800).
Check NVIDIA GeForce RTX 4060 Ti 16GB on Amazon→Buy on Shopee SG→Step 1: Identify your target model
The single most important question: what model do you want to run?
Ollama makes it easy to pull models, but each model has a fixed VRAM requirement. Start here:
| Model | Parameters | VRAM (Q4_K_M) | VRAM (Q8) |
|---|---|---|---|
| Llama 3 8B | 8B | ~5GB | ~8.5GB |
| Mistral 7B | 7B | ~4.5GB | ~7.5GB |
| Gemma 2 9B | 9B | ~5.5GB | ~9.5GB |
| Phi-3 Medium | 14B | ~8.5GB | ~14.5GB |
| Llama 2 13B | 13B | ~7.5GB | ~13GB |
| CodeLlama 34B | 34B | ~20GB | ~34GB |
| Qwen 2.5 32B | 32B | ~19GB | ~32GB |
| Llama 3 70B | 70B | 43GB | 75GB |
If you’re unsure, start with Llama 3 8B. It’s the most popular Ollama model, runs fast on modest hardware, and is genuinely useful for chat, coding, and writing tasks.
Step 2: Add VRAM headroom
Never buy a GPU that barely fits your model. You need headroom for:
- KV cache — grows with conversation length, but less than most guides say: at Ollama’s default 4K context a 7B needs about 0.5GB and a 14B about 0.75GB. Budget a gigabyte or two only once you raise the context past 16K
- System overhead — CUDA context uses 500MB-1GB
- Future models — the next popular model will probably be slightly larger
Rule of thumb: target 30-50% more VRAM than your model’s base Q4 size.
| Model target | Min VRAM | Recommended VRAM |
|---|---|---|
| 7B-8B models | 8GB | 12-16GB |
| 13B-14B models | 12GB | 16GB |
| 32B-34B models | 24GB | 24-32GB |
| 70B models | 48GB | 48GB+ (dual GPU) |
Step 3: Check bandwidth for speed
Once VRAM is sufficient, memory bandwidth determines your tokens-per-second speed. This is the number that makes Ollama feel fast or sluggish:
| GPU | VRAM | Bandwidth | Relative Speed |
|---|---|---|---|
| RTX 5090 | 32GB GDDR7 | 1,792 GB/s | Fastest |
| RTX 4090 | 24GB GDDR6X | 1,008 GB/s | Very fast |
| RTX 5080 | 16GB GDDR7 | 960 GB/s | Fast |
| RTX 3090 | 24GB GDDR6X | 936 GB/s | Fast |
| RTX 5070 Ti | 16GB GDDR7 | 896 GB/s | Fast |
| RTX 4070 Ti Super | 16GB GDDR6X | 672 GB/s | Good |
| RTX 3060 12GB | 12GB GDDR6 | 360 GB/s | Acceptable |
| RTX 4060 Ti 16GB | 16GB GDDR6 | 288 GB/s | Acceptable |
Notice the RTX 4060 Ti 16GB has low bandwidth but high VRAM. It fits more models but generates tokens slower. This is the classic trade-off for budget Ollama builds.
Step 4: Set your budget
| Budget | Best GPU | What it runs on Ollama |
|---|---|---|
| ~$230 | RTX 3060 12GB (used) | 7B fast, 13B slow |
| ~$425 | RTX 4060 Ti 16GB | 7B-13B comfortably |
| ~$800 | RTX 4070 Ti Super | 7B-13B fast |
| ~$850 | RTX 3090 (used) | Up to 34B quantized |
| ~$1,400 | RTX 5080 | 7B-13B very fast |
| ~$2,200 | RTX 4090 | Up to 34B comfortably |
| ~$4,900 | RTX 5090 | Up to 34B at high quality |
Step 5: Verify Ollama compatibility
Ollama works with any NVIDIA GPU that supports CUDA Compute Capability 5.0+. In practice, this means:
- All RTX 30-series — fully supported
- All RTX 40-series — fully supported
- All RTX 50-series — fully supported
- GTX 1060+ — supported but very limited VRAM
- AMD (ROCm) — supported on Linux, less reliable than NVIDIA
Check compatibility by running:
ollama list
nvidia-smiIf nvidia-smi shows your GPU and VRAM, Ollama will work.
Decision flowchart
- What model? Check VRAM needed from the table in Step 1
- Add 30-50% headroom to account for context and overhead
- Match to GPU using the budget table in Step 4
- Check bandwidth if you care about speed (Step 3)
- Buy the cheapest option that meets your VRAM and bandwidth needs
If you’re still unsure, the RTX 4060 Ti 16GB at $425 is the safest starting point. It runs every model up to 13B, works perfectly with Ollama out of the box, and won’t break the bank if you decide to upgrade later.
Check NVIDIA GeForce RTX 4060 Ti 16GB on Amazon→Buy on Shopee SG→ Check NVIDIA GeForce RTX 4090 on Amazon→Buy on Shopee SG→ Check NVIDIA GeForce RTX 3090 on Amazon→Buy on Shopee SG→Which GPU should you buy for Ollama?
If you just want to try Ollama with 7B models like Llama 3 8B and Mistral, a used RTX 3060 12GB at ~$230 gets you started without overspending. Plenty of VRAM for Q4 and Q8 quantization.
If you want a daily driver for 7B-13B models with fast response times, the RTX 4060 Ti 16GB ($425) or RTX 4070 Ti Super ($800) are the sweet spot. The 4070 Ti Super’s higher bandwidth makes a noticeable difference in token speed.
If you plan to run 34B models like CodeLlama 34B or Qwen 32B, you need 24GB minimum. The RTX 4090 ($2,200) or a used RTX 3090 ($820) are your only consumer options.
Common Ollama GPU mistakes
Buying 8GB in 2026 — Most interesting models need 10GB+ at good quantization. 8GB limits you to small 7B models at Q4, with no room for context.
Ignoring bandwidth — A model that fits but runs at 10 tok/s is frustrating. Check the bandwidth table before buying.
Buying AMD for simplicity — AMD can work with Ollama, but NVIDIA is still significantly easier to set up for local LLM work. If you are also deciding between Ollama and LM Studio as your inference interface, our LM Studio vs Ollama comparison breaks down which tool is better suited to your hardware and workflow. If you are also weighing whether to use Ollama, llama.cpp, or vLLM for your inference stack, see Ollama vs llama.cpp vs vLLM before finalizing your GPU choice.
For detailed VRAM planning, see our VRAM requirements guide. For a broader Ollama GPU overview, check Best GPU for Ollama. If you want to add a browser-based chat interface on top of Ollama, our best GPU for Open WebUI guide covers the setup and any added overhead. Considering LM Studio as a GUI alternative to the Ollama CLI? Our best GPU for LM Studio guide covers the right card for that interface, or for the oobabooga loader interface our best GPU for text-generation-webui guide has the equivalent picks.
The right GPU for Ollama is the one with enough VRAM for your target model and enough bandwidth to make it feel responsive. Everything else is secondary.