Quick answer: VRAM decides which models you can run at all, so pick that first and speed second. The RTX 4090 is the fastest card most people will consider, but a used RTX 3090 carries the same 24GB and reaches about 88% of its 13B throughput (full comparison) for roughly a third of the price — which makes the real question whether you want a warranty, not whether you want performance. If you also want to transcribe audio locally alongside your LLM stack, our local Whisper GPU guide covers what VRAM Whisper adds on top.
NVIDIA GeForce RTX 4090
24GB GDDR6X24GB VRAM handles everything from 7B to 34B models, with the highest throughput of any card most people will consider — and a warranty the used market cannot offer.
Affiliate links — we may earn a commission at no extra cost to you. Amazon ships globally; Shopee SG covers Singapore & ASEAN.
What matters most for Ollama
- VRAM for fitting your chosen model — our Ollama VRAM Requirements guide lists exact numbers per model and quant
- Memory bandwidth for faster inference
- Budget and availability
- Power and thermals for long-running sessions
Check your card before reading on:
Best GPUs for Ollama
| GPU | VRAM | Best For | Speed (13B Q4) | Price |
|---|---|---|---|---|
| RTX 5090 | 32GB | 34B+ models, maximum speed | ~55 tok/s | ~$4,900 |
| RTX 4090 | 24GB | Fastest 24GB card | ~40 tok/s | ~$2,200 |
| RTX 3090 (used) | 24GB | Same 24GB, 88% of the speed, a third of the price | ~35 tok/s | ~$820 |
| RTX 4070 Ti Super | 16GB | Best new 16GB card | ~24 tok/s | ~$800 |
| RTX 4060 Ti 16GB | 16GB | Cheapest usable 16GB | ~18 tok/s | ~$425 |
Note the third row. A used RTX 3090 runs a 13B model at roughly 88% of the RTX 4090’s speed, on the same 24GB of VRAM, for a little over a third of the price. It draws more power and carries no warranty, and those are the reasons to pay the difference — not throughput, and not capability.
For a detailed Ollama performance comparison between the 4090 and 3090, see RTX 4090 vs 3090 for Ollama. For the full generation leap from the used 3090 to the current flagship, see RTX 5090 vs 3090 for LLM.
Check RTX 4090 Price→Buy on Shopee SG→ Check RTX 4070 Ti Super Price→Buy on Shopee SG→How to choose
If your target is larger Llama-family models, prioritize VRAM first. If you mostly run smaller quantized models, value and power efficiency may matter more than flagship performance. For multi-step agentic workloads — where models plan, call tools, and loop autonomously — see our best GPU for AI agents guide for the additional VRAM considerations involved.
Which GPU should YOU buy for Ollama?
- Running 7B models (Llama 3 8B, Mistral 7B)? Get the RTX 4060 Ti 16GB (~$425). Plenty of VRAM and fast enough for interactive chat. Using it with a coding assistant like Continue.dev? Our Continue.dev GPU guide covers the exact latency targets you need, and for the broader workflow our local coding LLM GPU guide ties model choice and editor integration together.
- Running 13B models (CodeLlama 13B, Qwen 14B)? Get the RTX 4070 Ti Super (~$800), or a used RTX 3090 (~$820) if you want 24GB at the same money. Running Google’s Gemma family? Our best GPU for Gemma guide covers the 2B/7B/27B lineup, with separate Gemma 3 and Gemma 4 deep-dives for the latest releases.
- Running 34B+ models (Qwen 32B, Llama 70B)? Get the RTX 4090 minimum for 34B; RTX 5090 or dual GPUs for 70B. Weighing whether the RTX 5070 is a viable cheaper alternative to the 4090? See RTX 5070 vs 4090 for LLM for a VRAM and speed comparison. Running the latest Qwen 3.6? See our Qwen 3.6 GPU guide for updated VRAM numbers.
- Running Mistral 7B or Mistral variants? See our best GPU for Mistral guide for model-specific VRAM and speed numbers.
- Pairing Ollama with a retrieval pipeline? Our best GPU for RAG guide covers the extra VRAM the embedding model and long context window need on top of base inference.
- Only need occasional access to large models? Try cloud GPUs — cheaper than buying flagship hardware for occasional use.
- Considering a Mac Mini instead of a discrete GPU? See our can the Mac Mini run LLMs guide for a realistic assessment of what the current M6 and M5 Pro chips handle well, and our Mac vs NVIDIA for LLM head-to-head for the broader platform decision.
- Building an air-gapped or fully on-prem deployment? Our best GPU for private AI guide covers VRAM picks where data never leaves the machine.
Before you buy: is Ollama using the GPU you have?
Worth confirming before you spend anything, because Ollama falls back to CPU silently. Ollama’s GPU documentation lists the cards it supports and the driver stack each one needs — every NVIDIA card on this page is on that list, but it is the first thing to check if your tokens are arriving at CPU speed.
A surprising share of “Ollama is slow” turns out to be Ollama not running on the GPU at all, or running on it only partly. That is worth ruling out before spending anything, because no upgrade fixes a configuration problem.
$ ollama ps
NAME ID SIZE PROCESSOR UNTIL
qwen3:14b a1b2c3d4e5f6 11 GB 100% GPU 4 minutes from now100% GPU is what you want. A split like 38%/62% CPU/GPU means the model did
not fit and Ollama offloaded the remainder to system memory, where it runs at a
fraction of the speed — that is the 5-10x slowdown people usually blame on the
card. If you see a split, the fix is a smaller model or a lower quantization,
not a bigger GPU, and our Ollama GPU requirements guide
has the per-model VRAM figures to pick one.
Only once ollama ps reads 100% GPU and the speed still bothers you is the
card actually the bottleneck.
Common mistakes to avoid
- Buying an 8GB VRAM GPU for Ollama — 8GB limits you to small 7B models at low quantization with almost no context window. You will outgrow it within weeks. Wondering if an older card like the RTX 3060 is enough to start? Our can the RTX 3060 run Ollama guide answers that question model by model, including which 2026 releases its 12GB just misses.
- Ignoring memory bandwidth — two cards may have the same VRAM, but higher bandwidth means faster token generation. The RTX 3090’s 936 GB/s crushes the RTX 4060 Ti’s 288 GB/s in tokens per second. Choosing between the RTX 5080 and 4090 for Ollama? See RTX 5080 vs 4090 for LLM for a bandwidth and VRAM breakdown.
- Not accounting for context length overhead — Ollama’s KV cache grows with context. A model that “fits” at 2K context may OOM at 8K. Budget 2-4GB extra VRAM beyond model size. Choosing the right quantization level is key to fitting your model — our best quantization for local LLM guide breaks down the quality-vs-VRAM tradeoffs. This is especially critical for LLM summarization workloads, where long documents push context windows to their limits.
- Choosing AMD without checking Ollama compatibility — Ollama’s ROCm support is improving but still inconsistent. Verify your specific AMD card works before buying. For a practical breakdown of how Ollama performs differently on Windows versus Linux, including ROCm driver behavior, see our Windows vs Linux for local LLM guide. If you plan to run Ollama with a web interface, see our best GPU for Open WebUI guide — the GPU requirements are the same but there are configuration tips specific to that stack. If you are still deciding between Ollama and other inference engines, see Ollama vs llama.cpp vs vLLM compared to understand which tool best matches your use case.
Final verdict
NVIDIA GeForce RTX 4090
24GB GDDR6XFor most serious Ollama users, this is the safest recommendation. 24GB VRAM and fast inference handle everything from casual chat to 34B model experiments.
Affiliate links — we may earn a commission at no extra cost to you. Amazon ships globally; Shopee SG covers Singapore & ASEAN.
The best GPU for Ollama is the one that fits your target model size and usage pattern without overspending on performance you will not use. If you are choosing between Ollama and LM Studio as your inference frontend, our LM Studio vs Ollama comparison covers the GPU requirements, model format support, and usability tradeoffs of each tool. If you have settled on LM Studio specifically, our best GPU for LM Studio guide covers which cards deliver the best VRAM-to-speed ratio for that interface. Prefer a traditional model loader GUI over Ollama? See our text-generation-webui GPU guide for hardware recommendations tailored to that interface. For budget-focused picks at specific price points, see our best GPU for LLM under $1500 guide.
Match your GPU to the model you actually run, not the one you might try someday. You can always upgrade — but you can’t refund wasted headroom.
Frequently Asked Questions
What is the best budget GPU for Ollama?
The RTX 3060 12GB (around $250 used) is the best budget GPU for Ollama. It handles all 7B models at Q4_K_M or higher quantization with speeds fast enough for interactive chat. For a modest step up, the RTX 4060 Ti 16GB at ~$425 adds 13B model support and is the best new budget card for Ollama in 2026.
What Ollama models can I run on an RTX 3060 12GB?
With 12GB VRAM, the RTX 3060 comfortably runs all 7B models (Llama 3 8B, Mistral 7B, Gemma 7B) at Q4_K_M to Q8 quantization. You can also run 13B models like Llama 2 13B at Q3_K_M or Q4_K_M, though context length will be limited. Models larger than 13B will not fit.
What Ollama models can I run on an RTX 4090?
The RTX 4090’s 24GB VRAM handles all 7B and 13B models at full Q8 or FP16 precision, plus 34B models like CodeLlama 34B and Qwen 32B at Q4_K_M quantization. Expect fast, conversational-speed inference for 13B Q4 models — comfortably above 40 tok/s. For 70B models, even the 4090 falls short — you would need dual GPUs or cloud.
Does Ollama support AMD GPUs?
Yes, Ollama supports AMD GPUs through the ROCm framework on Linux. However, ROCm compatibility is inconsistent across AMD card models and driver versions, and performance is generally noticeably slower than equivalent NVIDIA CUDA setups — expect a meaningful speed penalty that varies by card and model. Always verify your specific AMD GPU is supported before purchasing. NVIDIA remains the safer choice for a hassle-free Ollama experience.