Can a $250 used GPU actually run local LLMs through Ollama? Yes — and better than most people expect. The RTX 3060 12GB handles 7B models at ~25 tok/s and even squeezes in some 13B models at lower quantization. Here is exactly what works and what does not.
Check NVIDIA GeForce RTX 3060 12GB on Amazon→Buy on Shopee SG→Who this is for
You already own an RTX 3060 12GB or you are considering buying one used as a cheap entry point into local LLM inference. You want honest answers about what this card can and cannot do with Ollama before spending money.
What the RTX 3060 12GB can run
Weight sizes below are the published ollama pull downloads, checked 2026-09-13 — not estimates from parameter counts. Add roughly half a gigabyte of KV cache on top at Ollama’s default 4K context, plus about a gigabyte of runtime overhead.
| Model | Quantization | Weights | Speed | Verdict |
|---|---|---|---|---|
| Llama 3.1 8B | Q4_K_M | 4.9GB | ~25 tok/s | Comfortable |
| Mistral 7B | Q4_K_M | 4.4GB | ~25 tok/s | Comfortable |
| Gemma 7B | Q4_K_M | 5.5GB | ~25 tok/s | Comfortable |
| Phi-3 Mini (3.8B) | Q4_K_M | 2.4GB | ~35 tok/s | Fast |
| CodeLlama 7B | Q4_K_M | 4.1GB | ~25 tok/s | Comfortable |
| Qwen 2.5 14B | Q4_K_M | 9.0GB | ~15 tok/s | Tight but works |
| Llama 3.1 8B | Q8_0 | 8.5GB | ~20 tok/s | Tight context |
| DeepSeek-R1 7B | Q4_K_M | 4.7GB | ~25 tok/s | Comfortable |
Sources: Llama 3.1, Mistral, Gemma, Phi-3, CodeLlama, DeepSeek-R1.
Note the spread. “A 7B model is about 5GB” is the usual shorthand and it is wrong in both directions here — CodeLlama 7B is 4.1GB and Gemma 7B is 5.5GB, a third apart for the same nominal size, because tokenizer and embedding dimensions differ. At Q4_K_M the 3060 clears every model in that list with 6.5 to 8GB left for context, which is more headroom than the shorthand suggests. You can push into 14B territory, though context windows shrink sharply.
What to actually pull in 2026
The table above is the compatibility picture. If you just want to know which
models to install, this is the short list, in the order most people should try
them. Figures are Q4_K_M, the default ollama pull gives you, except where a row names another build.
| Use it for | Model | Weights | Why this one |
|---|---|---|---|
| General chat, daily driver | Qwen 3 14B | 9.3GB | The largest model that fits properly. Tight, so keep context near 4K |
| General chat, safer fit | Llama 3.1 8B | 4.9GB | Leaves over 7GB for context. The one to pick if 14B feels cramped |
| Coding | CodeLlama 7B | 4.1GB | The smallest of the lot, and the 3060’s bandwidth keeps completions responsive |
| Fast and light | Gemma 4 E4B | 9.6GB (6.1GB QAT) | Quick responses; take the QAT build to leave room for context |
| Best quality that fits | Llama 3.1 8B at Q8_0 | 8.5GB | Higher precision on a smaller model beats Q4 on a bigger one |
The one that hurts: Qwen 3.6’s 35B-A3B ships as a 23-24GB file and Gemma 4’s 26B-A4B as an 18GB one (Qwen tags, Gemma tags). Their names advertise 3B and 4B active parameters, which is what they compute per token — but routing is decided at inference time, so every expert stays resident.
No quantization brings either within reach of 12GB: the smallest Gemma 4 26B build is the 16GB QAT one, and the smallest Qwen 3.6 35B build is 22GB. But “get a 24GB card” is too glib, and the two models differ:
- Gemma 4 26B-A4B at 18GB genuinely works on a used RTX 3090, leaving roughly 6GB for context.
- Qwen 3.6 35B-A3B at 23-24GB does not. A 24GB card that is 23GB full has nothing left for KV cache. You would need the 22GB nvfp4 build and short context to even try, and realistically this one wants more than 24GB.
Does 16GB of system RAM matter?
Not much, as long as the model fits in VRAM. Ollama loads the weights into the GPU and system RAM is only used while reading the file from disk. A 12GB card with 16GB of system RAM runs every model in the table above without touching it.
System RAM starts to matter the moment a model does not fit. Ollama then offloads layers to the CPU, and those layers run out of system memory at a fraction of the speed. That is the failure mode to avoid, not a reason to buy more RAM: 32GB will not make a 32B model usable on this card, it will only let it load before crawling.
For a model that is not on this list, the VRAM calculator will tell you whether 12GB clears it once the KV cache is counted.
VRAM capacity memory bandwidth Specs are manufacturer figures. Bar lengths are scaled independently per metric.
What the RTX 3060 cannot run
- Any 32B+ model. Qwen 3 32B is a 20GB download at Q4_K_M, and Q2 lands near 12GB — which is the card’s entire capacity before the desktop or a single token of context. Quantizing down does not rescue this tier.
- 70B models. Not even with CPU offloading at usable speeds. Offloaded layers crawl at 1-3 tok/s.
- 7B and 8B models at FP16. Full precision runs 14GB for Mistral 7B, 15GB for DeepSeek-R1 7B and 16GB for Llama 3.1 8B — all past a 12GB card. Stick to Q4 or Q8.
- Long context on larger models. Qwen 2.5 14B at Q4_K_M is 9.0GB, leaving about 3GB for KV cache. At 8K context you will hit OOM.
Real-world performance expectations
The RTX 3060 12GB has 360 GB/s memory bandwidth — actually higher than the newer RTX 4060 Ti 16GB (288 GB/s). This means for models that fit in VRAM, the 3060 generates tokens surprisingly fast relative to its age and price.
For interactive chat with a 7B model at 25 tok/s, the experience is smooth. Responses feel immediate. You will not notice the speed difference versus a $2,200 RTX 4090 for short answers. The gap shows up in longer generation tasks and larger models.
Check NVIDIA GeForce RTX 3060 12GB on Amazon→Buy on Shopee SG→Should you buy an RTX 3060 for Ollama?
If you want a dirt-cheap entry into local LLM and you only plan to run 7B models, the used RTX 3060 12GB at $250 is genuinely hard to beat. No other card at this price gives you 12GB of VRAM. If you want room to grow, the extra $175 for an RTX 4060 Ti 16GB (~$425) buys a 14B at Q8 instead of Q4, an 8B at FP16, and enough headroom that long context stops being a problem. It does not buy the 2026 MoE models — those start at 24GB — so weigh it as a precision-and-context upgrade rather than a new tier of capability.
Common mistakes to avoid
- Buying the RTX 3060 8GB variant. The 8GB version exists and it is drastically worse for LLMs. Always get the 12GB model. Double-check before purchasing.
- Expecting to run 13B models comfortably. They technically fit at Q4, but your context window shrinks to 2-4K tokens. For serious 13B use, you need 16GB.
- Comparing the RTX 3060 to the RTX 4060 (8GB). The RTX 4060 has only 8GB of VRAM — fewer than the older 3060 12GB. For LLM workloads, the 3060 12GB is the better card despite being two generations older.
- Ignoring power draw. The RTX 3060 draws 170W under load. Not a huge issue, but factor it into your electricity costs if running inference for hours daily.
Our recommendation
The RTX 3060 12GB is the best sub-$300 GPU for Ollama in 2026. It runs every major 7B model at usable speeds and costs less than two months of cloud GPU rental. If your budget allows $425, the RTX 4060 Ti 16GB is a better long-term investment. But if $250 is the ceiling, the 3060 12GB gets you into local LLM inference without compromise on the models that matter most.
Check NVIDIA GeForce RTX 3060 12GB on Amazon→Buy on Shopee SG→ Check NVIDIA GeForce RTX 4060 Ti 16GB on Amazon→Buy on Shopee SG→NVIDIA has also relaunched the RTX 3060 with updated stock for the AI inference market — see our RTX 3060 relaunch for LLM guide for what changed and whether the new batches are worth buying over used cards.
Twelve gigabytes of VRAM at $250 is the reason the RTX 3060 refuses to die. For 7B models, it is still enough.
For a full Ollama GPU comparison, see our best GPU for Ollama guide. If you want more budget options, check our best budget GPU for local LLM roundup.
Frequently asked questions
Is an RTX 3060 12GB still worth buying for local LLMs?
For 7B and 8B models, yes — it is hard to beat at around $250 used. Those models are roughly 4 to 5.5GB at Q4, so they sit in 12GB with plenty of room for context, and the card’s 360 GB/s of bandwidth is actually higher than the newer RTX 4060 Ti 16GB. Buy it knowing the ceiling is real: 14B fits tightly, and nothing above that does.
What is the largest model an RTX 3060 12GB can run?
The 14B class, and only just. Qwen 3 14B is a 9.3GB download at Q4_K_M, which leaves a couple of gigabytes for the KV cache and means keeping context short. Anything in the 32B class is out — quantizing one down to Q2 still lands around the card’s entire capacity, so there is nothing to step down to.
Does the 8GB version of the RTX 3060 work as well?
No, and the two are easy to confuse when buying used. Eight gigabytes puts the comfortable 7B models on the edge and rules out the 14B class entirely. The 12GB card is the one worth having for this; check the listing carefully, because both were sold under the same name.
Do I need a lot of system RAM?
Not while the model fits in VRAM — Ollama loads the weights onto the GPU and system memory only carries the file on its way there. It starts to matter the moment a model does not fit, because Ollama then runs the overflow layers on the CPU, at a small fraction of GPU speed.
Why do some newer models not fit despite small parameter counts?
Because mixture-of-experts models keep every expert resident even though only a few compute each token. A model advertised with a few billion active parameters can still ship as a download several times the size of a dense model you would expect to compare it with — which is why two of 2026’s headline releases are out of reach of a 12GB card.