How we evaluate GPUs for local LLM

Every recommendation on Best GPU for LLM goes through the same structured evaluation. This page explains what we actually check, where the data comes from, and how we decide what to recommend for local inference.

What we measure

Local LLM performance depends on a specific set of hardware characteristics. Every buyer guide and comparison on this site considers at least these factors:

• VRAM capacity — the single most important number. We map it against target model sizes (7B, 13B, 34B, 70B) at every realistic quantization level (Q3 to Q8, FP16).
• Memory bandwidth — determines tokens-per-second during inference. GDDR6 vs GDDR6X vs GDDR7 creates meaningful gaps for the same VRAM capacity.
• KV cache overhead — context length burns VRAM. We factor in realistic context sizes (4K, 16K, 32K+) rather than just base model weights.
• Quantization quality tradeoff — Q4_K_M vs Q5_K_M vs Q8 at different VRAM budgets.
• Software compatibility — Ollama, llama.cpp, vLLM, and Text Generation WebUI behavior per GPU family. ROCm maturity for AMD, MLX for Apple Silicon.
• Power and thermals — real constraint for always-on inference servers and home builds.
• Street price — including used-market pricing, which often changes the recommendation dramatically (the used RTX 3090 is a recurring example).

Data sources

We rely on a combination of primary benchmark sources, tool-specific community data, and manufacturer specs. We synthesize across multiple trusted sources rather than running every GPU in a private lab:

• Manufacturer specifications — NVIDIA, AMD, and Intel official spec sheets for VRAM, bandwidth, TDP, compute.
• Independent benchmarks — Tom's Hardware, TechPowerUp and Phoronix for cross-referenced performance data.
• LLM-specific community data — r/LocalLLaMA benchmark threads, LM Studio community results, and Ollama discussions.
• Tool behavior — Ollama, llama.cpp, vLLM and Text Generation WebUI issue trackers for compatibility reports and real-world speed numbers.
• Model-specific hardware guides — Unsloth documentation, Meta's Llama hardware guidance, and Hugging Face model card recommendations.
• Pricing — Amazon, Newegg, Best Buy, eBay sold listings, r/hardwareswap averages.

Our evaluation process

1. Model-first framing. Every guide starts with a specific model (Llama 3 8B, Qwen 14B, Mistral 7B) and works backward to hardware, not forward from a GPU.
2. VRAM fit check. We map model weights plus realistic KV cache overhead at multiple context lengths before recommending a card.
3. Quantization sanity check. We assume Q4_K_M as the default unless otherwise stated — this is what most Ollama and llama.cpp users actually run.
4. Where our tok/s numbers come from. They are modelled, not measured. Token generation on a local GPU is bound by memory bandwidth far more than by compute, so we derive tok/s from each card's published bandwidth and apply a per-vendor efficiency factor for software maturity — CUDA is further along than ROCm, which is further along than oneAPI. That is why an RX 7900 XTX lands below an RTX 4090 here despite comparable bandwidth. Use these figures to rank cards, not to predict what you will see in your own terminal; your quantization, context length and runtime all move the result. VRAM, bandwidth and TDP are manufacturer specifications, not estimates.
5. Cloud comparison. For large models (70B+) we always compare against cloud GPU rental cost before recommending a local hardware purchase.

What we don't do

• We don't run first-party benchmarks on every GPU. Honest buyer guides need breadth, and we'd rather cite multiple published sources than pretend to have hardware we haven't tested.
• We don't chase peak TFLOPS. For LLM inference, memory bandwidth dominates raw compute. A "slower" GPU with more VRAM and bandwidth beats a faster card with less.
• We don't push flagship cards when a $400 GPU is enough. If you only run 7B models, the RTX 4060 Ti 16GB is the right answer, regardless of affiliate payout differences.
• We don't fill keyword slots. If a topic doesn't have a distinct answer, we don't write a page just to rank.

How we handle updates

LLM hardware guidance changes quickly — new models, new quantization schemes, shifting pricing. We refresh articles when:

• A new GPU launches that changes the recommendation (RTX 5090, RTX 5060 Ti, etc.)
• A major new model launches with different VRAM characteristics (Llama 4, Qwen 3, Gemma 3)
• Street prices shift meaningfully
• A reader flags outdated information

When we update an article, we update the dateModified timestamp. We do not artificially bump dates to appear fresh.

Corrections

If you spot a factual error, outdated benchmark, or bad recommendation, we want to fix it. Reach out via the channels listed on our About page. Corrections are published transparently.

Affiliate disclosure

Best GPU for LLM participates in Amazon Associates and cloud GPU referral programs (RunPod, Vast.ai). If you buy through a link on this site, we may earn a commission at no extra cost to you. Commission differences do not change our recommendations — we recommend based on VRAM, throughput, and value, not payout rate. Full details on the Affiliate Disclosure page.