Two used RTX 3090s for $1,640 total. 48GB combined VRAM. Llama 70B at Q4 running at 18-22 tokens per second. That is the pitch — and it actually works. Dual 3090s are the cheapest way to run 70B-class models locally in 2026, and the setup is simpler than most people expect. No NVLink required. No exotic drivers. Just two cards, the right motherboard, and a beefy PSU.
NVIDIA GeForce RTX 3090
24GB GDDR6X24GB VRAM per card. Two for ~$1,640 gives you 48GB — enough for Llama 70B at Q4. The cheapest path to 70B.
Affiliate links — we may earn a commission at no extra cost to you. Amazon ships globally; Shopee SG covers Singapore & ASEAN.
Why dual 3090s?
The math is straightforward:
| Setup | VRAM | Can run 70B Q4? | Cost |
|---|---|---|---|
| 1x RTX 4090 | 24GB | No (43GB needed) | ~$2,200 |
| 1x RTX 5090 | 32GB | No (43GB needed) | ~$4,900 |
| 2x RTX 3090 (used) | 48GB | Yes | ~$1,640 |
| 2x RTX 4090 | 48GB | Yes | ~$4,400 |
A single RTX 4090 maxes out at 24GB — well short of the 43GB that Llama 70B at q4_K_M downloads as. The only way to fit 70B on consumer hardware is multiple GPUs. And two used 3090s at ~$820 each cost less than one new 4090.
What you need
Hardware checklist
| Component | Requirement | Why |
|---|---|---|
| GPUs | 2x RTX 3090 | 24GB each = 48GB total |
| Motherboard | 2 physical x16 PCIe slots | Both must run at x8 or x16 |
| PSU | 1000W minimum, 1200W recommended | Each 3090 draws up to 350W — 700W before the rest of the machine |
| CPU | Any modern 6+ core | Not the bottleneck for inference |
| RAM | 32GB minimum | 64GB recommended for large context |
| Case | Full tower with good airflow | 3090s are triple-slot cards — check clearance |
| PCIe risers | Optional | Can help with spacing if slots are too close |
NVIDIA specifies 750W of system power for a single RTX 3090 at 350W board power. A second card adds another 350W, which is why 850W — a figure you will see repeated for dual-3090 builds — leaves nothing for the CPU, drives and fans. Our PSU guide for dual-GPU LLM builds has specific units.
Motherboard notes
This is where most builds fail. Many consumer motherboards have two x16-length slots, but the second slot runs at x4 electrically. That works but costs ~15% performance. Look for boards where both slots run at x8/x8 minimum when populated. ATX boards with Intel Z690/Z790 or AMD X670 chipsets usually support this.
Do NOT buy NVLink bridges. The RTX 3090 supports NVLink, but it buys you nothing here — and the reason is worth understanding, because it is what makes dual 3090s work on ordinary hardware. llama.cpp’s --split-mode documentation lists layer as the default: it splits layers and KV cache across the cards and runs them pipelined. The GPUs largely take turns rather than working on the same tensors at once, so the link between them carries little traffic and PCIe is ample. (row and tensor modes are genuinely parallel — and tensor is marked experimental — but they are not what you get by default.) NVLink is wasted money for this use case.
Software setup
Option 1: Ollama (easiest)
Ollama automatically detects multiple GPUs and splits the model across them. No configuration needed.
# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh
# Run a 70B model — Ollama auto-splits across both GPUs
ollama run llama3.1:70b-instruct-q4_K_MVerify both GPUs are being used:
nvidia-smi
# Both GPUs should show VRAM usageOption 2: llama.cpp (more control)
llama.cpp gives you explicit control over layer splitting:
# Auto-split across GPUs
./llama-server -m llama-70b-Q4_K_M.gguf --n-gpu-layers 99
# Manual split: 40 layers on GPU 0, 40 on GPU 1
./llama-server -m llama-70b-Q4_K_M.gguf --n-gpu-layers 80 --tensor-split 0.5,0.5The --tensor-split flag controls how layers are distributed. Equal split (0.5,0.5) is usually optimal for two identical GPUs. If one card is slightly faster or has more free VRAM, adjust the ratio.
Do not confuse --tensor-split with --split-mode tensor. They are different flags: --tensor-split sets the ratio of the model each GPU receives, while --split-mode sets how the split works (layer, the pipelined default, or row/tensor, which are parallel). Passing a ratio does not switch you into a parallel mode.
Performance expectations
Llama 3.1 70B at Q4_K_M on dual RTX 3090s, with throughput estimated from combined memory bandwidth (methodology):
| Metric | Value |
|---|---|
| Prompt processing | ~350 tok/s |
| Token generation | ~18-22 tok/s |
| VRAM usage (per GPU) | ~21.5GB each |
| Total VRAM used | ~43GB (weights) |
| Power draw (both GPUs) | ~500-600W |
18-22 tok/s on a 70B model is comfortable for interactive chat. It is not blazing fast, but responses stream smoothly and you will not feel like you are waiting.
For comparison:
| Setup | 70B Q4 tok/s | Cost |
|---|---|---|
| 2x RTX 3090 | ~18-22 tok/s | ~$1,640 |
| 2x RTX 4090 | ~30-35 tok/s | ~$4,400 |
| Cloud (RunPod A100) | ~40-50 tok/s | ~$2-4/hr |
Dual 4090s are ~60% faster, but at nearly 3x the cost. The 3090 setup is the value play.
VRAM capacity memory bandwidth Specs are manufacturer figures. Bar lengths are scaled independently per metric.
What models fit on 48GB?
| Model | q4_K_M download | Fits on 2x 3090? | tok/s |
|---|---|---|---|
| Llama 3.1 70B | 43GB | Yes, ~5GB spare | ~18-22 |
| Qwen 2.5 72B | 47GB | No — 1GB short before any cache | — |
| Mixtral 8x7B | 28GB | Yes, comfortably | ~25-30 |
| Nemotron TwoTower | ~40GB at Q4 | Yes — both towers resident | ~48 |
| Llama 4 Scout (109B MoE) | ~67GB | No — needs four cards, not two | — |
| Mixtral 8x22B | 86GB | No — nearly double 48GB | — |
| Any model under 34B | Under 24GB | Yes (single GPU) | Varies |
Sizes are the published downloads for llama3.1, qwen2.5 and mixtral.
A correction worth stating plainly, because this page had it backwards until 2026-09-12. A mixture-of-experts model does not need less VRAM than its parameter count implies. Every expert has to be resident in memory, because the router can send the next token to any of them; what is sparse is the compute, not the storage. That is why Mixtral 8x22B is an 86GB file — 141B total parameters — even though only about 39B are active per token. It is fast for its size, not small for its size. If you want a Mixtral that fits in 48GB, it is 8x7B at 28GB.
The 48GB sweet spot opens up the entire 70B class of dense models and many larger MoE models. This is the key advantage over single-GPU setups.
Common issues and fixes
”Only one GPU is being used”
Check that both GPUs are detected: nvidia-smi should show two devices. If Ollama only uses one, try setting CUDA_VISIBLE_DEVICES=0,1 before starting. In llama.cpp, explicitly set --n-gpu-layers 99 to force full GPU offloading.
Thermal throttling
Two 3090s generate serious heat — up to 700W combined. Ensure your case has strong front-to-back airflow. Leave at least one slot gap between the cards if possible. Consider aftermarket GPU coolers or a case with 140mm fans if you see temperatures hitting 83C+ consistently.
PCIe bandwidth bottleneck
If your second slot runs at x4, you will see one GPU process tokens slower than the other. The impact is ~15% on overall throughput. Upgrading to a motherboard with proper x8/x8 bifurcation fixes this. For most users, the 15% loss is acceptable given the cost savings.
Who should NOT do this?
- Gamers who occasionally run LLMs. Dual 3090s draw 700W and generate significant heat. If you primarily game, a single RTX 4090 is a better all-rounder (though it cannot do 70B).
- Anyone who needs 70B at 30+ tok/s. Dual 3090s cap at ~22 tok/s. If speed is critical, dual 4090s or cloud are your options.
- Small form factor builders. Two triple-slot 3090s need a full tower case with good airflow. mITX and mATX builds cannot accommodate this.
NVIDIA GeForce RTX 3090
24GB GDDR6X~$820 used for 24GB VRAM. Two of these unlock 70B models for less than the price of a single RTX 4090.
Affiliate links — we may earn a commission at no extra cost to you. Amazon ships globally; Shopee SG covers Singapore & ASEAN.
For used 3090 buying tips, see our used RTX 3090 buying guide. Planning to run Llama specifically? The best GPU for Llama 70B guide covers all options. PSU sizing for multi-GPU is covered in PSU for dual GPU LLM. And for motherboard compatibility, see best motherboard for dual GPU LLM.
Frequently asked questions
Do I need NVLink to run a model across two RTX 3090s?
No, and it is the single most common thing people overspend on here. Layer splitting sends a small activation tensor between the cards once per layer, not the weights, so PCIe carries it comfortably. The 3090 does support a bridge and it is the last consumer card that does, but for inference it buys you very little.
What does a dual RTX 3090 build actually cost?
Around $1,640 for the pair at roughly $820 each used, which is less than one new RTX 4090. Budget separately for the parts that make two cards workable: a board with two usable x16 slots, a 1000W power supply with 1200W preferable, and a case that can move air across both.
What size power supply do two 3090s need?
1000W is the floor and 1200W is the comfortable answer. Each card is a 350W board, so the pair draws about 700W before the processor, drives and fans are counted, and it is the transient spikes rather than the steady draw that trip an undersized unit.
Which models fit in 48GB?
The whole 70B class at Q4_K_M — Llama 3.1 70B is a 43GB download, leaving roughly five gigabytes for context — plus Mixtral 8x7B at 28GB with room to spare. What does not fit is closer than people expect: Qwen 2.5 72B is 47GB, which is over before any cache, and Mixtral 8x22B is 86GB, nearly double the pair’s capacity.
How fast is 70B on two 3090s?
Expect the high teens to low twenties in tokens per second at Q4, which streams smoothly enough for conversation without feeling quick. Two RTX 4090s are roughly half again as fast for well over twice the money, and a rented A100 is faster still. Buy the pair for capacity; if speed is the constraint, this is not the build.