The “24GB” in the model’s name is the size of the model, not the size of the card you need. That one misreading is going to cost people money this month, so it goes first.
Quick answer: buy a 32GB RTX 5090 if you want to run Nemotron 3.5 Bolt the way NVIDIA documents it. A 24GB RTX 4090 can load it only by giving things up — vision, the speculative drafter, most of the context — and NVIDIA publishes no recipe for a 24GB discrete card at all.
Our Top Pick
NVIDIA GeForce RTX 5090
32GB GDDR7
32GB is the smallest discrete card NVIDIA's own Bolt recipe targets: 65K context, q8_0 KV cache, drafter on.
Affiliate links. We may earn a commission at no extra cost to you. Amazon ships worldwide; Shopee SG covers Singapore and ASEAN.
What NVIDIA actually shipped
NVIDIA put Nemotron 3.5 Bolt 74B-A9B on Hugging Face on October 15, 2026. It is a vision-language model compressed from the larger Nemotron 3.5 Super VL (120B total, 12B active), and the compression is aggressive on purpose:
| Property | Value | Source |
|---|---|---|
| Parameters | 74B total, 9B active per token | NVIDIA model card |
| Architecture | Mamba-2 + Transformer hybrid MoE, plus a C-RADIO vision encoder | NVIDIA model card |
| Expert weights | IQ2_XXS, 2.0625 bits per weight | NVIDIA model card |
| Mamba layers / embeddings | NVFP4 / Q8_0 | NVIDIA model card |
| Main GGUF file | 22GB | NVIDIA GGUF repo |
| Vision projector (mmproj) | 1.73GB | NVIDIA GGUF repo |
Ollama nemotron-3.5-bolt:latest | 25GB | ollama.com library |
| Max context | 256K tokens | NVIDIA model card |
| License | OpenMDW-1.1, commercial use allowed | NVIDIA model card |
Check the arithmetic, because it closes: 74 billion parameters at a little over 2 bits each is roughly 19GB, and the layers kept at NVFP4 and Q8_0 push the file to 22GB. There is no hidden full-precision copy waiting to blow up your VRAM. The weights really are that small.
NVIDIA says the router was tuned to keep the deployment footprint under 24GB at batch size 1 with 128K of context — on RTX Spark, its unified-memory machine, where “24GB” is a slice of a larger pool shared with everything else. A discrete graphics card is a different situation, and NVIDIA’s documentation treats it as one.
Who this is for
You want a single local model that reads screenshots and documents, runs agent-style tool calls, and holds long context, and you are willing to buy (or already own) an NVIDIA card from the Ada or Blackwell generation. If you only chat with text and own a 16GB card, skip to the decision section — there is a smaller Bolt for you.
Two recipes, and the one that is missing
NVIDIA’s GGUF card lists exactly two deployment recipes for llama.cpp:
- 28GB recipe, for RTX discrete GPUs with 32GB or more (or a 64GB RTX Spark): 65,536-token context, q8_0 KV cache, up to six DFlash draft tokens.
- 32GB recipe, for RTX discrete GPUs with 48GB or more (or a 128GB RTX Spark): the full 262,144-token context, up to seven draft tokens.
There is no 24GB row. That absence is the real answer to the question in the title. Here is how the arithmetic plays out on a 24GB card:
- The main file (22GB) plus the vision projector (1.73GB) is already about 23.7GB, before a single token of context is cached.
- The speculative drafter that both NVIDIA recipes switch on lives in a separate repository and needs memory of its own.
- A card that also drives your monitors loses some VRAM to the desktop before the model loads.
- Ollama’s default tag is 25GB, which is larger than the card. Ollama will still run it, by spilling layers into system RAM — and every layer that spills slows each token down.
So a 4090 can run Bolt if you drop the vision projector, skip the drafter and keep context short. That is a usable text model, but it is not the model NVIDIA describes.
VRAM capacity memory bandwidth Specs are manufacturer figures. Bar lengths are scaled independently per metric.
GPUs ranked for Nemotron 3.5 Bolt
1. RTX 5090 (32GB, Blackwell). The cheapest discrete card that meets NVIDIA’s 28GB recipe, with vision, drafter and 65K context all switched on. Blackwell also opens the second path: NVIDIA’s vLLM instructions target Blackwell (SM120/SM121) and currently need a nightly build, so treat that as an early-adopter option rather than the default.
2. RTX 6000 Ada (48GB, Ada). The card for the full 262K-token recipe. Ada is on NVIDIA’s supported list for the llama.cpp build, so this is the one to buy if you feed the model whole codebases or long PDFs. It is not on the vLLM path, which is Blackwell-only for now.
Check NVIDIA RTX 6000 Ada Generation 48GB on AmazonBuy on Shopee SG3. RTX 4090 (24GB, Ada). Supported architecture, wrong capacity. Good if you already own one and accept text-only use at short context; a poor reason to buy one today. Our RTX 5090 vs 4090 comparison covers the general trade-off, and Bolt is a model where the extra 8GB decides whether the feature list survives.
4. 16GB cards (RTX 5080, 5070 Ti, 4060 Ti 16GB). The 74B model is out of reach without heavy offloading. Pull nemotron-3.5-bolt:e4b instead: Ollama lists it at 7.2–7.9GB, but note that it is a different model — an 8B (4B effective) model NVIDIA post-trained from Gemma 4 E4B, not a cut-down 74B.
5. RTX 3090 (24GB, Ampere). NVIDIA’s GGUF card lists Ada Lovelace and Blackwell as supported, and Ampere is not on that list. Combine that with the 24GB problem and the used 3090 — still our favourite value card for most local LLM work, per the used RTX 3090 guide — is the wrong purchase for this particular model.
Apple Silicon is out too: Ollama’s page states that Metal is not currently supported for the 74B-A9B build.
Which GPU should you buy for Bolt?
- You want vision, the drafter and 65K context, as documented: RTX 5090.
- You want the full 256K context for long documents or repositories: RTX 6000 Ada 48GB.
- You already own a 4090 and mostly use text: keep it; load the model without the vision projector and with a short context.
- You own a 16GB card: run
nemotron-3.5-bolt:e4b, or a dense 27B such as Qwen 3.8 if you can reach 24GB. - You own a used 3090 or a Mac: not this model. Pick something on the Ollama VRAM guide that fits your memory.
Common mistakes to avoid
- Reading the model name as a hardware spec. “24GB” describes the packed weights on NVIDIA’s unified-memory box. It does not mean a 24GB card runs the whole feature set.
- Assuming active parameters set the memory bill. Only 9B parameters fire per token, but the router can pick any expert, so all 74B have to sit in memory. MoE saves compute, not capacity.
- Pulling the
e4btag and expecting the 74B model. Same family name, different base model, very different capability. - Re-quantizing it yourself. The experts were trained to work at 2 bits (quantization-aware distillation). Community re-quantized uploads produce a bigger file and are not what NVIDIA evaluated.
Not sure how much headroom your context length needs? The VRAM calculator works the cache arithmetic for conventional transformer models; Bolt’s Mamba layers keep a fixed-size state, so treat its estimate as a ceiling here. For agent workloads more broadly, see best GPU for agent AI, and for NVIDIA’s earlier experiment in an unusual architecture, Nemotron TwoTower.
Final verdict
| Need | Best pick | What you get |
|---|---|---|
| Bolt as NVIDIA documents it | RTX 5090 (32GB) | 28GB recipe: vision, drafter, 65K context |
| Full 256K context | RTX 6000 Ada (48GB) | 32GB recipe: everything on |
| Already own a 24GB Ada card | RTX 4090 | Text-only, short context, no drafter |
| 16GB or less | Any 16GB card | The separate e4b model, 7.2–7.9GB |
Best Overall
NVIDIA GeForce RTX 5090
32GB GDDR7
The smallest discrete card with an official Bolt recipe, and the only one that also runs NVIDIA's vLLM path.
Affiliate links. We may earn a commission at no extra cost to you. Amazon ships worldwide; Shopee SG covers Singapore and ASEAN.
Nemotron 3.5 Bolt is genuinely small for a 74B model, but “small” means 32GB to run it properly. Buy the RTX 5090, or keep your 4090 and accept a text-only Bolt.
Frequently asked questions
Can an RTX 4090 run Nemotron 3.5 Bolt?
Partly. The 22GB main file fits on a 24GB card, but adding the vision projector leaves almost nothing for context, and NVIDIA publishes no recipe for 24GB discrete GPUs. Expect to run it text-only, without the speculative drafter, at a short context length. Ollama’s default tag is larger than 24GB, so it will spill part of the model into system RAM and slow down.
How much VRAM does Nemotron 3.5 Bolt need?
NVIDIA’s smallest discrete-GPU recipe targets cards with 32GB or more, which gives a 65K-token context with vision and speculative decoding on. The full 256K-token context recipe targets cards with 48GB or more. The weights themselves are about 22GB because the experts are stored at roughly 2 bits.
Does Nemotron 3.5 Bolt work on an RTX 3090 or a Mac?
Neither is a good choice. NVIDIA lists Ada Lovelace and Blackwell as the supported architectures for the GGUF build, and the RTX 3090 is Ampere. Ollama states that Apple Silicon is not currently supported for the 74B model. Owners of either should pick a different model that fits their memory.