Can You Run Mistral Large 4 Locally? The 1T-Parameter Math

Mistral Large 4's open weights are 1.06TB at FP8, and even the smallest 1-bit GGUF is 233GB. What it takes to run locally, and what to run instead.

On this page
Build of Mistral Large 4Size on disk
Official FP8 checkpoint (Mistral)1.06TB
Unsloth Q8_0 GGUF1.12TB
Unsloth UD-Q4_K_XL GGUF605GB
Unsloth UD-Q2_K_XL GGUF378GB
Unsloth UD-IQ2_XXS GGUF302GB
Unsloth UD-IQ1_S GGUF (smallest)233GB

Read the bottom row first. The most aggressively compressed build anyone has published is still more than seven RTX 5090s’ worth of memory.

Short answer: not on graphics cards you can put in a desktop. Mistral Large 4 runs locally only on a machine with 256GB or more of system or unified memory, at 1- or 2-bit quality and slow speeds — or on a rented multi-GPU server node. For a home GPU, run a model that was built to fit one.

Rent an 8-GPU node on RunPod

What Mistral released

Mistral opened the API preview on October 6, 2026 and published the weights on October 29. The model card and config.json describe:

  • About 1 trillion parameters in total, with 49B active per token (52B if you count the embedding and output layers, which is the figure in the repository name).
  • A mixture-of-experts design: 72 layers, the first dense and the other 71 with 128 routed experts, 4 of them active per token plus one shared expert.
  • A 1M-token context window and a Pixtral-style vision encoder, so it reads images as well as text.
  • Two official checkpoints: FP8 (the 1.06TB one above) and NVFP4, which Mistral says targets Blackwell datacenter GPUs (B200, B300, GB200, GB300).

On licensing, the model card names the Mistral Ecosystem Open-Weight License, a modified MIT licence, and says running the model as a hosted service needs a separate commercial licence from Mistral. Running it for yourself does not.

Who this is for

You saw “open weights” in the announcement and want to know whether the hardware you own, or could buy, runs it. If you only need Large 4’s answers rather than custody of its weights, Mistral’s own API is the cheaper route and this page will mostly confirm that.

Why the active-parameter count does not save you

Only 49B parameters do work on each token, which sounds like a 70B-class memory bill. It is not. The router can send the next token to any of the 128 experts in any layer, so every expert must be loaded and ready. MoE cuts the compute per token. It does not cut the memory needed to hold the model. That is the same trap that made Kimi K3 look runnable for about a day.

The arithmetic closes: a trillion parameters at FP8 (one byte each) is about 1TB, matching the 1.06TB checkpoint. At roughly 2 bits per weight it is about 250-300GB, matching the IQ2 builds.

The tiers, honestly

HardwareMemorySmallest build that fitsVerdict
RTX 509032GBnoneabout 14% of the 1-bit build
4x RTX 5090128GBnoneabout 55% of the 1-bit build
Workstation, 256GB RAM256GBUD-IQ1_S (233GB)runs, slowly, at 1-bit quality
Workstation or unified-memory machine, 384-512GB384-512GBUD-IQ2_XXS to UD-Q2_K_XLUnsloth’s suggested tier for 2-bit
8x H100 80GB node640GBup to UD-Q4_K_XLnot the native FP8 checkpoint
8x H200 141GB node1,128GBFP8 checkpointfits, little left for long context
Blackwell node (B200 class)192GB per GPUNVFP4 checkpointMistral’s own SGLang example uses 4 GPUs

The memory tiers for the GGUF builds come from Unsloth’s guidance on its repository; the server rows follow from Mistral’s examples, which use a tensor-parallel size of 8 for FP8.

How fast is “slowly”?

On a CPU-and-RAM build, every token has to read the active weights out of memory once. At about 2 bits, 49B active parameters is roughly 12-13GB per token. Desktop dual-channel DDR5-6000 moves at most about 96GB/s, which caps generation near 7 tokens per second in theory, and real numbers land below the cap. An 8-channel server board raises the ceiling about threefold. These are bandwidth ceilings, not measurements — see our methodology for how we model speed.

Seven tokens per second is readable for chat. It is painful for coding agents, which generate and re-read thousands of tokens per step.

What to run on a home GPU instead

Two recent models were designed for single-GPU memory budgets, and both are better uses of the money than chasing Large 4 at 1-bit:

  • Qwen 3.8 27B: a dense model whose default Ollama Q4_K_M build is 18GB, so it needs a 24GB card. See best GPU for Qwen 3.8.
  • Nemotron 3.5 Bolt 74B-A9B: NVIDIA packed a 74B MoE into a 22GB file, with vision. NVIDIA’s own recipes start at 32GB cards. See best GPU for Nemotron 3.5 Bolt.
Check NVIDIA GeForce RTX 4090 on AmazonBuy on Shopee SG

If it is specifically a Mistral model you want locally, the smaller Mistral family still fits consumer cards — the Mistral GPU guide covers those.

Best Local Alternative

NVIDIA GeForce RTX 5090

32GB GDDR7

32GB runs Qwen 3.8 27B with long context and meets NVIDIA's recipe for Nemotron 3.5 Bolt — the realistic ceiling for one desktop GPU.

Affiliate links. We may earn a commission at no extra cost to you. Amazon ships worldwide; Shopee SG covers Singapore and ASEAN.

Which GPU should you buy?

  • You want Large 4’s quality, not its weights: no GPU. Use Mistral’s API.
  • You need the weights under your own control: rent an 8-GPU H200 node, or a Blackwell node for the NVFP4 build. Our cloud GPU guide compares providers, and RTX 5090 vs H100 explains why datacenter cards are a separate class.
  • You want to try it at home anyway: the deciding spec is 256GB+ of RAM, not the graphics card. A GPU only helps by holding the shared and dense layers.
  • You want the best model one desktop GPU can run: RTX 5090 with Qwen 3.8 27B or Nemotron 3.5 Bolt; a 24GB RTX 4090 for Qwen 3.8.
Compare multi-GPU node prices on Vast.ai

Common mistakes to avoid

  • Treating “open weights” as “runs on my PC”. The licence lets you run it. The 1.06TB checkpoint decides whether you can.
  • Sizing hardware from the 52B active figure. Every expert stays resident; size from the total.
  • Buying four GPUs to chase it. Four RTX 5090s hold about half the smallest build. RAM capacity, not GPU count, is what makes the 1-bit build load.
  • Judging the model on a 1-bit build. The IQ1 file exists so the model can load at all. Its quality is not what Mistral’s published results describe.

Final verdict

Your situationBest path
Want the answersMistral API
Need weight custodyRented 8x H200 (FP8) or Blackwell node (NVFP4)
Want to experiment at home256GB+ RAM workstation, 1- or 2-bit GGUF
Want a strong model on one GPURTX 5090 with Qwen 3.8 27B or Nemotron 3.5 Bolt

Best Budget

NVIDIA GeForce RTX 3090

24GB GDDR6X

A used 24GB card is the cheapest way to run Qwen 3.8 27B at Q4, a dense 27B that fits comfortably in 24GB.

Affiliate links. We may earn a commission at no extra cost to you. Amazon ships worldwide; Shopee SG covers Singapore and ASEAN.

Mistral Large 4 is a server model with downloadable weights. Rent the node when you need custody, use the API when you need answers, and spend your GPU budget on a model built to fit one card.

Frequently asked questions

How much memory does Mistral Large 4 need?

The official FP8 checkpoint is about 1.06TB. The smallest community GGUF build, a 1-bit quantization from Unsloth, is about 233GB and is meant for machines with 256GB of RAM. Two-bit builds run from roughly 300GB to 380GB. No single consumer graphics card comes close.

Can I run Mistral Large 4 on an RTX 5090 or 4090?

Not on its own. Even the smallest build is several times larger than a 32GB card, so the model would have to live in system RAM with the GPU helping on a few layers. A home machine needs 256GB or more of RAM to load it at all, and speed is then limited by memory bandwidth to a few tokens per second.

What should I run locally instead of Mistral Large 4?

A model built for one GPU. Qwen 3.8 27B fits a 24GB card at Q4, and NVIDIA’s Nemotron 3.5 Bolt packs a 74B mixture-of-experts model into about 22GB, with official recipes starting at 32GB cards. Both are far more practical than Mistral Large 4 at 1-bit quality.

Check a model against your own card

Pick a model size, a quantization and a context length. The calculator adds the KV cache and the runtime overhead, then lists the cards in our data that hold the total.

Open the VRAM calculator

This article contains affiliate links. If you buy through them we may earn a commission, at no extra cost to you. Read the full disclosure.

On this page
↑