Mixtral 8x7B
Mixtral 8x7B is an open-weights language model from Mistral, released December 11, 2023. It has 45B parameters (dense). On the Epoch Capabilities Index it scores 113.2, rank 198 of 256 rated models.
Check it against your own PC →
Can you run Mixtral 8x7B locally?
The smallest common GPU it runs on is 24 GB of VRAM (with 32 GB of RAM): IQ3_XXS, fully on GPU, roughly 32 tokens per second. Estimates assume 32 GB of DDR5 RAM, a 16K context and llama.cpp-style GGUF quantization. Enter your exact hardware in the live app.
| GPU memory | Verdict | Best quant | VRAM | RAM | tok/s |
|---|---|---|---|---|---|
| 6 GB VRAM | Very slow | IQ3_XXS (noticeable loss) | 5.2 GB | 16.3 GB | ~2 |
| 8 GB VRAM | Very slow | IQ3_XXS (noticeable loss) | 7.2 GB | 14.3 GB | ~2 |
| 12 GB VRAM | Very slow | IQ3_XXS (noticeable loss) | 11.2 GB | 10.3 GB | ~3 |
| 16 GB VRAM | Very slow | IQ3_XXS (noticeable loss) | 15.2 GB | 6.3 GB | ~4 |
| 24 GB VRAM | Runs on GPU | IQ3_XXS (noticeable loss) | 21.5 GB | – | ~32 |
| 32 GB VRAM | Runs on GPU | IQ4_XS (good) | 28.2 GB | – | ~43 |
Strengths by category
ModelTape's 0–100 fit on Epoch AI benchmark results.
| Category | Score | Based on |
|---|---|---|
| Knowledge | 24.2 | GPQA diamond, MMLU |
Benchmark results
| Benchmark (Epoch AI) | Score |
|---|---|
| MMLU | 60.8% |
| MATH level 5 | 10.0% |
| GPQA Diamond | 7.8% |
Links
Models with a similar score
Claude 2 · phi-3-small 7.4B · phi-3-medium 14B · Claude Instant · PaLM 2-L · Qwen2.5-Coder-14B · Mistral NeMo · GPT-3.5 Turbo (Jan 2024)
More from Mistral
Mistral Large 4 · Mistral Large 4 Preview · Mistral Large 4 1T A52B · Leanstral 1.5 · Mistral Medium 3.5 · Mistral Small 4 · Devstral Small 2 · Devstral 2