Mistral NeMo
Mistral NeMo is an open-weights language model from Mistral, released July 18, 2024. It has 12B parameters (dense). The context window is 128K tokens. On the Epoch Capabilities Index it scores 113.9, rank 197 of 256 rated models.
Check it against your own PC →
Can you run Mistral NeMo locally?
The smallest common GPU it runs on is 6 GB of VRAM (with 32 GB of RAM): IQ4_XS, split GPU / CPU, roughly 6 tokens per second. Estimates assume 32 GB of DDR5 RAM, a 16K context and llama.cpp-style GGUF quantization. Enter your exact hardware in the live app.
| GPU memory | Verdict | Best quant | VRAM | RAM | tok/s |
|---|---|---|---|---|---|
| 6 GB VRAM | Runs, slowly | IQ4_XS (good) | 5.2 GB | 4.6 GB | ~6 |
| 8 GB VRAM | Runs, slowly | Q5_K_M (excellent) | 7.2 GB | 4.8 GB | ~5 |
| 12 GB VRAM | Runs on GPU | Q4_K_M (very good) | 10.8 GB | – | ~28 |
| 16 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 13.3 GB | – | ~29 |
| 24 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 13.3 GB | – | ~53 |
| 32 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 13.3 GB | – | ~97 |
Ready-made quantizations on Hugging Face: dphn/dolphin-2.9.3-mistral-nemo-12b-gguf · XeyonAI/MN-Helcyon-GPT-4o-12b-v4.5-GGUF · ltg/normistral-11b-warm · QuantFactory/Dans-PersonalityEngine-V1.1.0-12b-GGUF · QuantFactory/Mistral-Nemo-Japanese-Instruct-2408-GGUF · mradermacher/Mistral-Nemo-Base-2407-i1-GGUF
Benchmark results
| Benchmark (Epoch AI) | Score |
|---|---|
| Balrog | 17.6% |
| MATH level 5 | 10.8% |
| GPQA Diamond | 6.8% |
Links
Hugging Face: mistralai/Mistral-Nemo-Base-2407 · Epoch AI benchmarks
Models with a similar score
Claude 3 Haiku · Claude 2 · phi-3-small 7.4B · phi-3-medium 14B · Claude Instant · PaLM 2-L · Qwen2.5-Coder-14B · Mixtral 8x7B
More from Mistral
Mistral Large 4 · Mistral Large 4 Preview · Mistral Large 4 1T A52B · Leanstral 1.5 · Mistral Medium 3.5 · Mistral Small 4 · Devstral Small 2 · Devstral 2