Mistral 7B v0.1
Mistral 7B v0.1 is an open-weights language model from Mistral, released September 27, 2023. It has 7B parameters (dense). On the Epoch Capabilities Index it scores 102.8, rank 211 of 256 rated models.
Check it against your own PC →
Can you run Mistral 7B v0.1 locally?
The smallest common GPU it runs on is 6 GB of VRAM (with 32 GB of RAM): Q6_K, split GPU / CPU, roughly 8 tokens per second. Estimates assume 32 GB of DDR5 RAM, a 16K context and llama.cpp-style GGUF quantization. Enter your exact hardware in the live app.
| GPU memory | Verdict | Best quant | VRAM | RAM | tok/s |
|---|---|---|---|---|---|
| 6 GB VRAM | Runs, slowly | Q6_K (≈ lossless) | 5.2 GB | 3.3 GB | ~8 |
| 8 GB VRAM | Runs on GPU | Q4_K_M (very good) | 7.0 GB | – | ~37 |
| 12 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 8.5 GB | – | ~36 |
| 16 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 8.5 GB | – | ~50 |
| 24 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 8.5 GB | – | ~89 |
| 32 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 8.5 GB | – | ~160 |
Benchmark results
| Benchmark (Epoch AI) | Score |
|---|---|
| MMLU | 50.0% |
Links
Models with a similar score
GPT-3.5 Turbo (Jun 2023) · Llama 2-70B · Gemma 7B · Yi-9B · Qwen-14B · LLaMA-65B · DeepSeek-Coder-V2-Lite-Base · Falcon 2 11B
More from Mistral
Mistral Large 4 · Mistral Large 4 Preview · Mistral Large 4 1T A52B · Leanstral 1.5 · Mistral Medium 3.5 · Mistral Small 4 · Devstral Small 2 · Devstral 2