MPT-30B
MPT-30B is an open-weights language model from MosaicML, released June 22, 2023. It has 30B parameters (dense). On the Epoch Capabilities Index it scores 85.5, rank 232 of 256 rated models.
Check it against your own PC →
Lab
MosaicML
Released
June 22, 2023
Weights
Open unrestricted
Parameters
30B dense
Context
–
Price / M tokens
–
Epoch ECI
85.5 #232
Can you run MPT-30B locally?
The smallest common GPU it runs on is 12 GB of VRAM (with 32 GB of RAM): IQ3_XXS, split GPU / CPU, roughly 6 tokens per second. Estimates assume 32 GB of DDR5 RAM, a 16K context and llama.cpp-style GGUF quantization. Enter your exact hardware in the live app.
| GPU memory | Verdict | Best quant | VRAM | RAM | tok/s |
|---|---|---|---|---|---|
| 6 GB VRAM | Very slow | IQ3_XXS (noticeable loss) | 5.2 GB | 10.2 GB | ~3 |
| 8 GB VRAM | Very slow | IQ3_XXS (noticeable loss) | 7.2 GB | 8.2 GB | ~3 |
| 12 GB VRAM | Runs, slowly | IQ3_XXS (noticeable loss) | 11.2 GB | 4.2 GB | ~6 |
| 16 GB VRAM | Runs, slowly | IQ4_XS (good) | 15.2 GB | 4.6 GB | ~5 |
| 24 GB VRAM | Runs on GPU | Q4_K_M (very good) | 22.2 GB | – | ~30 |
| 32 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 28.5 GB | – | ~42 |
Benchmark results
| Benchmark (Epoch AI) | Score |
|---|---|
| MMLU | 30.5% |
Links
Models with a similar score
Qwen2.5-Coder (1.5B) · Qwen-7B · LLaMA-13B · internlm-7b · Baichuan 2-7B · INTELLECT-1 · Llama 2-7B