Dolly 2.0-12b
Dolly 2.0-12b is an open-weights language model from Databricks, released April 11, 2023. It has 12B parameters (dense). On the Epoch Capabilities Index it scores 55.8, rank 255 of 256 rated models.
Check it against your own PC →
Lab
Databricks
Released
April 11, 2023
Weights
Open unrestricted
Parameters
12B dense
Context
–
Price / M tokens
–
Epoch ECI
55.8 #255
Can you run Dolly 2.0-12b locally?
The smallest common GPU it runs on is 6 GB of VRAM (with 32 GB of RAM): Q4_K_M, split GPU / CPU, roughly 5 tokens per second. Estimates assume 32 GB of DDR5 RAM, a 16K context and llama.cpp-style GGUF quantization. Enter your exact hardware in the live app.
| GPU memory | Verdict | Best quant | VRAM | RAM | tok/s |
|---|---|---|---|---|---|
| 6 GB VRAM | Runs, slowly | Q4_K_M (very good) | 5.2 GB | 5.2 GB | ~5 |
| 8 GB VRAM | Runs, slowly | Q5_K_M (excellent) | 7.2 GB | 4.4 GB | ~6 |
| 12 GB VRAM | Runs on GPU | Q4_K_M (very good) | 10.4 GB | – | ~29 |
| 16 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 12.9 GB | – | ~30 |
| 24 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 12.9 GB | – | ~54 |
| 32 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 12.9 GB | – | ~99 |
Benchmark results
| Benchmark (Epoch AI) | Score |
|---|---|
| MMLU | 1.6% |