StarCoder 2 3B
StarCoder 2 3B is an open-weights language model from Hugging Face, released February 22, 2024. It has 3B parameters (dense). On the Epoch Capabilities Index it scores 66.9, rank 251 of 256 rated models.
Check it against your own PC →
Lab
Hugging Face
Released
February 22, 2024
Weights
Open restricted use
Parameters
3B dense
Context
–
Price / M tokens
–
Epoch ECI
66.9 #251
Can you run StarCoder 2 3B locally?
The smallest common GPU it runs on is 6 GB of VRAM (with 32 GB of RAM): Q6_K, fully on GPU, roughly 45 tokens per second. Estimates assume 32 GB of DDR5 RAM, a 16K context and llama.cpp-style GGUF quantization. Enter your exact hardware in the live app.
| GPU memory | Verdict | Best quant | VRAM | RAM | tok/s |
|---|---|---|---|---|---|
| 6 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 4.8 GB | – | ~45 |
| 8 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 4.8 GB | – | ~62 |
| 12 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 4.8 GB | – | ~81 |
| 16 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 4.8 GB | – | ~110 |
| 24 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 4.8 GB | – | ~190 |
| 32 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 4.8 GB | – | ~300 |
Benchmark results
| Benchmark (Epoch AI) | Score |
|---|---|
| MMLU | 15.5% |
Links
Models with a similar score
XGen-7B · Phi-1.5 · Qwen2.5-Coder-0.5B · DeepSeek Coder 6.7B · open_llama_7b