Yi 6B
Yi 6B is an open-weights language model from 01.AI, released November 22, 2023. It has 6B parameters (dense). On the Epoch Capabilities Index it scores 94.4, rank 225 of 256 rated models.
Check it against your own PC →
Lab
01.AI
Released
November 22, 2023
Weights
Open restricted use
Parameters
6B dense
Context
–
Price / M tokens
–
Epoch ECI
94.4 #225
Can you run Yi 6B locally?
The smallest common GPU it runs on is 6 GB of VRAM (with 32 GB of RAM): Q6_K, split GPU / CPU, roughly 10 tokens per second. Estimates assume 32 GB of DDR5 RAM, a 16K context and llama.cpp-style GGUF quantization. Enter your exact hardware in the live app.
| GPU memory | Verdict | Best quant | VRAM | RAM | tok/s |
|---|---|---|---|---|---|
| 6 GB VRAM | Runs, slowly | Q6_K (≈ lossless) | 5.2 GB | 2.4 GB | ~10 |
| 8 GB VRAM | Runs on GPU | Q5_K_M (excellent) | 6.9 GB | – | ~37 |
| 12 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 7.6 GB | – | ~42 |
| 16 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 7.6 GB | – | ~57 |
| 24 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 7.6 GB | – | ~100 |
| 32 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 7.6 GB | – | ~180 |
Benchmark results
| Benchmark (Epoch AI) | Score |
|---|---|
| MMLU | 52.0% |
Links
Models with a similar score
PaLM 2-M · Qwen2.5-Coder-3B · Llama 2-13B · LLaMA-33B · Phi-2 · StarCoder 2 15B · PaLM 2-S · Llama 2-34B