Falcon-40B
Falcon-40B is an open-weights language model from TII, released May 25, 2023. It has 40B parameters (dense). On the Epoch Capabilities Index it scores 90.9, rank 229 of 256 rated models.
Check it against your own PC →
Lab
TII
Released
May 25, 2023
Weights
Open unrestricted
Parameters
40B dense
Context
–
Price / M tokens
–
Epoch ECI
90.9 #229
Can you run Falcon-40B locally?
The smallest common GPU it runs on is 16 GB of VRAM (with 32 GB of RAM): IQ3_XXS, split GPU / CPU, roughly 5 tokens per second. Estimates assume 32 GB of DDR5 RAM, a 16K context and llama.cpp-style GGUF quantization. Enter your exact hardware in the live app.
| GPU memory | Verdict | Best quant | VRAM | RAM | tok/s |
|---|---|---|---|---|---|
| 6 GB VRAM | Very slow | IQ3_XXS (noticeable loss) | 5.2 GB | 14.4 GB | ~2 |
| 8 GB VRAM | Very slow | IQ3_XXS (noticeable loss) | 7.2 GB | 12.4 GB | ~2 |
| 12 GB VRAM | Very slow | IQ3_XXS (noticeable loss) | 11.2 GB | 8.4 GB | ~3 |
| 16 GB VRAM | Runs, slowly | IQ3_XXS (noticeable loss) | 15.2 GB | 4.4 GB | ~5 |
| 24 GB VRAM | Runs on GPU | IQ3_XXS (noticeable loss) | 19.6 GB | – | ~35 |
| 32 GB VRAM | Runs on GPU | Q4_K_M (very good) | 28.7 GB | – | ~42 |
Benchmark results
| Benchmark (Epoch AI) | Score |
|---|---|
| MMLU | 42.5% |
Links
Models with a similar score
Yi 6B · PaLM 2-S · Llama 2-34B · Baichuan2-13B · Qwen2.5-Coder (1.5B) · Qwen-7B