Falcon-180B
Falcon-180B is an open-weights language model from TII, released September 6, 2023. It has 180B parameters (dense). On the Epoch Capabilities Index it scores 107.1, rank 207 of 256 rated models.
Check it against your own PC →
Lab
TII
Released
September 6, 2023
Weights
Open restricted use
Parameters
180B dense
Context
–
Price / M tokens
–
Epoch ECI
107.1 #207
Can you run Falcon-180B locally?
Even with 32 GB of VRAM and 32 GB of RAM it is too large to run at a usable speed on a single consumer GPU. Estimates assume 32 GB of DDR5 RAM, a 16K context and llama.cpp-style GGUF quantization. Enter your exact hardware in the live app.
| GPU memory | Verdict | Best quant | VRAM | RAM | tok/s |
|---|---|---|---|---|---|
| 6 GB VRAM | Too large | – | – | – | – |
| 8 GB VRAM | Too large | – | – | – | – |
| 12 GB VRAM | Too large | – | – | – | – |
| 16 GB VRAM | Too large | – | – | – | – |
| 24 GB VRAM | Too large | – | – | – | – |
| 32 GB VRAM | Very slow | IQ2_XXS (heavy loss) | 31.2 GB | 20.1 GB | ~1 |
Benchmark results
| Benchmark (Epoch AI) | Score |
|---|---|
| MMLU | 60.8% |
Links
Models with a similar score
phi-3-mini 3.8B · Stable Beluga 2 · Llama 3-8B · Qwen2.5-Coder (7B) · GPT-3.5 Turbo (Jun 2023) · Llama 2-70B · Gemma 7B