Granite-4.0-H-Micro
Granite-4.0-H-Micro is an open-weights language model from IBM, released October 2, 2025. It has 3.2B parameters (dense). The context window is 128K tokens.
We’re launching Granite 4, the next generation of IBM language models. Granite 4.0 features a new hybrid Mamba/transformer architecture that greatly reduces memory requirements without sacrificing performance.
Check it against your own PC →
Can you run Granite-4.0-H-Micro locally?
The smallest common GPU it runs on is 6 GB of VRAM (with 32 GB of RAM): Q6_K, fully on GPU, roughly 42 tokens per second. Estimates assume 32 GB of DDR5 RAM, a 16K context and llama.cpp-style GGUF quantization. Enter your exact hardware in the live app.
| GPU memory | Verdict | Best quant | VRAM | RAM | tok/s |
|---|---|---|---|---|---|
| 6 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 3.4 GB | – | ~42 |
| 8 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 3.4 GB | – | ~59 |
| 12 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 3.4 GB | – | ~76 |
| 16 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 3.4 GB | – | ~100 |
| 24 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 3.4 GB | – | ~180 |
| 32 GB VRAM | Runs on GPU | Q6_K (≈ lossless) | 3.4 GB | – | ~290 |
Ready-made quantizations on Hugging Face: ibm-granite/granite-4.0-h-micro-GGUF · unsloth/granite-4.0-h-micro-GGUF · bartowski/ibm-granite_granite-4.0-h-micro-GGUF · taronaeo/Granite-4.0-H-Micro-BE-GGUF · mradermacher/granite-4.0-h-micro-i1-GGUF · mradermacher/granite-4.0-h-micro-GGUF
Benchmark results
Epoch AI has not published results for Granite-4.0-H-Micro yet. New models are usually evaluated within days to weeks.
Links
Announcement · Hugging Face: ibm-granite/granite-4.0-h-micro
More from IBM
granite switch 4.1 8b · granite switch 4.1 30b · Granite 4.1 30B · Granite 4.1 8B · Granite-4.0-H-Tiny · Granite-4.0-H-Small · Granite-Docling