ModelTape
RTX 4060, RTX 3070, RTX 4060 Laptop, RX 7600

Best LLMs for 8 GB of VRAM

111 open-weights models run on a GPU with 8 GB of VRAM and 32 GB of system RAM, 56 of them fully on the GPU. The strongest by benchmark score: Qwen 3.6 35B-A3B, Qwen3.5-35B-A3B, Gemma 4 26B A4B.

Each row shows the best quantization that fits, where it runs and a speed estimate at a 16K context, assuming a card with about 272 GB/s of memory bandwidth. Bigger mixture-of-experts models can keep their experts in system RAM, which is why some run well above 8 GB. Other sizes: 6 GB · 12 GB · 16 GB · 24 GB · 32 GB.

Ranked by benchmarks

#ModelLabReleasedSizeECIBest quantRunstok/s
1Qwen 3.6 35B-A3BQwen2026-04-1436B MoE144.0Q6_KGPU + experts in RAM~28
2Qwen3.5-35B-A3BQwen2026-02-2436B MoE144.0Q6_KGPU + experts in RAM~28
3Gemma 4 26B A4BGoogle2026-04-0226B MoE141.6Q6_KGPU + experts in RAM~15
4gpt-oss-120bOpenAI2025-08-05117B MoE140.2IQ2_XXSGPU + experts in RAM~31
5Qwen3.5-9BQwen2026-02-249.7B139.3Q4_K_Mfully on GPU~27
6Qwen3-30B-A3B-Thinking (Jul 2025)Qwen2025-07-3030B MoE138.3Q6_KGPU + experts in RAM~20
7Qwen3-14BQwen2025-04-2915B136.1Q4_K_Msplit GPU / CPU~5
8Qwen3-30B-A3B-Instruct (Jul 2025)Qwen2025-07-2930B MoE135.2Q6_KGPU + experts in RAM~20
9gpt-oss-20bOpenAI2025-08-0521B MoE135.0Q6_KGPU + experts in RAM~16
10Qwen3-30B-A3BQwen2025-04-2931B MoE134.1Q6_KGPU + experts in RAM~22
11Qwen3-8BQwen2025-04-288.2B132.1Q3_K_Mfully on GPU~39
12Phi-4Microsoft2024-12-1215B129.0IQ4_XSsplit GPU / CPU~6
13phi-3-small 7.4BMicrosoft2024-04-237.4B116.5IQ4_XSfully on GPU~40
14phi-3-medium 14BMicrosoft2024-04-2314B114.3Q4_K_Msplit GPU / CPU~6
15Mistral NeMoMistral2024-07-1812B113.9Q5_K_Msplit GPU / CPU~5
16Gemma 2 9BGoogle2024-06-249.2B112.7Q6_Ksplit GPU / CPU~7
17Llama 3.1-8BMeta2024-07-238B111.1IQ4_XSfully on GPU~37
18phi-3-mini 3.8BMicrosoft2024-04-233.8B110.6Q6_Kfully on GPU~50
19Llama 3-8BMeta2024-04-188B109.4IQ4_XSfully on GPU~37
20Gemma 7BGoogle2024-02-217B103.3Q4_K_Mfully on GPU~37
21Mistral 7B v0.1Mistral2023-09-277B102.8Q4_K_Mfully on GPU~37
22Qwen-14BQwen2023-09-2414B101.9Q4_K_Msplit GPU / CPU~6
23Falcon 2 11BTII2024-05-0911B99.3Q6_Ksplit GPU / CPU~5
24Llama 2-13BMeta2023-07-1813B96.7Q4_K_Msplit GPU / CPU~6
25StarCoder 2 15BHugging Face2024-02-2015B96.3IQ4_XSsplit GPU / CPU~6
26Yi 6B01.AI2023-11-226B94.4Q5_K_Mfully on GPU~37
27Baichuan2-13BBaichuan2023-09-0613B92.0Q4_K_Msplit GPU / CPU~6
28Qwen-7BQwen2023-09-287B89.2Q4_K_Mfully on GPU~37
29LLaMA-13BMeta2023-02-2413B85.4Q4_K_Msplit GPU / CPU~6
30Baichuan 2-7BBaichuan2023-09-207B83.8Q4_K_Mfully on GPU~37
31Llama 2-7BMeta2023-07-187B83.4Q4_K_Mfully on GPU~37
32LLaMA-7BMeta2023-02-247B77.5Q4_K_Mfully on GPU~37
33Gemma 2BGoogle2024-02-212B76.0Q6_Kfully on GPU~91
34MPT-7BMosaicML2023-05-057B73.7Q4_K_Mfully on GPU~37
35Falcon-7BTII2023-04-247B73.4Q4_K_Mfully on GPU~37
36StarCoder 2 7BHugging Face2024-02-207B72.7Q4_K_Mfully on GPU~37
37XGen-7BSalesforce2023-06-277B70.6Q4_K_Mfully on GPU~37
38DeepSeek Coder 6.7BDeepSeek2023-11-026.7B67.8Q4_K_Mfully on GPU~38
39StarCoder 2 3BHugging Face2024-02-223B66.9Q6_Kfully on GPU~62
40Dolly 2.0-12bDatabricks2023-04-1112B55.8Q5_K_Msplit GPU / CPU~6
41Cerebras-GPT-13BCerebras2023-03-2013B49.3Q4_K_Msplit GPU / CPU~6

Newer models without benchmark results yet

ModelLabReleasedSizeECIBest quantRunstok/s
KLM 1.6MinimaLabs2026-11-090.1B–Q6_Kfully on GPU~670
Cola2ByteDance2026-11-068.8B–Q3_K_Mfully on GPU~37
S5 nanoMaritaca-ai2026-11-024.7B–Q6_Kfully on GPU~41
Raon 1 nanoKRAFTON2026-10-293.5B–Q6_Kfully on GPU~53
SenseNova Loop 9B PreviewSenseNova2026-10-289.7B–Q4_K_Mfully on GPU~27
LBS 27BinclusionAI2026-10-2627B–IQ3_XXSsplit GPU / CPU~5
WeVisOne VL 4BTencent2026-10-224.8B–Q6_Kfully on GPU~40
WeVisOne VL 2BTencent2026-10-222.4B–Q6_Kfully on GPU~75
flintRecursiveSI2026-10-2140B MoE–Q5_K_MGPU + experts in RAM~54
Axion 1.0 35BAxion1-02026-10-1635B MoE–Q6_KGPU + experts in RAM~22
NVIDIA Nemotron Labs Dory1.0 4B BF16NVIDIA2026-10-164B–Q6_Kfully on GPU~48
NVIDIA Nemotron Labs Dory1.0 4B MXFP8NVIDIA2026-10-164B–Q6_Kfully on GPU~48
Hy Memory 30B A3BTencent2026-10-1530B MoE–Q6_KGPU + experts in RAM~16
granite switch 4.1 8bIBM2026-10-148.8B–Q6_Ksplit GPU / CPU~7
Youtu Parsing OmniTencent2026-10-095.3B–Q6_Kfully on GPU~36
MiniCPM5 2BOpenbmb2026-09-062.5B–Q6_Kfully on GPU~73
tiny aya en thinkerCohere2026-09-023.4B–Q6_Kfully on GPU~56
tiny aya l2 thinkerCohere2026-09-023.4B–Q6_Kfully on GPU~56
Qwen3.8 Flash NextQwen2026-08-24125B MoE–IQ2_XXSGPU + experts in RAM~43
Ling 3.0 tinyinclusionAI2026-08-107.9B MoE–Q5_K_Mfully on GPU~140
North Micro Vision InstructCohere2026-08-102.5B–Q6_Kfully on GPU~74
Mage VLMicrosoft2026-07-254.7B–Q6_Kfully on GPU~40
Gemma 4 E4BGoogle2026-07-028B–Q5_K_Mfully on GPU~28
Gemma 4 E2BGoogle2026-07-025.1B–Q6_Kfully on GPU~37
DiffusionGemma 26B-A4BDeepMind2026-06-1026B MoE–Q6_KGPU + experts in RAM~16
Gemma 4 12BGoogle2026-06-0312B–Q5_K_Msplit GPU / CPU~6
Granite 4.1 8BIBM2026-04-298.8B–Q6_Ksplit GPU / CPU~7
Qwen3.5-4BQwen2026-02-244.7B–Q6_Kfully on GPU~41
Qwen3.5-2BQwen2026-02-242.3B–Q6_Kfully on GPU~81
Qwen3.5-0.8BQwen2026-02-240.9B–Q6_Kfully on GPU~190
Qwen3-Coder-NextQwen2026-02-0280B MoE–IQ3_XXSGPU + experts in RAM~40
GLM-4.7-FlashZ.ai2026-01-1931B MoE–Q6_KGPU + experts in RAM~16
Solar Open 100BUpstage2025-12-31103B MoE–IQ2_XXSGPU + experts in RAM~16
VAETKINC AI2025-12-30112B MoE–IQ2_XXSGPU + experts in RAM~17
Nemotron 3-Nano-30B-A3BNVIDIA2025-12-1530B MoE–Q6_KGPU + experts in RAM~20
Nemotron-Cascade 14BNVIDIA2025-12-1514B–Q4_K_Msplit GPU / CPU~6
GLM-4.6V-FlashZ.ai2025-12-0810B–IQ4_XSfully on GPU~29
GLM-4.6VZ.ai2025-12-08108B MoE–IQ2_XXSGPU + experts in RAM~15
Ministral 3 3BMistral2025-12-023.9B–Q6_Kfully on GPU~49
Ministral 3 8BMistral2025-12-028.9B–Q6_Ksplit GPU / CPU~8