ModelTape
RTX 5090

Best LLMs for 32 GB of VRAM

162 open-weights models run on a GPU with 32 GB of VRAM and 32 GB of system RAM, 140 of them fully on the GPU. The strongest by benchmark score: Qwen 3.8 27B, Qwen3.6 27B, Gemma 4 31B IT.

Each row shows the best quantization that fits, where it runs and a speed estimate at a 16K context, assuming a card with about 1792 GB/s of memory bandwidth. Bigger mixture-of-experts models can keep their experts in system RAM, which is why some run well above 32 GB. Other sizes: 6 GB · 8 GB · 12 GB · 16 GB · 24 GB.

Ranked by benchmarks

#ModelLabReleasedSizeECIBest quantRunstok/s
1Qwen 3.8 27BQwen2026-08-1428B152.6Q6_Kfully on GPU~45
2Qwen3.6 27BQwen2026-04-2128B146.7Q6_Kfully on GPU~45
3Gemma 4 31B ITGoogle2026-04-0231B144.3Q6_Kfully on GPU~41
4Qwen 3.6 35B-A3BQwen2026-04-1436B MoE144.0Q6_Kfully on GPU~300
5Qwen3.5-35B-A3BQwen2026-02-2436B MoE144.0Q6_Kfully on GPU~300
6Qwen3-235B-A22B-Thinking (Jul 2025)Qwen2025-07-25235B MoE143.2IQ1_MGPU + experts in RAM~20
7Gemma 4 26B A4BGoogle2026-04-0226B MoE141.6Q6_Kfully on GPU~250
8gpt-oss-120bOpenAI2025-08-05117B MoE140.2IQ3_XXSGPU + experts in RAM~50
9Qwen3.5-9BQwen2026-02-249.7B139.3Q6_Kfully on GPU~120
10Qwen3-235B-A22BQwen2025-04-28235B MoE138.4IQ1_MGPU + experts in RAM~18
11Qwen3-30B-A3B-Thinking (Jul 2025)Qwen2025-07-3030B MoE138.3Q6_Kfully on GPU~300
12Qwen3-235B-A22B-Instruct (Jul 2025)Qwen2025-07-25235B MoE138.1IQ1_MGPU + experts in RAM~20
13Qwen3-32BQwen2025-04-2933B136.9Q5_K_Mfully on GPU~44
14QwQ-32BQwen2025-03-0532B136.5Q6_Kfully on GPU~39
15Qwen3-14BQwen2025-04-2915B136.1Q6_Kfully on GPU~82
16Qwen3-30B-A3B-Instruct (Jul 2025)Qwen2025-07-2930B MoE135.2Q6_Kfully on GPU~300
17gpt-oss-20bOpenAI2025-08-0521B MoE135.0Q6_Kfully on GPU~240
18Qwen3-30B-A3BQwen2025-04-2931B MoE134.1Q6_Kfully on GPU~300
19Qwen3-8BQwen2025-04-288.2B132.1Q6_Kfully on GPU~140
20Phi-4Microsoft2024-12-1215B129.0Q6_Kfully on GPU~82
21Gemma 3 27BGoogle2025-03-1227B127.5Q6_Kfully on GPU~46
22Qwen2.5-72BQwen2024-09-1973B127.2IQ2_Mfully on GPU~42
23Llama 3.3 70BMeta2024-12-0670B126.1IQ2_Mfully on GPU~44
24Llama 3.1-70BMeta2024-07-2371B124.7IQ2_Mfully on GPU~43
25Llama 3.2 90BMeta2024-09-2490B122.8IQ2_XXSfully on GPU~44
26Qwen2-72BQwen2024-06-0773B122.3IQ2_Mfully on GPU~42
27Llama 3-70BMeta2024-04-1870B121.2IQ2_Mfully on GPU~44
28Qwen2.5-Coder-32BQwen2024-09-1832B118.9Q6_Kfully on GPU~39
29Gemma 2 27BGoogle2024-06-2427B118.6Q6_Kfully on GPU~46
30phi-3-small 7.4BMicrosoft2024-04-237.4B116.5Q6_Kfully on GPU~150
31phi-3-medium 14BMicrosoft2024-04-2314B114.3Q6_Kfully on GPU~86
32Mistral NeMoMistral2024-07-1812B113.9Q6_Kfully on GPU~97
33Mixtral 8x7BMistral2023-12-1145B113.2IQ4_XSfully on GPU~43
34Gemma 2 9BGoogle2024-06-249.2B112.7Q6_Kfully on GPU~120
35Llama 3.1-8BMeta2024-07-238B111.1Q6_Kfully on GPU~140
36phi-3-mini 3.8BMicrosoft2024-04-233.8B110.6Q6_Kfully on GPU~260
37Llama 3-8BMeta2024-04-188B109.4Q6_Kfully on GPU~140
38Llama 2-70BMeta2023-07-1870B105.8IQ2_Mfully on GPU~44
39Gemma 7BGoogle2024-02-217B103.3Q6_Kfully on GPU~160
40Mistral 7B v0.1Mistral2023-09-277B102.8Q6_Kfully on GPU~160
41Qwen-14BQwen2023-09-2414B101.9Q6_Kfully on GPU~86
42LLaMA-65BMeta2023-02-2465B101.7IQ3_XXSfully on GPU~41
43Falcon 2 11BTII2024-05-0911B99.3Q6_Kfully on GPU~110
44Llama 2-13BMeta2023-07-1813B96.7Q6_Kfully on GPU~92
45LLaMA-33BMeta2023-02-2433B96.6Q6_Kfully on GPU~38
46StarCoder 2 15BHugging Face2024-02-2015B96.3Q6_Kfully on GPU~80
47Yi 6B01.AI2023-11-226B94.4Q6_Kfully on GPU~180
48Baichuan2-13BBaichuan2023-09-0613B92.0Q6_Kfully on GPU~92
49Falcon-40BTII2023-05-2540B90.9Q4_K_Mfully on GPU~42
50Qwen-7BQwen2023-09-287B89.2Q6_Kfully on GPU~160
51MPT-30BMosaicML2023-06-2230B85.5Q6_Kfully on GPU~42
52LLaMA-13BMeta2023-02-2413B85.4Q6_Kfully on GPU~92
53Baichuan 2-7BBaichuan2023-09-207B83.8Q6_Kfully on GPU~160
54Llama 2-7BMeta2023-07-187B83.4Q6_Kfully on GPU~160
55LLaMA-7BMeta2023-02-247B77.5Q6_Kfully on GPU~160
56DeepSeek Coder 33BDeepSeek2023-11-0233B76.6Q6_Kfully on GPU~38
57Gemma 2BGoogle2024-02-212B76.0Q6_Kfully on GPU~400
58MPT-7BMosaicML2023-05-057B73.7Q6_Kfully on GPU~160
59Falcon-7BTII2023-04-247B73.4Q6_Kfully on GPU~160
60StarCoder 2 7BHugging Face2024-02-207B72.7Q6_Kfully on GPU~160

Newer models without benchmark results yet

ModelLabReleasedSizeECIBest quantRunstok/s
KLM 1.6MinimaLabs2026-11-090.1B–Q6_Kfully on GPU~930
Cola2ByteDance2026-11-068.8B–Q6_Kfully on GPU~130
S5 nanoMaritaca-ai2026-11-024.7B–Q6_Kfully on GPU~220
Raon 1 nanoKRAFTON2026-10-293.5B–Q6_Kfully on GPU~270
SenseNova Loop 9B PreviewSenseNova2026-10-289.7B–Q6_Kfully on GPU~120
LBS 27BinclusionAI2026-10-2627B–Q6_Kfully on GPU~46
WeVisOne VL 4BTencent2026-10-224.8B–Q6_Kfully on GPU~210
WeVisOne VL 2BTencent2026-10-222.4B–Q6_Kfully on GPU~350
flintRecursiveSI2026-10-2140B MoE–Q5_K_Mfully on GPU~550
MiniMax-M3.1-FlashMiniMax2026-10-19220B MoE–IQ1_MGPU + experts in RAM~190
Ling 3.0 flash SanteinclusionAI2026-10-19127B MoE–IQ3_XXSGPU + experts in RAM~52
Axion 1.0 35BAxion1-02026-10-1635B MoE–Q6_Kfully on GPU~250
NVIDIA Nemotron Labs Dory1.0 4B BF16NVIDIA2026-10-164B–Q6_Kfully on GPU~250
NVIDIA Nemotron Labs Dory1.0 4B MXFP8NVIDIA2026-10-164B–Q6_Kfully on GPU~250
Hy Memory 30B A3BTencent2026-10-1530B MoE–Q6_Kfully on GPU~300
granite switch 4.1 8bIBM2026-10-148.8B–Q6_Kfully on GPU~130
granite switch 4.1 30bIBM2026-10-1430B–Q6_Kfully on GPU~42
Youtu Parsing OmniTencent2026-10-095.3B–Q6_Kfully on GPU~200
MiniCPM5 2BOpenbmb2026-09-062.5B–Q6_Kfully on GPU~340
tiny aya en thinkerCohere2026-09-023.4B–Q6_Kfully on GPU~280
tiny aya l2 thinkerCohere2026-09-023.4B–Q6_Kfully on GPU~280
Qwen3.8 Flash NextQwen2026-08-24125B MoE–Stratavia Strata~73
Muse GlimmerMeta2026-08-1030B–Q6_Kfully on GPU~42
Ling 3.0 tinyinclusionAI2026-08-107.9B MoE–Q6_Kfully on GPU~490
North Micro Vision InstructCohere2026-08-102.5B–Q6_Kfully on GPU~350
Muse Glimmer 30B assistantMeta2026-08-0930B–Q6_Kfully on GPU~42
Mage VLMicrosoft2026-07-254.7B–Q6_Kfully on GPU~220
Gemma 4 E4BGoogle2026-07-028B–Q6_Kfully on GPU~140
Gemma 4 E2BGoogle2026-07-025.1B–Q6_Kfully on GPU~200
DiffusionGemma 26B-A4BDeepMind2026-06-1026B MoE–Q6_Kfully on GPU~250
Gemma 4 12BGoogle2026-06-0312B–Q6_Kfully on GPU~99
Granite 4.1 30BIBM2026-04-2929B–Q6_Kfully on GPU~43
Granite 4.1 8BIBM2026-04-298.8B–Q6_Kfully on GPU~130
MiniMax-M2.7MiniMax2026-03-18229B MoE–IQ1_MGPU + experts in RAM~34
Qwen3.5-4BQwen2026-02-244.7B–Q6_Kfully on GPU~220
Qwen3.5-2BQwen2026-02-242.3B–Q6_Kfully on GPU~370
Qwen3.5-0.8BQwen2026-02-240.9B–Q6_Kfully on GPU~600
Qwen3.5 27BQwen2026-02-2328B–Q6_Kfully on GPU~45
Qwen3.5 122B-A10BQwen2026-02-23125B MoE–IQ3_XXSGPU + experts in RAM~47
MiniMax-M2.5MiniMax2026-02-12229B MoE–IQ1_MGPU + experts in RAM~34