Hardware guide
What every popular GPU, Apple-silicon Mac and RAM tier can run locally — memory, bandwidth, and the biggest model each handles well.
NVIDIA
| Hardware | Memory | Bandwidth | Biggest model it runs well |
|---|---|---|---|
| RTX 5090 | 32 GB | 1792 GB/s | Mixtral 8x7B (MoE) (46.7B) |
| RTX 5080 | 16 GB | 960 GB/s | Qwen3 14B (14.8B) |
| RTX 5070 Ti | 16 GB | 896 GB/s | Qwen3 14B (14.8B) |
| RTX 5070 | 12 GB | 672 GB/s | Qwen3 14B (14.8B) |
| RTX 4090 | 24 GB | 1008 GB/s | Qwen3 32B (32.8B) |
| RTX 4080 Super | 16 GB | 736 GB/s | Qwen3 14B (14.8B) |
| RTX 4070 Ti Super | 16 GB | 672 GB/s | Qwen3 14B (14.8B) |
| RTX 4070 Super | 12 GB | 504 GB/s | Qwen3 14B (14.8B) |
| RTX 4060 Ti 16 GB | 16 GB | 288 GB/s | Qwen3 14B (14.8B) |
| RTX 4060 Ti 8 GB | 8 GB | 288 GB/s | Qwen3 8B (8.2B) |
| RTX 3090 | 24 GB | 936 GB/s | Qwen3 32B (32.8B) |
| RTX 3080 10 GB | 10 GB | 760 GB/s | Mistral Nemo 12B (12.2B) |
| RTX 3060 12 GB | 12 GB | 360 GB/s | Qwen3 14B (14.8B) |
| RTX 6000 Ada | 48 GB | 960 GB/s | Qwen2.5 72B (72.7B) |
| RTX A6000 | 48 GB | 768 GB/s | Qwen2.5 72B (72.7B) |
| A100 80 GB | 80 GB | 2039 GB/s | Llama 4 Scout (MoE) (109B) |
| H100 80 GB | 80 GB | 3350 GB/s | Llama 4 Scout (MoE) (109B) |
AMD
| Hardware | Memory | Bandwidth | Biggest model it runs well |
|---|---|---|---|
| Radeon RX 7900 XTX | 24 GB | 960 GB/s | Qwen3 32B (32.8B) |
| Radeon RX 7900 XT | 20 GB | 800 GB/s | Qwen3 30B-A3B (MoE) (30.5B) |
Apple
| Hardware | Memory | Bandwidth | Biggest model it runs well |
|---|---|---|---|
| Mac · M1/M2/M3 (base), 8 GB | 8 GB | 100 GB/s | Qwen2.5-Coder 7B (7.6B) |
| Mac · M1/M2/M3 (base), 16 GB | 16 GB | 100 GB/s | Qwen3 14B (14.8B) |
| Mac · M4 (base), 24 GB | 24 GB | 120 GB/s | Mistral Small 3 24B (23.6B) |
| Mac · M4 Pro, 48 GB | 48 GB | 273 GB/s | Mixtral 8x7B (MoE) (46.7B) |
| Mac · M1/M2/M3 Max, 32 GB | 32 GB | 400 GB/s | Qwen3 32B (32.8B) |
| Mac · M4 Max, 64 GB | 64 GB | 546 GB/s | Qwen2.5 72B (72.7B) |
| Mac · M1/M2/M3 Max, 64 GB | 64 GB | 400 GB/s | Qwen2.5 72B (72.7B) |
| Mac · M3/M4 Max, 128 GB | 128 GB | 546 GB/s | Llama 4 Scout (MoE) (109B) |
| Mac Studio · M1/M2 Ultra, 128 GB | 128 GB | 800 GB/s | Llama 4 Scout (MoE) (109B) |
| Mac Studio · M3 Ultra, 256 GB | 256 GB | 819 GB/s | Qwen3 235B-A22B (MoE) (235B) |
| Mac Studio · M3 Ultra, 512 GB | 512 GB | 819 GB/s | Qwen3 235B-A22B (MoE) (235B) |
System RAM
| Hardware | Memory | Bandwidth | Biggest model it runs well |
|---|---|---|---|
| CPU only · 8 GB RAM | 8 GB | 50 GB/s | Gemma 3 4B (4.3B) |
| CPU only · 16 GB RAM | 16 GB | 60 GB/s | Qwen3 14B (14.8B) |
| CPU only · 32 GB RAM | 32 GB | 70 GB/s | Qwen3 32B (32.8B) |
| CPU only · 64 GB RAM | 64 GB | 80 GB/s | Qwen2.5 72B (72.7B) |
| CPU only · 128 GB RAM | 128 GB | 90 GB/s | Llama 4 Scout (MoE) (109B) |