Which AI models can your computer run?
Pick your GPU, Mac or RAM and see exactly which open LLMs fit — the best quantisation, context headroom and rough speed, computed live in your browser.
23 models run well and 1 run tight on RTX 4090 (any context — just fitting the weights) — 23 GB usable.
| Model | Size | Fit | Best quant | Needs | Memory | Speed |
|---|---|---|---|---|---|---|
| Qwen3 32B | 32.8B | Runs well | Q5_K_M | 23 GB | ~31 tok/s | |
| Qwen2.5 32B | 32.8B | Runs well | Q5_K_M | 23 GB | ~31 tok/s | |
| Qwen2.5-Coder 32B | 32.8B | Runs well | Q5_K_M | 23 GB | ~31 tok/s | |
| DeepSeek-R1 Distill Qwen 32B | 32.8B | Runs well | Q5_K_M | 23 GB | ~31 tok/s | |
| Qwen3 30B-A3B (MoE)MoE | 30.5B | Runs well | Q5_K_M | 22 GB | ~202 tok/s | |
| Gemma 3 27B | 27.4B | Runs well | Q6_K | 23 GB | ~32 tok/s | |
| Gemma 2 27B | 27.2B | Runs well | Q6_K | 22 GB | ~33 tok/s | |
| Mistral Small 3 24B | 23.6B | Runs well | Q6_K | 19 GB | ~38 tok/s | |
| Qwen3 14B | 14.8B | Runs well | Q8_0 | 16 GB | ~46 tok/s | |
| Qwen2.5 14B | 14.8B | Runs well | Q8_0 | 16 GB | ~46 tok/s | |
| DeepSeek-R1 Distill Qwen 14B | 14.8B | Runs well | Q8_0 | 16 GB | ~46 tok/s | |
| Phi-4 14B | 14.7B | Runs well | Q8_0 | 16 GB | ~46 tok/s | |
| Mistral Nemo 12B | 12.2B | Runs well | Q8_0 | 13 GB | ~56 tok/s | |
| Gemma 3 12B | 12.2B | Runs well | Q8_0 | 13 GB | ~56 tok/s | |
| Gemma 2 9B | 9.2B | Runs well | FP16 | 19 GB | ~39 tok/s | |
| Qwen3 8B | 8.2B | Runs well | FP16 | 17 GB | ~44 tok/s | |
| Llama 3.1 8B | 8.0B | Runs well | FP16 | 16 GB | ~45 tok/s | |
| Qwen2.5-Coder 7B | 7.6B | Runs well | FP16 | 15 GB | ~48 tok/s | |
| Mistral 7B v0.3 | 7.3B | Runs well | FP16 | 15 GB | ~50 tok/s | |
| Gemma 3 4B | 4.3B | Runs well | FP16 | 9.1 GB | ~84 tok/s | |
| Qwen3 4B | 4B | Runs well | FP16 | 8.5 GB | ~91 tok/s | |
| Llama 3.2 3B | 3.2B | Runs well | FP16 | 7.0 GB | ~113 tok/s | |
| Llama 3.2 1B | 1.2B | Runs well | FP16 | 3.2 GB | ~293 tok/s | |
| Mixtral 8x7B (MoE)MoE | 46.7B | Runs (tight) | Q3_K_M | 23 GB | ~75 tok/s | |
| DeepSeek-R1 671B-A37B (MoE)MoE | 671B | Won't fit | — | 273 GB | — | |
| Qwen3 235B-A22B (MoE)MoE | 235B | Won't fit | — | 96 GB | — | |
| Llama 4 Scout (MoE)MoE | 109B | Won't fit | — | 45 GB | — | |
| Qwen2.5 72B | 72.7B | Won't fit | — | 30 GB | — | |
| Llama 3.3 70B | 70.6B | Won't fit | — | 29 GB | — | |
| DeepSeek-R1 Distill Llama 70B | 70.6B | Won't fit | — | 29 GB | — |
Estimates, computed in your browser — VRAM, quantisation and speed vary with your runtime and settings. How we estimate →
Popular questions
Can I run Llama 3.3 70B?
70.6B params · what hardware it needs
Can I run Qwen3 32B?
32.8B params · what hardware it needs
Can I run Qwen3 8B?
8.2B params · what hardware it needs
Can I run DeepSeek-R1 671B-A37B (MoE)?
671B params · what hardware it needs
Can I run Gemma 3 27B?
27.4B params · what hardware it needs
Can I run Llama 3.1 8B?
8.03B params · what hardware it needs
Best LLMs for your hardware
Best local LLMs for RTX 4090
24 GB · which models fit & how fast
Best local LLMs for RTX 5090
32 GB · which models fit & how fast
Best local LLMs for RTX 4070 Super
12 GB · which models fit & how fast
Best local LLMs for RTX 3060 12 GB
12 GB · which models fit & how fast
Best local LLMs for Mac · M4 Max, 64 GB
64 GB · which models fit & how fast
Best local LLMs for RTX 3090
24 GB · which models fit & how fast
Stop guessing whether a model will run
Every week brings a new open-weights model, and the same question follows it across r/LocalLLaMA, Hacker News and Discord: will it run on my machine? The honest answer depends on three numbers most write-ups skip — how big the weights are at the quantisation you'll actually use, how much memory the KV cache eats at your context length, and how much your runtime reserves on top. Intecca does that arithmetic for your specific hardware so you don't have to.
General chat assistants can't answer this reliably: they don't track this month's models and they won't do the per-card memory math. That's the gap this tool fills — a current model database plus a calculator that knows the difference between a 24 GB RTX 4090, a 64 GB Mac and 32 GB of system RAM.
The 30-second version
- Weights ≈ params × bits ÷ 8. At Q4 (~4.8 bits) that's roughly 0.6 GB per billion parameters — an 8B needs ~5 GB, a 32B ~20 GB, a 70B ~40 GB.
- Leave room for context. The KV cache grows with context length; long-context use can add several GB.
- Quantise to fit, not below Q4 if you can help it. Q4_K_M is the sweet spot; Q3/Q2 only to rescue a bigger model.
- Bandwidth sets speed. Once a model fits, tokens-per-second tracks your memory bandwidth, not raw compute.
Memory tiers at a glance
| You have | Comfortable at Q4 | Examples |
|---|---|---|
| 8 GB VRAM | up to ~8B | Llama 3.1 8B, Qwen3 8B |
| 12 GB VRAM | up to ~14B | Qwen3 14B, Phi-4 14B |
| 16 GB VRAM | up to ~14–24B | Mistral Small 24B (tight) |
| 24 GB VRAM | up to ~32B | Qwen3 32B, Gemma 3 27B |
| 48 GB VRAM / 64 GB Mac | up to ~70B | Llama 3.3 70B |
| 128 GB+ unified | 100B-class MoE | Llama 4 Scout, Qwen3 235B (192 GB+) |
Rules of thumb — use the calculator above for your exact card and context. See how we estimate.
Why run an LLM locally at all?
A cloud API is cheaper and easier to start with, so it’s worth being honest about why people still run models on their own hardware. The reasons that actually hold up:
- Privacy and data control. Your prompts and documents never leave the machine. For regulated work, personal data, or anything you simply don’t want logged on someone else’s server, that’s decisive.
- No rate limits and no per-token bill. Once the model is on your disk, you can run it as hard as you like — bulk classification, agent loops, synthetic-data generation — without watching a meter.
- Offline and air-gapped use. It keeps working on a plane, in a lab, or behind a firewall with no internet.
- It doesn’t change under you. A local model is frozen — no silent version bumps, deprecations or refusals shifting beneath your application.
- Learning and tinkering. Fine-tuning, quantising and wiring models into your own tools is simply more fun and more instructive when you own the whole stack.
What local won’t reliably give you is frontier-model quality on consumer hardware — see the size expectations below, and our best-model picks by use case for what’s realistic.
How to actually run a local model
It’s three steps, and the first two take about ten minutes:
- Install a runner. Ollama is the quickest (one command, runs as a local server); LM Studio and Jan are friendly graphical apps. All three are built on
llama.cppand load GGUF files. - Check what fits, then pull the model. Use the calculator above to find the largest model your hardware runs well, then download that model at the quantisation it recommends (e.g.
ollama pull qwen3:8b). - Match context to your memory. Long context inflates the KV cache; if you’re tight on memory, run a shorter context or a smaller quant. The calculator shows the trade-off live.
Local model vs. a cloud API — an honest comparison
| Local (your hardware) | Cloud API (GPT/Claude/Gemini) | |
|---|---|---|
| Privacy | Total — nothing leaves your machine | Prompts sent to a third party |
| Cost model | Upfront hardware, then free to run | Pay per token, forever |
| Top quality | Strong, but below the frontier on consumer gear | Highest available |
| Rate limits | None | Yes (quota / throttling) |
| Offline | Works fully offline | Needs internet |
| Stability | Frozen — you control upgrades | Changes under you |
Most serious local users run a hybrid: a local model for the bulk, private and high-volume work, and a frontier API for the hardest one-off questions.
What to expect at each model size
Capability scales with size, and so does the hardware bill. Roughly what you get:
| Size | What it’s good for | Reality check |
|---|---|---|
| 1–4B | Autocomplete, classification, simple chat, on-device | Fast everywhere; makes mistakes a bigger model wouldn’t |
| 7–14B | A genuinely useful daily assistant, RAG, drafting | The sweet spot for an 8–12 GB card; the most popular tier |
| 24–32B | Strong coding and reasoning; the best single-GPU experience | Wants a 24 GB card or a 32 GB+ Mac |
| 70B | Closest to hosted-model feel for general use | Needs ~40 GB+ — dual GPUs or a 64 GB Mac |
| 100B+ MoE | Frontier-adjacent quality, fast for its size | High-memory Macs or multi-GPU servers only |
Browse the full model database, see what your hardware runs, or jump to the best models for your use case.
Frequently asked questions
How do I know if my GPU can run a specific LLM?
A model fits if its weights, KV cache and overhead fit in your available memory. As a rule of thumb, the weights need about (parameters in billions) × 0.6 GB at the common Q4 quantisation — so an 8B model needs ~5 GB and a 70B needs ~40 GB. Intecca computes this for your exact card, the right quantisation and your chosen context length.
What is quantisation and which one should I use?
Quantisation shrinks a model by storing its weights at lower precision. Q4_K_M (about 4.8 bits per weight) is the default sweet spot — roughly a quarter the size of full precision with little quality loss. Use Q5 or Q6 if they fit, and only drop to Q3/Q2 to squeeze a larger model onto limited memory.
Do I need an NVIDIA GPU, or can I use a Mac?
Both work. Apple-silicon Macs share one pool of fast unified memory, so a 64 GB Mac can run models that need two NVIDIA cards — though raw speed still favours high-bandwidth GPUs. Intecca models Apple unified memory, NVIDIA/AMD VRAM and CPU-only RAM separately.
How much VRAM do I need to run a 70B model locally?
About 40 GB at Q4 for the weights plus a few GB for context — so a single 48 GB card, two 24 GB cards, or a 64 GB+ Apple-silicon Mac. It will not fit on one 24 GB consumer card at a useful quality.
Are the numbers exact?
They are careful estimates, not guarantees — real usage depends on your runtime (llama.cpp, Ollama, vLLM…), cache settings and OS overhead. We explain every assumption on the methodology page and keep the model list current as new releases land.
Is running an LLM locally free?
The models and the software are free and open. The cost is hardware: a GPU or a high-memory Mac. Once you own it, inference is unlimited and private with no per-token fees — which is exactly why heavy users and the privacy-conscious run locally even though a cloud API is cheaper to start.
Can I run an LLM without a GPU?
Yes — llama.cpp and Ollama run on CPU using system RAM. A modern desktop with 16–32 GB can run 7B–14B models, just slowly (a few tokens per second). Mixture-of-experts models like Qwen3 30B-A3B are the best CPU bet because only a few billion parameters are active per token.
What software do I use to run a local LLM?
The easiest starting points are Ollama (a one-command CLI/server) and LM Studio or Jan (graphical apps). All three sit on top of llama.cpp and use GGUF model files. Pick a model that fits your hardware here, then pull its GGUF in your tool of choice.
What is the best local LLM right now?
For most people on a single 24 GB card, Qwen3 32B is the strongest all-rounder; on 8–12 GB, Qwen3 8B or 14B; for coding, Qwen2.5-Coder 32B; for reasoning, the DeepSeek-R1 distills. The honest answer is "the best one that fits your hardware" — which is what the calculator above is for.