12 GB vs 16 GB vs 24 GB of VRAM for local LLMs
What each VRAM tier actually runs, from 8 GB to 32 GB, with the largest model that fits entirely on the GPU.
VRAM decides how big a model fits on the GPU; bandwidth decides how fast it runs. Between 12 and 24 GB you move from 8–14B models to 27–32B models, a clear jump in quality.
The table below gives, for common cards, the largest model in our catalogue that runs entirely in VRAM at its default quantization with an 8K context.
| GPU | VRAM | Largest model at Q4_K_M, 8K |
|---|---|---|
| RTX 4060 | 8 GB | Qwen3 8B (Q4_K_M) |
| RTX 3060 12 GB | 12 GB | Qwen3 14B (Q4_K_M) |
| RTX 4070 | 12 GB | Qwen3 14B (Q4_K_M) |
| RTX 4060 Ti 16 GB | 16 GB | gpt-oss-20b (MXFP4) |
| RTX 5070 Ti | 16 GB | gpt-oss-20b (MXFP4) |
| RTX 3090 | 24 GB | Qwen3 32B (Q4_K_M) |
| RTX 4090 | 24 GB | Qwen3 32B (Q4_K_M) |
| RTX 5090 | 32 GB | Mixtral 8x7B Instruct (Q4_K_M) |
A used RTX 3090 has the same 24 GB as an RTX 4090: for model size they are equivalent; the 4090 generates a little faster (1008 vs 936 GB/s). Mixture-of-experts models such as Qwen3 30B-A3B or gpt-oss-20b change the picture: they read few weights per token and stay fast even when partly offloaded.
EstimateData updated 24 days agoSource: Run LLM Locally engine