GPUs
VRAM decides which models fit; memory bandwidth decides how fast they generate.
| GPU | VRAM | Bandwidth | Board power | Largest model at Q4_K_M, 8K |
|---|---|---|---|---|
| RTX PRO 6000 BlackwellNVIDIA · Blackwell | 96 GB | 1,792 GB/s | 600 W | gpt-oss-120b |
| H100 PCIeNVIDIA · Hopper | 80 GB | 2,000 GB/s | 350 W | gpt-oss-120b |
| A100 80GBNVIDIA · Ampere | 80 GB | 1,935 GB/s | 300 W | gpt-oss-120b |
| RTX 6000 AdaNVIDIA · Ada Lovelace | 48 GB | 960 GB/s | 300 W | Llama 3.3 70B Instruct |
| Radeon PRO W7900AMD · RDNA 3 | 48 GB | 864 GB/s | 295 W | Llama 3.3 70B Instruct |
| RTX A6000NVIDIA · Ampere | 48 GB | 768 GB/s | 300 W | Llama 3.3 70B Instruct |
| RTX 5090NVIDIA · Blackwell | 32 GB | 1,792 GB/s | 575 W | Mixtral 8x7B Instruct |
| RTX 4090NVIDIA · Ada Lovelace | 24 GB | 1,008 GB/s | 450 W | Qwen3 32B |
| RTX 3090 TiNVIDIA · Ampere | 24 GB | 1,008 GB/s | 450 W | Qwen3 32B |
| RX 7900 XTXAMD · RDNA 3 | 24 GB | 960 GB/s | 355 W | Qwen3 32B |
| RTX 3090NVIDIA · Ampere | 24 GB | 936 GB/s | 350 W | Qwen3 32B |
| Tesla P40NVIDIA · Pascal | 24 GB | 346 GB/s | 250 W | Qwen3 32B |
| RX 7900 XTAMD · RDNA 3 | 20 GB | 800 GB/s | 315 W | Gemma 3 27B IT |
| RTX 5080NVIDIA · Blackwell | 16 GB | 960 GB/s | 360 W | gpt-oss-20b |
| RTX 5070 TiNVIDIA · Blackwell | 16 GB | 896 GB/s | 300 W | gpt-oss-20b |
| RTX 4080 SUPERNVIDIA · Ada Lovelace | 16 GB | 736 GB/s | 320 W | gpt-oss-20b |
| RTX 4080NVIDIA · Ada Lovelace | 16 GB | 716.8 GB/s | 320 W | gpt-oss-20b |
| RTX 4070 Ti SUPERNVIDIA · Ada Lovelace | 16 GB | 672 GB/s | 285 W | gpt-oss-20b |
| RX 9070 XTAMD · RDNA 4 | 16 GB | 640 GB/s | 304 W | gpt-oss-20b |
| RX 9070AMD · RDNA 4 | 16 GB | 640 GB/s | 220 W | gpt-oss-20b |
| RX 7800 XTAMD · RDNA 3 | 16 GB | 624 GB/s | 263 W | gpt-oss-20b |
| RX 7900 GREAMD · RDNA 3 | 16 GB | 576 GB/s | 260 W | gpt-oss-20b |
| Arc A770 16 GBIntel · Xe-HPG (Alchemist) | 16 GB | 560 GB/s | 225 W | gpt-oss-20b |
| RX 6800 XTAMD · RDNA 2 | 16 GB | 512 GB/s | 300 W | gpt-oss-20b |
| RTX 5060 Ti 16 GBNVIDIA · Blackwell | 16 GB | 448 GB/s | 180 W | gpt-oss-20b |
| RTX 4060 Ti 16 GBNVIDIA · Ada Lovelace | 16 GB | 288 GB/s | 165 W | gpt-oss-20b |
| RX 7600 XTAMD · RDNA 3 | 16 GB | 288 GB/s | 190 W | gpt-oss-20b |
| RTX 3080 TiNVIDIA · Ampere | 12 GB | 912 GB/s | 350 W | Qwen3 14B |
| RTX 5070NVIDIA · Blackwell | 12 GB | 672 GB/s | 250 W | Qwen3 14B |
| RTX 4070 TiNVIDIA · Ada Lovelace | 12 GB | 504 GB/s | 285 W | Qwen3 14B |
| RTX 4070 SUPERNVIDIA · Ada Lovelace | 12 GB | 504 GB/s | 220 W | Qwen3 14B |
| RTX 4070NVIDIA · Ada Lovelace | 12 GB | 504 GB/s | 200 W | Qwen3 14B |
| Arc B580Intel · Xe2 (Battlemage) | 12 GB | 456 GB/s | 190 W | Qwen3 14B |
| RTX 3060 12 GBNVIDIA · Ampere | 12 GB | 360 GB/s | 170 W | Qwen3 14B |
| RTX 2080 TiNVIDIA · Turing | 11 GB | 616 GB/s | 250 W | Mistral Nemo 12B Instruct |
| RTX 3080NVIDIA · Ampere | 10 GB | 760 GB/s | 320 W | Mistral Nemo 12B Instruct |
| RTX 5060 Ti 8 GBNVIDIA · Blackwell | 8 GB | 448 GB/s | 180 W | Qwen3 8B |
| RTX 5060NVIDIA · Blackwell | 8 GB | 448 GB/s | 145 W | Qwen3 8B |
| RTX 3070NVIDIA · Ampere | 8 GB | 448 GB/s | 220 W | Qwen3 8B |
| RTX 4060 Ti 8 GBNVIDIA · Ada Lovelace | 8 GB | 288 GB/s | 160 W | Qwen3 8B |
| RTX 4060NVIDIA · Ada Lovelace | 8 GB | 272 GB/s | 115 W | Qwen3 8B |
Official specData updated 24 days agoSource: NVIDIA, AMD, Intel