RTX 4090 vs RTX 5090 for local LLMs
VRAM decides which models fit entirely on the GPU; bandwidth decides how fast they generate. Here is how the two cards compare.
Which one for local LLMs?
The RTX 5090 has more VRAM (32 GB vs 24 GB), so it runs larger models entirely on the GPU.
The RTX 5090 has the higher memory bandwidth (1,792 GB/s vs 1,008 GB/s), so it should generate faster when a model fits on both.
RTX 4090 · Specifications
- VRAM
- 24 GB GDDR6X
- Bandwidth
- 1,008 GB/s
- Board power
- 450 W
- Architecture
- Ada Lovelace
- Largest model fully in VRAM
- Qwen3 32B
Official specData updated 38 days agoSource: NVIDIA specifications
RTX 5090 · Specifications
- VRAM
- 32 GB GDDR7
- Bandwidth
- 1,792 GB/s
- Board power
- 575 W
- Architecture
- Blackwell
- Largest model fully in VRAM
- Mixtral 8x7B Instruct
Official specData updated 38 days agoSource: NVIDIA specifications
| Model | RTX 4090 · 24 GB | RTX 5090 · 32 GB |
|---|---|---|
| Qwen3 32B | Compatible33–45 tok/s | Excellent53–77 tok/s |
| gpt-oss-20b | Excellent212–243 tok/s | Excellent264–291 tok/s |
| Qwen3 30B-A3B | Excellent125–209 tok/s | Excellent156–284 tok/s |
| Llama 3.1 8B Instruct | Excellent112–163 tok/s | Excellent171–269 tok/s |
| Qwen3 14B | Excellent68–95 tok/s | Excellent107–160 tok/s |
| Qwen3 8B | Excellent107–157 tok/s | Excellent161–256 tok/s |
| Llama 3.3 70B Instruct | Not recommended0.9–1.7 tok/s | Possible but slow1.4–2.5 tok/s |
| Gemma 3 27B IT | Excellent39–54 tok/s | Excellent62–92 tok/s |
For each model, the highest-precision quantization that runs entirely in fast memory; otherwise the one with the best result. Speeds are ranges; "estimate" unless a benchmark exists for this exact hardware.
EstimateData updated 38 days agoSource: Run LLM Locally engine