GeForce RTX 4060
NVIDIA
ExcellentRuns Qwen3 4B Q4_K_M entirely in memory
Model · Alibaba Qwen
Qwen3 4B: VRAM and hardware requirements
At Q4_K_M with an 8K context, Qwen3 4B needs about 4.5 GB: a GPU with 8 GB or more runs it entirely in VRAM.
- Parameters4B
- Max context128K
- LicenseApache-2.0
Specifications
- Organization
- Alibaba Qwen
- Architecture
- qwen3
- Released
- April 2025
- Layers
- 36
- Hidden size
- 2560
- Attention heads
- 32
- KV heads
- 8
- Head dimension
- 128
- Vocabulary
- 151,936
- KV cache per 1K tokens (f16)
- 0.141 GB
- Repository
- Qwen/Qwen3-4B
Official specData updated 24 days agoSource: Hugging Face config.json + model card
Quantizations and memory
| Quantization | File size | Memory needed | Minimum GPU memory | Quality |
|---|---|---|---|---|
| BF16 | 7.5 GB | ~9.8 GB | 11 GB | Reference (unquantized) |
| Q8_0 | 4 GB | ~6.2 GB | 8 GB | Near-lossless |
| Q6_K | 3.1 GB | ~5.3 GB | 8 GB | Near-lossless |
| Q5_K_M | 2.7 GB | ~4.9 GB | 8 GB | Very good |
| Q4_K_MMost used | 2.3 GB | ~4.5 GB | 8 GB | Good — the usual choice |
| IQ4_XS | 2.1 GB | ~4.3 GB | 8 GB | Good — the usual choice |
| Q3_K_M | 1.9 GB | ~4.1 GB | 8 GB | Noticeable loss |
| Q2_K | 1.6 GB | ~3.7 GB | 8 GB | Significant loss |
File sizes: not available until imported from the model repository. Memory = weights (estimated from bits per weight) + KV cache + runtime overhead + safety margin. (BF16 ≈ 16 bpw, Q8_0 ≈ 8.5 bpw, Q6_K ≈ 6.57 bpw, Q5_K_M ≈ 5.7 bpw, Q4_K_M ≈ 4.9 bpw, IQ4_XS ≈ 4.46 bpw, Q3_K_M ≈ 4 bpw, Q2_K ≈ 3.17 bpw)
Q4, Q5 or Q8: how to choose a quantization
GPUs that run Qwen3 4B
| RTX 509032 GB · 64 GB RAM~4.5 GB · 227–405 tok/s estimate | Excellent |
|---|---|
| RTX 409024 GB · 64 GB RAM~4.5 GB · 163–261 tok/s estimate | Excellent |
| RTX 309024 GB · 64 GB RAM~4.5 GB · 155–245 tok/s estimate | Excellent |
| RX 7900 XTX24 GB · 64 GB RAM~4.6 GB · 123–206 tok/s estimate | Excellent |
| RTX 5060 Ti 16 GB16 GB · 64 GB RAM~4.5 GB · 90–129 tok/s estimate | Excellent |
| RTX 5070 Ti16 GB · 64 GB RAM~4.5 GB · 151–237 tok/s estimate | Excellent |
| RTX 508016 GB · 64 GB RAM~4.5 GB · 158–251 tok/s estimate | Excellent |
| RTX 4060 Ti 16 GB16 GB · 64 GB RAM~4.5 GB · 62–86 tok/s estimate | Excellent |
| RTX 4070 Ti SUPER16 GB · 64 GB RAM~4.5 GB · 123–185 tok/s estimate | Excellent |
| RX 9070 XT16 GB · 64 GB RAM~4.6 GB · 96–152 tok/s estimate | Excellent |
| RTX 3060 12 GB12 GB · 64 GB RAM~4.5 GB · 75–106 tok/s estimate | Excellent |
| RTX 407012 GB · 64 GB RAM~4.5 GB · 99–144 tok/s estimate | Excellent |
| RTX 507012 GB · 64 GB RAM~4.5 GB · 123–185 tok/s estimate | Excellent |
| RTX 40608 GB · 64 GB RAM~4.5 GB · 59–82 tok/s estimate | Excellent |
Macs that run Qwen3 4B
| M3 Ultra (80-core GPU) · 512 GBMac Studio~4.3 GB · 108–244 tok/s estimate | Excellent |
|---|---|
| M4 Max (40-core GPU) · 128 GBMacBook Pro 14, MacBook Pro 16~4.3 GB · 82–163 tok/s estimate | Excellent |
| M3 Max (40-core GPU) · 128 GBMacBook Pro 14, MacBook Pro 16~4.3 GB · 66–119 tok/s estimate | Excellent |
| M4 Pro (20-core GPU) · 64 GBMac mini, MacBook Pro 14~4.3 GB · 48–81 tok/s estimate | Excellent |
| M4 Max (32-core GPU) · 36 GBMacBook Pro 14, MacBook Pro 16~4.3 GB · 67–122 tok/s estimate | Excellent |
| M4 · 32 GBMacBook Air, MacBook Pro 14~4.3 GB · 24–36 tok/s estimate | Excellent |
| M5 · 32 GBMacBook Pro 14~4.3 GB · 29–46 tok/s estimate | Excellent |
| M3 · 24 GBMacBook Air, MacBook Pro 14~4.3 GB · 20–30 tok/s estimate | Excellent |
EstimateData updated 24 days agoSource: Run LLM Locally engine
Limits in plain words
The KV cache grows with context: at 32K it adds about 4.5 GB on top of the weights.
Where to buy
Prices and links follow the market you choose.
Radeon RX 7600 XT
AMD
ExcellentRuns Qwen3 4B Q4_K_M entirely in memory
AMD Ryzen AI Max+ 395
Unified memory PC
ExcellentRuns Qwen3 4B Q4_K_M entirely in memory
No current offer imported for this market: the link opens a marketplace search.
Frequently asked questions
How much VRAM does Qwen3 4B need?
About 4.5 GB at Q4_K_M with an 8K context, including the KV cache and runtime overhead. At Q8_0 it needs about 6.2 GB. A longer context adds more: about 4.5 GB of KV cache at 32K.
Can Qwen3 4B run on a Mac?
Yes on a Mac whose GPU can use at least 4.3 GB, e.g. M3 24 GB. macOS lets the GPU use about two thirds to three quarters of unified memory by default.
How fast is Qwen3 4B?
On a RTX 4090, we estimate 163–261 tok/s at Q4_K_M. This is an estimate from memory bandwidth (confidence low) until benchmarks are imported.