GeForce RTX 5090
NVIDIA
ExcellentRuns Qwen3.6 35B-A3B Q4_K_M entirely in memory
Model · Alibaba Qwen
Qwen3.6 35B-A3B: VRAM and hardware requirements
At Q4_K_M with an 8K context, Qwen3.6 35B-A3B needs about 22.8 GB: a GPU with 24 GB or more runs it entirely in VRAM.
- Parameters36B
- Active3B
- Max context256K
- LicenseApache-2.0
Specifications
- Organization
- Alibaba Qwen
- Architecture
- qwen3_5_moe
- Released
- April 2026
- Layers
- 40
- Hidden size
- 2048
- Attention heads
- 16
- KV heads
- 2
- Head dimension
- 256
- Vocabulary
- 248,320
- KV cache per 1K tokens (f16)
- 0.081 GB
- Repository
- Qwen/Qwen3.6-35B-A3B
Official specData updated todaySource: Hugging Face config.json + model card
Quantizations and memory
| Quantization | File size | Memory needed | Minimum GPU memory | Quality |
|---|---|---|---|---|
| BF16 | 66.2 GB | ~71.5 GB | 80 GB | Reference (unquantized) |
| Q8_0 | 35.2 GB | ~38.6 GB | 48 GB | Near-lossless |
| Q6_K | 28.8 GB | ~30.1 GB | 48 GB | Near-lossless |
| Q5_K_M | 24.1 GB | ~26.3 GB | 32 GB | Very good |
| Q4_K_MMost used | 20.8 GB | ~22.8 GB | 24 GB | Good — the usual choice |
| IQ4_XS | 18.3 GB | ~20.8 GB | 24 GB | Good — the usual choice |
| Q3_K_M | 15.9 GB | ~18.8 GB | 20 GB | Noticeable loss |
| Q2_K | 12.6 GB | ~15.2 GB | 16 GB | Significant loss |
File sizes: not available until imported from the model repository. Memory = weights (estimated from bits per weight) + KV cache + runtime overhead + safety margin. (BF16 ≈ 16 bpw, Q8_0 ≈ 8.5 bpw, Q6_K ≈ 6.57 bpw, Q5_K_M ≈ 5.7 bpw, Q4_K_M ≈ 4.9 bpw, IQ4_XS ≈ 4.46 bpw, Q3_K_M ≈ 4 bpw, Q2_K ≈ 3.17 bpw)
Q4, Q5 or Q8: how to choose a quantization
GPUs that run Qwen3.6 35B-A3B
| RTX 509032 GB · 64 GB RAM~23 GB · 188–343 tok/s estimate | Excellent |
|---|---|
| RTX 409024 GB · 64 GB RAM~23 GB · 151–254 tok/s estimate | Compatible with limits |
| RTX 309024 GB · 64 GB RAM~23 GB · 145–243 tok/s estimate | Compatible with limits |
| RX 7900 XTX24 GB · 64 GB RAM~23.1 GB · 98–181 tok/s estimate | Compatible with limits |
| RTX 5060 Ti 16 GB16 GB · 64 GB RAM~23 GB · 23–46 tok/s estimate | CPU/GPU offload |
| RTX 5070 Ti16 GB · 64 GB RAM~23 GB · 25–50 tok/s estimate | CPU/GPU offload |
| RTX 508016 GB · 64 GB RAM~23 GB · 25–50 tok/s estimate | CPU/GPU offload |
| RTX 4060 Ti 16 GB16 GB · 64 GB RAM~23 GB · 22–42 tok/s estimate | CPU/GPU offload |
| RTX 4070 Ti SUPER16 GB · 64 GB RAM~23 GB · 24–48 tok/s estimate | CPU/GPU offload |
| RX 9070 XT16 GB · 64 GB RAM~23.1 GB · 22–45 tok/s estimate | CPU/GPU offload |
| RTX 3060 12 GB12 GB · 64 GB RAM~23 GB · 17–33 tok/s estimate | CPU/GPU offload |
| RTX 407012 GB · 64 GB RAM~23 GB · 18–34 tok/s estimate | CPU/GPU offload |
| RTX 507012 GB · 64 GB RAM~23 GB · 18–35 tok/s estimate | CPU/GPU offload |
| RTX 40608 GB · 64 GB RAM~23 GB · 13–26 tok/s estimate | Possible but slow |
Macs that run Qwen3.6 35B-A3B
| M3 Ultra (80-core GPU) · 512 GBMac Studio~22.8 GB · 73–135 tok/s estimate | Excellent |
|---|---|
| M4 Max (40-core GPU) · 128 GBMacBook Pro 14, MacBook Pro 16~22.8 GB · 64–112 tok/s estimate | Excellent |
| M3 Max (40-core GPU) · 128 GBMacBook Pro 14, MacBook Pro 16~22.8 GB · 56–95 tok/s estimate | Excellent |
| M4 Pro (20-core GPU) · 64 GBMac mini, MacBook Pro 14~22.8 GB · 46–75 tok/s estimate | Excellent |
| M4 Max (32-core GPU) · 36 GBMacBook Pro 14, MacBook Pro 16~22.8 GB · 56–96 tok/s estimate | Compatible |
| M4 · 32 GBMacBook Air, MacBook Pro 14~22.8 GB · No data | Compatible with limits |
| M5 · 32 GBMacBook Pro 14~22.8 GB · No data | Compatible with limits |
| M3 · 24 GBMacBook Air, MacBook Pro 14~22.8 GB · No data | Incompatible |
EstimateData updated 40 days agoSource: Run LLM Locally engine
Limits in plain words
Qwen3.6 35B-A3B is a mixture-of-experts model: all 36B parameters must be in memory, but only 3B are read per token, so it generates much faster than a dense model of the same size.
The KV cache grows with context: at 32K it adds about 0.7 GB on top of the weights.
With under 8 GB of VRAM, use a smaller quantization or a smaller model of the same family.
Where to buy
Prices and links follow the market you choose.
Radeon PRO W7900
AMD
ExcellentRuns Qwen3.6 35B-A3B Q4_K_M entirely in memory
MacBook Pro 14 M3 Pro
Apple Silicon
CompatibleRuns Qwen3.6 35B-A3B Q4_K_M entirely in memory
AMD Ryzen AI Max+ 395
Unified memory PC
CompatibleRuns Qwen3.6 35B-A3B Q4_K_M entirely in memory
No current offer imported for this market: the link opens a marketplace search.
Frequently asked questions
How much VRAM does Qwen3.6 35B-A3B need?
About 22.8 GB at Q4_K_M with an 8K context, including the KV cache and runtime overhead. At Q8_0 it needs about 38.6 GB. A longer context adds more: about 0.7 GB of KV cache at 32K.
Can Qwen3.6 35B-A3B run on a Mac?
Yes on a Mac whose GPU can use at least 22.8 GB, e.g. M4 32 GB. macOS lets the GPU use about two thirds to three quarters of unified memory by default.
How fast is Qwen3.6 35B-A3B?
On a RTX 4090, we estimate 151–254 tok/s at Q4_K_M. This is an estimate from memory bandwidth (confidence low) until benchmarks are imported.