GeForce RTX 3060 12 GB
12 GB GDDR6 · 360 GB/s · 170 W
RTX 3060 12 GB · 64 GB RAM → Qwen3 14B
GPU · NVIDIA
RTX 3060 12 GB for local LLMs: which models run?
With 12 GB of VRAM and 360 GB/s of bandwidth, the RTX 3060 12 GB runs models up to about Qwen3 14B entirely on the GPU.
- VRAM12 GB GDDR6
- Bandwidth360 GB/s
- Board power170 W
- ArchitectureAmpere
Specifications
- VRAM
- 12 GB
- Memory type
- GDDR6
- Memory bus
- 192-bit
- Bandwidth
- 360 GB/s
- Board power
- 170 W
- Backends
- CUDA, Vulkan
- Interface
- PCIe 4.0 x16
- Released
- February 2021
- Segment
- Consumer
- FP16 compute
- Not available
Official specData updated 24 days agoSource: NVIDIA specifications
What the RTX 3060 12 GB runs
| Qwen2.5 7B InstructQ8_0 · 8K~9.2 GB · 32–42 tok/s estimate | Excellent |
|---|---|
| Mistral 7B Instruct v0.3Q8_0 · 8K~9.2 GB · 32–42 tok/s estimate | Excellent |
| Gemma 3 4B ITQ8_0 · 8K~5.4 GB · 57–78 tok/s estimate | Excellent |
| Qwen3 4BQ8_0 · 8K~6.2 GB · 51–69 tok/s estimate | Excellent |
| Phi-4 Mini Instruct 3.8BQ8_0 · 8K~6 GB · 54–73 tok/s estimate | Excellent |
| Llama 3.2 3B InstructQ8_0 · 8K~5.1 GB · 64–87 tok/s estimate | Excellent |
| Qwen3 1.7BQ8_0 · 8K~3.7 GB · 100–142 tok/s estimate | Excellent |
| Llama 3.2 1B InstructQ8_0 · 8K~2.5 GB · 162–227 tok/s estimate | Excellent |
| Qwen3 0.6BQ8_0 · 8K~2.5 GB · 177–272 tok/s estimate | Excellent |
| Qwen3 14BQ4_K_M · 8K~11 GB · 27–36 tok/s estimate | Compatible |
| DeepSeek R1 Distill Qwen 14BIQ4_XS · 8K~10.4 GB · 29–38 tok/s estimate | Compatible |
| Qwen2.5 14B InstructIQ4_XS · 8K~10.4 GB · 29–38 tok/s estimate | Compatible |
| Phi-4 14BIQ4_XS · 8K~10.2 GB · 29–39 tok/s estimate | Compatible |
| Mistral Nemo 12B InstructQ5_K_M · 8K~10.7 GB · 28–37 tok/s estimate | Compatible |
| Gemma 3 12B ITQ5_K_M · 8K~10.2 GB · 29–39 tok/s estimate | Compatible |
| Gemma 2 9B ITQ6_K · 8K~10.5 GB · 30–40 tok/s estimate | Compatible |
| Qwen3 8BQ8_0 · 8K~10.5 GB · 28–37 tok/s estimate | Compatible |
| Llama 3.1 8B InstructQ8_0 · 8K~10.2 GB · 29–38 tok/s estimate | Compatible |
| DeepSeek R1 Distill Llama 8BQ8_0 · 8K~10.2 GB · 29–38 tok/s estimate | Compatible |
| Mixtral 8x7B InstructQ3_K_M · 8K~21.6 GB · 6–12 tok/s estimate | CPU/GPU offload |
| Qwen3 32BQ3_K_M · 8K~18.6 GB · 2.5–5 tok/s estimate | CPU/GPU offload |
| DeepSeek R1 Distill Qwen 32BQ3_K_M · 8K~18.5 GB · 2.5–5 tok/s estimate | CPU/GPU offload |
| Qwen2.5 32B InstructQ3_K_M · 8K~18.5 GB · 2.5–5 tok/s estimate | CPU/GPU offload |
| Qwen2.5 Coder 32B InstructQ3_K_M · 8K~18.5 GB · 2.5–5 tok/s estimate | CPU/GPU offload |
| QwQ 32BQ3_K_M · 8K~18.5 GB · 2.5–5 tok/s estimate | CPU/GPU offload |
| Qwen3 30B-A3BQ3_K_M · 8K~15.4 GB · 27–53 tok/s estimate | CPU/GPU offload |
| Qwen3 Coder 30B-A3B InstructQ3_K_M · 8K~16 GB · 25–48 tok/s estimate | CPU/GPU offload |
| Gemma 3 27B ITQ3_K_M · 8K~15.3 GB · 4.5–8.5 tok/s estimate | CPU/GPU offload |
| Gemma 2 27B ITQ3_K_M · 8K~16.5 GB · 4–7 tok/s estimate | CPU/GPU offload |
| Mistral Small 3.2 24B InstructQ3_K_M · 8K~13.3 GB · 7.5–14 tok/s estimate | CPU/GPU offload |
| gpt-oss-20bMXFP4 · 8K~13 GB · 41–75 tok/s estimate | CPU/GPU offload |
| Llama 4 Scout 17B-16E InstructQ3_K_M · 8K~55.7 GB · 2.5–5 tok/s estimate | Possible but slow |
| Qwen2.5 72B InstructQ3_K_M · 8K~38.2 GB · 0.7–1.4 tok/s estimate | Not recommended |
| Llama 3.3 70B InstructQ3_K_M · 8K~37 GB · 0.8–1.5 tok/s estimate | Not recommended |
| DeepSeek R1 Distill Llama 70BQ3_K_M · 8K~37 GB · 0.8–1.5 tok/s estimate | Not recommended |
| DeepSeek R1 (671B)Q8_0 · 8K~699 GB · No data | Incompatible |
| Qwen3 235B-A22BQ8_0 · 8K~247 GB · No data | Incompatible |
| gpt-oss-120bMXFP4 · 8K~63.2 GB · No data | Incompatible |
For each model, the highest-precision quantization that runs entirely in fast memory; otherwise the one with the best result. Speeds are ranges; "estimate" unless a benchmark exists for this exact hardware.
EstimateData updated 24 days agoSource: Run LLM Locally engine
Limits in plain words
12 GB of VRAM holds the weights and KV cache of models up to about Qwen3 14B at Q4_K_M. Bigger models spill into system RAM and slow down sharply: generation speed is then limited by RAM bandwidth, not by the GPU.
Where to buy a RTX 3060 12 GB
Prices and links follow the market you choose.
No current offer imported for this market: the link opens a marketplace search.
Frequently asked questions
Which LLMs can the RTX 3060 12 GB run?
Entirely on the GPU: Qwen3 14B (Q4_K_M), DeepSeek R1 Distill Qwen 14B (IQ4_XS), Qwen2.5 14B Instruct (IQ4_XS), Phi-4 14B (IQ4_XS), Mistral Nemo 12B Instruct (Q5_K_M), Gemma 3 12B IT (Q5_K_M), Gemma 2 9B IT (Q6_K), Qwen3 8B (Q8_0). Larger models need CPU/GPU offload.
Can the RTX 3060 12 GB run a 70B model?
Not recommended: a 70B model at Q4_K_M needs about 45.1 GB; the RTX 3060 12 GB has 12 GB.
How is the speed estimated?
Token generation reads every active weight once per token, so it is limited by memory bandwidth (360 GB/s here). We apply an efficiency range per backend and label the result as an estimate until real benchmarks exist.