Models
Memory at the default quantization (Q4_K_M, or the native format) with an 8K context. Open a model to see every quantization and compatible hardware.
| Model | Family | Parameters | Max context | Memory (Q4_K_M / MXFP4, 8K) |
|---|---|---|---|---|
| DeepSeek R1 Distill Llama 8BDeepSeek | DeepSeek R1 Distill | 8.03B | 128K | ~6.7 GB |
| DeepSeek R1 Distill Qwen 14BDeepSeek | DeepSeek R1 Distill | 14.8B | 128K | ~11.3 GB |
| DeepSeek R1 Distill Qwen 32BDeepSeek | DeepSeek R1 Distill | 32.8B | 128K | ~22.6 GB |
| DeepSeek R1 Distill Llama 70BDeepSeek | DeepSeek R1 Distill | 70.6B | 128K | ~45.8 GB |
| DeepSeek R1 (671B)DeepSeek | DeepSeek V3 | 671B37B active | 160K | ~403 GB |
| Gemma 3 4B ITGoogle | Gemma | 4.3B | 128K | ~4 GB |
| Gemma 2 9B ITGoogle | Gemma | 9.24B | 8K | ~8.6 GB |
| Gemma 3 12B ITGoogle | Gemma | 12.2B | 128K | ~9.2 GB |
| Gemma 2 27B ITGoogle | Gemma | 27.2B | 8K | ~19.6 GB |
| Gemma 3 27B ITGoogle | Gemma | 27.4B | 128K | ~18.6 GB |
| gpt-oss-20bOpenAI | gpt-oss | 20.9B3.6B active | 128K | ~13.6 GB |
| gpt-oss-120bOpenAI | gpt-oss | 116.8B5.1B active | 128K | ~71.2 GB |
| Llama 2 7BMeta | Llama 2 | 6.74B | 4K | ~8.6 GB |
| Llama 3.2 1B InstructMeta | Llama 3 | 1.24B | 128K | ~2 GB |
| Llama 3.2 3B InstructMeta | Llama 3 | 3.21B | 128K | ~3.7 GB |
| Llama 3.1 8B InstructMeta | Llama 3 | 8.03B | 128K | ~6.7 GB |
| Llama 3.3 70B InstructMeta | Llama 3 | 70.6B | 128K | ~45.8 GB |
| Llama 4 Scout 17B-16E InstructMeta | Llama 4 | 109B17B active | 10M | ~67.8 GB |
| Mistral 7B Instruct v0.3Mistral AI | Mistral | 7.25B | 32K | ~6 GB |
| Mistral Nemo 12B InstructMistral AI | Mistral | 12.2B | 128K | ~9.4 GB |
| Mistral Small 3.2 24B InstructMistral AI | Mistral | 24B | 128K | ~16.5 GB |
| Mixtral 8x7B InstructMistral AI | Mistral | 46.7B12.9B active | 32K | ~29.6 GB |
| Phi-4 Mini Instruct 3.8BMicrosoft | Phi | 3.8B | 128K | ~4.3 GB |
| Phi-4 14BMicrosoft | Phi | 14.7B | 16K | ~11.2 GB |
| Qwen2.5 7B InstructAlibaba Qwen | Qwen 2.5 | 7.61B | 128K | ~5.9 GB |
| Qwen2.5 14B InstructAlibaba Qwen | Qwen 2.5 | 14.7B | 128K | ~11.2 GB |
| Qwen2.5 32B InstructAlibaba Qwen | Qwen 2.5 | 32.5B | 128K | ~22.4 GB |
| Qwen2.5 Coder 32B InstructAlibaba Qwen | Qwen 2.5 | 32.5B | 128K | ~22.4 GB |
| QwQ 32BAlibaba Qwen | Qwen 2.5 | 32.5B | 128K | ~22.4 GB |
| Qwen2.5 72B InstructAlibaba Qwen | Qwen 2.5 | 72.7B | 128K | ~47.1 GB |
| Qwen3 0.6BAlibaba Qwen | Qwen 3 | 0.75B | 32K | ~2.4 GB |
| Qwen3 1.7BAlibaba Qwen | Qwen 3 | 2.03B | 32K | ~3.1 GB |
| Qwen3 4BAlibaba Qwen | Qwen 3 | 4B | 128K | ~4.5 GB |
| Qwen3 8BAlibaba Qwen | Qwen 3 | 8.2B | 128K | ~6.9 GB |
| Qwen3 14BAlibaba Qwen | Qwen 3 | 14.8B | 128K | ~11 GB |
| Qwen3 30B-A3BAlibaba Qwen | Qwen 3 | 30.5B3.3B active | 128K | ~19.9 GB |
| Qwen3 Coder 30B-A3B InstructAlibaba Qwen | Qwen 3 | 30.5B3.3B active | 256K | ~19.9 GB |
| Qwen3 32BAlibaba Qwen | Qwen 3 | 32.8B | 128K | ~22.6 GB |
| Qwen3 235B-A22BAlibaba Qwen | Qwen 3 | 235B22B active | 128K | ~143 GB |
EstimateData updated 24 days agoSource: Run LLM Locally engine