Skip to content
Run LLM Locally

Local LLM configurator

Q4_K_M is the right trade-off for most uses. What is this?

Around 10 tok/s reads comfortably; 30+ feels instant.

Advanced options8K tokens · KV cache f16

The defaults suit most uses. You can skip this section.

How much text the model keeps in memory. 8K is enough for chat; long documents need 32K or more. What is this?

Leave f16 if unsure. q8_0 roughly halves the conversation's memory, q4_0 nearly quarters it; the option must also be enabled in your app. What is this?

Compatible machines

What the model needs

DeepSeek R1 (671B) Q4_K_M with a 8K context needs about 403 GB (up to 428 GB) of GPU or unified memory.

Compatible configurations for DeepSeek R1 (671B) Q4_K_M · 8K context
Apple SiliconM3 Ultra (80-core GPU) · 512 GBCompatible with limits
  • Mac Studio
  • 80 GPU · 819 GB/s

Smallest configuration of this platform that runs the model entirely in fast memory.

Memory for the model384 GB
Estimated speedNo data
Power draw (GPU/board)–
PriceNo current offer
Confidencelow

Leaders are marked per criterion; there is no overall winner. Speeds are estimates unless marked measured. Power is the GPU's rated board power (or the platform's, when known).

Where to buy

Prices and links follow the market you choose.

Mac Studio M3 Ultra

Mac Studio · 80 GPU · 819 GB/s

Compatible with limitsRuns DeepSeek R1 (671B) Q4_K_M entirely in memory

Price not available

Availability unknown

View on amazon.com

Affiliate link

No current offer imported for this market: the link opens a marketplace search.