Skip to content
Run LLM Locally

Local LLM configurator

Q4_K_M is the right trade-off for most uses. What is this?

Around 10 tok/s reads comfortably; 30+ feels instant.

Advanced options8K tokens · KV cache f16

The defaults suit most uses. You can skip this section.

How much text the model keeps in memory. 8K is enough for chat; long documents need 32K or more. What is this?

Leave f16 if unsure. q8_0 roughly halves the conversation's memory, q4_0 nearly quarters it; the option must also be enabled in your app. What is this?

Compatible machines

What the model needs

Qwen3 235B-A22B Q4_K_M with a 8K context needs about 143 GB (up to 152 GB) of GPU or unified memory.

Compatible configurations for Qwen3 235B-A22B Q4_K_M · 8K context
NVIDIA2 × RTX PRO 6000 BlackwellExcellent
  • RTX PRO 6000 Blackwell Workstation Edition · 96 GB GDDR7 ECC
  • 1,792 GB/s
  • System RAM 64 GB

Smallest configuration of this platform that runs the model entirely in fast memory.

Apple SiliconM3 Ultra (80-core GPU) · 256 GBExcellent
  • Mac Studio
  • 80 GPU · 819 GB/s

Smallest configuration of this platform that runs the model entirely in fast memory.

Memory for the model186 GB192 GBMost memory
Estimated speed39–72 tok/sestimateFastest20–32 tok/sestimate
Power draw (GPU/board)~1200 W–
PriceNo current offerNo current offer
Confidencelowlow

Leaders are marked per criterion; there is no overall winner. Speeds are estimates unless marked measured. Power is the GPU's rated board power (or the platform's, when known).

Suggested PC build

GPU
2 × RTX PRO 6000 Blackwell Workstation Edition
Processor
Ryzen 7 9700X
RAM
192 GB DDR5
Power supply
1750 W

The processor matters little when the model runs on the GPU; any current 8-core CPU is enough. PSU sized with 30% headroom for power spikes.

Compatibility checks: 2 GPUs on a desktop platform share PCIe lanes (x8/x8 or less); fine for inference.

Where to buy

Prices and links follow the market you choose.

RTX PRO 6000 Blackwell Workstation Edition

RTX PRO 6000 Blackwell Workstation Edition · 96 GB GDDR7 ECC · 1,792 GB/s

ExcellentRuns Qwen3 235B-A22B Q4_K_M entirely in memory

Price not available

Availability unknown

View on amazon.com

Affiliate link

Mac Studio M3 Ultra

Mac Studio · 80 GPU · 819 GB/s

ExcellentRuns Qwen3 235B-A22B Q4_K_M entirely in memory

Price not available

Availability unknown

View on amazon.com

Affiliate link

No current offer imported for this market: the link opens a marketplace search.