Local LLM configurator

Model

Quantization

Q4_K_M is the right trade-off for most uses. What is this?

Target speed

Around 10 tok/s reads comfortably; 30+ feels instant.

Platform

Advanced options8K tokens · KV cache f16

The defaults suit most uses. You can skip this section.

Context length

How much text the model keeps in memory. 8K is enough for chat; long documents need 32K or more. What is this?

KV cache type

Leave f16 if unsure. q8_0 roughly halves the conversation's memory, q4_0 nearly quarters it; the option must also be enabled in your app. What is this?

Compatible machines

What the model needs

Phi-4 Mini Instruct 3.8B Q4_K_M with a 8K context needs about 4.3 GB (up to 4.9 GB) of GPU or unified memory.

Compatible configurations for Phi-4 Mini Instruct 3.8B Q4_K_M · 8K context
	NVIDIARTX 4060Excellent GeForce RTX 4060 · 8 GB GDDR6 272 GB/s System RAM 64 GB Smallest configuration of this platform that runs the model entirely in fast memory.	AMDRX 7600 XTExcellent Radeon RX 7600 XT · 16 GB GDDR6 288 GB/s System RAM 64 GB Smallest configuration of this platform that runs the model entirely in fast memory.	Apple SiliconM1 · 8 GBExcellent MacBook Air, MacBook Pro 13, Mac mini, iMac 8 GPU · 68.3 GB/s Smallest configuration of this platform that runs the model entirely in fast memory.	Unified memory PCAMD Ryzen AI Max+ 395 · 32 GBExcellent Radeon 8060S 256 GB/s Smallest configuration of this platform that runs the model entirely in fast memory.	Highest estimated speedH100 PCIeExcellent H100 PCIe 80GB · 80 GB HBM2e 2,000 GB/s System RAM 64 GB Highest estimated generation speed among configurations that fit.
Memory for the model	8 GB	16 GB	5 GB	24 GB	78 GBMost memory
Estimated speed	61–84 tok/sestimate	55–87 tok/sestimate	15–21 tok/sestimate	49–78 tok/sestimate	258–460 tok/sestimateFastest
Power draw (GPU/board)	~115 WLowest power	~190 W	–	–	~350 W
Price	No current offer	No current offer	No current offer	No current offer	No current offer
Confidence	low	low	low	low	low

Leaders are marked per criterion; there is no overall winner. Speeds are estimates unless marked measured. Power is the GPU's rated board power (or the platform's, when known).

Suggested PC build

GPU: GeForce RTX 4060
Processor: Ryzen 7 9700X
RAM: 32 GB DDR5
Power supply: 350 W

The processor matters little when the model runs on the GPU; any current 8-core CPU is enough. PSU sized with 30% headroom for power spikes.

Compatibility checks: No compatibility problem detected between these parts.

Where to buy

Your market

Prices and links follow the market you choose.

GeForce RTX 4060

GeForce RTX 4060 · 8 GB GDDR6 · 272 GB/s

ExcellentRuns Phi-4 Mini Instruct 3.8B Q4_K_M entirely in memory

Price not available

Availability unknown

View on amazon.com

Affiliate link

Radeon RX 7600 XT

Radeon RX 7600 XT · 16 GB GDDR6 · 288 GB/s

ExcellentRuns Phi-4 Mini Instruct 3.8B Q4_K_M entirely in memory

Price not available

Availability unknown

View on amazon.com

Affiliate link

AMD Ryzen AI Max+ 395

Radeon 8060S · 256 GB/s

ExcellentRuns Phi-4 Mini Instruct 3.8B Q4_K_M entirely in memory

Price not available

Availability unknown

View on amazon.com

Affiliate link

No current offer imported for this market: the link opens a marketplace search.