Skip to content
Run LLM Locally
New to this?Read our local LLM guide for beginners

Find the LLM that fits your machine

Enter your GPU or Mac to see which models run, with the memory they need and an estimated speed. Or pick a model and see what hardware runs it.

Advanced options8K tokens · KV cache f16

The defaults suit most uses. You can skip this section.

How much text the model keeps in memory. 8K is enough for chat; long documents need 32K or more. What is this?

Leave f16 if unsure. q8_0 roughly halves the conversation's memory, q4_0 nearly quarters it; the option must also be enabled in your app. What is this?

Used to estimate RAM speed, which only matters for the part of the model that does not fit on the graphics card.

Browse modelsBrowse GPUsBrowse Macs