The defaults suit most uses. You can skip this section.
How much text the model keeps in memory. 8K is enough for chat; long documents need 32K or more. What is this?
Leave f16 if unsure. q8_0 roughly halves the conversation's memory, q4_0 nearly quarters it; the option must also be enabled in your app. What is this?
Used to estimate RAM speed, which only matters for the part of the model that does not fit on the graphics card.
