Llamacpp on Rost Glukhov | Personal site and technical blog https://www.glukhov.org/tags/llamacpp/ Recent content in Llamacpp on Rost Glukhov | Personal site and technical blog Hugo en-us Tue, 08 Sep 2026 00:00:00 +0000 KV Cache on 16 GB GPUs: Making Long Context Actually Fit https://www.glukhov.org/llm-performance/optimization/kv-cache-16gb-long-context/ Tue, 08 Sep 2026 00:00:00 +0000 https://www.glukhov.org/llm-performance/optimization/kv-cache-16gb-long-context/ <p>A model can advertise a 128K context window and still fail at 40K tokens on a 16 GB GPU. The architecture ceiling never promised that weights, KV cache, compute buffers, and the desktop compositor would fit on your card at the same time.</p>