KV Cache

3 articles

AI Security • Nov 5, 2026

Fix a KV Cache Leak: 155 Responses Exposed Other Users' Data

Check your LLM server for a KV cache leak: compare pool size to context length and send a canary prompt. Cloudflare's GLM-5.3 bug exposed 155 responses.

9 min read

LLM Engineering • Sep 20, 2026

How to Fix Multi-Agent LLM Latency: Text Handoffs Cost 2.5x

Multi-agent LLM latency isn't random: an ICLR 2026 paper measured the cause — text handoffs cost 2.5x more time and 3-5% less accuracy than skipping text.

11 min read

LLM Engineering • Aug 15, 2026

Qwen3.8 27B VRAM: how to fit 262K context in 16 GiB, not 64

Qwen3.8 27B VRAM math: 25.9 GiB of FP8 weights plus 16 GiB of KV cache at 262,144 tokens, not 64. The arithmetic, and where a 48 GB card breaks.

10 min read