KV Cache

2 articles

LLM Engineering • Sep 20, 2026

How to Fix Multi-Agent LLM Latency: Text Handoffs Cost 2.5x

Multi-agent LLM latency isn't random: an ICLR 2026 paper measured the cause — text handoffs cost 2.5x more time and 3-5% less accuracy than skipping text.

11 min read

LLM Engineering • Aug 15, 2026

Qwen3.8 27B VRAM: how to fit 262K context in 16 GiB, not 64

Qwen3.8 27B VRAM math: 25.9 GiB of FP8 weights plus 16 GiB of KV cache at 262,144 tokens, not 64. The arithmetic, and where a 48 GB card breaks.

10 min read