LLM Engineering •
How to Compare LLM Architectures: 16 Models, One File Each
Compare LLM architectures fast: GQA cuts KV cache 8x on Llama 3 70B, MLA cuts it 93.3% on DeepSeek-V2. One PyTorch repo shows why, file by file.
8 min read
Read more →Compare LLM architectures fast: GQA cuts KV cache 8x on Llama 3 70B, MLA cuts it 93.3% on DeepSeek-V2. One PyTorch repo shows why, file by file.
Loop transformer blocks or add more layers? Reusing weights cuts training compute 6.8-18% at equal loss, but forward passes and KV cache never shrink.