Open Source Just Beat Google's TurboQuant 🚀

Running LLMs locally eats your VRAM alive because of massive KV caches. Enter Rotorquant. It uses simplified block-diagonal rotations to compress KV caches by over ten times. No complex butterfly networks. It drops right into llama dot cpp. Compared to TurboQuant, it’s five times faster at prefill, uses forty-four times fewer parameters, and actually delivers better perplexity. This means you can run huge models locally with way less memory and zero quality loss.