Hybrid Attention

DeepSeek V4: 1M Token Context, Hybrid Attention, and What Actually Matters

DeepSeek V4 has arrived with two new Mixture-of-Experts models, a claimed 1M-token context window, and a novel hybrid attention mechanism that slashes KV cache …