DeepSeek Releases DeepSeek-V4.1-Flash: 552B Multimodal MoE with Causal Encoder-Decoder Architecture and 1M Context

DeepSeek AI has officially released the open weights and technical report for **DeepSeek-V4.1-Flash**, setting a transformative benchmark for long-context inference and multi-agent systems.

The Causal Encoder-Decoder (CED) Breakthrough Conventional Transformers suffer from quadratic KV cache explosion as context lengths expand to hundreds of thousands of tokens. DeepSeek-V4.1-Flash bypasses this bottleneck with a 40-layer **Causal Encoder-Decoder (CED)** architecture: - **20-layer Causal Encoder**: Compresses and pre-processes input prompts. - **20-layer Autoregressive Decoder**: Projects global KV cache directly from final encoder hidden states rather than recalculating across all decoder layers.

As a result, DeepSeek-V4.1-Flash activates only **8B parameters per token during prefill** and **16B parameters during decode**, while possessing a massive **552-billion parameter backbone**.

SWA Bounded Replay & Native Multimodal Support In addition to CED, DeepSeek introduced **Sliding Window Attention (SWA) Bounded Replay**. By replaying only the most recent *n*-window tokens dynamically during generation, the persistent KV cache footprint on SSDs and RAM is cut by over 75%, allowing full **1,000,000-token context inference** on standard enterprise GPU clusters.

DeepSeek-V4.1-Flash is natively multimodal (text and image comprehension) and is published under the fully permissive **MIT license** on Hugging Face.