DeepSeek Releases DeepSeek-V4.1-Flash: 552B Multimodal MoE with Causal Encoder-Decoder Architecture and 1M Context

DeepSeek AI has officially released the open weights and technical report for DeepSeek-V4.1-Flash, setting a transformative benchmark for long-context inference and multi-agent systems.

The Causal Encoder-Decoder (CED) Breakthrough

Conventional Transformers suffer from quadratic KV cache explosion as context lengths expand to hundreds of thousands of tokens. DeepSeek-V4.1-Flash bypasses this bottleneck with a 40-layer Causal Encoder-Decoder (CED) architecture:

• 20-layer Causal Encoder: Compresses and pre-processes input prompts.
• 20-layer Autoregressive Decoder: Projects global KV cache directly from final encoder hidden states rather than recalculating across all decoder layers.

As a result, DeepSeek-V4.1-Flash activates only 8B parameters per token during prefill and 16B parameters during decode, while possessing a massive 552-billion parameter backbone.

SWA Bounded Replay & Native Multimodal Support

In addition to CED, DeepSeek introduced Sliding Window Attention (SWA) Bounded Replay. By replaying only the most recent n-window tokens dynamically during generation, the persistent KV cache footprint on SSDs and RAM is cut by over 75%, allowing full 1,000,000-token context inference on standard enterprise GPU clusters.

DeepSeek-V4.1-Flash is natively multimodal (text and image comprehension) and is published under the fully permissive MIT license on Hugging Face.