How Xiaomi’s HySparse2 Cuts Long-Context Prefill Without Giving Up Retrieval

Long-running agents often add large tool outputs to an already substantial history. HySparse2 is Xiaomi’s proposed architecture for making that growing context less expensive to process while retaining access to both recent and distant tokens. The released work is architecture research associated with plans for MiMo-V3, not evidence of a new MiMo-V3 open-weight release. 34

Why prefill can stop after the self-decoder

HySparse2 divides the model into a YOCO-style self-decoder and cross-decoder. The self-decoder processes the long input. Through KV Bridging, full-attention layers in the cross-decoder construct their keys and values from self-decoder hidden states. Because the cross-decoder’s KV data comes from those states, prefill of the existing long context can finish after the self-decoder rather than running that context through the cross-decoder too. The cross-decoder still participates when the model generates new tokens. 4615

Within each cross-decoder hybrid block, KV Reuse lets sparse layers share the preceding full-attention layer’s KV cache and token-selection indices. HySparse2 selects individual tokens rather than blocks. It also forces recent tokens into the sparse selection, replacing a separate sliding-window-attention branch so local and selected distant tokens can use the same cache. Together, these choices target repeated cache storage and selection work without excluding nearby context. 4615

What the comparisons and ablation show

For a reported matched MoE configuration with 80B total parameters and roughly 3B active per token, HySparse2’s prefill FLOPs at 1 million tokens are 5.02× lower than Hybrid SWA’s. The reported KV-cache sizes are 2.69 GB versus 12.09 GB, or about 4.5× smaller. Xiaomi also reports higher MRCRv2 and RULER-v2 retrieval scores and lower AgentPPL and LongPPL; a report of the evaluation says HySparse2 leads both HySparse and Hybrid SWA on the evaluated long-context retrieval tests. 615

One reported ablation holds the backbone and attention budget constant while changing block-level selection to token-level selection. On tests at 32k tokens or less, the report gives gains of 6.57 percentage points on RULER-v2, 8.14 points on two-needle MRCR-v2, and 5.55 points on GraphWalks. That supports the value of finer-grained selection in that setting; it does not isolate the contribution of KV Bridging, KV Reuse, or the forced local window. 6

The available evidence does not establish a numerical HySparse-versus-HySparse2 result at 1 million tokens, component-by-component gains for every mechanism, or whether the matched-model improvements persist at a larger model scale. Nor do the cited excerpts specify the precision behind the 2.69 GB cache figure, so it should not be presented as a verified FP8-specific measurement. Prefill FLOPs and cache size also should not be mistaken for measured production latency. 615