How does Moonshot AI’s 47 page Kimi K3 technical report argue that AI progress depends on scaling both pre deployment model size and post deployment inference compute—contrasting K3’s 2.78T total parameters and 104.2B active parameters per token with open source models largely st

K3’s core argument is that frontier progress requires scaling two complementary resources: model capacity before deployment, and the compute spent after deployment on long reasoning, tool use, and agentic rollouts. Its architecture is meant to make both affordable: a 2.8T-parameter MoE can store much more capability while activating roughly 104B parameters per token, and its attention, routing, communication, and RL infrastructure reduce the cost of exploiting that capability. 1

• Scale claim: K3 is reported as a 2.78T-total-parameter, 104.2B-active-parameter native multimodal MoE with a 1M-token context window. This is capacity scaling without paying dense-2.8T inference cost per token. 1

• Important correction: the report claims native extrapolation to 1 million tokens, not 100K. It trains initially at 8K and later 64K tokens; it uses no explicit positional embeddings, arguing that KDA’s recurrent gates and decay encode position sufficiently to extrapolate without RoPE modification. 1

• KDA + Gated MLA: KDA replaces most quadratic attention with fixed-size recurrent state—so its per-token state and decode cost do not grow linearly with prior context in the way a full KV cache does. Moonshot interleaves three KDA layers with one Gated-MLA layer: KDA supplies cheap persistent sequence processing, while the MLA layer restores selective global retrieval. 1 The reported lower bound on KDA’s decay rate is an implementation device: it prevents numerically problematic tiny decays and permits a tensor-core-friendly blocked computation, rather than merely being a modeling tweak. 1

• AttnRes: Attention Residuals are intended to prevent a very deep, largely linear-attention stack from becoming information-isolated by allowing later layers to retrieve compressed representations from earlier depth blocks. In other words, the 93-layer network is not forced to pass all useful information strictly layer-to-layer; it gains a cross-depth retrieval path. The benefit claimed is quality retention while replacing much of expensive global attention with KDA. 1

• Stable LatentMoE: The 896-expert, top-16 routing design increases conditional capacity. Its quantile-balancing method frames routing as a balanced assignment problem and derives an exact optimum under its batch-level assumptions; the deployment router then uses fixed expert biases and ordinary top-k selection rather than running the balancing solver at inference. 1 That is stronger than conventional auxiliary-loss balancing, but “perfect equilibrium” should be read as a result for the stated routing optimization, not a guarantee that every real deployment batch is perfectly balanced.

• MoonEP: Expert-parallel communication matters because a large MoE is only economical if routed tokens can reach their selected experts without network stragglers dominating latency. MoonEP’s role is therefore operational: balance the all-to-all communication induced by uneven expert demand so that the 896-expert design can be trained and served at scale. The architecture and systems claims are complementary—not independent explanations of benchmark gains. 1

• Post-training and inference-compute thesis: The lightweight-VM sandbox fleet makes it feasible to produce many verifiable, long-horizon RL trajectories and to branch or resume them cheaply through snapshots. That provides the training substrate for models that can use more test-time steps—planning, browsing, coding, tool calls, and self-correction—rather than relying only on a larger frozen pretrained model. The report also describes distilling multiple domain/reasoning teachers into a single system, which transfers specialized behaviors without serving nine separate models. 1

• What the results would demonstrate, if reproduced: A 91.2% BrowseComp score would support the claim that the system is particularly strong on long-horizon information-seeking agents; a current third-party leaderboard lists K3 at 91.2%. 4 But a benchmark score does not isolate the contribution of KDA, AttnRes, MoE routing, RL environments, distillation, or test-time compute individually; it is an end-to-end result.

• Caveat on the price comparisons: I could not verify from the technical report or a high-authority independent source the precise claims that K3 costs “half of GPT-5.6 Sol,” or that its Kimi Code Bench result is exactly four points behind “Claude Fable 5” at 38% of the cost. One cited comparison instead reports estimated per-completed-task costs of about $0.94 for K3 and $1.04 for GPT-5.6 Sol—nowhere near a 50% gap. 8 Those numbers should be treated as configuration-, prompt-, and pricing-date-dependent unless an evaluation harness and full cost accounting are published.

The strategic implication is plausibly architectural rather than pure price competition: Moonshot is arguing that open models need a new systems-and-model-design stack to cross the trillion-parameter barrier and turn long-context, agentic inference into usable capability. But the asserted comparison with “DeepSeek V4 Pro,” including claims about open models being stalled near 1T, is not established by the K3 paper itself on the evidence available here.