Moonshot AI presents Kimi K3 as a case for two-axis scaling: build a much larger model before deployment, then give that model enough compute at deployment to reason, use tools, browse, code, and recover from mistakes. The central challenge is economic. A model cannot simply be made denser and deeper if its long-context serving cost makes those capabilities impractical. K3’s design is intended to increase total capacity while containing the compute and memory used for each generated token. 1

The core distinction: total capacity versus active compute

Kimi K3 is described as a native multimodal mixture-of-experts (MoE) model with 2.8 trillion total parameters, about 104 billion activated parameters per token, and a context window of up to one million tokens. 117

Those two parameter counts answer different questions:

• Total parameters indicate how much capacity exists across the full collection of experts.
• Activated parameters per token indicate the portion used for a particular token after routing.

That is the practical appeal of MoE: K3 can be far larger in aggregate than a dense model with a comparable per-token compute footprint. It does not make serving cheap—104B active parameters remains substantial—but it avoids paying the inference cost of a dense 2.8T-parameter network on every token. 1

Moonshot’s argument is therefore not simply “more parameters are better.” It is that conditional computation can make a new scale of model capacity usable for long-running tasks.

Why long-context inference needs a different attention stack

A million-token context is difficult not only because of training, but because conventional attention can accumulate costly memory and compute as the sequence grows. K3 uses a hybrid design that combines Kimi Delta Attention (KDA) with Gated Multi-head Latent Attention (Gated MLA). The reported backbone composition is 69 KDA layers and 24 Gated MLA layers, arranged primarily in a three-KDA-to-one-MLA pattern. 112

KDA handles persistent sequence processing

KDA is the model’s linear-attention component. Rather than retaining a conventional growing key-value cache across the whole history in every such layer, it uses a fixed-size recurrent state. The intended benefit is that memory state and decode work do not expand with prior context in the same way as full attention. 1

The report also describes a lower bound on KDA’s decay rate. This is not only a modeling preference: it prevents extremely small decay values that are problematic for efficient numerical computation and enables a blocked implementation suited to tensor-core execution. 1

Gated MLA restores global retrieval

Linear recurrence alone may not provide the same selective global lookup behavior as attention over a full context. K3 therefore retains Gated MLA layers throughout the network. In the reported hybrid, KDA supplies inexpensive persistent processing while MLA provides periodic global retrieval. 1

The design is notable because it treats long-context capability as a systems problem as well as an architecture problem. A large advertised context window matters most when the model can still retrieve relevant information and generate at usable cost.

AttnRes tries to preserve information through 93 layers

K3’s second architectural component is Attention Residuals (AttnRes). The model has 93 layers, and the report uses AttnRes to let later layers retrieve compressed representations from earlier depth blocks rather than forcing all information to travel strictly one layer at a time. 1

Conceptually, KDA addresses information flow across sequence length, while AttnRes addresses information flow across depth. That pairing is important in a deep model where much of the stack uses linear attention: replacing expensive global attention only helps if useful information is not lost or isolated as activations pass through the network. 1

Stable LatentMoE supplies conditional capacity

K3’s MoE layer is reported to use 896 routed experts, selecting 16 experts per token. 16 This is the mechanism behind the gap between total and active parameters.

The report’s Stable LatentMoE approach focuses on keeping this routing stable and balanced. It frames batch-level expert assignment as an optimization problem and introduces quantile-based balancing. The report derives an exact optimum under its stated assumptions; at deployment, however, routing uses fixed expert biases with ordinary top-k selection rather than solving the balancing problem afresh for every inference batch. 1

That distinction matters. A claimed optimal equilibrium is a result for the report’s routing formulation, not a guarantee that every real production workload will distribute tokens perfectly evenly across experts.

MoonEP makes sparse routing operationally viable

Large MoE models introduce a networking problem: selected tokens often need to be sent across devices to reach their assigned experts. If some experts receive disproportionate demand, the slowest communication path can determine latency.

Moonshot positions MoonEP as the complementary systems layer for expert-parallel communication. Its job is to reduce imbalance in the all-to-all exchanges created by sparse routing, helping the large expert pool remain trainable and serveable. 1

This is a key part of K3’s broader claim. Architecture, routing, and communication cannot be evaluated as independent explanations of performance; the model needs all three to turn theoretical sparse capacity into real throughput.

The post-deployment half of the strategy: compute for agentic work

The other axis in K3’s thesis is what happens after pretraining. Moonshot describes infrastructure for reinforcement-learning trajectories involving long-horizon tasks, tools, coding, browsing, and iterative problem solving. The report also describes distilling multiple domain and reasoning teachers into one system. 1

The supporting environment is a lightweight virtual-machine sandbox system designed to create, snapshot, and resume task environments at scale. Reported figures include 51.2 million sandbox instances, 133 ms snapshots, and 49 ms restores. 1

The strategic purpose is straightforward: if a model is meant to spend more test-time compute planning, calling tools, checking work, or continuing a long task, then the training setup must expose it to those kinds of trajectories. Cheaply pausing, branching, and restoring environments makes more such training rollouts feasible.

This does not mean a single model response automatically uses unlimited inference compute. Rather, K3’s report argues for a system that can convert additional steps and context into better task performance instead of relying solely on a larger pretrained model.

What the BrowseComp result supports—and what it cannot isolate

Kimi K3 is reported at 91.2% on BrowseComp, a benchmark aimed at long-horizon information seeking. A third-party leaderboard also lists K3 at 91.2%, although rankings and reporting configurations can change over time. 18

That result is consistent with the product goal: a model designed around long context, agentic post-training, and test-time reasoning should be evaluated on complex information-seeking tasks. But it is not a clean ablation of the individual components.

A single benchmark score cannot tell readers how much of the result comes from KDA, AttnRes, expert routing, communication infrastructure, RL environments, distillation, prompting, or inference-time settings. The strongest interpretation is that the reported system performs well as an integrated stack—not that one architectural innovation alone explains the score. 118

Why the DeepSeek comparison is mainly architectural

K3 is often framed as a challenge to other large open models, including DeepSeek systems. The defensible takeaway from the report is not a settled league table of open-model progress, but a different design emphasis: Moonshot is advocating a stack that combines very large sparse capacity, efficient long-context processing, and post-training infrastructure for agents. 117

The paper itself supports K3’s specifications and design choices. It does not, by itself, establish that other open models are “stalled,” nor does it isolate a decisive architectural advantage over a named competitor. Those are broader market claims that require consistent, independently reproduced comparisons.

Treat cost claims as configuration-dependent

Price-per-token and cost-per-task comparisons are especially easy to overread. They can vary with prompting, reasoning effort, context length, caching, tool usage, throughput assumptions, pricing date, and the task harness.

One published comparison cited in the available material estimates completed-task costs of approximately $0.94 for K3 and $1.04 for GPT-5.6 Sol. That may suggest a modest advantage in that specific setup, but it does not substantiate a universal “half the cost of Sol” claim. 20

The useful conclusion is less dramatic and more durable: K3 is an attempt to make long-context, agentic capability economically credible at a much larger total model scale. Whether it achieves that advantage in practice depends on reproducible evaluations and deployment-specific cost accounting.

Bottom line

Kimi K3’s technical report makes a full-stack scaling argument. Pre-deployment scale comes from a 2.8T-parameter MoE with about 104B parameters activated per token. Post-deployment scale comes from training and serving a model that can use long context, tools, reasoning steps, and agentic rollouts. 117

KDA, Gated MLA, AttnRes, Stable LatentMoE, MoonEP, and the RL sandbox infrastructure are all parts of the same proposition: frontier-quality agentic behavior needs not just a bigger model, but a design that makes more context and more inference-time work usable. The benchmark claims are promising evidence for that integrated approach, but they should be read as reported system-level results rather than final proof of each component’s contribution. 118

Moonshot AI presents Kimi K3 as a case for two-axis scaling: build a much larger model before deployment, then give that model enough compute at deployment to reason, use tools, browse, code, and recover from mistakes. The central challenge is economic. A model cannot simply be made denser and deeper if its long-context serving cost makes those capabilities impractical. K3’s design is intended to increase total capacity while containing the compute and memory used for each generated token. 1

The core distinction: total capacity versus active compute

Kimi K3 is described as a native multimodal mixture-of-experts (MoE) model with 2.8 trillion total parameters, about 104 billion activated parameters per token, and a context window of up to one million tokens. 117

Those two parameter counts answer different questions:

• Total parameters indicate how much capacity exists across the full collection of experts.
• Activated parameters per token indicate the portion used for a particular token after routing.

That is the practical appeal of MoE: K3 can be far larger in aggregate than a dense model with a comparable per-token compute footprint. It does not make serving cheap—104B active parameters remains substantial—but it avoids paying the inference cost of a dense 2.8T-parameter network on every token. 1

Moonshot’s argument is therefore not simply “more parameters are better.” It is that conditional computation can make a new scale of model capacity usable for long-running tasks.

Why long-context inference needs a different attention stack

A million-token context is difficult not only because of training, but because conventional attention can accumulate costly memory and compute as the sequence grows. K3 uses a hybrid design that combines Kimi Delta Attention (KDA) with Gated Multi-head Latent Attention (Gated MLA). The reported backbone composition is 69 KDA layers and 24 Gated MLA layers, arranged primarily in a three-KDA-to-one-MLA pattern. 112

KDA handles persistent sequence processing

KDA is the model’s linear-attention component. Rather than retaining a conventional growing key-value cache across the whole history in every such layer, it uses a fixed-size recurrent state. The intended benefit is that memory state and decode work do not expand with prior context in the same way as full attention. 1

The report also describes a lower bound on KDA’s decay rate. This is not only a modeling preference: it prevents extremely small decay values that are problematic for efficient numerical computation and enables a blocked implementation suited to tensor-core execution. 1

Gated MLA restores global retrieval

Linear recurrence alone may not provide the same selective global lookup behavior as attention over a full context. K3 therefore retains Gated MLA layers throughout the network. In the reported hybrid, KDA supplies inexpensive persistent processing while MLA provides periodic global retrieval. 1

The design is notable because it treats long-context capability as a systems problem as well as an architecture problem. A large advertised context window matters most when the model can still retrieve relevant information and generate at usable cost.

AttnRes tries to preserve information through 93 layers

K3’s second architectural component is Attention Residuals (AttnRes). The model has 93 layers, and the report uses AttnRes to let later layers retrieve compressed representations from earlier depth blocks rather than forcing all information to travel strictly one layer at a time. 1

Conceptually, KDA addresses information flow across sequence length, while AttnRes addresses information flow across depth. That pairing is important in a deep model where much of the stack uses linear attention: replacing expensive global attention only helps if useful information is not lost or isolated as activations pass through the network. 1

Stable LatentMoE supplies conditional capacity

K3’s MoE layer is reported to use 896 routed experts, selecting 16 experts per token. 16 This is the mechanism behind the gap between total and active parameters.

The report’s Stable LatentMoE approach focuses on keeping this routing stable and balanced. It frames batch-level expert assignment as an optimization problem and introduces quantile-based balancing. The report derives an exact optimum under its stated assumptions; at deployment, however, routing uses fixed expert biases with ordinary top-k selection rather than solving the balancing problem afresh for every inference batch. 1

That distinction matters. A claimed optimal equilibrium is a result for the report’s routing formulation, not a guarantee that every real production workload will distribute tokens perfectly evenly across experts.

MoonEP makes sparse routing operationally viable

Large MoE models introduce a networking problem: selected tokens often need to be sent across devices to reach their assigned experts. If some experts receive disproportionate demand, the slowest communication path can determine latency.

Moonshot positions MoonEP as the complementary systems layer for expert-parallel communication. Its job is to reduce imbalance in the all-to-all exchanges created by sparse routing, helping the large expert pool remain trainable and serveable. 1

This is a key part of K3’s broader claim. Architecture, routing, and communication cannot be evaluated as independent explanations of performance; the model needs all three to turn theoretical sparse capacity into real throughput.

The post-deployment half of the strategy: compute for agentic work

The other axis in K3’s thesis is what happens after pretraining. Moonshot describes infrastructure for reinforcement-learning trajectories involving long-horizon tasks, tools, coding, browsing, and iterative problem solving. The report also describes distilling multiple domain and reasoning teachers into one system. 1

The supporting environment is a lightweight virtual-machine sandbox system designed to create, snapshot, and resume task environments at scale. Reported figures include 51.2 million sandbox instances, 133 ms snapshots, and 49 ms restores. 1

The strategic purpose is straightforward: if a model is meant to spend more test-time compute planning, calling tools, checking work, or continuing a long task, then the training setup must expose it to those kinds of trajectories. Cheaply pausing, branching, and restoring environments makes more such training rollouts feasible.

This does not mean a single model response automatically uses unlimited inference compute. Rather, K3’s report argues for a system that can convert additional steps and context into better task performance instead of relying solely on a larger pretrained model.

What the BrowseComp result supports—and what it cannot isolate

Kimi K3 is reported at 91.2% on BrowseComp, a benchmark aimed at long-horizon information seeking. A third-party leaderboard also lists K3 at 91.2%, although rankings and reporting configurations can change over time. 18

That result is consistent with the product goal: a model designed around long context, agentic post-training, and test-time reasoning should be evaluated on complex information-seeking tasks. But it is not a clean ablation of the individual components.

A single benchmark score cannot tell readers how much of the result comes from KDA, AttnRes, expert routing, communication infrastructure, RL environments, distillation, prompting, or inference-time settings. The strongest interpretation is that the reported system performs well as an integrated stack—not that one architectural innovation alone explains the score. 118

Why the DeepSeek comparison is mainly architectural

K3 is often framed as a challenge to other large open models, including DeepSeek systems. The defensible takeaway from the report is not a settled league table of open-model progress, but a different design emphasis: Moonshot is advocating a stack that combines very large sparse capacity, efficient long-context processing, and post-training infrastructure for agents. 117

The paper itself supports K3’s specifications and design choices. It does not, by itself, establish that other open models are “stalled,” nor does it isolate a decisive architectural advantage over a named competitor. Those are broader market claims that require consistent, independently reproduced comparisons.

Treat cost claims as configuration-dependent

Price-per-token and cost-per-task comparisons are especially easy to overread. They can vary with prompting, reasoning effort, context length, caching, tool usage, throughput assumptions, pricing date, and the task harness.

One published comparison cited in the available material estimates completed-task costs of approximately $0.94 for K3 and $1.04 for GPT-5.6 Sol. That may suggest a modest advantage in that specific setup, but it does not substantiate a universal “half the cost of Sol” claim. 20

The useful conclusion is less dramatic and more durable: K3 is an attempt to make long-context, agentic capability economically credible at a much larger total model scale. Whether it achieves that advantage in practice depends on reproducible evaluations and deployment-specific cost accounting.

Bottom line

Kimi K3’s technical report makes a full-stack scaling argument. Pre-deployment scale comes from a 2.8T-parameter MoE with about 104B parameters activated per token. Post-deployment scale comes from training and serving a model that can use long context, tools, reasoning steps, and agentic rollouts. 117

KDA, Gated MLA, AttnRes, Stable LatentMoE, MoonEP, and the RL sandbox infrastructure are all parts of the same proposition: frontier-quality agentic behavior needs not just a bigger model, but a design that makes more context and more inference-time work usable. The benchmark claims are promising evidence for that integrated approach, but they should be read as reported system-level results rather than final proof of each component’s contribution. 118