Kimi K3's 2.8 Trillion Parameters: What the Number Means

Moonshot AI’s Kimi K3 has 2.8 trillion parameters, the internal values a neural network tunes during training so it can predict text. That is more total parameters than any other openly released large language model to date. What that total does not tell you is how much of the model actually runs each time it answers a question. Kimi K3 touches only a sliver of those parameters, about 1.8 percent, for every word it generates, and that fraction, not the trillion-scale headline, is what actually decides how much a model costs to run. This piece stays on what that scale means technically; Kimi K3’s price and its rocky first two weeks are covered separately.

The trick has a name: mixture of experts, or MoE, an architecture that activates only a handful of experts per word instead of running the entire model every time. Moonshot published the full technical breakdown on its own blog when it released Kimi K3’s complete weights on July 27, 2026. Understanding that mechanism explains what the 2.8 trillion number actually buys, and why the same trick already runs inside other large language models a reader may recognize by name.

The difference between total size and what actually runs

A parameter is one adjustable number inside a neural network, tuned during training until the model gets better at predicting the next word. Kimi K3 holds 2.8 trillion of them in total, spread across its full architecture. In a dense model, the kind that powered the earliest wave of large language models, every one of those parameters gets used on every calculation. Double a dense model’s parameter count and you roughly double the compute it takes to answer one question.

Kimi K3 does not work that way. It is a Mixture-of-Experts (MoE) model, which means that total is not one continuous block that always runs together. It is split into 896 separate specialist sub-networks, called experts, and a routing mechanism decides, word by word, which of those experts get used. That headline number describes the full pool Kimi K3 can draw on, not the amount of math it performs to generate any single word.

That distinction gets lost in coverage that treats a model’s total parameter count as a straightforward stand-in for how big or how capable it is, the framing behind headlines calling Kimi K3 the largest AI model released so far. DecodeStack already examined how that hype held up against Kimi K3’s actual benchmark results. The parameter count is a real number, but on its own it says nothing about what the model costs to run, which is the part the next section unpacks.

The trick: only 16 of 896 experts fire for a single word

Picture a large hospital that employs 896 doctors, one for nearly every medical specialty imaginable. When a patient walks in, the hospital does not send their file to all 896 doctors and ask each one to weigh in. A triage step matches the patient’s symptoms to the handful of specialists actually relevant to that case, maybe 16 of them, and only those doctors do any work. The other 880 stay free for other patients.

Kimi K3’s mixture-of-experts architecture works on the same logic, applied to every word it generates. The model holds 896 experts, small specialized sub-networks that each learn to handle certain kinds of patterns in language. For each word, a router component looks at the context so far and picks the experts best suited to that specific word. Only those 16 of the 896 experts actually do any computation for that word; the rest sit idle until the next word, when the router can pick an entirely different set of 16.

Sixteen out of 896 works out to roughly 1.8 percent, the sparsity figure Moonshot cites for Kimi K3. It is why a model with a 2.8 trillion parameter total can still respond in something closer to the time and cost of a much smaller model: only a sliver, about 1.8 percent, actually does the work for any given word, not the whole 2.8 trillion.

The routing happens automatically, learned during training rather than hand-programmed, so Moonshot does not manually decide which experts handle which topics. The model discovers its own division of labor.

Why this matters for what you pay and how fast it runs

Running a language model costs money and time roughly in proportion to how much math it performs per word, not how many parameters it owns in total. Because inference cost tracks the parameters actually activated, not the full 2.8 trillion total, Kimi K3’s real running cost per response looks much closer to a mid-sized dense model than its headline parameter count suggests. A dense model built to the same 2.8 trillion parameter total would need to run all of it on every word, and would be dramatically slower and more expensive to serve.

That gap between total size and running cost is also why Moonshot can offer Kimi K3 through an API at a price competitive with much smaller models, and why the model can hold a conversation at a usable speed rather than the crawl a dense model of that scale would require. DecodeStack covered the pricing side directly, including the price war it triggered among Chinese AI labs in the two weeks after launch. The short version is that sparsity, not any pricing trick, is what makes those numbers possible in the first place.

Kimi K3 isn’t the only model doing this

Mixture-of-experts is a well established technique that predates Kimi K3 by several years, used across large language models as labs looked for a way to keep growing a model’s total capacity without a proportional jump in what it costs to run. Mistral AI’s Mixtral is a widely reported example of another openly released model built on the same principle, activating a subset of its experts per token rather than running its full parameter count every time.

What changes from model to model is the specific tuning: how many total experts, how many activate per word, and how the routing gets trained. Kimi K3’s particular numbers, 896 experts with 16 active, are Moonshot’s own choice within that shared design, not a different architecture altogether. Several of the strongest open-weight language models available in 2026 now lean on some version of this same sparse-activation idea, worth remembering before judging any model by its raw parameter total alone.

What Moonshot claims, and what’s independently confirmed

Some facts about Kimi K3’s architecture are public and verifiable because Moonshot documented them itself: the 2.8 trillion total parameters, the 896 experts with 16 active per word, and a native context window of 1 million tokens, the amount of text the model can consider at once. Those numbers come from Moonshot’s own architecture blog post and describe the model as built, not a claim requiring outside confirmation.

One figure in that same post is different in kind. Moonshot writes that its architectural changes deliver “an approximate 2.5x improvement in overall scaling efficiency compared to Kimi K2,” its previous model. That is Moonshot’s own claim, stated in its own announcement, not a number produced by an outside lab running its own benchmark. Moonshot’s 2.5x efficiency claim is the company’s own number, not an independently measured benchmark, and as of this writing no third party has published a test confirming or disputing that specific ratio. That does not make the claim false. It means the number belongs in the category of things a vendor says about its own product, worth noting as exactly that until someone outside Moonshot checks it.

The measure that actually matters

Kimi K3’s 2.8 trillion parameters are real, and Moonshot has documented exactly how the model is built to reach that total. But the sliver that actually fires per word, not its total size, is what determines the real cost of running it, and Kimi K3 activates only about 1.8 percent of itself for any single word. That is the number worth remembering the next time a language model’s headline size gets reported as if it were the whole story.

Frequently asked questions

What is Kimi K3?

Kimi K3 is Moonshot AI’s large language model, released with full open weights on July 27, 2026. It is built as a Mixture-of-Experts model with 2.8 trillion total parameters, though only a small fraction of that total is active for each response.

How many experts does Kimi K3 activate per word?

Kimi K3 activates 16 of its 896 total experts for each word it generates, roughly 1.8 percent of the full model. A router picks a different set of 16 for each word based on context, so the exact experts used can change from one word to the next in the same sentence.

Is Kimi K3 the biggest AI model?

Kimi K3 carries more total parameters than most publicly released large language models, which is where the “biggest” framing comes from. But because it only activates a small slice of that total per word, being that large on paper doesn’t make it the slowest or most expensive model to run.

What other AI models use a mixture-of-experts design?

Mixture-of-experts predates Kimi K3 and appears in several other well known large language models. Mistral AI’s Mixtral is a widely reported example, activating a subset of specialized sub-networks per word instead of running its entire parameter count on every request, the same principle behind Kimi K3.

Is Moonshot’s 2.5x efficiency claim independently verified?

Moonshot’s claimed 2.5x improvement in scaling efficiency over Kimi K2 has not been independently verified. That figure comes from the company’s own architecture blog post, not from an outside lab’s benchmark, and no independent test confirming or challenging it has been published as of this writing.