Skip to content

Search across models, tools, services, companies, providers, APIs, plans, articles and news.

Architecture & models

Mixture of Experts

Mixture of Experts (MoE)

Mixture of Experts

An architecture that splits a model into several "experts", activating only the relevant ones per input — yielding much larger models at lower inference cost.

Mixture of Experts splits a model into pieces: several “experts,” each versed in a slice of knowledge, plus a router that wakes only the few relevant experts per input. A 400-billion-parameter model may activate just 10 billion per token.

The attractive equation: giant-model knowledge at mid-model inference cost. That is why MoE dominates 2025–2026: DeepSeek V3, GPT-4 and Gemini are all MoE.

The technical caveat to know: “total parameters” and “active parameters per token” are two different numbers; compare models on the second (and data quality), not the first.

Back to glossary