Skip to content

Search across models, tools, services, companies, providers, APIs, plans, articles and news.

Architecture & models

Transformer

Transformer

Transformer

The neural architecture that processes whole text in parallel via the attention mechanism — the foundation of every large language model since 2017.

The transformer is the neural architecture introduced in Google’s 2017 paper “Attention Is All You Need” that changed everything: instead of slow sequential processing, its attention mechanism handles all tokens of a text simultaneously, in relation to one another.

The payoff: parallel training on hardware, grasp of long-range dependencies (“which noun does this pronoun refer to?”) and fearsome scalability. Every major language model today — GPT, Claude, Gemini, Llama — is a direct descendant.

The stack in one glance: embeddings in, stacked attention + feed-forward layers, positional encoding to keep order. The 2026 variants (MoE, FlashAttention, hybrid windows) optimize that skeleton rather than replace it.

Back to glossary