Skip to content

Search across models, tools, services, companies, providers, APIs, plans, articles and news.

Architecture & models

Multimodal

Multimodal

Multimodal

A model that understands or generates images, audio and video in addition to text — like Gemini viewing a chart and explaining it.

A multimodal model takes and combines inputs through several senses like a human does: text, images, audio, screenshots, charts — and in newer generations, video. The practical turning point in 2024–2026 was when “send an image and ask about it” became default behavior.

Behind the scenes: each modality is mapped into a shared representation space (embeddings) where the model reasons and generates — which is why it can read a chart into code or turn a snapshot into a to-do list.

For users the real value is deleting the “convert and copy-paste” step: invoice photos, terminal error screenshots and whiteboard snaps become direct input.

Back to glossary