Multimodal Reasoning, Agentic Systems, and Embedding Models: A Survey of Recent Advances
Multimodal models increasingly reason over what they perceive, act in digital and physical environments, and retrieve content across modalities. This survey reviews recent advances in the three families of models behind these abilities: multimodal reasoning models, which reach answers through chains of thought trained with reinforcement learning; multimodal agentic systems, which run such models in a loop of perception, reasoning and action with tools, graphical interfaces, robots and search engines; and multimodal embedding models, which map images, text, audio, video and documents into a shared space for retrieval and memory. We present them in one mathematical framework in which every model composes modality encoders, a connector and a foundation-model backbone ending in a generative or an embedding head. Within it we trace reasoning from attention-based visual question answering to group-relative policy optimization and ``thinking with images''; agents from tool-augmented programs to GUI, embodied and search agents trained with verifiable rewards; and embeddings from canonical correlation analysis and contrastive encoders (CLIP, SigLIP) to instruction-following universal embedders. More than 350 works are organized along five taxonomies, and we summarize benchmarks and reported results and close with open problems, including the perception bottleneck, faithful reasoning, agent safety, efficiency and evaluation.
Authors
- Mohammed Ayalew Belay (ORCID: https://orcid.org/0000-0003-2144-9790)
- Amirshayan Haghipour (ORCID: https://orcid.org/0009-0006-2670-6337)
- Michael Endale
Publication Details
- Journal
- Preprints.org
- Published
- 2026-11-03
- DOI
- https://doi.org/10.20944/preprints202611.0215.v1
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- preprint