Position: Prospective of Autonomous Driving - Multimodal LLMs, World Models, Embodied Intelligence, AI Alignment, and Mamba

With the emergence of Generative AI, multimodal AI systems that leverage foundation models are beginning to demonstrate enormous potential for perceiving the real world, collecting new data, making decisions, and using tools like humans. In recent years, the use of Large Language Models and World Models in autonomous driving has received widespread attention. However, despite their enormous potential, there is still a lack of comprehensive understanding regarding the key challenges, opportunities, and future applications of these new foundation models in driving systems. In this paper, we provide an outlook on this field, summarizing existing methods and exploring their limitations. In addition, we further discuss the applicability of emerging approaches, such as Reinforcement Learning from Human Feedback and Mamba for applications in autonomous driving. Finally, we highlight open questions and offer insights into promising directions for future research. This paper is part of a living document that will be updated based on the LLVM-AD workshop series to reflect the latest developments in the field.

Authors

Institutions

Publication Details

Published
2025-02-28
DOI
https://doi.org/10.1109/wacvw65960.2025.00114
Citations
12
Primary Topic
Artificial Intelligence in Law
Type
article
Field-Weighted Citation Impact
102.27
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Position: Prospective of Autonomous Driving - Multimodal LLMs, World Models, Embodied Intelligence, AI Alignment, and Mamba

Jintai Chen, Guangtao Zheng, Burhaneddin Yaman, Hamid Rezatofighi et al.
12 citations
Artificial Intelligence in Law
102.27
article

Position: Prospective of Autonomous Driving - Multimodal LLMs, World Models, Embodied Intelligence, AI Alignment, and Mamba

Jintai Chen, Guangtao Zheng, Burhaneddin Yaman, Hamid Rezatofighi, Can Cui, Manmohan Chandraker, Shuo Xing, Tianjiao He, Wenqian Ye, Fucai Ke, Chao Zheng, Zhen Li, Xin Ye, Hang Zhao, Xu Cao, Jinhong Wang, Yunsheng Ma, Haiming Zhang, Chenglin Miao
article en
12 citations

Abstract

With the emergence of Generative AI, multimodal AI systems that leverage foundation models are beginning to demonstrate enormous potential for perceiving the real world, collecting new data, making decisions, and using tools like humans. In recent years, the use of Large Language Models and World Models in autonomous driving has received widespread attention. However, despite their enormous potential, there is still a lack of comprehensive understanding regarding the key challenges, opportunities, and future applications of these new foundation models in driving systems. In this paper, we provide an outlook on this field, summarizing existing methods and exploring their limitations. In addition, we further discuss the applicability of emerging approaches, such as Reinforcement Learning from Human Feedback and Mamba for applications in autonomous driving. Finally, we highlight open questions and offer insights into promising directions for future research. This paper is part of a living document that will be updated based on the LLVM-AD workshop series to reflect the latest developments in the field.

University of Illinois Urbana-Champaign (US), Iowa State University (US), University of Toronto (CA), Purdue University West Lafayette (US), Tencent (China) (CN), Australian Regenerative Medicine Institute (AU), Universidad Católica Santo Domingo (DO), Robert Bosch (India) (IN), University of Virginia (US), Monash University (AU), Zhejiang University (CN), University of Hong Kong (HK), Texas A&M University (US), Tsinghua University (CN)
Openalex Percentile: Top 0%
Artificial Intelligence in Law
102.27
DOISourceOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.