Foundation Models for Autonomous Robots in Unstructured Environments: Current State-of-the-Art, Challenges, and Future Pathways

Automating activities in unstructured environments, such as construction sites, has been challenging due to unpredictable events, limiting robot adoption compared to structured settings. Recently, pre-trained foundation models, particularly Large Language Models (LLMs), have shown promise in addressing this challenge through superior generalization capabilities. This study employed a multi-dimensional method that systematically reviews the field from different perspectives of foundation models in robotics and unstructured environments, and synthesizes them with deliberative acting theory. The findings revealed that LLMs’ linguistic capabilities are primarily used to improve perception and human–robot interactions in robotic tasks, while applications in project management, safety, and natural hazard detection are the most utilized applications of foundation models in unstructured environments. Our synthesis shows an empirical gap in the field where fewer identified studies within unstructured environments validated their foundation model applications using physically deployed robots. We positioned the current state-of-the-art on a five-level automation scale of conditional automation. These findings inform future scenarios, challenges, and solutions toward autonomous safe unstructured environments. Our study serves as a benchmark to track our progress toward that future.