An Intelligent Loading System for Standardized Mining Material Transportation Based on Multimodal Perception and Multi-Arm Collaboration

Currently, mining material transportation in warehouses relies heavily on manual operations, which pose safety hazards and suffer from low standardization and automation. Existing automated attempts using single-sensor perception or single-arm manipulators lack robustness and adaptability in harsh mine environments. To address these gaps, this paper proposes an intelligent loading system for standardized mining material transportation based on multimodal perception and multi-arm collaboration. First, the overall architecture of the transportation and loading system is introduced, comprising five modules: a standardized carrier platform and modular transport boxes, a box locking and spreader module, a multi-sensor recognition and positioning module, a multi-manipulator collaborative loading/unloading module, and a perception feedback and (human-controlled) overhead crane module. Next, a standardized hardware system is designed, focusing on the standardization of the separable and easily detachable carrier platform and the modularization of transport boxes, along with the locking mechanism between them, establishing the hardware foundation for the system. Subsequently, a multimodal perception data fusion and recognition positioning technology based on multiple depth cameras, UWB, and IMU is investigated to provide perceptual feedback for automated loading/unloading. Following this, a multi-manipulator collaborative control technology based on multi-agent error consensus is developed, designing a “two-master, two-slave” structure and a collaborative control algorithm to achieve automated loading/unloading of transport boxes. An information-based interactive monitoring software is then designed to monitor system perception data in real time and control the system’s operational status, ensuring safety and controllability. Finally, the feasibility and effectiveness of the system are validated through simulations and prototype experiments. This work provides a foundation for standardized transportation and storage of mining materials and outlines a practical system-level approach.