Image Robotic Auto-photographing System for Remote Visual Train Bogie Inspection: A Multi-Environment Gaze-Speed and 3D Clustering Approach

We propose a multi-environment motion-teaching approach for a robotic auto-photographing system designed to replicate an inspector’s viewpoint during remote visual inspection of train bogies. The system utilizes a head-mounted device (HMD) to record the inspector’s gaze points and head pose, and then determine the robot’s camera pose based on them Previous works extracted camera poses based on dense clusters of gaze points recorded by the HMD, assuming that the inspector’s gaze points are concentrated. However, this approach fails to capture large components, where the gaze naturally moves along the surface. Additionally, these methods require the environments for HMD data collection and robot operation to be identical, limiting inspections in locations inaccessible to human inspectors. To address these limitations, we propose a method that leverages gaze speed information in addition to a 3D clustering method to identify both small and large components. The method also adjusts camera poses, enabling the robot to capture images in diverse environments. Experiments conducted in a mock-up environment replicating a real train bogie for HMD data collection, as well as simulations using robot and operational environment models, confirm that the proposed method allows the robot to successfully capture images of all the small and large components, even in environments different from the one where the HMD data was collected.