A Real-Time Pedestrian Situation Detection Method Using CNN and DeepSORT with Rule-Based Analysis for Autonomous Mobility

This paper presents a real-time pedestrian situation detection framework for autonomous mobility platforms. The proposed approach extracts pedestrians from images acquired by a camera mounted on an autonomous mobility system, classifies their postures, tracks their trajectories, and subsequently detects pedestrian situations. A convolutional neural network (CNN) is employed for pedestrian detection and posture classification, where the YOLOv12 model is fine-tuned via transfer learning for this purpose. To improve detection and classification performance, a region of interest (ROI) is defined using camera calibration data, enabling robust detection of small-scale pedestrians over long distances. Using a custom-labeled dataset, the proposed method achieves a precision of 96.6% and a recall of 97.0% for pedestrian detection and posture classification. The detected pedestrians are tracked using the DeepSORT algorithm, and their situations are inferred through a rule-based analysis module. Experimental results demonstrate that the proposed system operates at an execution speed of 58.11 ms per frame, corresponding to 17.2 fps, thereby satisfying the real-time requirements for autonomous mobility applications. These results confirm that the proposed framework enables reliable real-time pedestrian extraction and situation awareness in real-world autonomous mobility environments.