Fixed Monocular Object Localization via Geometric and Vision Multi-Modal Learning

This study presents a hybrid monocular object localization system designed for autonomous waste collection using fixed urban CCTV infrastructure. Waste objects are first detected using YOLO, and their initial 3D positions are estimated via ray-plane intersection, leveraging the camera's intrinsic and extrinsic parameters. To address the limitations of ray-plane intersection, including systematic distance overestimation and amplification of detection inaccuracies at longer ranges, we introduce a multimodal refinement network. The network employs a ConvNeXt backbone to extract visual features from cropped object regions and fuses them with ray-plane derived geometric embeddings to regress position offsets. The proposed method estimates 3D object positions using only monocular CCTV input, without the need for external depth sensors or stereo vision setups. Experimental validation in the Isaac Sim environment demonstrates the system's practicality and indicates strong potential for real-world deployment.