Localization and Situational Awareness for Autonomous Ship Using Trinocular Stereo Vision

Similar to the demand for automated driving vehicles and autonomous mobile robots on land, marine autonomous surface ships are desired in the maritime field. An autonomous ship must be capable of recognizing objects in its vicinity and accurately determining the relative positions between the ship and objects. In this paper, we propose a novel method for creating a map of the ship’s surroundings using trinocular stereo vision, along with accurately estimating its location and motion within this map. In the proposed method, the world coordinates of feature points—such as land-based structures—are measured at the ship’s initial position. As the ship moved, these feature points in the images are tracked using optical flow. The ship’s world coordinates are then estimated based on the pixel and world coordinates of these feature points. Furthermore, the feature points in the three-dimensional map are classified using deep learning. An experiment involving ship navigation over a distance of 1000 m confirmed that the system can accurately estimate the position, heading, and speed of the ship and can also identify quay walls, other ships, and bridges in three-dimensions. The results indicate that the proposed method is suitable for automating berthing systems.