3D-omgevingsdetectie en voertuigherkenning voor toepassingen in autonoom rijden

Autonomous driving systems require a comprehensive understanding of their surroundings to ensure safety and effective decision-making. This includes recognizing objects, understanding their structure and motion, and reasoning about drivable areas, obstacles, lanes, and so on. Traditionally, this complex perception stack, comprising tasks such as object detection, tracking, semantic segmentation, and depth estimation, has relied on a variety of sensors. In particular, systems that fuse information from LiDAR, radar, and cameras typically achieve superior performance compared to using any single modality alone. However, inspired by human visual perception, this manuscript pushes the boundaries of what can be achieved in robust perception using only camera inputs. We focus on two fundamental perception tasks that are key to 3D surround vehicle awareness: multi-object tracking, which involves detecting, re-identifying, and following objects over time, and 3D semantic scene completion, which consists of reconstructing a semantically labeled 3D representation of the surrounding environment. First, we propose a camera-only 3D multi-object tracker that learns to robustly combine appearance and motion cues, overcoming limitations of prior methods that compromise object representation quality in order to perform joint detection and tracking. Our method is agnostic to both the object detector and camera setup, and achieves state-of-the-art performance on widely used automotive benchmarks. By effectively modeling object appearance and motion, our approach enables consistent identity assignment and robust re-identification over time, even under temporary occlusions or missed detections. Second, we present a transformer-based tracking framework that integrates tracking capabilities into modern detection models through a contrastive loss and a revised sampling strategy, without sacrificing detection accuracy. We also introduce a pretraining scheme on static image datasets (i.e., detection datasets), enhancing tracking performance. This pretraining technique enables the exploitation of large and diverse detection datasets, which are typically much more extensive than tracking datasets. Third, we address the emerging task of 3D panoptic semantic scene completion. Current methods often rely on visibility masks during training to indicate which parts of the scene are observed by the cameras. As a result, they tend to accurately reconstruct only visible areas while neglecting occluded regions, which limits their ability to estimate complete object shapes. This is a critical limitation for reliable path planning and decision-making. We propose a novel approach that detects and reconstructs full object shapes, including occluded parts, using camera inputs alone. By representing objects with learned spatial vectors, our top-down method naturally extends 3D semantic scene completion to a panoptic setting, contributing to a more complete 3D representation of the surrounding environment. In summary, this manuscript presents several contributions to the fields of multi-object tracking and 3D semantic scene completion for autonomous driving. The innovations focus on effective leverage of appearance and motion cues, the design of a joint detection and tracking model based on modern transformer-based object detectors, and the introduction of a novel approach for panoptic 3D semantic scene completion. Together, these contributions advance the state of the art in multi-object tracking and introduce a novel conceptual framework for 3D panoptic semantic scene completion, offering new perspectives on camera-based perception in autonomous driving.