Παθητική απτική σε περιβάλλον επαυξημένης πραγματικότητας : ανίχνευση αντικειμένων σε επαυξημένη πραγματικότητας με χρήση στερεοσκοπικής κάμερας βάθους

Markerless augmented reality (AR) – overlaying virtual content that is registered to physical objects without relying on printed fiducial markers – remains a substantially harder tracking problem than marker-based AR, because the system must segment, identify, and estimate the pose of an object purely from its natural visual appearance and orientation. This thesis presents a real-time markerless AR tracking system built around a ZED stereo camera and the Unity 6 engine, which tracks three physical objects (a pen, a cup and a coaster ) using a dependencyfree, hand-implemented computer-vision pipeline: HSV color thresholding, Gaussian smoothing, morphological filtering, connected-component contour extraction, principal-axis endpoint disambiguation, sub-pixel refinement, per-endpoint Kalman filtering, and pyramidal Lucas-Kanade optical flow as an occlusion fallback. Pixel-space detections are converted into metric 3D poses using the stereo camera’s per-pixel depth channel and a pinhole-camera unprojection, and are used to drive a virtual 3D object overlay (a sword for the pen, a bin for the cup and a mat for the coaster) attached to the tracked objects, demonstrated through an interactive fruit-slicing mini-game. The three objects deliberately share the same magenta color, so color alone cannot tell them apart and discrimination must rely on shape and previous tracking results. Beyond the core single, dual- and triple-object tracking system, this work documents an extension undertaken as part of the ongoing thesis work and reported here at its current state of completion: a proposed generalization pipeline that lets an operator calibrate a new trackable object at runtime from a single reference photograph, rather than through hand tuned HSV constants, removing the need to re-compile or hand-calibrate the system for every new physical prop. The thesis further situates the system within the broader history of augmented reality and object-tracking research, from Sutherland’s 1968 head-mounted display to contemporary spatial computing, and a state-of-the-art chapter traced each technical technique used throughout implementation to its modern successor, explaining why the classical choice suited real-time, CPU-only constraints. The thesis also reports the system’s limits and the lessons learned while building it. Future work includes a full 6-DOF pose estimation, a GPU implementation, a completion of the generalization pipeline and a broader passive-haptics experience. Overall, the work shows that real-time, markereless tracking of several same-colored objects, are achievable with classical computer vision and a depth stereo camera.