Direct TPS-based 3D non-rigid motion estimation on 3D colored point cloud in eye-in-hand configuration

In this paper, a method for 3D non-rigid motion estimation of a surface using an RGB-D camera in eye-in-hand configuration is presented. The eye-in-hand configuration eliminates errors typically associated with camera-end-effector calibration, and is thus desirable for task on moving surfaces such as bioprinting. However, its implementation is challenging since camera and surface of interest are moving, making mesh-based approaches unsuitable. Thus, the proposed method operates directly on point clouds, benefiting from accurate and simplified data processing. A point cloud contains both intensity and depth data, with the former used to estimate in-plane deformation and the latter to compute full 3D deformation. Surface deformation is modeled via a Thin Plate Spline model. The method accuracy is assessed at 0.1 mm accuracy in simulated datasets, rendering it suitable for precision tasks, and its feasibility is validated experimentally on a moving platform that deforms at a rate of 0.8 Hz with a 4 mm in-plane amplitude and a 20 mm elevation amplitude.