Synthetic-to-Real 5D Pose Estimation for Acoustically Levitated Micro-Calibrators in Lab-on-a-Chip Devices

Established acoustofluidic lab-on-a-chip devices enable contactless, label-free trapping of cells, but do not provide a quantitative readout of a levitated sample’s orientation, restricting their analytical reach. We present a synthetic-to-real deep learning framework that addresses this gap by estimating the five-dimensional (5D) pose of a trident-shaped micro-calibrator co-trapped with the cells. Here, 5D pose denotes the two-dimensional (2D) bounding-box center in the image plane and the calibrator’s three-dimensional (3D) orientation. Printed by two-photon polymerization with 10, 15, and 20 μm arms, the calibrator provides the rigid, asymmetric geometry that a translucent, deformable spheroid lacks. The framework cascades detection, foreground segmentation, and rotation regression, with the normalized bounding-box center as a positional auxiliary input, and optionally applies temporal smoothing; it is trained on domain-randomized renderings, eliminating manual 3D pose annotation. On 15,000 synthetic test images, it reached a mean geodesic rotation error of 0.82°. The ablation study showed that the positional input substantially improves rotation estimation and background removal further lowers the error, and identified segmentation, not detection, as the dominant upstream error source. Pseudo-label adaptation raised the expert-assessed correctness rate from 34.71% to 83.62% on 5281 real microscopy frames from which the pseudo-labels were drawn (adaptation performance) and from 2.48% to 29.34% on 242 held-out fluorescence frames with adherent cells (generalization performance). This demonstrates the feasibility of a calibrator-based orientation readout on an established acoustofluidic platform without hardware change.