Fourier lightfield multiview stereoscope for large field-of-view 3D imaging in microsurgical settings

This work presents the Fourier Lightfield Multi-view Stereoscope (FiLM-Scope), a novel imaging device that combines concepts from Fourier Light Field Microscopy and Multi-view Stereo imaging to capture high-resolution 3D videos over large fields-of-view. The FiLM-Scope optical hardware consists of a multi-camera array, with 48 individual micro-cameras, placed behind a high-throughput primary lens. This allows the FiLM-Scope to simultaneously capture 48 unique 12.8 megapixel images of a 28 × 37 <i>mm</i> field-of-view, from unique angular perspectives over a 21° × 29° range, with down to 22 <i>μm</i> lateral resolution. We additionally describe a self-supervised algorithm to reconstruct 3D height maps from these images. Our approach demonstrates height accuracy down to 11 <i>μm</i>. To showcase the utility of our system, we perform tool tracking over the surface of an ex-vivo rat skull and visualize the 3D deformation in stretching human skin, with videos captured at up to 100 frames-per-second. The FiLM-Scope has the potential to improve 3D visualization in a range of micro-surgical settings.