Utilizing Fisheye Cameras and Mass Spectrometry Imaging to Obtain Probing Coordinates for Histological Samples

Detecting chemical compositions in histological samples using mass spectrometry often requires time-consuming user interaction with a probe to ensure it is positioned as closely as possible to a sampling region. Fisheye cameras can obtain images of samples with a wide field-of-view; however, barrel distortion makes it difficult to analyze them as a flat plane. This research project aimed to develop an application that unwarps live images of histological samples captured with a fisheye camera and calculates the exact image-to-real-world probing coordinates with minor user interaction. Python’s PyQt5 and OpenCV libraries were utilized to create interactive software that removes fisheye distortion, finds image-to-real-world coordinates, and segments images. Connected hardware consisted of a Prusa 3D printer, liquid micro junction-surface sampling probe, fisheye camera, and LED light strips. The printer provided a bed for capturing sample images and control of the camera and probe within 1/100 of a millimetre. The software was outlined to process at least 2 photos of checkerboards for the fisheye undistortion, capture a photo of a sample to be unwarped, and require the user to map 4 points from the image to probe coordinates. Using this approach, real-world coordinates could be found for any image location. The application successfully unwarps live photos and converts image-to-real-world coordinates with sub-millimeter accuracy. Users can take new calibration photos, control lighting, save photos, and check undistortion entirely within the application to ensure all parameters are satisfactory. Coordinates of significant regions are displayed or may be manually selected, making it easier to sample specific areas. The success of unwarping relies on the camera’s focus and height, lighting, and object positioning. Future development plans to improve robustness and coordinate accuracy, and reduce the amount of calibration needed.