A Lightweight Data-Augmented Deep Learning Framework for Real-Time Instance Segmentation in Liquid-Phase In Situ Transmission Electron Microscopy

Liquid-phase transmission electron microscopy (LPTEM) offers critical insights into the dynamic behaviors of materials, but automated analysis is hindered by the scarcity of annotated data and the high cost of image acquisition. To address these challenges, we propose an LPTEM deep learning framework that (i) closes the annotation gap, (ii) localizes nanoparticles through real-time detection, and (iii) preserves per-pixel detail. Through CycleGAN, we are able to generate a large number of images that are style-consistent with real LPTEM frames, effectively expanding the data set without manual labeling. After augmenting the data set, our fine-tuned YOLOv11n model achieves 97.66% precision and 99.05% mAP50 for detecting nanoparticles, closely approaching the performance of models trained on real data. Three Mobile-UNet variants, optimized for different computational efficiency and accuracy trade-offs, demonstrate instance segmentation for in situ TEM with intersection over union (IoU) scores ranging from 0.8685 to 0.9207 (on a scale where 1.0 indicates perfect overlap with the ground truth), indicating highly accurate particle shapes. Our integrated framework significantly enhances real-time detection and segmentation performance of nanoparticles in LPTEM, with YOLO + Mobile-UNet Slim achieving a real-time processing speed of 102.34 FPS. This framework establishes a scalable, annotation-efficient route to high-precision, real-time LPTEM analysis, opening the door to autonomous in situ experiments and accelerated materials discovery.