Fast closed-loop STEM navigation using vision-language models with typed decisions

Closed-loop scanning transmission electron microscopy (STEM), in which each acquired image determines the next field of view and magnification, offers a route to autonomous atomic-scale materials analysis. Such a loop would automate navigation to defects and grain boundaries, a task that has long relied on operator judgment, and would enable systematic analysis of diverse local structures at scale. Realizing it hinges on the decision step, which must translate each acquired image into the next operation both quickly and correctly; yet which decision method can deliver both remains unclear. Here, we isolate this decision step. Using simulated atomic-resolution STEM images of a substitutional W dopant and a grain boundary in monolayer MoS$_2$, where the true target position is known and every method starts from identical conditions, we compared four vision-language models (VLMs) and three decision methods. Our results demonstrate the effectiveness of Typed Decision, which directly obtains the probabilities of candidate labels from a pretrained VLM, for closed-loop STEM navigation. Typed Decision showed the shortest decision latency for all models (0.13-0.21 s) and generally achieved the highest navigation success rate among the three methods: it was the best for all models in W dopant navigation (66-91%) and for the best-performing model, Cosmos-Reason2 32B, in grain boundary navigation (43%). Furthermore, navigation to the different targets, the W dopant and the grain boundary, was possible by natural-language specification alone, without task-specific training of the VLM. These results indicate that Typed Decision is an effective decision method for realizing fast, accurate, and flexible closed-loop STEM.

Authors

Publication Details

Journal
arXiv (Cornell University)
Published
2026-11-18
DOI
https://doi.org/10.48550/arxiv.2611.23698
Primary Topic
Advanced Electron Microscopy Techniques and Applications
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Fast closed-loop STEM navigation using vision-language models with typed decisions

Izumi Takahara, Shuichiro Ozawa, Fumihiko Uesugi, Teruyasu Mizoguchi
arXiv (Cornell University)
Advanced Electron Microscopy Techniques and Applications
preprint

Fast closed-loop STEM navigation using vision-language models with typed decisions

Izumi Takahara, Shuichiro Ozawa, Fumihiko Uesugi, Teruyasu Mizoguchi
preprint en

Abstract

Closed-loop scanning transmission electron microscopy (STEM), in which each acquired image determines the next field of view and magnification, offers a route to autonomous atomic-scale materials analysis. Such a loop would automate navigation to defects and grain boundaries, a task that has long relied on operator judgment, and would enable systematic analysis of diverse local structures at scale. Realizing it hinges on the decision step, which must translate each acquired image into the next operation both quickly and correctly; yet which decision method can deliver both remains unclear. Here, we isolate this decision step. Using simulated atomic-resolution STEM images of a substitutional W dopant and a grain boundary in monolayer MoS$_2$, where the true target position is known and every method starts from identical conditions, we compared four vision-language models (VLMs) and three decision methods. Our results demonstrate the effectiveness of Typed Decision, which directly obtains the probabilities of candidate labels from a pretrained VLM, for closed-loop STEM navigation. Typed Decision showed the shortest decision latency for all models (0.13-0.21 s) and generally achieved the highest navigation success rate among the three methods: it was the best for all models in W dopant navigation (66-91%) and for the best-performing model, Cosmos-Reason2 32B, in grain boundary navigation (43%). Furthermore, navigation to the different targets, the W dopant and the grain boundary, was possible by natural-language specification alone, without task-specific training of the VLM. These results indicate that Typed Decision is an effective decision method for realizing fast, accurate, and flexible closed-loop STEM.

arXiv (Cornell University)
Advanced Electron Microscopy Techniques and Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon