Who Should Look at This Image? Small Language Agents Choosing the Tools of Inspection Drones
Inspection drones can run small vision models on board and send images to larger models in the cloud. The drone must choose models for each image, with limited time and a weak radio link. We show that a small language model can learn to make this choice for power-line inspection. Step by step, the model picks one of about 60 vision tools and reads its answer. At the end it writes a report with three parts: the component type, whether there is a defect, and the condition. We train this agent by supervised fine-tuning and reinforcement learning and compare its reports with the true labels. We report four results. First, supervised fine-tuning turns a model that writes no correct report into an agent with 88.1% fully correct reports. Second, the set of tools matters: when we replace one large general model with a small component classifier, the agent learns to use two small classifiers on the drone, and 95.9% of its reports are fully correct, in 0.09 seconds per image and with no upload. Third, reinforcement learning with a reward on the final report raises fully correct reports from 79.8% to 88.4% and the share of defects found from 68% to 83%. Fourth, reinforcement learning helps the agents that we tested to adapt: when the general model is removed at test time, an agent with 1.7 billion parameters uses a tool that it never saw in training, and 92.4% of its reports are fully correct, against 76.2% without reinforcement learning.
Authors
- Mahdi Khashan (ORCID: https://orcid.org/0009-0000-6281-0736)
Institutions
- TU Wien (AT)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-14
- DOI
- https://doi.org/10.5281/zenodo.24212599
- Primary Topic
- Power Line Inspection Robots
- Type
- article
- Field-Weighted Citation Impact
- 0.00