Accessing the universe via algorithm: a baseline study for automating alt-text generation with NASA data and AI
Introduction “Alt-text” is a textual substitute for visual information on digital platforms, primarily by users of screen readers. Effective image description is crucial for inclusive, accessible communication; unfortunately, alt-text creation is often limited by the time and skill constraints of the humans writing it. This baseline study aimed to develop an AI-driven method to semi-automate alt-text generation, with minimal expert moderation, for publicly released science images from NASA's Chandra X-ray Observatory. Methods We used pre-trained vision-language models and image metadata to generate alt-text; however, we found the results to be unreliable with this method alone. We iterated the process to include a two-tier fact-checking system of “exogenous” and “endogenous” checks to ensure the alt-text accurately represented the image and image caption. We then evaluated the alt-texts by using an automated “rubric” and investigated the tool's ability to discern “Good” from “Bad” alt-text using developed examples. Results We compared these scores with human evaluators and found that although human and AI scoring does not always agree, the AI rarely rates alt-text low when a human rates it high, indicating that the AI is proficient at noticing high-quality alt-text. Furthermore, our quantified multi-dimensional evaluation step enables adaptability to automated corrections or training improvements. Finally, we find that although the resulting alt-texts still need human editing, the time required for this (compared with writing the alt-text from scratch) reduces the human time burden by 96%. Discussion This multistage approach aligns with current AI research and demonstrates the role AI can play in creating alt-text for large scientific datasets, while our findings reinforce the importance of using these tools alongside human expertise. In addition to performing the research, we add our real-world results to the Chandra X-ray Observatory public database to improve access to the images.
Authors
- Tianze Hua
- Jessica Schonhut-Stasik (ORCID: https://orcid.org/0000-0002-1043-8853)
- Nancy Wolk
- Kimberly K. Arcand (ORCID: https://orcid.org/0000-0002-1436-6607)
- Areshva Aisha Mir (ORCID: https://orcid.org/0009-0007-5602-7260)
- Anna Ohrt (ORCID: https://orcid.org/0009-0007-2307-4036)
- Tom Sgouros
- Sarah Kane
Institutions
- Harvard University (US)
- Smithsonian Astrophysical Observatory (US)
- Vanderbilt University (US)
- University of Cambridge (GB)
- Brown University (US)
- Center for Autism and Related Disorders (US)
Publication Details
- Journal
- Frontiers in Artificial Intelligence
- Published
- 2026-11-18
- DOI
- https://doi.org/10.3389/frai.2026.1740372
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Marshall Space Flight Center