Accessing the universe via algorithm: a baseline study for automating alt-text generation with NASA data and AI

Introduction “Alt-text” is a textual substitute for visual information on digital platforms, primarily by users of screen readers. Effective image description is crucial for inclusive, accessible communication; unfortunately, alt-text creation is often limited by the time and skill constraints of the humans writing it. This baseline study aimed to develop an AI-driven method to semi-automate alt-text generation, with minimal expert moderation, for publicly released science images from NASA's Chandra X-ray Observatory. Methods We used pre-trained vision-language models and image metadata to generate alt-text; however, we found the results to be unreliable with this method alone. We iterated the process to include a two-tier fact-checking system of “exogenous” and “endogenous” checks to ensure the alt-text accurately represented the image and image caption. We then evaluated the alt-texts by using an automated “rubric” and investigated the tool's ability to discern “Good” from “Bad” alt-text using developed examples. Results We compared these scores with human evaluators and found that although human and AI scoring does not always agree, the AI rarely rates alt-text low when a human rates it high, indicating that the AI is proficient at noticing high-quality alt-text. Furthermore, our quantified multi-dimensional evaluation step enables adaptability to automated corrections or training improvements. Finally, we find that although the resulting alt-texts still need human editing, the time required for this (compared with writing the alt-text from scratch) reduces the human time burden by 96%. Discussion This multistage approach aligns with current AI research and demonstrates the role AI can play in creating alt-text for large scientific datasets, while our findings reinforce the importance of using these tools alongside human expertise. In addition to performing the research, we add our real-world results to the Chandra X-ray Observatory public database to improve access to the images.

Authors

Institutions

Publication Details

Journal
Frontiers in Artificial Intelligence
Published
2026-11-18
DOI
https://doi.org/10.3389/frai.2026.1740372
Primary Topic
Multimodal Machine Learning Applications
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Accessing the universe via algorithm: a baseline study for automating alt-text generation with NASA data and AI

Tianze Hua, Jessica Schonhut-Stasik, Nancy Wolk, Kimberly K. Arcand et al.
2 links
Frontiers in Artificial Intelligence
Multimodal Machine Learning Applications
article

Accessing the universe via algorithm: a baseline study for automating alt-text generation with NASA data and AI

Tianze Hua, Jessica Schonhut-Stasik, Nancy Wolk, Kimberly K. Arcand, Areshva Aisha Mir, Anna Ohrt, Tom Sgouros, Sarah Kane
article en

Abstract

Introduction “Alt-text” is a textual substitute for visual information on digital platforms, primarily by users of screen readers. Effective image description is crucial for inclusive, accessible communication; unfortunately, alt-text creation is often limited by the time and skill constraints of the humans writing it. This baseline study aimed to develop an AI-driven method to semi-automate alt-text generation, with minimal expert moderation, for publicly released science images from NASA's Chandra X-ray Observatory. Methods We used pre-trained vision-language models and image metadata to generate alt-text; however, we found the results to be unreliable with this method alone. We iterated the process to include a two-tier fact-checking system of “exogenous” and “endogenous” checks to ensure the alt-text accurately represented the image and image caption. We then evaluated the alt-texts by using an automated “rubric” and investigated the tool's ability to discern “Good” from “Bad” alt-text using developed examples. Results We compared these scores with human evaluators and found that although human and AI scoring does not always agree, the AI rarely rates alt-text low when a human rates it high, indicating that the AI is proficient at noticing high-quality alt-text. Furthermore, our quantified multi-dimensional evaluation step enables adaptability to automated corrections or training improvements. Finally, we find that although the resulting alt-texts still need human editing, the time required for this (compared with writing the alt-text from scratch) reduces the human time burden by 96%. Discussion This multistage approach aligns with current AI research and demonstrates the role AI can play in creating alt-text for large scientific datasets, while our findings reinforce the importance of using these tools alongside human expertise. In addition to performing the research, we add our real-world results to the Chandra X-ray Observatory public database to improve access to the images.

Frontiers in Artificial IntelligenceVol. 9
Harvard University (US), Smithsonian Astrophysical Observatory (US), Vanderbilt University (US), University of Cambridge (GB), Brown University (US), Center for Autism and Related Disorders (US)
Marshall Space Flight Center
Multimodal Machine Learning Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon