"Just a Strange Pic": Rethinking 'Safety' in GenAI Image Safety Annotation Tasks from Diverse Annotators' Perspectives

Understanding what constitutes safety in AI-generated content is complex. While developers often rely on predefined taxonomies, real-world safety judgments also involve personal, social, and cultural perceptions of harm. This paper examines how annotators evaluate the safety of AI-generated images, focusing on the qualitative reasoning behind their judgments. Analyzing 5,372 open-ended comments, we find that annotators consistently invoke moral, emotional, and contextual reasoning that extends beyond structured safety categories. Many reflect on potential harm to others more than to themselves, grounding their judgments in lived experience, collective risk, and sociocultural awareness. Beyond individual perceptions, we also find that the structure of the task itself--including annotation guidelines--shapes how annotators interpret and express harm. Guidelines influence not only which images are flagged, but also the moral language used in justifications. Annotators frequently cite factors such as image quality, visual distortion, and mismatches between prompt and output as contributing to perceived harm-dimensions often overlooked in standard evaluation frameworks. Our findings reveal that existing safety pipelines miss critical forms of reasoning that annotators bring to the task. We argue for evaluation designs that scaffold moral reflection, differentiate types of harm, and make space for subjective, context-sensitive interpretations of AI-generated content.

Authors

Institutions

Publication Details

Journal
Proceedings of the AAAI/ACM Conference on AI Ethics and Society
Published
2025-10-15
DOI
https://doi.org/10.1609/aies.v8i3.36742
Primary Topic
Visual Attention and Saliency Detection
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

"Just a Strange Pic": Rethinking 'Safety' in GenAI Image Safety Annotation Tasks from Diverse Annotators' Perspectives

Aida Mostafazadeh Davani, Parul Mishra, Mark Díaz, Lora Aroyo et al.
Proceedings of the AAAI/ACM Conference on AI Ethics and Society
Visual Attention and Saliency Detection
article

"Just a Strange Pic": Rethinking 'Safety' in GenAI Image Safety Annotation Tasks from Diverse Annotators' Perspectives

Aida Mostafazadeh Davani, Parul Mishra, Mark Díaz, Lora Aroyo, Vinodkumar Prabhakaran, Ding Wang, Tian Huey Teh, Charvi Rastogi, Verena Rieser, Alicia Parrish, Zoe C. Ashwood, Roma Patel, Michela Paganini
article en

Abstract

Understanding what constitutes safety in AI-generated content is complex. While developers often rely on predefined taxonomies, real-world safety judgments also involve personal, social, and cultural perceptions of harm. This paper examines how annotators evaluate the safety of AI-generated images, focusing on the qualitative reasoning behind their judgments. Analyzing 5,372 open-ended comments, we find that annotators consistently invoke moral, emotional, and contextual reasoning that extends beyond structured safety categories. Many reflect on potential harm to others more than to themselves, grounding their judgments in lived experience, collective risk, and sociocultural awareness. Beyond individual perceptions, we also find that the structure of the task itself--including annotation guidelines--shapes how annotators interpret and express harm. Guidelines influence not only which images are flagged, but also the moral language used in justifications. Annotators frequently cite factors such as image quality, visual distortion, and mismatches between prompt and output as contributing to perceived harm-dimensions often overlooked in standard evaluation frameworks. Our findings reveal that existing safety pipelines miss critical forms of reasoning that annotators bring to the task. We argue for evaluation designs that scaffold moral reflection, differentiate types of harm, and make space for subjective, context-sensitive interpretations of AI-generated content.

Proceedings of the AAAI/ACM Conference on AI Ethics and SocietyVol. 8(3)
Google (United States) (US), Google DeepMind (United Kingdom) (GB)
Openalex Percentile: Top 71%
Visual Attention and Saliency Detection
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.