Training Data Diversity and Distribution Shift in AI-Generated Image Detection: An Empirical Study

The proliferation of high-fidelity generative models such as Stable Diffusion, DALL-E, MidJourney, and Gemini has made automated detection of AI-generated images an increasingly important research problem.While many published detectors report high in-domain accuracy, their performance under distribution shift is often overlooked.This paper presents an empirical study of binary real-versus-AI image classifiers, examining how training data composition and architecture choice affect accuracy on images from sources not seen during training.Models trained only on public data (ResNet18 and EfficientNet-B3) reached 53.33% and 61.67% on a 60-image held-out set of images from web-based generators and personal photographs, whereas a ResNet18 trained with an additional 360 self-collected images reached 75.00%, rising to 81.67% after threshold calibration.On unseen Tiny-GenImage samples (n=800) the three models scored between 72.12% and 74.75%, and the data-augmented model scored lower than the public-only ResNet18 on a 2,000image slice (68.85% vs. 72.60%),indicating a trade-off between the added sources and the original public distribution.Because the two architectures have near-identical parameter counts (11.18M and 10.70M) and the runs differ in training volume and schedule, we draw no conclusion about model capacity.The results suggest that adding even a small amount of data from the target sources can matter more than the choice between these two architectures.