Eye Tracking-Based Substitution of Human Feedback for Image Quality Assessment in Diffusion Models

As diffusion models rapidly evolve, the need to incorporate manual feedback into their retraining processes has become increasingly urgent. This paper proposes a novel framework, ED3PO, to address this limitation and improve the objectivity and efficiency of image quality assessment in diffusion model retraining. The framework integrates the D3PO diffusion model with eye-tracking-based deep learning to automate the evaluation of image quality. Eye-tracking experiments were conducted with 200 participants to collect visual attention metrics during paired comparisons of image quality. Seven machine learning models were subsequently trained, with the best-performing Transformer model used to capture the relationship between eye-tracking data and human image quality assessments. The effectiveness of ED3PO retraining was validated through computational metrics and eye-tracking experiments, and it was further applied to four additional domains. By incorporating eye-tracking into the diffusion model retraining process, this study enables the automated assessment of image quality and the generation of higher-quality images. Furthermore, it offers a reference framework and future research directions for applying eye-tracking technology in the development of diffusion models.