Diffusion-based Data Augmentation for Object Counting Problems

Crowd counting, an important problem in computer vision, is commonly solved with deep learning approaches, such as convolutional networks and transformers. However, Deep networks often overfit when the available labeled crowd data is scarce. To overcome this, we have designed a pipeline that utilizes a diffusion model to generate extensive training data. We pioneer using diffusion models to generate images from high-density head location dot maps (a binary dot map that specifies the location of human heads) and are the first to use these diverse synthetic data to augment the crowd counting models. Our proposed smoothed density map input for ControlNet significantly improves ControlNet’s performance in generating crowds in the correct locations. Also, our proposed counting loss and guidance sampling for the diffusion model effectively minimize the discrepancies between the location dot map and the crowd images generated. Moreover, our versatile framework can be easily adapted to all kinds of counting problems. Extensive experiments demonstrate that our framework improves the counting performance on the ShanghaiTech, NWPU-Crowd, UCF-QNRF, and TRANCOS datasets.