Multi-label Image Classification of Protest Scenes in Bangladesh Using Vision Transformer on a Novel Dataset

In this study, we explore the analysis of images to detect visual attributes of protest scenes in Bangladesh through a multi-label classification model. By utilizing the powerful Vision Transformer (ViT) architecture with transfer learning, we identify multiple attributes simultaneously from visual data. Our research addresses the critical need for automated, objective analysis of crowd dynamics and law enforcement responses in the context of Bangladesh’s changing socio-political landscape. We introduce a dataset consisting of 5022 images collected from various protests across Bangladesh, each manually labeled with one or more of nine categories. These images cover scenarios from peaceful gatherings to violent confrontations, and capture various degrees of interaction involving police and civilians. They also include instances of property damage, aggressive actions, and broader contextual elements, providing a comprehensive view of protest scenes in the country. The ViT model, pre-trained on large-scale image datasets, was fine-tuned on our custom dataset to adapt to the specific nuances of Bangladesh’s protest scenarios. Our results demonstrate the efficacy of the proposed approach, achieving a high F1 score of 0.91 in multi-label classification across nine distinct categories. This research contributes to the field of computer vision by enhancing the understanding of protest scenes. The implications of our work extend to policy-making, human rights monitoring, and conflict resolution, offering a data-driven approach to understanding and potentially mitigating tensions in public demonstrations.