Automatic Sentiment Analysis of Citizen Comments: The Case of the Albania Earthquake

Collecting and analysing data after an earthquake is essential to determine its impact. In 2014, the European Mediterranean Seismological Centre launched the LastQuake system. Its app collects reports on the intensity users feel, along with comments that provide situational awareness. However, text data collected through crowdsourcing platforms is unstructured. Therefore, natural language processing techniques such as sentiment analysis and aspect-based sentiment analysis are necessary to extract meaningful information. On the 26 November 2019, following an earthquake in Albania, the LastQuake app recorded 28,220 reports with user comments. For the current analysis, we sampled comments posted on the exact day of the earthquake, in Albanian: 1678 comments (6%). The most frequent polarity detected in comments from LastQuake app users was negative (52%), followed by positive and neutral. However, manual classification is time-consuming and not feasible during the emergency phase. Therefore, we tested the accuracy of two automatic classification models for sentiment analysis: ‘troberta’ and ‘txlm’. These models were fine-tuned using already-classified text data from the 2020 Aegean earthquake. Using the manual classification as the reference to evaluate the accuracy of the automatic classification models for sentiment analysis yields accuracies of 71% for the ‘troberta’ model and 56% for the ‘txlm’ model.