PORTRAIT: A Hybrid Approach to Create Extractive Ground-truth Summary for Disaster Event

Nowadays, X (formerly known as Twitter) is an important source of information and latest updates during ongoing events, such as disaster events. However, the huge number of tweets posted during a disaster makes identification of relevant information highly challenging. Therefore, a summary of the tweets can help the decision-makers to ensure efficient allocation of resources among the affected population. There exist several automated summarization approaches that can generate a summary given the tweets related to a disaster. Development of these automated summarization approaches require availability of ground-truth summary of the dataset for verification. However, the number of publicly available datasets along with the ground-truth summary for disaster events are still inadequate. To improve this situation, we need to create more ground-truth summaries. Existing approaches for ground-truth summary generation rely on the annotators’ wisdom and intuition. This process requires immense human effort and significant time. Moreover, the selection of the important tweets from the humongous set of input tweets often results in sub-optimal choice of tweets in the final summary. Therefore, to handle these challenges, we propose a hybrid approach (PORTRAIT) for ground-truth summary generation, where we partly automate the procedure to improve the quality of ground-truth summary and reduce human effort and time. We validate the effectiveness of PORTRAIT on nine disaster events through quantitative and qualitative analysis. We prepare and release the ground-truth summaries for nine disaster events, which consist of both natural and man-made disaster events belonging to five different continents.