Enhancing Social Media Summarization with Instruction Tuned Large Language Models and Adaptive Summary Ranking

This study presents a multifaceted analysis of social network datasets, coupled with an evaluation of the efficacy of cutting-edge Large Language Models when applied to these datasets. Moreover, we conduct an in-depth analysis of the correlation between decoding parameters and the qualitative aspects of the generated summaries, with a particular focus on coherence and factual accuracy. As a novel contribution, we introduce an adaptive summary ranking mechanism that enables the selection of summaries from different models, enhancing the efficiency and adaptability of our approach. Our best-performing model, Llama-3.1-70B, achieves state-of-the-art results with ROUGE-1, ROUGE-2, and ROUGE-L scores of 0.3099, 0.1165, and 0.2552 on Reddit TIFU and of 0.2170, 0.0712, and 0.1799 on Webis-TLDR-17 datasets. By leveraging non-reference-based ASR techniques, we further enhance the ROUGE-1 score of the generated summaries from different models to 0.3142 for Reddit TIFU and to 0.2197 for Webis-TLDR-17.