MSEED: Human Preference Dataset for Multidimensional Safety Enhancement and Evaluation of Large Language Models

Large language models (LLMs) acquire extensive knowledge from vast datasets, showcasing exceptional performance across diverse tasks, including natural language processing. However, these datasets inevitably include a proportion of low-quality content, which can lead generative models to produce inappropriate outputs when dealing with sensitive information. Such outputs not only compromise user experience but also entail potential legal risks. To address these concerns, research institutions have actively pursued the development of safety alignment and evaluation datasets tailored for LLMs. Despite these efforts, existing datasets face significant limitations in terms of scale, applicability, and coverage, rendering them insufficient for the comprehensive safety alignment and rigorous evaluation required by LLMs. To bridge this gap, this study introduces the MSEED, a dataset designed to encompass multidimensional safety scenarios while maintaining high adaptability. MSEED provides robust support for both enhancing and evaluating the safety of LLMs. Our findings highlight pronounced deficiencies in the safety alignment of current LLMs. Through fine-grained alignment optimization, it is possible to significantly improve multi-dimensional security performance without affecting the overall capabilities of LLMs. Furthermore, it exhibits strong transferability across the languages supported by the model, significantly improving the security of model replies across multiple languages.